hans/party-player
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Screenshot from the server instead of the browser

Browse Source 
getDisplayMedia costs a picker, a permission, and a "Sharing this tab" bar
parked over the projector all night. None of it can be suppressed from the
page — the indicator is a security surface with no API, and the Chromium
switches that hide it were no help here.

The server runs on the machine the screen is plugged into, so it can take the
screenshot itself. server/screen.js drives grim for one region and /shot
returns the finished crop, which means grab() hands back a URL for an <img>
rather than a data URL off a canvas: same synchronous contract, so the
animations are untouched, and no tainting to work around either.

Coordinates line up because a fullscreen page's CSS pixels are the
compositor's logical pixels, which is the space grim's -g takes. The monitor
layout comes from hyprctl, so a projector is measured against itself rather
than the laptop panel, and -s pins the scale to that monitor: it keeps a
mixed-DPI desk deterministic, and it keeps grim native, where resampling costs
440ms against 35ms.

Needs a wlroots compositor and grim on PATH; without them the route reports
501 and the player falls back to getDisplayMedia, probed at load because that
call cannot be reached through an await without losing transient activation.
Also skipped when the page is not on localhost, where the server would be
photographing its own screen rather than the viewer's.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9v2Xoa6eoeGYpK6gp95k9
This commit is contained in:
simon
2026-08-10 22:49:36 +02:00
co-authored by Claude Opus 5
parent cc6ea58c7c
commit 0683acd69e
5 changed files with 351 additions and 68 deletions
Download Patch File Download Diff File Expand all files Collapse all files
public/player/capture.js
+130 -49
View File
@@ -1,30 +1,137 @@
// Screen capture, used to photograph the video frame.
// Screenshots of the video frame, from one of two backends.
//
// The decks are cross-origin YouTube iframes, so nothing on the page can read
// their pixels — canvas taints, html2canvas sees an empty box. Capturing the
// screen through getDisplayMedia and cropping out the frame's rect is the only
// way to get the actually-playing image.
// their pixels — canvas taints, html2canvas sees an empty box. Photographing
// the screen is the only way to get the actually-playing image.
//
// The crop assumes the captured surface *is* the viewport, so stream pixels map
// to CSS pixels by a single scale factor. Chrome can guarantee that with tab
// capture. Firefox has no tab capture at all — its picker offers only windows
// and screens — so there the operator shares the whole screen and the page goes
// fullscreen to match it. A shared window can never be mapped: it carries the
// tab bar and URL bar, and Wayland reports window.screenX/screenY as 0, so
// there is no offset to subtract.
// Preferred: the server takes the screenshot (see server/screen.js). It runs on
// the machine the projector is plugged into, so it can grab the region itself
// with no picker, no permission, and no "Sharing this tab" bar sitting over the
// party. The response is the finished crop, so `grab` hands back a URL and an
// <img> does the rest.
//
// Fallback: getDisplayMedia, for machines the server cannot screenshot. That
// path crops a live stream on a canvas, and carries all the caveats the server
// path removes — a picker, a sharing indicator, and on Firefox no tab capture
// at all, only whole windows and screens.
//
// Both need the page fullscreen, for the same reason: only then does a CSS
// pixel equal a screen pixel, and a shared window can never be mapped at all —
// it carries the tab bar and URL bar, and Wayland reports window.screenX as 0,
// so there is no offset to subtract.
// The server screenshots its own screen, which is the player's screen only when
// the two are the same machine. Opened from a laptop across the network, /shot
// would photograph the projector and crop the laptop's coordinates out of it.
const LOCAL_HOSTS = new Set(['localhost', '127.0.0.1', '[::1]', '::1']);
/**
* Ask for the screen-share stream. Resolves to a `{ grab, stop }` handle, or to
* null if the operator declines or the API is missing — callers degrade rather
* than fail.
* Whether the server can screenshot for us. Probed at load rather than at the
* click, because getDisplayMedia needs the click's transient activation and
* cannot be reached through an await.
*/
let serverShots = null;
if (LOCAL_HOSTS.has(location.hostname)) {
// Any rect will do; only the status code is read.
fetch('/shot?x=0&y=0&w=2&h=2&sw=0&sh=0', { method: 'GET', cache: 'no-store' })
.then((response) => {
serverShots = response.status !== 501 && response.status !== 404;
})
.catch(() => {
serverShots = false;
});
}
/**
* Ask for a screenshot source. Resolves to a `{ grab, stop }` handle, or to
* null if there is none — callers degrade rather than fail.
*
* `onIssue` receives a human-readable reason whenever screenshots are
* unavailable or stop lining up; without it these failures are invisible.
*
* Must be called synchronously from a user gesture (getDisplayMedia needs
* transient activation), so kick it off before any await and keep the promise.
* Must be called synchronously from a user gesture: the fallback's
* getDisplayMedia needs transient activation, so kick this off before any await
* and keep the promise.
*/
export function requestCapture({ onIssue = () => {} } = {}) {
if (serverShots) return Promise.resolve(openServerCapture(onIssue));
return requestDisplayMedia(onIssue);
}
/**
* True while the viewport covers the whole screen, which is the one case the
* crop maths can handle. Tested by dimensions rather than by
* `document.fullscreenElement`, because F11 fullscreen is browser-level and
* leaves that property null.
*/
function fillsScreen() {
return window.innerWidth === screen.width && window.innerHeight === screen.height;
}
/**
* Watch the viewport-covers-screen condition and report only its transitions:
* grab() runs as often as once per animation frame, and a stuck state would
* otherwise repeat the same notice forever.
*/
function trackAlignment(onIssue, initial) {
let mapped = initial;
if (!mapped) {
onIssue('Screenshots need the page fullscreen to line up with the screen — press F11.');
}
return function aligned() {
const now = fillsScreen();
if (now !== mapped) {
mapped = now;
onIssue(
mapped
? 'Screenshots are lined up again.'
: 'Screenshots paused: the page no longer fills the screen — press F11.',
);
}
return mapped;
};
}
// --- server screenshots -------------------------------------------------
function openServerCapture(onIssue) {
const aligned = trackAlignment(onIssue, fillsScreen());
let nonce = 0;
return {
/**
* A URL for a fresh photograph of `rect`, in CSS pixels. Safe to hand
* straight to an <img>: the server returns the crop itself, already sized.
* Null while the viewport does not cover the screen.
*/
grab({ x, y, width, height }) {
if (!aligned()) return null;
if (width <= 0 || height <= 0) return null;
const query = new URLSearchParams({
x: Math.round(x),
y: Math.round(y),
w: Math.round(width),
h: Math.round(height),
sw: screen.width,
sh: screen.height,
// The server sends no-store, but a nonce also keeps two stills taken at
// the same spot from collapsing into one request in flight.
n: String(++nonce),
});
return `/shot?${query}`;
},
// Nothing is held open: each screenshot is one request.
stop() {},
};
}
// --- getDisplayMedia fallback -------------------------------------------
function requestDisplayMedia(onIssue) {
// Undefined on a plain-HTTP origin, which is exactly how the README tells
// people to reach this page from other machines on the network.
if (!navigator.mediaDevices?.getDisplayMedia) {
@@ -67,21 +174,6 @@ function describe(error) {
return `Screen capture failed (${name}: ${error?.message || 'no detail'}) — running without screenshots.`;
}
/**
* True while the viewport covers the whole captured surface, which is the one
* case the crop maths can handle.
*
* `browser` is Chrome's tab capture: the stream is the viewport by definition.
* For a screen it holds only while the page is fullscreen — tested by
* dimensions rather than `document.fullscreenElement`, because F11 fullscreen
* is browser-level and leaves that property null.
*/
function mapsToViewport(surface) {
if (surface === 'browser') return true;
if (surface === 'window') return false;
return window.innerWidth === screen.width && window.innerHeight === screen.height;
}
async function attach(stream, onIssue) {
const [track] = stream.getVideoTracks();
if (!track) {
@@ -89,12 +181,16 @@ async function attach(stream, onIssue) {
return null;
}
// 'browser' is Chrome's tab capture: the stream is the viewport by
// definition, so it lines up whether or not the page is fullscreen.
const surface = track.getSettings?.().displaySurface ?? null;
if (surface === 'window') {
onIssue('Sharing a single window cannot be cropped — restart and share the whole screen instead.');
} else if (!mapsToViewport(surface)) {
onIssue('Screenshots need the page fullscreen to line up with the shared screen — press F11.');
}
const aligned =
surface === 'browser'
? () => true
: trackAlignment(onIssue, surface !== 'window' && fillsScreen());
const video = document.createElement('video');
video.srcObject = stream;
@@ -111,11 +207,6 @@ async function attach(stream, onIssue) {
const canvas = document.createElement('canvas');
const context = canvas.getContext('2d');
// The operator can leave fullscreen mid-party, so the mapping is rechecked on
// every grab. Only the transitions are reported: grab() runs per animation
// frame, and a stuck state would otherwise repeat the notice forever.
let mapped = mapsToViewport(surface);
return {
/**
* Crop a viewport rect out of the live stream. `rect` is in CSS pixels.
@@ -123,17 +214,7 @@ async function attach(stream, onIssue) {
* or while the viewport cannot be located inside the captured surface.
*/
grab({ x, y, width, height }) {
const nowMapped = mapsToViewport(surface);
if (nowMapped !== mapped) {
mapped = nowMapped;
onIssue(
mapped
? 'Screenshots are lined up again.'
: 'Screenshots paused: the page no longer fills the shared screen — press F11.',
);
}
if (!mapped) return null;
if (!aligned()) return null;
if (video.readyState < 2 || !video.videoWidth) return null;
if (width <= 0 || height <= 0) return null;
public/player/player.js
+8 -6
View File
@@ -49,14 +49,16 @@ async function start() {
if (started) return;
started = true;
// Before any await: getDisplayMedia needs the click's transient activation,
// which is gone the moment this function yields. It goes first for the same
// reason — it is the call that must not lose the activation to another API.
// Before any await: the getDisplayMedia fallback needs the click's transient
// activation, which is gone the moment this function yields. It goes first for
// the same reason — it is the call that must not lose the activation to
// another API. The server-screenshot path needs no gesture, but cannot be
// chosen without one, so this ordering has to hold either way.
const capturing = requestCapture({ onIssue: showNotice });
// Screenshots only line up while the viewport covers the whole captured
// surface. Firefox cannot share a bare tab, so the operator shares the screen
// and this makes the page match it. Harmless on the tab-capture path.
// Screenshots only line up while the viewport covers the whole screen, since
// that is what makes a CSS pixel a screen pixel. Chrome's tab capture is the
// one exception, and fullscreen does it no harm.
document.documentElement.requestFullscreen?.().catch(() => {
showNotice('Could not go fullscreen — press F11 so screenshots line up.');
});