lytedev/nix

Fork 2

fix(deck-slice): stop fabricating hand speed from a collapsed interval #911

Closed

lytedev wants to merge 2 commits from deck-slice-speed into deck-slice-cut

lytedev commented

2026-08-03 11:09:24 -05:00

Owner

Fixes item 2 — and it is not just a diagnostic bug. Now covers two independent mechanisms; the second was found because your run's intermittency did not fit the first.

Stacked on #910.

The cut path shares the computation

You asked me to check. It does — blade.speed feeds both the diagnostic and is_cutting(), so inflated values bypass the threshold and cause accidental slices. That is the real severity.

Mechanism 1 — a fabricated interval

Your body-scale hypothesis was reasonable but was not the cause; it was already floored. The culprit was the time denominator: speed was measured against the game's receive time, Time.get_ticks_msec() is millisecond-resolution, and the client drains every buffered line in one poll — so batched poses shared one timestamp, dt collapsed to zero, and the old code clamped it to 0.1ms. A normalised movement of 0.0001 became 4 body-heights/sec. The unbounded spikes also explain the monotonic-then-pinned shape.

Reproduced with a deliberately gentle wave (~1 body-height/sec) sent in batches: before 4.00 peak and one spurious slice; after 0.63 and none.

Fixed by timing against the daemon's ts_ms (the right clock was already in the protocol) and dropping unmeasurable intervals rather than clamping.

Mechanism 2 — a teleported keypoint

Correct timing does not help when the position jumps. A third of the frame in one entirely honest 16.7ms interval computes ~79–94 body-heights/sec — nearly 120x the cut threshold. Causes: MoveNet swapping left/right wrists as arms cross, losing and reacquiring joints through occlusion, and the tracker handing an existing player id to a different body when two people pass.

This is the better fit for your intermittency. A steady timing bug would be steady; tracking discontinuities are sporadic by construction.

Demonstrated, against a stream whose wrist jumps a third of the frame twice a second:

	peak	sliced
without the guard	94.29 body-heights/sec	yes
with it	0.00 (14 samples rejected)	no

And the batched-message reproduction is unchanged at 0.62 with zero rejections, so the guard costs nothing it should not.

Samples above 30 body-heights/sec are rejected. Measured reality is 7–14, so the ceiling is over twice the fastest motion ever observed. Rejected, not clamped — clamping to the ceiling would still be ~37x the threshold and would still slice. Position is kept and re-anchored so the next frame does not spike off the same jump. Rejections are counted into the debug readout, since this class of fault was hard to find precisely because it left no trace.

Body scale was hardened too, as you suggested: full 2D torso length rather than the vertical span alone (which collapses when a player leans while the torso is plainly still there), floored at 0.10.

Should this be fixed upstream too? Yes — and one part is a live daemon bug

You asked me to record the judgement, not act on it.

The game-side filter should stay regardless. A consumer that trusts a stream absolutely is fragile, the game owns its own threshold semantics, and defence in depth is right at a process boundary.

But two things belong in the daemon:

Player identity is squarely the daemon's job. PlayerId is documented as stable across occlusion and never recycled — a game should not have to defend against that guarantee being violated. Bbox IoU alone cannot disambiguate two people crossing, which is exactly when it breaks. The fix is adding keypoint similarity (OKS) to the assignment cost in track.rs, not just box overlap.
The daemon has the same exposure in its own recognisers, and it is currently live. recognise.rs:325 computes (dx*dx+dy*dy).sqrt() / dt / scale — the identical formula — and compares it against swipe_speed 1.5 and punch_speed 2.0. A teleport at 79–94 blows past both, so the daemon is emitting phantom swipe and punch events to every consumer on tracking discontinuities today. That is not hypothetical and it is not fixed by anything in this PR; it needs the same plausibility rejection applied daemon-side, where it would protect every future game instead of each one reinventing the guard.

A per-keypoint plausibility guard in the daemon would cover both, and the daemon already computes body scale so it has everything it needs. Happy to take that as a follow-up whenever you want it.

Item 3 — window geometry

Not a game bug. project.godot sets stretch/aspect="expand", which keeps the base width and expands the other axis to the window's aspect. Under a tiling compositor the game never gets the 1280x720 it requests — niri sizes it to the column — so 1280x4551 implies roughly 1:3.6. The letterbox coping is the system working as intended. Removing the oddity would mean going fullscreen, which is a behaviour change beyond a bug fix; say the word.

Checks

nix build .#deck-slice green (includes the framing check phase). Both reproductions verified from source. No daemon started and the camera untouched — synthetic streams only.

Untested

Not play-tested by a person since these fixes.
The 30 body-heights/sec ceiling is derived from measurements on one person at one distance. It has over 2x headroom, but a much faster player or a much closer one could in principle approach it — the rejection counter in the debug readout is how that would show up.

Fixes item 2 — and it is **not** just a diagnostic bug. Now covers **two** independent mechanisms; the second was found because your run's intermittency did not fit the first. **Stacked on #910.** ## The cut path shares the computation You asked me to check. **It does** — `blade.speed` feeds both the diagnostic and `is_cutting()`, so inflated values bypass the threshold and cause accidental slices. That is the real severity. ## Mechanism 1 — a fabricated interval Your body-scale hypothesis was reasonable but was not the cause; it was already floored. The culprit was the **time** denominator: speed was measured against the *game's receive time*, `Time.get_ticks_msec()` is millisecond-resolution, and the client drains every buffered line in one poll — so batched poses shared one timestamp, dt collapsed to zero, and the old code clamped it to 0.1ms. A normalised movement of 0.0001 became 4 body-heights/sec. The unbounded spikes also explain the monotonic-then-pinned shape. Reproduced with a deliberately **gentle** wave (~1 body-height/sec) sent in batches: **before 4.00 peak and one spurious slice; after 0.63 and none.** Fixed by timing against the daemon's `ts_ms` (the right clock was already in the protocol) and **dropping** unmeasurable intervals rather than clamping. ## Mechanism 2 — a teleported keypoint Correct timing does not help when the *position* jumps. A third of the frame in one entirely honest 16.7ms interval computes **~79–94 body-heights/sec** — nearly 120x the cut threshold. Causes: MoveNet swapping left/right wrists as arms cross, losing and reacquiring joints through occlusion, and the tracker handing an existing player id to a different body when two people pass. **This is the better fit for your intermittency.** A steady timing bug would be steady; tracking discontinuities are sporadic by construction. Demonstrated, against a stream whose wrist jumps a third of the frame twice a second: | | peak | sliced | | --- | --- | --- | | without the guard | **94.29** body-heights/sec | **yes** | | with it | 0.00 (14 samples rejected) | no | And the batched-message reproduction is unchanged at 0.62 with zero rejections, so the guard costs nothing it should not. Samples above **30 body-heights/sec** are rejected. Measured reality is 7–14, so the ceiling is over twice the fastest motion ever observed. **Rejected, not clamped** — clamping to the ceiling would still be ~37x the threshold and would still slice. Position is kept and re-anchored so the next frame does not spike off the same jump. Rejections are counted into the debug readout, since this class of fault was hard to find precisely because it left no trace. Body scale was hardened too, as you suggested: full 2D torso length rather than the vertical span alone (which collapses when a player leans while the torso is plainly still there), floored at 0.10. ## Should this be fixed upstream too? Yes — and one part is a live daemon bug You asked me to record the judgement, not act on it. **The game-side filter should stay regardless.** A consumer that trusts a stream absolutely is fragile, the game owns its own threshold semantics, and defence in depth is right at a process boundary. **But two things belong in the daemon:** 1. **Player identity is squarely the daemon's job.** `PlayerId` is documented as stable across occlusion and never recycled — a game should not have to defend against that guarantee being violated. Bbox IoU alone cannot disambiguate two people crossing, which is exactly when it breaks. The fix is adding keypoint similarity (OKS) to the assignment cost in `track.rs`, not just box overlap. 2. **The daemon has the same exposure in its own recognisers, and it is currently live.** `recognise.rs:325` computes `(dx*dx+dy*dy).sqrt() / dt / scale` — the identical formula — and compares it against `swipe_speed` 1.5 and `punch_speed` 2.0. A teleport at 79–94 blows past both, so **the daemon is emitting phantom swipe and punch events to every consumer on tracking discontinuities today**. That is not hypothetical and it is not fixed by anything in this PR; it needs the same plausibility rejection applied daemon-side, where it would protect every future game instead of each one reinventing the guard. A per-keypoint plausibility guard in the daemon would cover both, and the daemon already computes body scale so it has everything it needs. Happy to take that as a follow-up whenever you want it. ## Item 3 — window geometry Not a game bug. `project.godot` sets `stretch/aspect="expand"`, which keeps the base width and expands the other axis to the *window's* aspect. Under a tiling compositor the game never gets the 1280x720 it requests — niri sizes it to the column — so 1280x4551 implies roughly 1:3.6. The letterbox coping is the system working as intended. Removing the oddity would mean going fullscreen, which is a behaviour change beyond a bug fix; say the word. ## Checks `nix build .#deck-slice` green (includes the framing check phase). Both reproductions verified from source. No daemon started and the camera untouched — synthetic streams only. ## Untested - Not play-tested by a person since these fixes. - The 30 body-heights/sec ceiling is derived from measurements on one person at one distance. It has over 2x headroom, but a much faster player or a much closer one could in principle approach it — the rejection counter in the debug readout is how that would show up.

lytedev added 1 commit

2026-08-03 11:09:24 -05:00

fix(deck-slice): stop fabricating hand speed from a collapsed interval

/ check-format (push) Has been cancelled

Details

/ build (push) Has been cancelled

Details

7d09abb06d

The peak-speed diagnostic reported 727 body-heights/sec. That is not a
diagnostic bug — the same number feeds the cut test, so it was silently
bypassing the threshold and causing accidental slices.

Cause: speed was measured against the GAME`s receive time. `Time.get_ticks_msec`
has millisecond resolution and the client drains every buffered line in one
poll, so a batch of poses that arrived together all shared one timestamp. The
interval collapsed to zero and was clamped to 0.1ms, turning a normalised
movement of 0.0001 into 4 body-heights/sec.

Reproduced before fixing: a deliberately gentle wave (~1 body-height/sec) sent
in batches reported 4.00 and registered a slice. After the fix the same stream
reports 0.63 and slices nothing, while a real swing still slices and now
measures 7.55 against an analytic prediction of 7.54 — the old code reported
10.57 for that same stream, inflated by the same mechanism.

Fixes:

* Speed is timed against the DAEMON`s `ts_ms`, stamped at capture, so batched
  frames keep their true ~16ms spacing.
* Intervals too short to divide by are DROPPED, not clamped. Positions still
  advance for drawing; the next valid interval measures across the gap.
  Fabricating a dt is what caused this.
* Body scale — the denominator of every speed — is now the full 2D torso
  length rather than the vertical span alone, which collapses toward zero when
  a player leans or turns while the torso is plainly still there. Floored at
  0.10, with a deliberately non-small fallback when the torso keypoints are not
  confident, consistent with treating low-confidence keypoints as absent.
* The reported peak is now a decaying 3-second window. A run-forever maximum
  pins itself to the first spike and is useless for tuning afterwards, which is
  exactly how the 727 reading presented.

lytedev referenced this pull request

2026-08-03 11:19:02 -05:00

feat(deck-pose,deck-slice): draw the camera feed as the game background #913

lytedev added 1 commit

2026-08-03 11:25:18 -05:00

fix(deck-slice): reject physically impossible hand speeds

/ check-format (push) Successful in 11s

Details

/ build (push) Successful in 6m31s

Details

17e5e3349e

The interval fix in the previous commit removes one way a speed can be
fictional. It does not remove the other, and the remaining one is the better
match for the intermittency actually observed.

Correct timing does not help when the POSITION jumps. MoveNet swaps left and
right wrists as arms cross, loses and reacquires joints through occlusion, and
the tracker can hand an existing player id to a different body when two people
pass each other. Each teleports a wrist. A third of the frame in one entirely
honest 16.7ms interval computes ~79-94 body-heights/sec — nearly 120x the cut
threshold — so it slices targets the player never touched. Tracking
discontinuities are sporadic by construction, which fits an intermittent
symptom in a way a steady timing bug does not.

Demonstrated rather than argued. Against a synthetic stream whose wrist jumps a
third of the frame twice a second:

  without this guard: peak 94.29 body-heights/sec, targets sliced
  with it:            peak 0.00, 14 samples rejected, nothing sliced

and the earlier batched-message reproduction is unchanged at 0.62 peak with
zero rejections, so the guard costs nothing it should not.

Samples above 30 body-heights/sec are rejected. Measured reality is 7-14, so
the ceiling is more than twice the fastest motion ever observed and refuses
nothing a person can do.

REJECTED, not clamped: clamping to the ceiling would still leave a sample ~37x
the cut threshold, which still slices — the trap this is avoiding. The position
is kept, since it is probably where the hand now is, and re-anchored so the
next frame measures from there rather than spiking a second time off the same
jump.

Rejections are counted and printed in the debug readout. This class of fault
was hard to pin down precisely because it left no trace.

lytedev referenced this pull request

2026-08-03 11:29:34 -05:00

WIP: markerless full-body pose input (deck-pose daemon + deck-slice game) #915

lytedev commented

2026-08-03 11:30:29 -05:00

Author

Owner

Superseded by #915, which consolidates the whole markerless-pose-input program into a single WIP branch against main. Every commit from this PR is preserved there — nothing was squashed, so the root-cause writeups in the commit messages are intact. Closing here; review happens on #915.

Superseded by #915, which consolidates the whole markerless-pose-input program into a single WIP branch against `main`. Every commit from this PR is preserved there — nothing was squashed, so the root-cause writeups in the commit messages are intact. Closing here; review happens on #915.

lytedev closed this pull request

2026-08-03 11:30:33 -05:00