deck-pose

Markerless full-body pose input: a camera-only body-tracking daemon that publishes a pose stream and discrete gesture events over Unix sockets, so a game can use a body as a controller. Modelled on the Nex Playground.

v4l2 capture (newest-frame-only) → MoveNet MultiPose Lightning via ONNX Runtime → IoU + Hungarian player tracking → One Euro smoothing → gesture recognisers → NDJSON on a socket, plus an optional second socket forwarding the camera's raw MJPEG for a video background.

Licence

GPL-3.0-or-later — see LICENSE.

Dependencies keep their own terms: ONNX Runtime is MIT, the ort bindings and most of the Rust tree are Apache-2.0/MIT, and the MoveNet weights are Apache-2.0. GPLv3 rather than GPLv2 is a requirement rather than a preference — Apache-2.0 is compatible with the former and not the latter, and the weights and ort are both Apache-2.0.

The consumer's THIRD-PARTY.md carries the full audit, including the provenance of the model re-export.

Read this first

• doc/protocol.md — the wire protocol. This is the API this repo owns and the contract consumers depend on. Consumers live in other repositories now, so a protocol change is not visible in one diff.
• doc/deck-pose.md — running it, the latency budget, the model rationale, and measurements from real cameras.
• doc/cross-platform.md — this is Linux only, and what it would take not to be. Read before promising a Windows or macOS build: capture and the publish socket are both platform-bound, and the cost is weeks rather than a weekend.
• doc/macos-port.md — the macOS port, measured on real hardware and deliberately deferred. Self-contained: build blockers, the capture contract a backend must implement, and the gotchas (CoreML is slower; camera permission fails silently).
• issues/open/ — known open defects. Notably, the daemon still emits phantom swipe/punch/jump events on tracking discontinuities.

Known consumer: deck-slice, a Fruit-Ninja-style slice game.

Using it

nix run git+https://git.lyte.dev/lytedev/deck-pose -- --help

As a flake input, with the NixOS module:

{
  inputs.deck-pose.url = "git+https://git.lyte.dev/lytedev/deck-pose";

  # in your host's modules:
  imports = [ inputs.deck-pose.nixosModules.default ];

  lyte.deck-pose = {
    enable = true;
    users = [ "you" ];              # camera access; granting it is a decision
    socket = "%t/deck-pose.sock";   # %t, never a literal /run/user/1000
    video = {
      enable = true;
      socket = "%t/deck-pose-video.sock";
    };
    service.enable = true;          # socket-activated; exits when idle
  };
}

The module pins the package itself, so importing it works without also applying overlays.default. Apply the overlay instead (or as well) if you want pkgs.deck-pose available generally.

Outputs

output	what
packages.<system>.deck-pose	the daemon (also .default)
overlays.default	adds pkgs.deck-pose
nixosModules.default	the lyte.deck-pose module (also .deck-pose)
checks.<system>	package (build + cargo test), clippy, fmt
devShells.<system>.default	rust toolchain, ORT_DYLIB_PATH preset, v4l-utils

Building and testing

nix build
nix flake check

The unit tests are pure arithmetic over the filter, tracker, config and publisher layers — no camera, no model file, no network, no timezone database — so they run inside the nix sandbox. Keep them that way: cargo test runs inside the build derivation, so a test that needs any of those does not fail loudly, it makes the package unbuildable.