Live Camera Coarse Spatial Detection

This skill provides coarse-grained spatial relationship detection: in an RTSP
camera feed, it decides whether a target of a given class appears where it
shouldn't be (a pet on the dining table, someone entering a restricted zone).
On a suspicious match, an annotated screenshot is saved to disk.

This skill cannot distinguish behavioral details — whether a dog on the table
is stealing food or just sleeping, or whether a pet near cables is passing by or
chewing them. Those fine-grained judgments are made by the Agent after reading
the screenshot. The skill itself never notifies the user.

User intent ("Watch the dog — don't let it steal food from the table")
       ↓  Agent maps to
  Spatial relationship (subject=dog, surface=dining table, on_or_over)
       ↓
  This skill: YOLO coarse detection → trigger → annotated screenshot saved
       ↓
  Agent reads screenshot → fine-grained judgment
       ↓
  Confirmed violation → notify user; false positive → stay silent

When to Use

• The user wants to monitor a live camera for coarse presence/spatial events:
  pets on furniture, someone entering a zone, objects where they don't belong.
• An RTSP stream URL is available (obtain it via @xpai-camera-control or any
  other source).
• The deliverable is "a trigger plus a screenshot worth looking at" — fine
  behavior judgment is delegated to the Agent.

When NOT to Use

• The user needs identity tracking across cameras ("where did this person
  go?") — use the sibling skill rtsp-multi-tracker instead.
• The target object is outside the model vocabulary (e.g. cables, knives) and no
  proxy surface class fits — a custom-trained model is required (see
  references/configuration.md).
• The user expects this skill to notify them directly — it doesn't; the Agent
  must review screenshots and decide.

Workflow

1. Obtain an RTSP stream URL

Use @xpai-camera-control (or the user's own source) to get a live
rtsp://... URL. This skill does not manage cameras — it only consumes streams.

2. Map intent to scenarios

Write the stream URL into scripts/settings.json → rtsp_url (or pass --rtsp)
and declare monitoring scenarios in scenarios[]. The Agent must translate the
user's intent into COCO-class combinations:

• "Dog stealing food from the table" → subject: dog, surface: dining table,
  relationship: on_or_over
• "Cat entered the bedroom" → subject: cat, surface: bed (bed as zone proxy),
  relationship: on_top
• "Person in a restricted zone" → subject: person, surface: <proxy object>,
  relationship: within

Full field reference plus the semantic mapping guide (COCO class table, mapping
principles, blind spots): references/configuration.md.

3. Get model weights (first run)

No model weights ship with the skill (publishing constraint) — neither
yolo26n.pt nor yolo26n.onnx is bundled. Obtain and verify them once:

python scripts/setup_model.py --onnx          # .pt (if missing) -> local export -> verify .onnx
python scripts/setup_model.py                 # .pt only (ultralytics backend)
python scripts/setup_model.py --from <path>   # import an existing local .pt, then verify it
python scripts/setup_model.py --check         # verify existing .pt offline
python scripts/setup_model.py --onnx --check  # verify existing .onnx offline

setup_model.py embeds no URLs — the .pt download goes through the installed
ultralytics official-asset mechanism and is checked against the SHA-256
fingerprint this skill was validated with (mismatch exits non-zero). If a
verified copy already sits elsewhere (old directory / cache / offline media),
--from <path> copies it into scripts/ and runs the same check — no download
needed. The .onnx is generated locally from that verified .pt via the
ultralytics exporter (no network) and checked against a semantic fingerprint
(weights + graph + attributes; metadata-independent — ONNX export embeds a
timestamp).

4. Start detection (persistent background process)

python scripts/main.py --rtsp "rtsp://..."

ONNX lightweight deployment (no torch; pip install opencv-python numpy onnxruntime):

python scripts/main.py --rtsp "rtsp://..." --model yolo26n.onnx --backend onnx

The skill then runs continuously: read stream → YOLO inference → spatial check →
save annotated screenshot on trigger. The Agent can work on other tasks and poll
the screenshot directory periodically.

Useful flags (full table in references/configuration.md):

5. Review triggers (Agent loop)

Each trigger writes into the output_dir tree (shipped default: ./captures,
resolved against the working directory — see
references/agent-review.md for details):

captures/
├── capture_<ts>.jpg            raw trigger frame
├── capture_<ts>_annotated.jpg  same frame with the relation drawn
├── pending/
│   ├── <event_id>.jpg          annotated frame for review
│   │                           (red box = subject / blue box = surface)
│   └── <event_id>.json         event metadata (scenario / subject / conf / ...)
└── reviewed/
    └── <event_id>.json         verdict written back by the Agent (optional)

Agent review loop:

1. Read pending/<event_id>.jpg and pending/<event_id>.json.
2. Judge against the user's original intent (stealing? just sleeping? passing by?).
3. Write the verdict to captures/reviewed/<event_id>.json — fields confirmed,
   verdict, reason, confidence; the running HUD picks it up.
4. Confirmed violation → notify the user with the screenshot; false positive →
   stay silent.

Known Limitations

• COCO 80 classes only — objects outside the vocabulary (e.g. cables) cannot
  be detected directly. Approximate with a proxy surface class, or plug in a
  custom-trained model via model.class_names.
• No ROI / polygon zones — zones are approximated by detected objects (use
  "bed" as a bedroom proxy).
• OR semantics — subject: ["cat", "dog"] means cat or dog; a single
  scenario cannot express AND / NOT / time windows.
• Single-frame geometry — a pet lying on the table and a pet standing on it
  produce the same trigger; this is exactly why Agent review exists.
• No notification — the skill only writes screenshots; notifying the user is
  the Agent's responsibility.

References

• references/configuration.md — settings.json & scenario fields, full CLI table, semantic mapping guide
• references/agent-review.md — pending/ content, event metadata, verdict write-back format, notification policy
• references/tuning.md — relationship thresholds, model choice, RTSP hardening, downstream integration
• references/troubleshooting.md — FAQ: config errors, offline model install, stream/connection diagnosis, anti-patterns