Component wiring · how it all connects
Pipeline integration.
How the APEX detector chain, telemetry adapter, IGPP fallback, and controller bolt together inside the sim client. This is the plumbing — less about algorithms, more about data flow, concurrency, and error handling.
Pattern
Async pipeline · single-frame tick
no queues
Entry
race_pipeline.py
orchestrator
Sim adapter
Stub today · real adapter May 2026
transport TBD
Budget
<50 ms per tick
Orin NX target
§ 01Component responsibilities
| Component | Owns | Emits |
|---|
sim_client (stub → real) | Transport to the sim. Connection, handshake, recv/send. | (frame_bgr, telemetry) per tick |
ApexDetectorChain | Phase 1 YOLO + Phase 2 keypoints models | Detection(bbox, corners, conf) |
PnPSolver | Camera intrinsics, gate-3D template | Gate pose in camera frame |
IGPPEKF | 16D EKF state, IMU propagation, vision update | Smoothed gate pose + visibility flag |
SAMDRefiner | K-frame window of (pose, corners) | Refined depth when baseline sufficient |
TargetTracker | Which gate is "next" | Single target Detection per tick |
Controller | PID (VQ1) or PPO (VQ2) | ControlCmd(throttle, roll, pitch, yaw) |
Logger | JSONL + PNG capture | (async, off-path) |
§ 02Data flow (per tick)
sim_client.next_frame()
│
▼
(frame_bgr, telemetry)
│
▼ ┌─► ApexDetectorChain ─► detections
│ │ │
│ │ ▼
│ │ TargetTracker ─► target
│ │ │
│ │ ▼
│ │ PnP ─► gate_cam
│ │ │
│ │ ┌───────────────────────────────────┘
│ │ │ ┌─► SAMD.refine ───┐
│ │ ▼ │ ▼
│ └─► IGPP.update_from_vision(gate_cam) gate_cam_refined
│ │
▼ ▼
IGPP.update_from_imu(telemetry)
│ │
└──────────┬───────────────────┘
▼
Controller.step(gate_cam_refined, telemetry)
│
▼
ControlCmd
│
▼
sim_client.send(cmd)
│
▼ (async)
Logger.write
§ 03Concurrency
- Per-tick pipeline is synchronous. No thread pools, no queues between stages. Deterministic ordering. Each stage awaits the previous.
- Logger runs async, off-path. Writes to disk don't block the next tick. Bounded in-memory ring buffer — if disk falls behind, drop the oldest entries, not the next command.
- Telemetry may arrive faster than frames. IMU telemetry at 100 Hz, camera at 10–60 Hz. A background coroutine accumulates telemetry between frames; the main loop samples the latest on each tick.
- PPO parallel env instances (training only): each sim instance runs in its own subprocess via
SubprocVecEnv. They don't share Python GIL.
§ 04Error handling
| Failure | Handling | User-visible effect |
|---|
| Detector returns empty | TargetTracker marks "lost", state → SEEK | Pilot slow-forwards + yaw sweep |
| PnP solve fails (degenerate corners) | Skip PnP, use IGPP prediction | Smooth continuity; no command jump |
| Telemetry payload malformed | Use last valid telemetry, log warning | One-frame stale; recoverable |
| sim_client.send fails | Retry once; if still fails, log + escalate | Sim assumed dropped; we abort run |
| IGPP EKF diverges (P matrix large) | Reset to last confident vision state | One-frame discontinuity, auto-recovers |
| CUDA OOM mid-run | Pipeline catches, logs, crashes the run | Attempt logged as failed; restart |
§ 05Config surface (race_config.py)
@dataclass
class RaceConfig:
vision: VisionConfig # detector mode, conf thresholds, SAMD window
controller: ControllerConfig # PID gains, PPO weights path
connection: ConnectionConfig # sim host/port, anti-cheat token, heartbeat
race: RaceRuleConfig # max_time_s, gate sequence, retry policy
logging: LoggingConfig # dirs, compression, frame capture rate
gate: GateConfig # width_m, height_m, transit threshold
@classmethod
def from_yaml(cls, path): ...
def snapshot(self) -> dict: ... # save with every race log
Every race log includes the full config snapshot. Reproducibility is not a nice-to-have for AIGP — Anduril can request code review. A reviewer should be able to re-run any artifact from deterministic inputs.