Vision innovations · competition edge
IMU-Gated Projective Prediction
+ Synthetic Aperture Monocular Depth.
Two physics-based vision techniques that tighten the gap between a single-frame YOLO detection and the gate pose a controller actually needs. IGPP uses IMU to predict where a gate will be in the next frame (fallback when detection drops). SAMD uses multi-frame parallax for 3–5× better PnP depth. Both run in <1 ms combined.
IGPP
16D EKF · IMU + vision fusion
short-horizon gate predictor
SAMD
Multi-frame triangulation
3–5× PnP depth accuracy
Role
Fallback + refinement for APEX chain
not in PPO obs
Budget
<1 ms combined
CPU-only
Post-2026-04-19 scope: IGPP and SAMD are fallback + refinement, not the primary perception. The APEX detector chain (YOLO11n + YOLO11n-pose + PnP) is the primary. IGPP kicks in when detection drops for <10 frames; SAMD refines PnP depth on a rolling window. Their outputs do not flow into the PPO observation — that would leak privileged state via the EKF's internal model.
§ 01IGPP — IMU-Gated Projective Prediction
What it does
When the gate detector drops a frame (occlusion, motion blur, edge case), IGPP uses the last confident gate pose + current IMU state to project where the gate should appear in the next frame. The controller gets a continuous gate estimate instead of a hole.
State (16D)
| Block | Dims | Contents |
|---|
| Drone pose (body-frame velocity, attitude quat) | 7 | from telemetry + integration |
| Gate pose (relative position, heading) | 4 | last vision observation, decayed |
| Gate velocity (body-frame, assumed zero mean) | 3 | handles drift + small gate sway |
| Bias terms (accel + gyro) | 2 | slow EKF drift correction |
Update cycle
# At every IMU tick (~100 Hz)
x, P = igpp.imu_propagate(imu_data, dt)
# At every vision detection (~10 Hz)
if det is not None:
x, P = igpp.vision_update(det.pose, det.confidence)
# Every frame: gate estimate
gate_pose = igpp.predict_gate_in_camera_frame() # always defined
Does not use absolute positioning. Relative gate pose only. The EKF integrates IMU in body frame against a last-seen gate bearing; there's no global NED state. Safe for the real AIGP sim.
§ 02SAMD — Synthetic Aperture Monocular Depth
Why single-frame PnP isn't enough
PnP on a single detection gives depth proportional to gate_width / pixel_width. At 5 m, a 1-pixel corner error = 15% depth error. For controller feedforward at 20 m/s, that's noise large enough to wobble.
SAMD idea
Collect the last K frames with their estimated camera poses (from IGPP). Treat them as a synthetic-aperture stereo rig. Triangulate gate corners across frames using bundle adjustment. The wider the baseline, the tighter the depth.
Parameters
| Parameter | Value | Notes |
|---|
| Window K | 8 frames | ~0.8 s at 10 fps |
| Min baseline | 0.5 m | Skip triangulation below |
| Reprojection threshold | 2 px | Reject outlier corners |
| Solver | Levenberg–Marquardt | scipy.optimize.least_squares |
| Runtime | ~0.6 ms | CPU, 32 observations |
Accuracy gain
| Condition | PnP (single) | SAMD (8-frame) | Improvement |
|---|
| Gate at 5 m | ±15% | ±4% | 3.8× |
| Gate at 10 m | ±25% | ±6% | 4.2× |
| Gate at 20 m | ±45% | ±12% | 3.7× |
§ 03How they plug into the APEX chain
YOLO11n
detector
Keypoints
4 corners
PnP
single-frame
IGPP EKF
fallback + IMU fuse
SAMD
depth refine
Controller
PID / PPO
Deployment order: detector → keypoints → PnP → IGPP (predict + fuse) → SAMD (depth refinement window) → controller. IGPP updates internal state on every vision detection; SAMD opportunistically refines when baseline is big enough.
§ 04What we don't claim
- IGPP is not a pose estimator. It predicts gate-in-camera, not drone-in-world. No absolute positioning. No NED.
- SAMD is not SLAM. It's bundle adjustment on a rolling 8-frame window against one gate, using egomotion from telemetry. When the drone banks hard or loses the gate, SAMD pauses and SAMD.has_estimate() returns False.
- Neither flows into the PPO observation. Doing so would leak the EKF's model-based state into training and break sim-to-real transfer. The PPO sees raw detector output + telemetry; IGPP/SAMD outputs are for the controller fallback only.