total frames
4,000
CURRENT TRAINING RUN
DETECTOR · EPOCH 142/200
apex-v2.0 · YOLO11n-pose
mAP50
0.952
▲ +0.043 vs v1.5
mAP50-95
0.687
▲ +0.062 vs v1.5
latency
5.8ms
RTX 5080 · batch 1
eta
1h 42m
to phase 1 done
APEX PIPELINE · detector → keypoints → segmentation → policy
per train_apex.py CLI view test course →PHASE 1 · APEX DETECTOR
①
RUNNING
YOLO11n · recall-maximized augmentation · single gate class · <7 ms on Orin NX target
PHASE 2 · APEX KEYPOINTS
②
QUEUED
YOLO11n-pose · bbox + 4-corner keypoints → PnP for 6-DoF gate pose
PHASE 3 · APEX POLICY
③
QUEUED
PPO · 24-D observation · perception-aware reward (cos boresight, gate) · MLP [256,256,256]
DATASET · pq-2026-05-26
SYNTHETIC · 640×360 · JPEG q=82 regenerate →labeled gates
11,847 · 2.96 / frame
train / val
3,600 / 400 · 90/10
course layouts
80 random · 50 fpc
pose jitter
±8° · ±0.3 m
light jitter
±15% · enabled
gate kinds covered
start, std, dgate, sturn, dive, finish
empty frames
~6% · negative samples
MODEL PERFORMANCE · LATEST EPOCH
epoch 142 of 200mAP@50
0.952
target ≥ 0.95
mAP@50-95
0.687
+0.062 vs v1.5
Precision
0.928
@ conf 0.18
Recall
0.971
recall-maximized
F1
0.948
PR-balanced
Inference (RTX 5080)
5.8 ms
172 fps · batch 1
Inference (Orin NX est)
6.8 ms
target < 7 ms
Box loss
0.024
flat 30 ep
Pose loss
—
phase 2
VRAM
3.2 GB
of 16 GB · batch 32
GPU util
94 %
RTX 5080 · 360W TGP
PCIe throughput
12 GB/s
data loader OK
TRAINING CURVES
phase 1 · livemAP@50 vs epoch
v2.0 (current)
v1.5 (baseline)
target
Box loss vs epoch
train
val
HYPERPARAMETERS · best-known
RTX 5080 · 16GB| PARAM | VALUE | REASONING |
|---|---|---|
| imgsz | 640 spec | Matches VADR-TS-002 §3.7 native resolution. Training & inference at the same resolution = no domain gap. Don't upscale. |
| batch (phase 1) | 32 | Comfortable on 16 GB at 640 imgsz with default YOLO11n. Larger batch (64+) helps stabilize but VRAM-bound on aug-heavy configs. |
| lr0 / lrf | 0.01 / 0.001 | Standard YOLO11n schedule. Cosine. Lower lr0 (0.005) only if loss diverges in first 5 epochs. |
| epochs (phase 1) | 200 w/ patience 30 | Synthetic data converges fast (mAP plateaus by ~120). Patience 30 stops early once val mAP plateaus. |
| conf threshold | 0.18 recall-max | Neyman-Pearson: FN cost (~2s, lose race) >> FP cost (~0.1s, verify+discard). Optimal is 0.15–0.25, NOT default 0.5. |
| iou threshold (NMS) | 0.45 | Default. Lower means more aggressive de-duplication; gates are large enough that 0.45 is fine. |
| fliplr | 0.5 aug | Critical for keypoint pose: respects flip_idx: [1, 0, 3, 2]. Horizontal flips double effective dataset. |
| flipud | 0.0 | Don't enable. Drone always upright in real footage; vertical flips create unrealistic poses. |
| mosaic | 1.0 → 0.0 (close at ep 10) final ep | Mosaic helps early, hurts later — gate keypoints get distorted by mosaicking. Disable in last 10 epochs. |
| mixup | 0.0 disabled | Mixup blends two images; corrupts keypoint geometry for pose head. Don't enable. |
| hsv_h / hsv_s / hsv_v | 0.015 / 0.6 / 0.4 | Color jitter helps generalize from synthetic to real lighting. Saturation jitter (0.6) is the most important — synthetic blues are too clean. |
| perspective | 0.0001 | Subtle perspective warp — represents real lens distortion. Don't go higher; deforms gate squares unrealistically. |
| pose loss weight (phase 2) | 12.0 tuned | Default 12.0 works. Higher (15+) makes keypoints more accurate but bbox accuracy drops. Don't tune unless you ship a multi-task evaluator. |
| workers | 8 | RTX 5080 fed by 8 cores → data loader is not the bottleneck. Lower if disk I/O saturates. |
⚠ PAST LEARNINGS · gotchas from prior runs
read before training
CRITICAL
W&B + Ultralytics on Windows · pre-init wandb.init()
Ultralytics 8.4.32 wb callback does
str(trainer.args.project).replace("/", "-") but doesn't strip :.
When project= is an absolute Windows path, the drive-letter colon survives and W&B's pydantic validator
rejects the project name. Training crashes before epoch 1.
Fix: always call
wandb.init(project="aigp-gate-detector", name=run_name, reinit=True) BEFORE model.train(...).
The callback has if not wb.run: guard that skips its broken path when pre-initialized.
RESOLVED · still relevant
Phase 3 observation mode · default flipped to
detector_telemetry 2026-04-24
Legacy Phase-3 PPO read privileged gate positions in world NED from
course_map_test.json.
That ground-truth is in SimDrone but NOT in the real AIGP sim.
Status:
--observation-mode detector_telemetry is now the DEFAULT
(28-D obs: FPV detections + telemetry, ships safely). The legacy 24-D privileged path is opt-in via
--observation-mode privileged — SimDrone-only, do NOT ship. The dashboard's PRIVILEGED
quick-command button warns you.
WARN
Mosaic augmentation breaks pose keypoints in late epochs
Mosaic stitches 4 images into one and resizes — gate keypoints get warped and ground-truth
coordinates no longer match the visual feature exactly. Hurts pose head accuracy after ~150 epochs.
Fix: set
close_mosaic=10 (default in Ultralytics 8.4+) so mosaic
is disabled for the last 10 epochs. Final epochs train on clean single-image batches.
WARN
Synthetic-to-real lighting gap · crank saturation jitter
Synthetic gates rendered uniform blue (per spec) are too clean. Real ADNEC Marina Hall has color
temperature shifts, ambient bleed, and CCD response variance.
Fix:
hsv_s=0.6 in augmentation. If real-world frames available,
mix in 10% real frames in the train set — synthetic→real domain gap closes by ~12 mAP.
INFO
Conf threshold · 0.18 not 0.50 (Neyman-Pearson)
FN cost (miss gate, enter blind-seek, lose ~2s) is ~20× FP cost (verify + discard, ~0.1s).
By the N-P lemma, optimal operating point shifts toward higher recall.
Fix: set
conf=0.18 at inference time. The detector outputs more
candidates; downstream PnP + tracker filter the false positives. Don't use Ultralytics default 0.50
— leaves ~3% recall on the table.
INFO
SLAM is not a fit for drone racing
Monocular SLAM (LingBot-Map, ORB-SLAM3, etc.) fails at racing velocities — keypoint tracking breaks
above ~8 m/s due to motion blur + parallax. The MonoRace winner used detect+keypoint+PnP+EKF instead.
Fix: stay with the detect → 4-corner keypoint → PnP pipeline.
EKF handles state estimation between gate detections. No SLAM backend needed.
INFO
Perception-aware reward · the Swift "secret sauce"
Add
cos(camera_boresight, gate_direction) as an auxiliary reward term in PPO.
Teaches seek-attack cycle without a state machine:
far/lost → pitch up + scan, found → pitch down + thrust, close → level + keep in FoV.
Fix: already in train_apex.py Phase 3. Don't disable. Coefficient 0.3 is the sweet spot.
QUICK START · commands
copy-paste ready
$
cd C:\Users\pc\Downloads\grandprix-latest
repo root
$
# 1. Download dataset from /train-pq-gen → extract → confirm structure
manual
$
python -c "from pathlib import Path; print(sum(1 for _ in Path('dataset_gates_synthetic_pose_pose/images/train').iterdir()))"
count train frames
$
python race/train_apex.py
run all phases · ~7.5h
$
python race/train_apex.py detector --epochs 200
Phase 1 · YOLO11n bbox · ~2h
$
python race/train_apex.py keypoints --epochs 150
Phase 2A · YOLO11n-pose corners · ~1.5h
$
python race/train_apex.py segmentation
Phase 2C · U-Net gate masks (optional)
$
python race/train_apex.py policy --steps 10000000
Phase 3 · PPO · default obs = detector_telemetry (safe to ship)
$
python race/train_apex.py policy --observation-mode privileged
⚠ priv obs · SimDrone dev only · DO NOT ship
$
python race/train_apex.py eval
evaluate all trained models
$
python race/train_apex.py export
export to TensorRT/ONNX for Orin NX deploy
$
python race/vq1_completion_pilot.py --probe runs/pose/apex_yolo11n_pose/weights/best.pt
smoke-test new weights on one sample frame
$
yolo export model=runs/pose/apex_yolo11n_pose/weights/best.pt format=engine half=True device=0
TensorRT FP16 export for Jetson Orin NX deploy
RUN HISTORY · last 8
W&B · aigp-gate-detector view all on W&B →| RUN | DATE | MODEL | DATASET | EPOCHS | mAP@50 | mAP@50-95 | RECALL | LATENCY | STATUS |
|---|---|---|---|---|---|---|---|---|---|
| apex-v2.0 | 2026-05-26 | YOLO11n-pose | pq-2026-05-26 (4k) | 142/200 | 0.952 | 0.687 | 0.971 | 5.8 ms | RUNNING |
| apex-v1.5 | 2026-05-18 | YOLO11n-pose | pq-2026-05-12 (2k) | 200/200 | 0.909 | 0.625 | 0.943 | 6.1 ms | DONE |
| rf-detr-base | 2026-05-10 | RF-DETR-base | pq-2026-05-12 (2k) | 50/50 | 0.924 | 0.654 | 0.918 | 18.7 ms | DONE |
| yolo26n-s1 | 2026-05-04 | YOLO26n | pq-2026-05-04 (2k) | 180/200 | 0.891 | 0.598 | 0.902 | 4.2 ms | DONE |
| apex-v1.4-mosaic | 2026-04-28 | YOLO11n-pose | pq-2026-04-22 (1.5k) | 180/200 | 0.864 | 0.541 | 0.882 | 6.0 ms | DONE |
| apex-v1.3-priv | 2026-04-19 | YOLO11n + PPO | pq-2026-04-12 (1k) | 10M st | — | — | — | — | DISCARDED · priv obs |
| baseline-yolo11n | 2026-04-10 | YOLO11n | kaggle-gates (open) | 100/100 | 0.812 | 0.476 | 0.851 | 5.5 ms | BASELINE |
| smoke-2ep | 2026-04-23 | YOLO11n | dataset_gates_yolo | 2/2 | 0.904 | 0.502 | 0.918 | 5.4 ms | W&B SMOKE |
HARDWARE · rig + deploy target
RTX 5080 · Orin NXTraining rig
RTX 5080 16GB
1000W PSU · 360W TGP stock
Phase 1 train time
~2 h
200 ep · batch 32
Phase 2 train time
~1.5 h
150 ep · batch 16
Phase 3 train time
~4 h
10M steps · 4 envs
Total · overnight
~7.5 h
start by 22:00, done by 06:00
Deploy target
Jetson Orin NX
100 TOPS · onboard
Target latency budget
19 ms
det 7 + pose 5 + PnP 2 + ctrl 5
Export format
TensorRT FP16
.engine · 2× speedup