The wire between sim and pilot.

1. Executive Summary

VQ1 is a downloaded Windows simulator that exposes a drone-racing course over the network. It streams a 640×360 forward-facing camera feed and a stream of telemetry packets; our process consumes both, runs perception and control, and sends four floats back. The interface is small. The geometry is fixed. The timing budget is ~50 ms per closed-loop iteration.

The AIGP team announced the launch for "late next week" (week of May 25, 2026) in their 2026-05-21 email. Roster lock is T−24h before the binary drops.

What VQ1 does not give you: GPS, depth, NED position, gate locations. This is by design — the same submission must run on the physical stage where none of those are available either.

Sections marked CONF are confirmed in VADR-TS-002. Sections marked ASSM are reasonable assumptions pending the sim binary drop. Sections marked UNK are flagged unknowns that first connect will resolve.

2. Interface Architecture

The sim and the pilot are two processes on the same Windows host. They share nothing but loopback sockets. There is no Python module embedded in the sim, no shared memory, no privileged callback into the renderer — the loop closes entirely through user-space network I/O.

This is the same pattern as PX4 SITL, AirSim, and Gazebo — there is no AIGP-specific SDK. Any language with a UDP socket and a JPEG decoder can implement a pilot. Our reference uses Python, OpenCV, and ultralytics.

2.1 One-tick sequence

The interaction across one render tick of the sim is best read as a UML-style sequence diagram. Wall-clock time flows top to bottom.

3. Video Channel

The forward camera frame arrives as motion-JPEG over UDP. CONF Geometry is fixed by VADR-TS-002 §3.7–3.8. Packet framing details below are TS-002-derived where possible, otherwise marked.

3.1 Receiver pattern

The pilot binds UDP:5600, receives one datagram per recvfrom(), and decodes with cv2.imdecode. No GStreamer pipeline, no RTP unpacking, no reassembly — keep it simple.

# vq1_completion_pilot.py — reference receiver
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 2_000_000)
sock.bind(('127.0.0.1', 5600))

while running:
    payload, _ = sock.recvfrom(65535)
    frame = cv2.imdecode(np.frombuffer(payload, np.uint8), cv2.IMREAD_COLOR)
    if frame is not None:
        on_frame(frame)            # 640×360 BGR ndarray

3.2 What a frame looks like

A representative annotated frame, showing the detector output, principal point, horizon line, and the 20° upward tilt offset that biases everything in image space toward the lower half of the frame.

3.3 Camera intrinsics

The intrinsic matrix is fixed and given in TS-002. The principal point sits at image center; pixels are square. Focal length is identical on both axes.

# Intrinsics — VADR-TS-002 §3.7
K = np.array([
    [320.0,   0.0, 320.0],
    [  0.0, 320.0, 180.0],
    [  0.0,   0.0,   1.0],
])
dist = np.zeros(5)                        # pinhole, no distortion
camera_tilt_rad = math.radians(20.0)     # +pitch up

4. Telemetry Channel

Telemetry is the privileged input we do get. It contains everything an on-board IMU + AHRS would produce — orientation, body angular rates, body linear acceleration — plus a simulator-time stamp for cross-channel synchronization. It does not contain absolute position, velocity, or any course landmarks.

4.1 Field schema (per Telemetry dataclass)

attitude_q

float32[4]

Orientation quaternion, body-relative-to-world, in (w, x, y, z) order. Unit-normalized.

body_rates

float32[3]

Angular velocity in body frame, radians per second. (p, q, r) about body X/Y/Z.

body_accel

float32[3]

Specific force in body frame, m/s². Includes gravity component projected into body axes.

t_sim

float64

Simulation time in seconds since run start. Monotonic. Use as the canonical timestamp for sync.

4.2 Wire layout — best-guess byte map

VADR-TS-002 fixes the field set but not the byte order. The most plausible packing is naturally-aligned little-endian — 48 bytes total, single datagram. Confirm against the first recvfrom().

4.3 Rate, ordering, derivation

What we derive from it:

1. Heading-hold — yaw from attitude_q as a setpoint reference between gate sightings.
2. Lost-sight propagation — when the detector misses a frame, propagate the last gate pose by integrating body_rates and body_accel for up to 30 frames.
3. Pitch / altitude trim — gravity vector projected through the attitude quaternion gives true-down; bias the altitude PID against that.
4. Control loop pacing — t_sim drives the PID integral term, not wall-clock.

5. Coordinate Frames

Three frames are in play. Getting these consistent is the single most common source of sign-flip bugs in a racing stack, so the convention is fixed here and referenced from every PnP and control call.

6. Gate Geometry & PnP

Two gate sizes exist in the AIGP catalog. VQ1 uses the smaller of the two exclusively. CONF

The detector emits four keypoints per gate, ordered TL → TR → BR → BL (image space). Combined with known metric corner coordinates and the camera intrinsic matrix, cv2.solvePnP with the IPPE square solver returns the gate's SE(3) pose in the camera frame.

7. Control Channel

Four floats, sent back to the sim. The body of the entire control surface is this small.

7.1 ControlCmd schema

throttle

float32

Normalized collective thrust, 0.0 (off) to 1.0 (max). Hover ≈ 0.55 at the SimDrone weight.

roll

float32

Roll rate setpoint, normalized −1.0 (max left) to +1.0 (max right). Body X axis.

pitch

float32

Pitch rate setpoint, normalized −1.0 (max nose-down) to +1.0 (max nose-up). Body Y axis.

yaw

float32

Yaw rate setpoint, normalized −1.0 (max CCW) to +1.0 (max CW). Body Z axis.

7.2 Throttle envelope

Throttle is the only non-symmetric axis. The thrust map is roughly affine in the operational range, but the hover point sits well above 0.5 — this is the single quirk most likely to bite a naïve controller.

7.3 Send pattern

Send one command per inbound frame — the pilot is frame-driven, not clock-driven. If the detector loses sight, the pilot still emits a fresh command derived from propagated state; never let the control channel go silent.

# vq1_completion_pilot.py:ControlCmd → wire
@dataclass
class ControlCmd:
    throttle: float            # [0, 1]
    roll:     float            # [-1, +1]
    pitch:    float            # [-1, +1]
    yaw:      float            # [-1, +1]

def send(cmd: ControlCmd):
    payload = struct.pack('<ffff', cmd.throttle, cmd.roll, cmd.pitch, cmd.yaw)
    sock.sendto(payload, sim_addr)   # <= 16 bytes

8. Closed-Loop Timing Budget

The pilot must complete photon-to-PWM in under 50 ms to track a 6 m/s drone through a 1.5 m gate with margin. Measured per-stage on the training rig (RTX 5080, i9, Windows), the budget breaks down as follows:

9. Pilot State Machine

VQ1 doesn't need a planner — the next gate is whichever gate is in front of us. The state machine is correspondingly small: two states and one timer.

10. What the Sim Drop Resolves

VADR-TS-002 pins almost the entire interface, but three details are not real until the binary is in hand. The pilot is structured so that all three are local edits, not redesigns.