ROS 2 bridge

vig_bridge connects ROS 2 camera topics to the governor. It subscribes to sensor_msgs/Image, turns each frame into the model's input tensor, and sends it over the Open Inference Protocol with the freshness information the governor needs: how old the frame is, which capture it belongs to, and which camera it replaces. Results come back on a topic next to the camera; every decision the governor makes about a frame is published as an event.

It is a separate Python process (rclpy + gRPC). Nothing native is linked into the governor (ADR-0033).

Package: integrations/ros2/vig_bridge (ament_python, ROS 2 Jazzy; Humble uses the same API).

Setup

# in your ROS 2 workspace
cp -r integrations/ros2/vig_bridge src/
pip install "tritonclient[grpc]"          # gRPC stubs for the Open Inference Protocol
colcon build --packages-select vig_bridge
source install/setup.bash

ros2 run vig_bridge vig_bridge --ros-args \
  -p governor:=127.0.0.1:9001 \
  -p topics:="['/front/image_raw', '/rear/image_raw']" \
  -p models:="['detector', 'detector']" \
  -p capture_groups:="['front', 'rear']" \
  -p supersession_keys:="[1, 2]"

The governor must list the models under the names you give in models — the bridge asks it for their input shape at start-up.

Parameters

topics, models, capture_groups and supersession_keys are parallel lists and must have the same length. Unequal lists are refused, not padded.

What gets published

A refusal is information, not an error. A frame that was superseded by a newer one, or that would have been too old when finished, is the governor doing its job. The bridge publishes it as an event of kind refusal and does not resend that frame — the next one is fresher. Only a transport error (the governor did not answer at all) leaves open whether a frame arrived.

A result the governor completed but marks as obsolete (vig_obsolete) is still published, with "obsolete": true. Whether to use it is the application's decision.

The JSON result message is deliberately generic: output decoding is model-specific and belongs to the application. A production consumer would decode the outputs into a typed message such as vision_msgs/Detection2DArray.

Clocks

The governor works on its own monotonic clock. A ROS stamp comes from another time base, so the bridge never sends an absolute time. It sends an age: now − header.stamp, both read from the bridge node's ROS clock, at the moment the request is sent — conversion and copying are included.

That age is only right if two things hold:

• Camera and bridge share a clock, or their clocks are synchronised. If the camera driver runs on another machine without PTP or chrony, every age is off by the offset between the two clocks.
• The ROS clock runs at real time. With use_sim_time, ages come out in simulated seconds, while the governor measures deadlines in real ones. A simulation that runs slower, faster or pauses produces ages that mean nothing. The bridge warns at start-up; set age_source: arrival in that case — the governor then counts from arrival, which is honestly wrong instead of quietly wrong.

A stamp in the future is clamped to age zero. Beyond max_clock_skew_us it is also reported as a clock_skew event: a negative age would promise the governor a fresher frame than exists.

Shared memory

When the governor is reachable on loopback, the bridge writes each frame into a POSIX shared-memory region and sends only a reference; the governor and the backend read it in place (ADR-0003). For camera frames this is the path the product is measured on — the copy path costs a multiple (datapath budgets).

Each camera gets one region with shm_slots slots. A slot is reused only once the request that uses it has provably ended; with one slot, a new frame would overwrite the one the backend may still be reading. If every slot is busy, the new frame is not sent and counted as client_backpressure.

An answer alone is not proof. After backend_timeout, execution_unknown, a cancellation or a transport error the backend may still be computing and reading the slot — the governor keeps its execution credit for exactly that reason. Such a slot goes into quarantine and is never handed out again (slot_quarantined); a timer would not prove the reader finished. A slot is freed by a delivered result, by a refusal the governor made before forwarding (superseded, stale, infeasible, capacity, graph) and by an error the backend reported itself. Until 11 September every answer freed the slot, timeouts included (review R04).

When every slot of a region is quarantined, the camera starts a new region (shm_new_epoch, at most shm_max_epochs times, default 4). The old region is never written again and stays registered and mapped until the bridge shuts down. After the last epoch the camera stays in backpressure and shm_epochs_exhausted counts the frames it drops.

In containers, bridge, governor and backend must share /dev/shm (--ipc=host or a common IPC namespace). An address that only looks local — a port forward into another container without shared IPC — needs transport: copy.

Fusion across cameras

Frames from the same capture_group whose stamps lie within capture_tolerance_us get the same vig_capture_id, and every request gets a numeric id. Both are in the result message. A fusion node that combines them sends its own request with vig_capture_id and vig_depends_on; the governor refuses a fusion across captures before it computes (results from the same capture).

Fixed: the capture graph filled up (releases up to 0.2.0)

The governor keeps one node per announced capture in a bounded graph of 256 nodes (MAX_NODES). Up to release 0.2.0 it never moved a node out of the open state once its request had finished: after 256 captures in the lifetime of a governor process, every request carrying vig_capture_id was refused with FAILED_PRECONDITION. At 30 Hz that is about eight seconds. The bridge reported these as fusion_refused events, and a live run against vig serve showed exactly this: 256 frames delivered, every later one refused.

Every terminal path now closes its node, and a completed result is kept for its model's max_age so a fusion node still finds it (details). Against 0.2.0 or older, run the bridge with send_capture_id: false unless a downstream node actually fuses results; freshness, supersession and the shared-memory path do not depend on it.

What the bridge does not do

• No preprocessing beyond resize and scale. Nearest-neighbour resize, channel order, scaling. A model that needs letterboxing or normalisation with mean and standard deviation needs its own preprocessing.
• No output decoding. It publishes tensors, not detections.
• No retries of refused frames, by design.
• No clock synchronisation. It reports skew; it cannot fix it.
• No compressed images. sensor_msgs/CompressedImage would need a decoder in the bridge.

Tests

docker build -t vig-ros2-test integrations/ros2/docker
docker run --rm -v "$PWD/integrations/ros2/vig_bridge:/ws/vig_bridge" vig-ros2-test

Unit tests for the logic (age, capture and request ids, parameters, refusal classification, image conversion, slot reuse), transport tests against a stub governor that reads the shared-memory region and answers with vig-reason, and a function test that runs the real ROS 2 node with two cameras of one capture group over both transports.

Against a real governor in front of Triton (pose_main on 127.0.0.1:8001):

integrations/ros2/scripts/live-smoke.sh shm target/release/vig
integrations/ros2/scripts/live-smoke.sh copy target/release/vig