AI Overlay API

AI Overlay Control Client

Control the platform’s AI overlay system that draws detection boxes, labels, and confidence scores directly on NV12 frames before encoding. Zero CPU cost — drawing happens in camera-daemon before encoding.

Two halves:

configuration — enable / disable / configure / apply via the UpdateAiOverlay RPC;
content — annotate / annotate_result push detection boxes (or classifications / landmarks / OCR lines) through the event bus; camera-daemon’s overlay subscriber renders them with its HAL draw ops.

Uses gRPC over Unix domain socket, consistent with other SDK clients.

class neoruntime_ipc_sdk.overlay.OverlayConfig(enabled=True, show_label=True, show_confidence=True, line_thickness=2, box_color=0, label_color=0, font_size=0)[source]

Bases: object

AI overlay configuration.

enabled: bool = True

show_label: bool = True

show_confidence: bool = True

line_thickness: int = 2

box_color: int = 0

label_color: int = 0

font_size: int = 0

to_proto()[source]

__init__(enabled=True, show_label=True, show_confidence=True, line_thickness=2, box_color=0, label_color=0, font_size=0)

class neoruntime_ipc_sdk.overlay.OverlayClient(*args, **kwargs)[source]

Bases: GrpcClient

AI Overlay Control Client

Uses gRPC to communicate with camera-daemon’s CameraControl service.

Usage:

from neoruntime_ipc_sdk import OverlayClient

oc = OverlayClient()

# Enable overlay with default settings
oc.enable()

# Customize appearance
oc.configure(
    show_label=True,
    show_confidence=True,
    line_thickness=3,
)

# Disable overlay
oc.disable()

Environment variables:

CAMERA_CONTROL_ENDPOINT: Camera control gRPC endpoint: (default: unix:///run/aipc/camera-control.sock)

__init__(*args, **kwargs)[source]

close()[source]

property channel_options: gRPC channel options applied at connect time.

enable(show_label=True, show_confidence=True, line_thickness=2)[source]

Enable AI overlay with specified settings.

disable()[source]

Disable AI overlay.

configure(enabled=True, show_label=True, show_confidence=True, line_thickness=2, box_color=0, label_color=0, font_size=0)[source]

Configure AI overlay with full control.

Parameters:

enabled (bool) – Enable or disable overlay
show_label (bool) – Show class label on detections
show_confidence (bool) – Show confidence score on detections
line_thickness (int) – Box line thickness (1-10)
box_color (int) – Box color in ARGB format (e.g. 0xFFFF0000 for red)
label_color (int) – Label color in ARGB format
font_size (int) – Font size (8-72)

apply(config)[source]

Apply an OverlayConfig object.

annotate(stream_id, detections)[source]

Draw detection boxes on stream_id’s encoded video.

Parameters:

stream_id (str) – the camera stream the boxes belong to (e.g. "main") — camera-daemon keys overlay results by it.
detections (list) – DetectedObject items or dicts with label / score (or confidence) and bbox ({"x","y","width","height"} or [x, y, w, h]).

Returns:

The published event id.

Return type:

str

The overlay expires results 500 ms after the last event, so call this at a few Hz while detections are fresh — and publish an empty detections list to clear the boxes. The overlay itself must be on: enable() first.

oc = OverlayClient()
oc.enable()
for result in inf.subscribe(stream, model):
    oc.annotate(stream, result.objects)

annotate_result(stream_id, result)[source]

Push whichever section of an InferenceResult is populated.

Detections take precedence, then classifications, landmarks and OCR lines — a payload carries one kind (that is how camera-daemon’s parser discriminates them). An empty result publishes zero detections, clearing stale boxes.

OverlayClient

class neoruntime_ipc_sdk.OverlayClient(*args, **kwargs)[source]

Bases: GrpcClient

AI Overlay Control Client

Uses gRPC to communicate with camera-daemon’s CameraControl service.

Usage:

from neoruntime_ipc_sdk import OverlayClient

oc = OverlayClient()

# Enable overlay with default settings
oc.enable()

# Customize appearance
oc.configure(
    show_label=True,
    show_confidence=True,
    line_thickness=3,
)

# Disable overlay
oc.disable()

Environment variables:

CAMERA_CONTROL_ENDPOINT: Camera control gRPC endpoint: (default: unix:///run/aipc/camera-control.sock)

__init__(*args, **kwargs)[source]

close()[source]

property channel_options: gRPC channel options applied at connect time.

enable(show_label=True, show_confidence=True, line_thickness=2)[source]

Enable AI overlay with specified settings.

disable()[source]

Disable AI overlay.

configure(enabled=True, show_label=True, show_confidence=True, line_thickness=2, box_color=0, label_color=0, font_size=0)[source]

Configure AI overlay with full control.

Parameters:

enabled (bool) – Enable or disable overlay
show_label (bool) – Show class label on detections
show_confidence (bool) – Show confidence score on detections
line_thickness (int) – Box line thickness (1-10)
box_color (int) – Box color in ARGB format (e.g. 0xFFFF0000 for red)
label_color (int) – Label color in ARGB format
font_size (int) – Font size (8-72)

apply(config)[source]

Apply an OverlayConfig object.

annotate(stream_id, detections)[source]

Draw detection boxes on stream_id’s encoded video.

Parameters:

stream_id (str) – the camera stream the boxes belong to (e.g. "main") — camera-daemon keys overlay results by it.
detections (list) – DetectedObject items or dicts with label / score (or confidence) and bbox ({"x","y","width","height"} or [x, y, w, h]).

Returns:

The published event id.

Return type:

str

The overlay expires results 500 ms after the last event, so call this at a few Hz while detections are fresh — and publish an empty detections list to clear the boxes. The overlay itself must be on: enable() first.

oc = OverlayClient()
oc.enable()
for result in inf.subscribe(stream, model):
    oc.annotate(stream, result.objects)

annotate_result(stream_id, result)[source]

Push whichever section of an InferenceResult is populated.

Detections take precedence, then classifications, landmarks and OCR lines — a payload carries one kind (that is how camera-daemon’s parser discriminates them). An empty result publishes zero detections, clearing stale boxes.

OverlayConfig

class neoruntime_ipc_sdk.OverlayConfig(enabled=True, show_label=True, show_confidence=True, line_thickness=2, box_color=0, label_color=0, font_size=0)[source]

AI overlay configuration.

enabled: bool = True

show_label: bool = True

show_confidence: bool = True

line_thickness: int = 2

box_color: int = 0

label_color: int = 0

font_size: int = 0

to_proto()[source]

__init__(enabled=True, show_label=True, show_confidence=True, line_thickness=2, box_color=0, label_color=0, font_size=0)

Usage Examples

Enable the overlay

from neoruntime_ipc_sdk import OverlayClient

overlay = OverlayClient()

# Enable the hardware overlay: draw detection boxes + labels +
# confidence on the video.
overlay.enable(show_label=True, show_confidence=True, line_thickness=2)

# Disable
overlay.disable()

Custom styling

# configure sets every style parameter in one call
overlay.configure(
    enabled=True,
    show_label=True,
    show_confidence=False,
    line_thickness=3,
    box_color=0x00FF00,     # green boxes
    label_color=0xFFFFFF,   # white labels
    font_size=16,
)

Structured configuration

from neoruntime_ipc_sdk import OverlayClient, OverlayConfig

config = OverlayConfig(
    enabled=True,
    show_label=True,
    show_confidence=True,
    line_thickness=2,
)
OverlayClient().apply(config)

Combining with inference results (annotate)

# annotate() pushes detections through the event bus to
# camera-daemon's overlay renderer: the app never touches video
# frames — the daemon draws the boxes onto the stream before
# encoding. Results expire after 500 ms, so call at inference
# cadence; publish an empty list to clear the screen.
from neoruntime_ipc_sdk import OverlayClient

overlay = OverlayClient()
overlay.enable()

for result in inference_results:      # e.g. InferenceClient.subscribe(...)
    overlay.annotate("main", result.objects)

overlay.annotate("main", [])          # clear

Other result kinds

# annotate_result picks whichever section of an InferenceResult is
# populated, with precedence objects > classifications > landmarks
# > ocr_lines
overlay.annotate_result("main", result)

Context manager

with OverlayClient() as overlay:
    overlay.enable()