Audio Streaming API

Audio Stream Client

Reads encoded audio frames from the camera-daemon EncodedPublisher UDS socket. Uses the same 30-byte header protocol as the video encoded publisher.

NOTE: the current platform daemon writes the VIDEO header layout on this socket (see decode_audio_format below); the audio-specific format fields are auto-detected and reported as 0/unknown until the daemon fills them.

Socket path: /run/aipc/encoded/audio_capture.sock

neoruntime_ipc_sdk.audio_stream.decode_audio_format(header, payload_size)[source]

Decode the format fields of a 30-byte audio-capture header.

The platform daemon currently writes the VIDEO EncHeader layout on audio_capture.sock (width/height at [14:22] = 0, dts at [22:30]) instead of the documented audio layout, so the tail [14:30] is first validated for audio plausibility; when that fails it is re-read as the video tail (width, height, dts_ns) and the format parameters are reported as unknown (0). A future daemon that fills the audio layout is picked up automatically, with no client change.

Returns (sample_rate, channels, bits_per_sample, dts_ns); dts_ns is 0 unless the video-layout fallback was taken.

class neoruntime_ipc_sdk.audio_stream.AudioFrame(codec, flags, pts_ns, sample_rate, channels, bits_per_sample, data, dts_ns=0)[source]

Bases: object

Encoded or raw audio frame from the audio capture pipeline.

codec: int

flags: int

pts_ns: int

sample_rate: int

channels: int

bits_per_sample: int

data: bytes

dts_ns: int = 0

property is_keyframe: bool

property codec_name: str

property duration_ms: float: Estimated frame duration in ms based on PCM parameters.

__init__(codec, flags, pts_ns, sample_rate, channels, bits_per_sample, data, dts_ns=0)

class neoruntime_ipc_sdk.audio_stream.AudioStreamClient(socket_path=None)[source]

Bases: UdsStreamClient

Audio frame subscriber via Unix Domain Socket.

Connects to the EncodedPublisher audio_capture socket and yields AudioFrame objects containing captured audio data.

Usage:

client = AudioStreamClient()

# Iterator pattern
for frame in client.subscribe():
    print(f"Audio: {frame.codec_name} {frame.sample_rate}Hz "
          f"{frame.channels}ch {len(frame.data)} bytes")

# Callback pattern
client.on_frame(lambda f: process(f))
# ... later ...
client.close()

__init__(socket_path=None)[source]

AudioStreamClient

class neoruntime_ipc_sdk.AudioStreamClient(socket_path=None)[source]

Bases: UdsStreamClient

Audio frame subscriber via Unix Domain Socket.

Connects to the EncodedPublisher audio_capture socket and yields AudioFrame objects containing captured audio data.

Usage:

client = AudioStreamClient()

# Iterator pattern
for frame in client.subscribe():
    print(f"Audio: {frame.codec_name} {frame.sample_rate}Hz "
          f"{frame.channels}ch {len(frame.data)} bytes")

# Callback pattern
client.on_frame(lambda f: process(f))
# ... later ...
client.close()

__init__(socket_path=None)[source]

AudioFrame

class neoruntime_ipc_sdk.AudioFrame(codec, flags, pts_ns, sample_rate, channels, bits_per_sample, data, dts_ns=0)[source]

Encoded or raw audio frame from the audio capture pipeline.

codec: int

flags: int

pts_ns: int

sample_rate: int

channels: int

bits_per_sample: int

data: bytes

dts_ns: int = 0

property is_keyframe: bool

property codec_name: str

property duration_ms: float: Estimated frame duration in ms based on PCM parameters.

__init__(codec, flags, pts_ns, sample_rate, channels, bits_per_sample, data, dts_ns=0)

Usage Examples

Iterate audio frames

from neoruntime_ipc_sdk import AudioClient, AudioStreamClient

# Start capture first, then subscribe
audio = AudioClient()
audio.start_capture(sample_rate=16000, channels=1, codec="aac")

stream = AudioStreamClient()
for frame in stream.subscribe():
    print(f"{frame.codec_name} frame: {len(frame.data)} bytes, "
          f"{frame.duration_ms:.1f}ms")

Fetch a single frame

frame = stream.get_frame(timeout_ms=2000)
if frame is not None:
    print(f"codec={frame.codec_name}, pts={frame.pts_ns}")

Callback subscription

def on_audio(frame):
    # Process the audio frame (forward, persist, feed to ASR, ...)
    save(frame.data)

thread = stream.on_frame(on_audio)

Raw PCM vs encoded frames

for frame in stream.subscribe():
    if frame.codec == 0:  # PCM
        # frame.data can be parsed directly using
        # sample_rate / channels / bits_per_sample
        process_pcm(frame.data)
    else:  # aac / g711a / g711u
        process_encoded(frame.data, frame.is_keyframe)

Error handling and cleanup

stream = AudioStreamClient()
try:
    for frame in stream.subscribe(reconnect=True):
        process(frame)
finally:
    stream.close()