TN-003: Exception Skip Policy in Streaming¶

Summary¶

During event streaming, EventTimeout and ProtocolError are caught, counted in StreamingStats, and skipped — allowing the stream to continue. DeviceError and CommandError propagate immediately and abort the stream.

Background¶

Long-running measurements (thousands to millions of events) inevitably encounter transient errors: a rare missed read, a brief USB glitch producing a malformed JSON line. Aborting the entire measurement for a single bad event is too strict. But silently swallowing all errors hides real hardware failures.

The challenge is distinguishing recoverable per-event errors from unrecoverable connection failures.

Decision¶

Two tiers of exception handling in Streamer.stream_by_count() and stream_by_time():

Tier 1 — skip and count (recoverable per-event errors):

Exception	Cause	Action
`EventTimeout`	No event within `event_timeout` seconds	Skip, increment `stats.skipped_timeout`
`ProtocolError`	Malformed JSON or type mismatch	Skip, increment `stats.skipped_protocol`
`ResponseError`	Invalid response structure	Skip, increment `stats.skipped_response`

Tier 2 — propagate immediately (unrecoverable connection errors):

Exception	Cause	Action
`DeviceError`	Port lost or hardware failure	Raise — stream aborts
`CommandError`	Cannot send command to device	Raise — stream aborts

StreamingStats accumulates counts for all skipped events and exposes success_rate so callers can assess data quality after collection.

Alternatives Considered¶

Alternative	Reason Not Chosen
Abort on any exception	Single bad JSON line kills a 10-hour measurement
Skip all exceptions including `DeviceError`	Hides hardware failures; stream loops forever on a disconnected device
Configurable retry count per event	Added complexity; `event_timeout` already provides retry duration control
Raise after N consecutive skips	Heuristic hard to tune; `StreamingStats` gives users the data to decide

Consequences¶

Measurements complete even when a small fraction of events are malformed or timed out.
StreamingStats.success_rate lets callers and users assess data quality post-collection.
A disconnected USB cable mid-stream still aborts immediately via DeviceError.
The --verbose flag prints a skip summary at the end of each collection, making data quality visible without requiring post-processing.