Skip to content

TN-001: Response Flattening and Output Formatting

Summary

Decided to implement a dedicated flattening layer for converting nested DeviceResponse objects into flat dictionaries with underscore-separated keys, and a separate formatter layer for output format conversion (JSONL, CSV, SSV, TSV, etc.).

Background

OSECHI detector responses are returned as nested JSON objects. Users working with pandas, databases, or Unix tools need flat key-value structures. The question was where to place flattening logic and how to expose it.

Decision

Flattening is implemented as a general-purpose mechanical transformation, separate from the command and parser layers:

  • ResponseView (formatter.py): wraps a DeviceResponse and exposes two perspectives — raw (original nested structure) and flat (underscore-separated flattened structure). Flattening is performed automatically in __post_init__.
  • ResponseFormatter: converts a DeviceResponse into the requested output format (table, JSONL, JSON, CSV, SSV, TSV, LTSV). Works with both raw and flat perspectives.
  • CLI exposes perspective selection via --use-flat / --use-raw and format selection via --format.

No public flatten() library function was added — the flattening is internal to ResponseView. Users who need flat data access it through ResponseView.flat.

Alternatives Considered

Alternative Reason Not Chosen
Flattening in the Command layer (per method) Duplicates logic; each method would need its own flatten call
FlatResponse as a separate Pydantic model Overly complex; same data with two model definitions
Public flatten() utility function Not needed — CLI and library users both access flat data via ResponseView
Embedding flatten logic in cli.py Couples output formatting to CLI; prevents reuse in scripts

Consequences

  • ResponseView adds a small overhead on construction (one extra model_dump() + recursive flatten per response), negligible for interactive and single-event use cases.
  • All output formatting (CLI, example scripts) goes through ResponseFormatter, ensuring consistent behavior across formats.
  • Adding a new output format only requires a new method in ResponseFormatter.
  • The --use-flat flag in the CLI reflects the ResponseView.flat perspective directly; field names are predictable (underscore-separated).