Architecture

Engine and host configuration

The codebase has two halves. The engine (src/metadata_tools/, excluding hosts/) is host-agnostic: it knows how to walk a volume tree, read labels, compute geometry, format columns, and write tables and PDS3 labels. A host configuration package (src/metadata_tools/hosts/<HOST>/) supplies the collection-specific knowledge: which files to include, how to derive certain columns, the spacecraft ID, the body-selection mission table, the meshgrids, and the label templates.

The engine never imports a specific host’s config modules directly (see issue #112). Instead, console-script entry points call metadata_tools.config.set_host() once, which package-qualifies the import of metadata_tools.hosts.<host_id>.host_config and index_config (geometry_config is imported lazily, on first use, to avoid paying the SPICE startup cost for stages that do not need it). Engine modules then call get_host_config(), get_index_config(), and get_geometry_config() instead of importing those modules directly. This works from any current working directory — no sys.path manipulation is involved — and is why the documentation build mocks the three module names as plugin-injected surfaces (see docs/conf.py) rather than needing a working directory trick.

Table classes

All table kinds derive from a single base class. The geometry tables for one volume are coordinated by a Suite, which builds one Record per observation and feeds it to each table.

        classDiagram
    class Table {
        +template_path
        +volume_id
        +level
        +qualifier
        +rows
        +filename
        +write(labels_only)
    }
    class IndexTable {
        +create(labels_only, pattern)
        +add(root, name)
    }
    class InventoryTable {
        +add(record)
    }
    class SkyTable {
        +add(record)
    }
    class SunTable {
        +add(record)
    }
    class RingTable {
        +add(record)
    }
    class BodyTable {
        +add(record)
    }
    class Suite {
        +tables
        +create(labels_only, pattern)
        +make_records(index)
        +add(records)
        +write(labels_only)
    }
    class Record {
        +primary
        +bodies
        +backplane
        +add(qualifier)
        +postprocess(columns, qualifier)
    }

    Table <|-- IndexTable
    Table <|-- InventoryTable
    Table <|-- SkyTable
    Table <|-- SunTable
    Table <|-- RingTable
    Table <|-- BodyTable
    Suite o-- Table : owns
    Suite ..> Record : produces
    BodyTable ..> Record : consumes

The base class

Table holds the state common to every table: the label template_path, the volume_id, the processing level ("summary", "detailed", or "index"), the qualifier ("sky", "sun", "ring", "body", "inventory", or "supplemental"), the accumulated rows, and the output filename. Its write() method writes the table file (unless labels_only is set) and then generates the PDS3 label through create(). Subclasses add the logic that fills rows.

The index table

IndexTable represents one volume’s supplemental index. create() iterates the volume’s data labels, and add() reads each PDS3 label and appends one formatted row. Its columns come from the supplemental label template, not from a Python column list. It does not use a Record.

The geometry tables

The geometry tables in metadata_tools.geometry_support.tables all extend Table and share a single contract: an add method that takes a Record and appends the appropriate rows. SkyTable, RingTable, and BodyTable each ask the record for the rows for their qualifier (the body table emits one row per selected body; the ring table emits rows only when a ring system is present), while InventoryTable writes the list of bodies in the field of view as a CSV row.

SunTable is defined (and shown in the diagram above) but is experimental and not wired into the pipeline: Suite does not build it and cumulative_support does not concatenate it. The Sun is a body, so a sun table would be structured like the body table but without illumination-based columns; however oops models the Sun as the sole illumination source and cannot currently evaluate a Sun-surface backplane (a surface event key for the Sun collapses to the illegal ('SUN<',)). See the SunTable docstring for the blocker and the recipe for enabling it once that support exists.

The volume coordinator

Suite is the geometry stage’s per-volume coordinator. It is not a table; it owns a list of table objects (one inventory table plus a sky, ring, and body table per requested level), reads the volume’s observations through the host’s from_index hook, builds the meshgrids, and in create() loops over observations, building records with make_records() and dispatching them to every table with add().

The row builder

Record represents one observation across all geometry tables. On construction it determines the primary body from the spacecraft clock (via get_primary()), selects the bodies in the field of view, and builds the oops backplane. Its add() method delegates to prep_row() to evaluate and format the columns for a qualifier, then postprocess() applies the inter-column null-linking rules.

Data flow

The three stages connect through files on disk, not in-memory objects:

The index stage (process_index()) writes a supplemental index table per volume.
The geometry stage (process_tables()) reads each volume’s supplemental index, builds a Suite, and writes the geometry tables.
The cumulative stage (create_cumulative_indexes()) walks the tree and concatenates the per-volume tables.

Each stage is documented in its own chapter: Index table subsystem, Geometry table subsystem, and Cumulative table subsystem. The shared machinery is covered in Shared support.