The index program

Each host ships an index program named <HOST>_index.py (for Galileo SSI, GO_0xxx_index.py). It generates the supplemental index table and its PDS3 label for every volume in a tree, by reading each data product’s PDS3 label. Under the hood it calls process_index().

Run it from inside the host directory (see Installation and setup).

Synopsis

python <HOST>_index.py [options] volume_tree metadata_tree output_tree

Positional arguments

Argument	Description
`volume_tree`	Path to the top of the tree containing the volume data files, specifically the PDS3 labels that are read for each row.
`metadata_tree`	Path to the top of the tree containing the metadata files, specifically the project-supplied corrected index files (`<volume>_index.tab`).
`output_tree`	Path to the top of the tree in which to write the new supplemental index files and labels.

All path arguments are expanded for environment variables.

Options

Selection

Option	Description
`--volumes VOL [VOL ...]`, `-vv`	Process only these volume IDs (e.g. `GO_0017`). Repeatable list. If omitted, every volume in the tree is processed.
`--pattern PATTERN`, `-p`	Glob pattern that further restricts which data files are processed within each volume.

Output / type

Option	Description
`--type TYPE`, `-t`	Type of index file to create, e.g. `supplemental`. Default: taken from the host’s template name (`supplemental` for the shipped hosts).
`--labels`, `-l`	Generate labels only for index tables that already exist; do not recompute the tables themselves.

Processing / distribution

Option	Description
`--task-output FILE`, `-to`	Write a task-queue file listing one task per volume and perform no processing. Used to drive the distributed cloud workers; see Distributed (cloud) runs.

Miscellaneous

Option	Description
`-h`, `--help`	Show the help message and exit.

Example

Generate supplemental index tables for every Galileo SSI volume, then for one volume only:

cd src/metadata_tools/hosts/GO_0xxx
python GO_0xxx_index.py "$RMS_VOLUMES/GO_0xxx/" "$RMS_METADATA/GO_0xxx/" \
    "$RMS_METADATA_TEST/GO_0xxx/"
python GO_0xxx_index.py "$RMS_VOLUMES/GO_0xxx/" "$RMS_METADATA/GO_0xxx/" \
    "$RMS_METADATA_TEST/GO_0xxx/" -vv GO_0017

Results

For each processed volume <volume>, the program writes <volume>_supplemental_index.tab and <volume>_supplemental_index.lbl into the corresponding volume directory of the output tree, and a processing log named <volume>_index-log.txt. Columns that never received a non-null value across the run are reported as a warning at the end of the log.