Documentation for rms-metadata-tools
Introduction
rms-metadata-tools (the importable package metadata_tools) generates PDS3
index, geometry, and cumulative metadata tables, and their PDS3
labels, for planetary science data collections. Each row of a table holds the
metadata for a single data product, such as one image.
The Planetary Data System (PDS) distributes data as collections of volumes.
Alongside the data, each collection ships flat ASCII metadata tables that
summarize every product: observation times, instrument settings, and the
geometry of what was observed. These tables feed the
OPUS search service and are downloaded
directly by PDS users. Producing them by hand is tedious and error-prone;
rms-metadata-tools generates them, and their validated labels, for a whole
volume tree from a small per-collection configuration.
rms-metadata-tools is a product of the
PDS Ring-Moon Systems Node at the SETI Institute.
Features
Three table kinds. Generates supplemental index tables (extra columns from PDS3 labels), geometry tables (body, ring, sky, and Sun quantities computed from SPICE through
oops), and cumulative tables that span a whole collection.PDS3 labels included. Writes a validated
.lbllabel for every table from reusable templates.Engine plus per-host configuration. A host-agnostic engine does the work; each collection plugs in through a small configuration package, so adding a new collection means writing config, not engine code.
Local or cloud. Run on one machine, or distribute per-volume work across Google Cloud with the included
rms-cloud-tasksworkers.Local and remote storage. Reads and writes through
rms-filecache, so local paths andgs:///s3://URIs are interchangeable.
Installation
rms-metadata-tools requires Python 3.11 or later and is tested on Python
3.11, 3.12, and 3.13. Generating geometry tables additionally requires the SPICE
kernels for your collection, installed where oops can find them.
Install from PyPI:
pip install rms-metadata-tools
To work from a checkout (recommended when adding or modifying a host configuration, since the runnable host scripts live in the source tree):
git clone https://github.com/SETI/rms-metadata-tools.git
cd rms-metadata-tools
python -m venv venv
source venv/bin/activate
pip install -e ".[dev]"
The optional extras are dev (linting, type-checking, tests, docs), docs
(Sphinx), and cloud (the GCP workers).
Quick Start
Each supported collection (“host”) has its own runnable programs in its
directory under src/metadata_tools/hosts/<HOST>/. They import their
configuration as top-level modules, so run them from inside the host directory.
The three stages, for the Galileo SSI host, are:
cd src/metadata_tools/hosts/GO_0xxx
# 1. Supplemental index tables (extra columns from the PDS3 labels)
python GO_0xxx_index.py "$RMS_VOLUMES/GO_0xxx/" "$RMS_METADATA/GO_0xxx/" \
"$RMS_METADATA_TEST/GO_0xxx/"
# 2. Geometry tables (body/ring/sky/Sun quantities from SPICE)
python GO_0xxx_geometry.py "$RMS_METADATA/GO_0xxx/" "$RMS_METADATA_TEST/GO_0xxx/"
# 3. Cumulative tables across the whole collection
python GO_0xxx_cumulative.py "$RMS_METADATA_TEST/GO_0xxx/GO_0999/"
Path arguments are expanded for environment variables. Restrict a run to one
volume with -vv GO_0017, or to a few images with --first 5. See the
user guide
for the full list of programs and options.
Documentation
Full documentation is hosted on ReadTheDocs. It includes a user guide (installation, configuration, and a reference for every command-line program) and a developer guide (architecture, per-subsystem internals, how to add a new collection or geometry column, and the API reference).
To build the documentation locally:
scripts/read-docs.sh
Contributing
Information on contributing to this package can be found in the Contributing Guide.
Links
Licensing
This code is licensed under the Apache License v2.0.
User Guide
Developer Guide