API Reference

All of the following can be imported from paramdb.

Parameter Data

class ParamData

Abstract base class for all parameter data.

property last_updated: datetime

When any parameter within this parameter data was last updated.

child_last_updated(child_name: ChildNameT) → datetime

Return the last updated time of the given child.

property parent: ParamData[Any]

Parent of this parameter data. The parent is defined to be the ParamData object that most recently had this object added as a child.

Raises a ValueError if there is currently no parent, which can occur if the parent is still being initialized.

property root: ParamData[Any]

Root of this parameter data. The root is defined to be the first object with no parent when going up the chain of parents.

abstract to_json() → Any

Convert the data stored in this object into a JSON serializable format, which will later be passed to from_json() to reconstruct this object.

The last updated timestamp is handled separately and does not need to be saved here.

Note that objects within a list or dictionary returned by this function do not need to be JSON serializable, since they will be processed recursively.

classmethod from_json(json_data: list[Any], last_updated_timestamp: float) → Self

Construct a parameter data object from the given last updated timestamp and JSON data originally constructed by to_json().

class ParamDataclass(*args: Any, **kwargs: Any)

Subclass of ParamData.

Base class for parameter data classes. Subclasses are automatically converted to data classes. For example:

class CustomParam(ParamDataclass):
    value1: float
    value2: int

Any class keyword arguments (other than those described below) given when creating a subclass are passed internally to the @dataclass() decorator.

If Pydantic is installed, then subclasses will have Pydantic runtime validation enabled by default. This can be disabled using the class keyword argument type_validation. The following Pydantic configuration values are set by default:

• extra: 'forbid' (forbid extra attributes)

• validate_assignment: True (validate on assignment as well as initialization)

• arbitrary_types_allowed: True (allow arbitrary type hints)

• strict: True (disable value coercion, e.g. ‘2’ -> 2)

• validate_default: True (validate default values)

Pydantic configuration options can be updated using the class keyword argument pydantic_config, which will merge new options with the existing configuration. See https://docs.pydantic.dev/latest/api/config for full configuration options.

to_json() → dict[str, Any]

Convert the data stored in this object into a JSON serializable format, which will later be passed to from_json() to reconstruct this object.

The last updated timestamp is handled separately and does not need to be saved here.

Note that objects within a list or dictionary returned by this function do not need to be JSON serializable, since they will be processed recursively.

class ParamFile(path: str, initial_data: InitVar[DataT | None] = None)

Subclass of ParamDataclass.

Base class for parameter file classes. Subclasses must implement the abstract methods _save_data() and _load_data(). For example:

class ParamText(ParamFile[str]):
    def _save_data(self, path: str, data: str) -> None:
        with open(path, "w", encoding="utf-8") as f:
            f.write(data)

    def _load_data(self, path: str) -> str:
        with open(path, "r", encoding="utf-8") as f:
            return f.read()

path: str

Path to the file represented by this object.

property data: DataT

Data stored in the file represented by this object.

update_data(data: DataT) → None

Update the data stored within the file represented by this object.

class ParamDataFrame(path: str, initial_data: InitVar[DataT | None] = None)

Subclass of ParamFile.

Parameter data Pandas DataFrame, stored in a Pickle file. This class will only be defined if Pandas is installed.

class ParamList(iterable: Iterable[ItemT] | None = None)

Subclass of ParamData and MutableSequence.

Mutable sequence that is also parameter data. It can be initialized from any iterable (like builtin list).

class ParamDict(mapping: Mapping[str, ItemT] | None = None, /, **kwargs: ItemT)

Subclass of ParamData and MutableMapping.

Mutable mapping that is also parameter data. It can be initialized from any mapping or using keyword arguments (like builtin dict).

Keys that do not refer to existing attributes or class type hints can be gotten, set, and deleted via dot notation.

class ParentType

Mixin for ParamData that sets the type hint for ParamData.parent to type parameter ParentT. For example:

class CustomParam(ParentType[ParentParam], Param):
    ...

Note that if the parent actually has a different type, the type hint will be incorrect.

class RootType

Mixin for ParamData that sets the type hint for ParamData.root to type parameter RootT. For example:

class CustomParam(RootType[RootParam], Param):
    ...

Note that if the root actually has a different type, the type hint will be incorrect.

Database

class ParamDB(path: str)

Parameter database. The database is created in a file at the given path if it does not exist. To work with type checking, this class can be parameterized with a root data type DataT. For example:

from paramdb import ParamDataclass, ParamDB

class Root(ParamDataclass):
    ...

param_db = ParamDB[Root]("path/to/param.db")

property path: str

Path of the database file.

commit(message: str, data: DataT, timestamp: datetime | None = None, *, raw_json: Literal[False] = False) → CommitEntry

commit(message: str, data: str, timestamp: datetime | None = None, *, raw_json: Literal[True]) → CommitEntry

Commit the given data to the database with the given message and return a commit entry for the new commit.

By default, the timestamp will be set to the current time. If a timestamp is given, it is used instead. Naive datetimes will be assumed to be in UTC time.

By default, the given data will be converted into a JSON string, which is then saved to the database; however, if raw_json is True, then the given data is assumed to already be a JSON string in the format specified by ParamDB.load(). Note that any string will be accepted, so be careful using this option. If the format is incorrect, loading this particular commit may fail.

property num_commits: int

Number of commits in the database.

load(commit_id: int | None = None, *, raw_json: Literal[False] = False) → DataT

load(commit_id: int | None = None, *, raw_json: Literal[True]) → str

Load and return data from the database. If a commit ID is given, load from that commit; otherwise, load from the most recent commit. Raise an IndexError if the specified commit does not exist. Note that commit IDs begin at 1.

By default, objects are reconstructed, which requires the relevant parameter data classes to be defined in the current program. However, if raw_json is True, the JSON data is returned directly from the database as a string. The format of the JSON data is as follows:

json_data =
    | int
    | float
    | bool
    | str
    | None
    | {"type": "datetime", "timestamp": float}
    | {"type": "Quantity", "value": float, "unit": str}
    | {"type": "list", "data": [json_data, ...]}
    | {"type": "dict", "data": {str: json_data, ...}}
    | {"type": "ParamData", "lastUpdated": float, "data": json_data}
    | {"type": "ParamData", "className": str, "lastUpdated": float, "data": json_data}

load_commit_entry(commit_id: int | None = None) → CommitEntry

Load and return a commit entry from the database. If a commit ID is given, load that commit entry; otherwise, load the most recent commit entry. Raise an IndexError if the specified commit does not exist. Note that commit IDs begin at 1.

commit_history(start: int | None = None, end: int | None = None) → list[CommitEntry]

Retrieve the commit history as a list of CommitEntry objects between the provided start and end indices, which work like slicing a Python list.

commit_history_with_data(start: int | None = None, end: int | None = None, *, raw_json: Literal[False] = False) → list[CommitEntryWithData[DataT]]

commit_history_with_data(start: int | None = None, end: int | None = None, *, raw_json: Literal[True]) → list[CommitEntryWithData[str]]

Retrieve the commit history with data as a list of CommitEntryWithData objects between the provided start and end indices, which work like slicing a Python list.

See ParamDB.load() for the behavior of raw_json.

dispose() → None

Dispose of the underlying SQLAlchemy connection pool. Usually this method does not need to be called since disposal happens automatically when the database is garbage collected, and typically only one global database object should be used. However, there are certain cases where it is useful to fully dispose of a database before the end of the program, such as when running a test suite. See https://docs.sqlalchemy.org/en/20/core/connections.html#engine-disposal for more information.

class CommitEntry(id: int, message: str, timestamp: datetime)

Entry for a commit containing the ID, message, and timestamp.

id: int

Commit ID.

message: str

Commit message.

timestamp: datetime

When this commit was created.

class CommitEntryWithData(id: int, message: str, timestamp: datetime, data: DataT)

Subclass of CommitEntry.

Entry for a commit containing the ID, message, and timestamp, as well as the data.

data: DataT

Data contained in this commit.

class ParamDBType

Type strings corresponding to different object types in the JSON representation of the data in a ParamDB commit.

DATETIME = 'datetime'

Type string for datetime.datetime objects.

QUANTITY = 'Quantity'

Type string astropy.units.Quantity objects.

LIST = 'list'

Type string ordinary lists.

DICT = 'dict'

Type string ordinary dictionaries.

PARAM_DATA = 'ParamData'

Type string for ParamData objects.

The JSON object should either include a parameter class name, or be None if wrapping a non-ParamData with parameter metadata (e.g. a last updated time).