Parameter Data

A ParamDB database stores parameter data. The abstract base class ParamData defines some core functionality for this data, including the last_updated, parent, and root properties. Internally, any subclasses of ParamData are automatically registered with ParamDB so that they can be loaded to and from JSON, which is how they are stored in the database.

All of the “Param” classes described on this page are subclasses of ParamData.

Important

Any data that is going to be stored in a ParamDB database must be a JSON serializable type (str, int, float, bool, None, dict, or list), a datetime, an astropy.units.Quantity, or an instance of a ParamData subclass. Otherwise, a TypeError will be raised when they are committed to the database.

Primitives

Primitives are the building blocks of parameter data. While builtin primitive types can be used in a ParamDB (int, float, str, bool, and None), they will not store a last_updated time and will not have parent or root properties. When these features are desired, we can wrap primitive values in the following types:

• ParamInt for integers

• ParamFloat for float

• ParamBool for booleans

• ParamStr for strings

• ParamNone for None

For example:

from paramdb import ParamInt

param_int = ParamInt(123)
param_int

ParamInt(123)

print(param_int.last_updated)

2024-05-08 18:17:50.650095+00:00

Tip

Methods from the builtin primitive types work on parameter primitives, with the caveat that they return the builtin type. For example:

param_int + 123

246

type(param_int + 123)

int

Data Classes

A parameter data class is defined from the base class ParamDataclass. This custom class is automatically converted into a data class, meaning that class variables with type annotations become object properties and the corresponding __init__ function is generated. An example of a defining a custom parameter Data Class is shown below.

from paramdb import ParamFloat, ParamDataclass

class CustomParam(ParamDataclass):
    value: ParamFloat

custom_param = CustomParam(value=ParamFloat(1.23))

These properties can then be accessed and updated.

custom_param.value = ParamFloat(1.234)
custom_param.value

ParamFloat(1.234)

The data class aspects of the subclass can be customized by passing keyword arguments when defining the custom class (the same arguments that would be passed to the @dataclass decorator), and by using the dataclass field function. The class arguments have the same default values as in @dataclass. An example of data class customization is shown below.

Note

The kw_only setting below only works in Python 3.10, but is useful for defining non-default arguments after those with default values (like in the example), especially when building up dataclasses through inheritance.

from dataclasses import field

class KeywordOnlyParam(ParamDataclass, kw_only=True):
    count: int
    values: list[int] = field(default_factory=list)

keyword_only_param = KeywordOnlyParam(count=123)
keyword_only_param

KeywordOnlyParam(count=123, values=[])

Warning

For mutable default values, default_factory should generally be used. See the Python data class documentation on mutable default values for more information.

Custom methods can also be added, including dynamic properties using the @property decorator. For example:

class ParamWithProperty(ParamDataclass):
    value: ParamInt

    @property
    def value_cubed(self) -> int:
        return self.value ** 3

param_with_property = ParamWithProperty(value=ParamInt(16))
param_with_property.value_cubed

4096

Important

Since __init__ is generated for data classes, other initialization must be done using the __post_init__ function. Furthermore, since __post_init__ is used internally by ParamDataclass to perform initialization, always call the superclass’s __post_init__. For example:

class ParamCustomInit(ParamDataclass):
    def __post_init__(self) -> None:
        print("Initializing...")  # Replace with custom initialization code
        super().__post_init__()

param_custom_init = ParamCustomInit()

Initializing...

Parameter data track when any of their properties were last updated, and this value can be accessed by the read-only last_updated property. For example:

print(custom_param.last_updated)

2024-05-08 18:17:50.693126+00:00

import time

time.sleep(1)
custom_param.value = ParamFloat(4.56)
print(custom_param.last_updated)

2024-05-08 18:17:51.740616+00:00

Parameter dataclasses can also be nested, in which case the ParamData.last_updated property returns the most recent last updated time stamp among its own last updated time and the last updated times of any ParamData it contains. For example:

class NestedParam(ParamDataclass):
    value: float
    child_param: CustomParam

nested_param = NestedParam(value=1.23, child_param=CustomParam(value=ParamFloat(4.56)))
print(nested_param.last_updated)

2024-05-08 18:17:51.750544+00:00

time.sleep(1)
nested_param.child_param.value = ParamFloat(2)
print(nested_param.last_updated)

2024-05-08 18:17:52.759055+00:00

You can access the parent of any parameter data using the ParamData.parent property. For example:

nested_param.child_param.parent is nested_param

True

Similarly, the root can be accessed via ParamData.root:

nested_param.child_param.root is nested_param

True

See Type Mixins for information on how to get the parent and root properties to work better with static type checkers.

Type Validation

If Pydantic is installed, parameter data classes will automatically be converted to Pydantic data classes, enabling runtime type validation. Some Pydantic configuration have modified defaults; see ParamDataclass for more information.

Pydantic type validation will enforce type hints at runtime by raising an exception. For example:

import pydantic

try:
    CustomParam(value="123")
except pydantic.ValidationError as exception:
    print(exception)

1 validation error for CustomParam
value
  Input should be an instance of ParamFloat [type=is_instance_of, input_value='123', input_type=str]
    For further information visit https://errors.pydantic.dev/2.7/v/is_instance_of

Type validation can be disabled for a particular parameter data class (and its subclasses) using the class keyword argument type_validation:

class NoTypeValidationParam(CustomParam, type_validation=False):
    pass

NoTypeValidationParam(value="123")

NoTypeValidationParam(value='123')

Files

ParamFile is an abstract base class that stores the path to a file. The data in the file can then be loaded by accessing ParamFile.data and updated using ParamFile.update_data(). In order to use ParamFile, it must be subclassed and the functions ParamFile._save_data() and ParamFile._load_data() must be defined.

Pandas DataFrames

One class that implements these functions is ParamDataFrame for saving and retrieving Pandas DataFrames. For example:

import pandas as pd
from paramdb import ParamDataFrame

data_frame = pd.DataFrame([[1, 2, 3], [4, 5, 6]], columns=["col1", "col2", "col3"])
param_data_frame = ParamDataFrame("data.csv", data_frame)
param_data_frame.data

	col1	col2	col3
0	1	2	3
1	4	5	6

Collections

Ordinary lists and dictionaries can be used within parameter data; however, any parameter data objects they contain will not have a last updated time or a parent object. Therefore, it is not recommended to use ordinary lists and dictionaries to store parameter data. Instead, ParamList and ParamDict can be used.

Parameter Lists

ParamList implements the abstract base class MutableSequence from collections.abc, so it behaves similarly to a list. It is also a subclass of ParamData, so the last updated, parent, and root properties will work properly. For example:

from paramdb import ParamList

param_list = ParamList([ParamInt(1), ParamInt(2), ParamInt(3)])
param_list[1].parent is param_list

True

print(param_list.last_updated)

2024-05-08 18:17:52.824976+00:00

time.sleep(1)
param_list[1] = ParamInt(4)
print(param_list.last_updated)

2024-05-08 18:17:53.841886+00:00

Parameter Dictionaries

Similarly, ParamDict implements MutableMapping from collections.abc, so it behaves similarly to a dictionary. Additionally, its items can be accessed via dot notation in addition to index brackets (unless they begin with an underscore). For example:

from paramdb import ParamDict

param_dict = ParamDict(p1=ParamFloat(1.23), p2=ParamFloat(4.56), p3=ParamFloat(7.89))
param_dict.p2.root == param_dict

True

print(param_dict.last_updated)

2024-05-08 18:17:53.849585+00:00

time.sleep(1)
param_dict.p2 = ParamFloat(0)
print(param_dict.last_updated)

2024-05-08 18:17:54.865889+00:00

Parameter collections can also be subclassed to provide custom functionality. For example:

class CustomDict(ParamDict[ParamFloat]):
    @property
    def total(self) -> float:
        return sum(param.value for param in self.values())

custom_dict = CustomDict(param_dict)
custom_dict.total

9.12

Type Mixins

The return type hint for ParamData.parent and ParamData.root is ParamData. Since the parent and root objects can change, it is not possible to automatically infer a more specific type for the parent or root. However, a type hint can be given using the ParentType and RootType mixins. For example:

from paramdb import ParentType

class ParentParam(ParamDataclass):
    child_param: ChildParam

class ChildParam(ParamDataclass, ParentType[ParentParam]):
    value: float

parent_param = ParentParam(child_param=ChildParam(value=1.23))

This does nothing to the functionality, but static type checkers will now know that parent_param.child_param.parent in the example above is a ParentParam object.