Loader

Loader interfaces for materializing data objects.

This module defines the abstract Loader contract and a concrete DataSourceLoader implementation that turns a mapping of pandas DataFrames into a DataSource while preserving validation messages.

class algomancy_data.loader.Loader(logger)[source]

Bases: ABC

Abstract interface for loading transformed data into a destination.

abstractmethod load(name, data, validation_messages, ds_type)[source]

Create the destination object from transformed data.

Parameters:

  • name (str) – Logical name of the dataset/destination.

  • data (Dict[str, DataFrame]) – Mapping from table name to pandas DataFrame.

  • validation_messages (List[ValidationMessage]) – Messages collected during validation.

  • ds_type (DataClassification) – Classification of the destination data.

Returns:

A destination object containing the data.

Return type:

BASEDATASOURCE

class algomancy_data.loader.DataSourceLoader(logger)[source]

Bases: Loader

Loader that builds and populates a DataSource.

load(name, data, validation_messages, ds_type=DataClassification.MASTER_DATA)[source]

Instantiate a DataSource and populate it with tables.

Parameters:

  • name (str) – Name of the resulting data source.

  • data (dict[str, DataFrame]) – Mapping of table names to DataFrames.

  • validation_messages (List[ValidationMessage]) – Messages collected during validation.

  • ds_type (DataClassification) – Data classification for the data source.

Returns:

The populated data source.

Return type:

DataSource