conmo.datasets.dataset.LocalDataset

class conmo.datasets.dataset.LocalDataset(path: str)[source]

Abstract base class for a LocalDataset (loadable).

__init__(path: str) → None[source]

Constructor of local dataset.

Parameters

path (str) – Absolute path to the folder where the dataset is located in your disk. See the example of batteries_degradation.py.

abstract dataset_files() → Iterable

Iterable of files included in the dataset.

abstract feed_pipeline(out_dir: str) → None[source]

Copy selected data file to pipeline step folder.

Parameters

out_dir – Directory where the dataset was originally stored.

fetch(out_dir: str) → None[source]

Fetch data to feed the pipeline.

Parameters

out_dir (str) – Directory where the dataset will be stored.

is_dataset_ready() → bool

Check if dataset has been already loaded/downloaded and parsed to package format.

abstract load() → None[source]

Parse raw dataset to package format. Data and labels must be saved in parquet format. More information about parquet format: https://parquet.apache.org/

show_start_message() → None

Show starting step info message.

Methods

`__init__`(path)	Constructor of local dataset.
`dataset_files`()	Iterable of files included in the dataset.
`feed_pipeline`(out_dir)	Copy selected data file to pipeline step folder.
`fetch`(out_dir)	Fetch data to feed the pipeline.
`is_dataset_ready`()	Check if dataset has been already loaded/downloaded and parsed to package format.
`load`()	Parse raw dataset to package format.
`show_start_message`()	Show starting step info message.