corelay.io

IO-related module for Processor data

class corelay.io.DataStorageBase(**kwargs)

Bases: Plugboard

Implements a key, value storage object.

at(**kwargs)

Return a copy of the instance where kwargs become the attributes of the class. I.e. a specific self.data_key is set so that self.write(data) automatically writes the data to correct key.

close()

Close opened io file object.

abstract exists()

Return True if data exists.

abstract keys()

Return keys of the io file object.

abstract read(data_in=None, meta=None)

Should implement read functionality.

abstract write(data_out, data_in=None, meta=None)

Should implement write functionality.

class corelay.io.HDF5Storage(path, mode='r', **kwargs)

Bases: DataStorageBase

HDF5 storage that stores data under different keys.

exists()

Returns True if key exists in self.io.

keys()

Return keys of the storage.

read(data_in=None, meta=None)

Return type

data for a given key

write(data_out, data_in=None, meta=None)

Parameters

data_out (np.ndarray, dict) – Data being stored. Dictionaries are pickled and stored as strings.

exception corelay.io.NoDataSource(message='No Data Source available.')

Bases: Exception

Raise when no data source available.

exception corelay.io.NoDataTarget

Bases: Exception

Raise when no target source available.

class corelay.io.NoStorage(**kwargs)

Bases: DataStorageBase

Stub class when no Storage is used.

exists()

Return True if data exists.

keys()

Return keys of the io file object.

read(data_in=None, meta=None)

Should implement read functionality.

write(data_out, data_in=None, meta=None)

Should implement write functionality.

class corelay.io.PickleStorage(path, mode='r', **kwargs)

Bases: DataStorageBase

Experimental pickle storage that uses pickle to store data.

exists()

Return True if key exists in self.keys().

keys()

Return keys from self.data. Need to load the complete pickle at first read.

read(data_in=None, meta=None)

Return data for a given key. Need to load the complete pickle at first read. After the data is cached.

Return type

data for a given key

write(data_out, data_in=None, meta=None)

Write and pickle the data as: {“data”: data, “key”: key}

Parameters

data_out (np.ndarray, dict) – Data being stored.

class corelay.io.Storable

Bases: object

Abstract class to check for write/ read attributes via isinstance

Modules

corelay.io.hashing	Persistent, non-cryptographic hashing of python objects.
corelay.io.storage	'io module contains classes to load and dump different files like hdf5, etc.