Pipelines

Collection of pipelines

Pipeline

class ml_investment.pipelines.Pipeline(data: Dict, feature, target, model, out_name=None)

Bases: object

Class incapsulate feature and target calculation, model training and validation during fit-phase and feature calculation and model prediction during execute-phase. Support multi-target with different models and metrics.

Parameters

• data – dict having needed for features and targets fields. This field should contain classes implementing load(index) -> pd.DataFrame interfaces

• feature – feature calculator implements calculate(data: Dict, index) -> pd.DataFrame interface

• target – target calculator implements calculate(data: Dict, index) -> pd.DataFrame interface OR List of such target calculators

• model – class implements fit(X, y) and predict(X) interfaces. Сopy of the model will be used for every single target if type of target is List. OR List of such classes(len of this list should be equal to len of target)

• out_name – str column name of result in pd.DataFrame after execute() OR List[str] (len of this list should be equal to len of target) OR None ( List['y_0', 'y_1'...] will be used in this case)

execute(index)

Interface for executing pipeline for tickers. Features will be based on data from data_loader

Parameters

index – execute identification(i.e. list of tickers to predict model for)

Returns

result values in columns named as out_name param in __init__()

Return type

pd.DataFrame

export_core(path=None)

Interface for saving pipelines core

Parameters

path – str with path to store pipeline core OR None (path will be generated automatically)

fit(index: typing.List[str], metric=None, target_filter_foo=<function nan_mask>)

Interface to fit pipeline model for tickers. Features and target will be based on data from data_loader

Parameters

• index – fit identification(i.e. list of tickers to fit model for)

• metric – function implements foo(gt, y) -> float interface. The same metric will be used for every single target if type of target is List. OR List of such functions(len of this list should be equal to len of target)

• target_filter_foo – function for filtering samples according target values/ Should implement foo(arr) -> np.array[bool] interface. Len of resulted array should be equal to len of arr. OR List of such functions(len of this list should be equal to len of target)

load_core(path)

Interface for loading pipeline core

Parameters

path – str with path to load pipeline core from

MergePipeline

class ml_investment.pipelines.MergePipeline(pipeline_list: List, execute_merge_on)

Bases: object

Class combining list of pipelines to single pipilene.

Parameters

• pipeline_list – list of classes implementing fit(index) and execute(index) -> pd.DataFrame() interfaces. Order is important: merging results during execute() will be done from left to right.

• execute_merge_on – column names for merging pipelines results on.

execute(index, batch_size=None) → pandas.core.frame.DataFrame

Interface for executing pipeline for tickers. Features will be based on data from data_loader

Parameters

• index – identifiers for executing pipelines. I.e. list of companies tickers

• batch_size – size of batch for execute separation(may be usefull for lower memory usage). OR None (for full-size executing)

Returns

combined pipelines execute result

Return type

pd.DataFrame

fit(index)

Interface for training all pipelines

Parameters

index – identifiers for fit pipelines. I.e. list of companies tickers

LoadingPipeline

class ml_investment.pipelines.LoadingPipeline(data_loader, columns: List[str])

Bases: object

Wrapper for data loaders for loading data in execute(index) -> pd.DataFrame interface

Parameters

• data_loader – class implements load(index) -> pd.DataFrame interface

• columns – column names for loading

execute(index)

Interface for executing pipeline(lading data) for tickers.

Parameters

index – inentification for loading data, i.e. list of tickers

Returns

resulted data

Return type

pd.DataFrame

fit(index)