pytabkit.bench.alg_wrappers package

Submodules

pytabkit.bench.alg_wrappers.general module

class pytabkit.bench.alg_wrappers.general.AlgWrapper

Bases: object

Base class for ML methods that can be run in the benchmarking code.

__init__(**config)

Constructor.

Parameters:: config – Configuration parameters.

get_max_n_vectorized(task_info)

Returns 1 by default, should be overridden in subclasses if they benefit from vectorization.

Parameters:: task_info (TaskInfo) – Information about the task that this method should run on.
Returns:: Maximum number of train/test splits that this method can be run on at once.
Return type:: int

get_pred_param_names(task_package)

Return the possible prediction parameter names, used as suffixes for alg names :param task_package: Task package. :return: List of the possible names.

Parameters:: task_package (TaskPackage)
Return type:: List[str]

get_required_resources(task_package)

Should be overridden in subclasses.

Parameters:: task_package (TaskPackage) – Information about the task that should be executed.
Returns:: Information about the estimated required resources that will be needed to run this task.
Return type:: RequiredResources

run(task_package, logger, assigned_resources, tmp_folders, metrics=None)

Run the ML method on the given task. Should be overridden in subclasses.

Parameters:

task_package (TaskPackage) – Information about the task to be run.
logger (Logger) – Logger.
assigned_resources (NodeResources) – Assigned resources (e.g. number of threads).
tmp_folders (List[Path]) – Temporary folders, one for each train/test split, to save temporary data to.
metrics (Metrics | None)

Returns:

A dictionary of lists of ResultManager objects. The dict key is the predict params name, which is used as a suffix for the alg_name, and each list contains ResultManagers for each train/test split.

Return type:

Dict[str, List[ResultManager]]

pytabkit.bench.alg_wrappers.interface_wrappers module

class pytabkit.bench.alg_wrappers.interface_wrappers.AlgInterfaceWrapper

Bases: AlgWrapper

Base class for wrapping AlgInterface classes for benchmarking.

__init__(create_alg_interface_fn, **config)

Constructor.

Parameters:

create_alg_interface_fn (Callable[[...], AlgInterface] | None) – Function to create an AlgInterface via create_alg_interface_fn(**config).
config – Configuration parameters.

create_alg_interface(task_package)

Method to create an AlgInterface.

Parameters:: task_package (TaskPackage) – Task information.
Returns:: An AlgInterface corresponding to an ML method.
Return type:: AlgInterface

get_pred_param_names(task_package)

Return the possible prediction parameter names, used as suffixes for alg names :param task_package: Task package. :return: List of the possible names.

Parameters:: task_package (TaskPackage)
Return type:: List[str]

get_required_resources(task_package)

Should be overridden in subclasses.

Parameters:: task_package (TaskPackage) – Information about the task that should be executed.
Returns:: Information about the estimated required resources that will be needed to run this task.
Return type:: RequiredResources

run(task_package, logger, assigned_resources, tmp_folders, metrics=None)

Run the ML method on the given task. Should be overridden in subclasses.

Parameters:

task_package (TaskPackage) – Information about the task to be run.
logger (Logger) – Logger.
assigned_resources (NodeResources) – Assigned resources (e.g. number of threads).
tmp_folders (List[Path]) – Temporary folders, one for each train/test split, to save temporary data to.
metrics (Metrics | None)

Returns:

A dictionary of lists of ResultManager objects. The dict key is the predict params name, which is used as a suffix for the alg_name, and each list contains ResultManagers for each train/test split.

Return type:

Dict[str, List[ResultManager]]

class pytabkit.bench.alg_wrappers.interface_wrappers.AlgorithmSelectionWrapper

Bases: AlgInterfaceWrapper

__init__(sub_wrappers, **config)

Constructor.

Parameters:

create_alg_interface_fn – Function to create an AlgInterface via create_alg_interface_fn(**config).
config – Configuration parameters.
sub_wrappers (List[AlgInterfaceWrapper])

get_required_resources(task_package)

Should be overridden in subclasses.

Parameters:: task_package (TaskPackage) – Information about the task that should be executed.
Returns:: Information about the estimated required resources that will be needed to run this task.
Return type:: RequiredResources

class pytabkit.bench.alg_wrappers.interface_wrappers.AutoGluonModelInterfaceWrapper

Bases: AlgInterfaceWrapper

__init__(**config)

Constructor.

Parameters:

create_alg_interface_fn – Function to create an AlgInterface via create_alg_interface_fn(**config).
config – Configuration parameters.

class pytabkit.bench.alg_wrappers.interface_wrappers.CaruanaEnsembleWrapper

Bases: AlgInterfaceWrapper

__init__(sub_wrappers, **config)

Constructor.

Parameters:

create_alg_interface_fn – Function to create an AlgInterface via create_alg_interface_fn(**config).
config – Configuration parameters.
sub_wrappers (List[AlgInterfaceWrapper])

get_required_resources(task_package)

Should be overridden in subclasses.

Parameters:: task_package (TaskPackage) – Information about the task that should be executed.
Returns:: Information about the estimated required resources that will be needed to run this task.
Return type:: RequiredResources

class pytabkit.bench.alg_wrappers.interface_wrappers.CatBoostHyperoptInterfaceWrapper

Bases: MultiSplitAlgInterfaceWrapper

create_single_alg_interface(n_cv, task_type)

Parameters:

n_cv (int)
task_type (TaskType)

Return type:

AlgInterface

class pytabkit.bench.alg_wrappers.interface_wrappers.CatBoostInterfaceWrapper

pytabkit.bench.alg_wrappers package

Submodules

pytabkit.bench.alg_wrappers.general module

pytabkit.bench.alg_wrappers.interface_wrappers module

Module contents