renate.benchmark.experiment_config module

renate.benchmark.experiment_config.model_fn(num_outputs, model_state_url=None, updater=None, model_name=None, num_inputs=None, num_hidden_layers=None, hidden_size=None, dataset_name=None, pretrained_model_name_or_path=None, prompt_size=10, clusters_per_task=5, per_task_classifier=True)

Returns a model instance.

Return type:

RenateModule

renate.benchmark.experiment_config.get_data_module(data_path, src_bucket, src_object_name, dataset_name, val_size, seed, pretrained_model_name_or_path, input_column, target_column)

Return type:

RenateDataModule

renate.benchmark.experiment_config.get_scenario(scenario_name, data_module, chunk_id, seed, num_tasks=None, groupings=None, degrees=None, input_dim=None, feature_idx=None, randomness=None, data_ids=None)

Function to create scenario based on name and arguments.

Parameters:

• scenario_name (str) – Name to identify scenario.

• data_module (RenateDataModule) – The base data module.

• chunk_id (int) – The data chunk to load in for the training or validation data.

• seed (int) – A random seed to fix the created scenario.

• num_tasks (Optional[int]) – The total number of expected tasks for experimentation.

• groupings (Optional[Tuple[Tuple[int]]]) – Used for scenario ClassIncrementalScenario to partition datasets into chunks by class. Used by DataIncrementalScenario to group domains to chunks..

• degrees (Optional[List[int]]) – Used for scenario ImageRotationScenario. Rotations applied for each chunk.

• input_dim (Union[List[int], Tuple[int], int, None]) – Used for scenario PermutationScenario. Input dimensionality.

• feature_idx (Optional[int]) – Used for scenario SoftSortingScenario. Index of feature to sort by.

• randomness (Optional[float]) – Used for all _SortingScenario. Randomness strength in [0, 1].

• data_ids (Optional[Tuple[Union[int, str]]]) – List of data_ids for the DataIncrementalScenario.

Return type:

Scenario

Returns:

An instance of the requested scenario.

Raises:

ValueError – If scenario name is unknown.

renate.benchmark.experiment_config.loss_fn(updater=None)

Return type:

Module

renate.benchmark.experiment_config.data_module_fn(data_path, chunk_id, seed, scenario_name, dataset_name, val_size=0.0, num_tasks=None, groupings=None, degrees=None, input_dim=None, feature_idx=None, randomness=None, src_bucket=None, src_object_name=None, pretrained_model_name_or_path=None, input_column=None, target_column=None, data_ids=None)

renate.benchmark.experiment_config.train_transform(dataset_name, model_name=None)

Returns a transform function to be used in the training.

Return type:

Optional[Callable]

renate.benchmark.experiment_config.test_transform(dataset_name, model_name=None)

Returns a transform function to be used for validation or testing.

Return type:

Optional[Callable]

renate.benchmark.experiment_config.lr_scheduler_fn(learning_rate_scheduler=None, learning_rate_scheduler_step_size=30, learning_rate_scheduler_gamma=0.1, learning_rate_scheduler_interval='epoch', learning_rate_scheduler_t_max=None, learning_rate_scheduler_eta_min=0)

Return type:

Tuple[Optional[Callable[[Optimizer], _LRScheduler]], str]

renate.benchmark.experiment_config.metrics_fn(num_outputs)

Return type:

Dict

renate.benchmark.experiment_config.optimizer_fn(optimizer, learning_rate, weight_decay, momentum=0.0)

Return type:

Callable