renate.benchmark.experiment_config module#
- renate.benchmark.experiment_config.model_fn(num_outputs, model_state_url=None, updater=None, model_name=None, num_inputs=None, num_hidden_layers=None, hidden_size=None, dataset_name=None, pretrained_model_name_or_path=None, prompt_size=10, clusters_per_task=5, per_task_classifier=True)[source]#
Returns a model instance.
- Return type:
- renate.benchmark.experiment_config.get_data_module(data_path, src_bucket, src_object_name, dataset_name, val_size, seed, pretrained_model_name_or_path, input_column, target_column)[source]#
- Return type:
- renate.benchmark.experiment_config.get_scenario(scenario_name, data_module, chunk_id, seed, num_tasks=None, groupings=None, degrees=None, input_dim=None, feature_idx=None, randomness=None, data_ids=None)[source]#
Function to create scenario based on name and arguments.
- Parameters:
scenario_name¶ (
str
) – Name to identify scenario.data_module¶ (
RenateDataModule
) – The base data module.chunk_id¶ (
int
) – The data chunk to load in for the training or validation data.seed¶ (
int
) – A random seed to fix the created scenario.num_tasks¶ (
Optional
[int
]) – The total number of expected tasks for experimentation.groupings¶ (
Optional
[Tuple
[Tuple
[int
]]]) – Used for scenarioClassIncrementalScenario
to partition datasets into chunks by class. Used byDataIncrementalScenario
to group domains to chunks..degrees¶ (
Optional
[List
[int
]]) – Used for scenarioImageRotationScenario
. Rotations applied for each chunk.input_dim¶ (
Union
[List
[int
],Tuple
[int
],int
,None
]) – Used for scenarioPermutationScenario
. Input dimensionality.feature_idx¶ (
Optional
[int
]) – Used for scenarioSoftSortingScenario
. Index of feature to sort by.randomness¶ (
Optional
[float
]) – Used for all_SortingScenario
. Randomness strength in [0, 1].data_ids¶ (
Optional
[Tuple
[Union
[int
,str
]]]) – List of data_ids for theDataIncrementalScenario
.
- Return type:
- Returns:
An instance of the requested scenario.
- Raises:
ValueError – If scenario name is unknown.
- renate.benchmark.experiment_config.data_module_fn(data_path, chunk_id, seed, scenario_name, dataset_name, val_size=0.0, num_tasks=None, groupings=None, degrees=None, input_dim=None, feature_idx=None, randomness=None, src_bucket=None, src_object_name=None, pretrained_model_name_or_path=None, input_column=None, target_column=None, data_ids=None)[source]#
- renate.benchmark.experiment_config.train_transform(dataset_name, model_name=None)[source]#
Returns a transform function to be used in the training.
- Return type:
Optional
[Callable
]
- renate.benchmark.experiment_config.test_transform(dataset_name, model_name=None)[source]#
Returns a transform function to be used for validation or testing.
- Return type:
Optional
[Callable
]
- renate.benchmark.experiment_config.lr_scheduler_fn(learning_rate_scheduler=None, learning_rate_scheduler_step_size=30, learning_rate_scheduler_gamma=0.1, learning_rate_scheduler_interval='epoch', learning_rate_scheduler_t_max=None, learning_rate_scheduler_eta_min=0)[source]#
- Return type:
Tuple
[Optional
[Callable
[[Optimizer
],_LRScheduler
]],str
]