renate.benchmark.experiment_config module#
- renate.benchmark.experiment_config.model_fn(num_outputs, model_state_url=None, updater=None, model_name=None, num_inputs=None, num_hidden_layers=None, hidden_size=None, dataset_name=None, pretrained_model_name_or_path=None, prompt_size=10, clusters_per_task=5, per_task_classifier=True)[source]#
Returns a model instance.
- Return type:
- renate.benchmark.experiment_config.get_data_module(data_path, src_bucket, src_object_name, dataset_name, val_size, seed, pretrained_model_name_or_path, input_column, target_column)[source]#
- Return type:
- renate.benchmark.experiment_config.get_scenario(scenario_name, data_module, chunk_id, seed, num_tasks=None, groupings=None, degrees=None, input_dim=None, feature_idx=None, randomness=None, data_ids=None)[source]#
Function to create scenario based on name and arguments.
- Parameters:
scenario_name¶ (
str) – Name to identify scenario.data_module¶ (
RenateDataModule) – The base data module.chunk_id¶ (
int) – The data chunk to load in for the training or validation data.seed¶ (
int) – A random seed to fix the created scenario.num_tasks¶ (
Optional[int]) – The total number of expected tasks for experimentation.groupings¶ (
Optional[Tuple[Tuple[int]]]) – Used for scenario ClassIncrementalScenario to partition datasets into chunks by class. Used by DataIncrementalScenario to group domains to chunks..degrees¶ (
Optional[List[int]]) – Used for scenario ImageRotationScenario. Rotations applied for each chunk.input_dim¶ (
Union[List[int],Tuple[int],int,None]) – Used for scenario PermutationScenario. Input dimensionality.feature_idx¶ (
Optional[int]) – Used for scenario SoftSortingScenario. Index of feature to sort by.randomness¶ (
Optional[float]) – Used for all _SortingScenario. Randomness strength in [0, 1].data_ids¶ (
Optional[Tuple[Union[int,str]]]) – List of data_ids for the DataIncrementalScenario.
- Return type:
- Returns:
An instance of the requested scenario.
- Raises:
ValueError – If scenario name is unknown.
- renate.benchmark.experiment_config.data_module_fn(data_path, chunk_id, seed, scenario_name, dataset_name, val_size=0.0, num_tasks=None, groupings=None, degrees=None, input_dim=None, feature_idx=None, randomness=None, src_bucket=None, src_object_name=None, pretrained_model_name_or_path=None, input_column=None, target_column=None, data_ids=None)[source]#
- renate.benchmark.experiment_config.train_transform(dataset_name, model_name=None)[source]#
Returns a transform function to be used in the training.
- Return type:
Optional[Callable]
- renate.benchmark.experiment_config.test_transform(dataset_name, model_name=None)[source]#
Returns a transform function to be used for validation or testing.
- Return type:
Optional[Callable]
- renate.benchmark.experiment_config.lr_scheduler_fn(learning_rate_scheduler=None, learning_rate_scheduler_step_size=30, learning_rate_scheduler_gamma=0.1, learning_rate_scheduler_interval='epoch', learning_rate_scheduler_t_max=None, learning_rate_scheduler_eta_min=0)[source]#
- Return type:
Tuple[Optional[Callable[[Optimizer],_LRScheduler]],str]