renate.models.layers.shared_linear module

class renate.models.layers.shared_linear.SharedMultipleLinear(in_features, out_features, bias=True, share_parameters=True, num_updates=0)

Bases: ModuleDict

This implements a linear classification layer for multiple tasks (updates). This linear layer can be shared across all tasks or can have a separate layer per task. This follows the _task_params in the RenateBenchmarkingModule that is a nn.ModuleDict that holds a classifier per task (as in TIL).

Parameters:

• in_features (int) – size of each input sample

• out_features (int) – size of each output sample

• bias (bool) – If set to False, the layer will not learn an additive bias. Default: True

• share_parameters (bool) – Flag whether to share parameters or use individual linears per task. The interface remains identical, and the underlying linear layer is shared (or not).

• num_updates (int) – Number of updates that have happened/is happening.

increment_task()

Return type:

None