L1Regularization

class lightning_ir.loss.regularization.L1Regularization(query_weight: float = 0.0001, doc_weight: float = 0.0001)

Bases: RegularizationLossFunction

L1 Regularization loss function for query and document embeddings.

L1 Regularization, also known as Lasso Regression, adds a penalty term to the loss function that is proportional to the absolute value of the model’s parameters (in this case, the query and document embeddings). This encourages sparsity in the embeddings, meaning that it pushes many of the embedding dimensions to be exactly zero. This can lead to more interpretable models and can also help with feature selection by effectively removing less important features.

Originally proposed in: Regression Shrinkage and Selection via the Lasso

Methods

compute_loss(output)

Compute the L1 regularization loss.

compute_loss(output: BiEncoderOutput) → torch.Tensor

Compute the L1 regularization loss.

Parameters:

output (BiEncoderOutput) – The output from the model containing query and document embeddings.

Returns:

The computed loss.

Return type:

torch.Tensor