MvrConfig

class lightning_ir.models.bi_encoders.mvr.MvrConfig(query_length: int | None = 32, doc_length: int | None = 512, similarity_function: 'cosine' | 'dot' = 'dot', normalization: 'l2' | None = None, add_marker_tokens: bool = False, embedding_dim: int | None = None, projection: 'linear' | 'linear_no_bias' | None = 'linear', num_viewer_tokens: int | None = 8, **kwargs)[source]

Bases: MultiVectorBiEncoderConfig

Configuration class for a MVR model.

__init__(query_length: int | None = 32, doc_length: int | None = 512, similarity_function: 'cosine' | 'dot' = 'dot', normalization: 'l2' | None = None, add_marker_tokens: bool = False, embedding_dim: int | None = None, projection: 'linear' | 'linear_no_bias' | None = 'linear', num_viewer_tokens: int | None = 8, **kwargs)[source]

A MVR model encodes queries and document separately. It uses a single vector to represent the query and multiple vectors to represent the document. The document representation is obtained from n viewer tokens ([VIE]) prepended to the document. During training, a contrastive loss pushes the viewer token representations away from one another, such that they represent different “views” of the document. Only the maximum similarity between the query vector and the viewer token vectors is used to compute the relevance score.

Parameters:

  • query_length (int | None) – Maximum number of tokens per query. If None does not truncate. Defaults to 32.

  • doc_length (int | None) – Maximum number of tokens per document. If None does not truncate. Defaults to 512.

  • similarity_function (Literal['cosine', 'dot']) – Similarity function to compute scores between query and document embeddings. Defaults to “dot”.

  • normalization (Literal['l2'] | None) – Whether to normalize query and document embeddings. Defaults to None.

  • add_marker_tokens (bool) – Whether to prepend extra marker tokens [Q] / [D] to queries / documents. Defaults to False.

  • embedding_dim (int | None) – Dimension of the final embeddings. If None, it will be set to the hidden size of the backbone model. Defaults to None.

  • projection (Literal["linear", "linear_no_bias"] | None) – Type of projection layer to apply on the pooled embeddings. If None, no projection is applied. Defaults to “linear”.

  • num_viewer_tokens (int | None) – Number of viewer tokens to prepend to the document. Defaults to 8.

Methods

__init__([query_length, doc_length, ...])

A MVR model encodes queries and document separately.

Attributes

model_type

Model type for a MVR model.
model_type: str = 'mvr'

Model type for a MVR model.