FaissIVFPQIndexConfig

class lightning_ir.retrieve.faiss.faiss_indexer.FaissIVFPQIndexConfig(num_train_embeddings: int | None = None, num_centroids: int = 262144, ef_construction: int = 40, num_subquantizers: int = 16, n_bits: int = 8)[source]

Bases: FaissIVFIndexConfig

Configuration class for FAISS IVFPQ indexers in the Lightning IR framework.

__init__(num_train_embeddings: int | None = None, num_centroids: int = 262144, ef_construction: int = 40, num_subquantizers: int = 16, n_bits: int = 8) → None[source]

Initialize the FaissIVFPQIndexConfig.

Parameters:

  • num_train_embeddings (int | None) – Number of embeddings to use for training the index. If None, it will be set later. Defaults to None.

  • num_centroids (int) – Number of centroids for the IVF index. Defaults to 262144.

  • ef_construction (int) – The size of the dynamic list used during construction. Defaults to 40.

  • num_subquantizers (int) – Number of subquantizers for the PQ index. Defaults to 16.

  • n_bits (int) – Number of bits for the PQ index. Defaults to 8.

Methods

__init__([num_train_embeddings, ...])

Initialize the FaissIVFPQIndexConfig.

Attributes

classmethod from_pretrained(index_dir: Path | str) → IndexConfig

Load the index configuration from a directory.

Parameters:

index_dir (Path | str) – Path to the directory containing the index configuration.

Returns:

An instance of the index configuration class.

Return type:

IndexConfig

Raises:

ValueError – If the index type in the configuration does not match the expected class name.

indexer_class

alias of FaissIVFPQIndexer

save(index_dir: Path) → None

Save the index configuration to a directory.

Parameters:

index_dir (Path) – The directory to save the index configuration.

to_dict() → dict

Convert the index configuration to a dictionary.

Returns:

A dictionary representation of the index configuration.

Return type:

dict