evaluate_run

lightning_ir.base.validation_utils.evaluate_run(run: DataFrame, qrels: DataFrame, measures: Sequence[str]) → Dict[str, float][source]

Convenience function to evaluate a run against qrels using a set of measures.

Parameters:

  • run (pd.DataFrame) – Parse TREC run.

  • qrels (pd.DataFrame) – Parse TREC qrels.

  • measures (Sequence[str]) – Metrics corresponding to ir-measures measure strings.

Returns:

Calculated metrics.

Return type:

Dict[str, float]