Evaluators

pydantic model retrieval_qa_benchmark.evaluators.MCMAEvaluator

Fields:

• dataset (retrieval_qa_benchmark.schema.dataset.BaseDataset)

• llm (retrieval_qa_benchmark.schema.model.BaseLLM)

• matcher (Callable[[str, retrieval_qa_benchmark.schema.datatypes.QARecord], float])

• out_file (str | None)

• transform (retrieval_qa_benchmark.schema.transform.TransformGraph)

field matcher: Callable[[str, QARecord], float] = <function mcma_fuzzy_matcher>

pydantic model retrieval_qa_benchmark.evaluators.MCSAEvaluator

Fields:

• dataset (retrieval_qa_benchmark.schema.dataset.BaseDataset)

• llm (retrieval_qa_benchmark.schema.model.BaseLLM)

• matcher (Callable[[str, retrieval_qa_benchmark.schema.datatypes.QARecord], float])

• out_file (str | None)

• transform (retrieval_qa_benchmark.schema.transform.TransformGraph)

field matcher: Callable[[str, QARecord], float] = <function mcsa_fuzzy_matcher>