Term Frequency Normalisation Tuning


The term frequency normalisation tuning estimates the free-parameter of a TermFrequencyNormalisation method. It is crucial issue in InformationRetrieval that significantly affects robustness and effectiveness of retrieval performance.

Term frequency normalisation tuning by measuring normalisation effect

The tuning method by measuring normalisation effect estimates the free-parameters of the normalisation component of BM25 and the Normalisation2 of the DivergenceFromRandomness models (He & Ounis, ECIR2005). The notion of normalisation effect refers to the variation of the term frequency with respect to the DocumentLength distribution, as defined in (He & Ounis, ECIR2005). The optimal normalisation effect stands for the normalisation effect measure that corresponding to the parameter setting that gives the highest mean average precision, which is a collection-independent constant. The tuning methodology can be summarised as follows:

On a given new collection, the normalisation effect is computed over a set of queries that are generated by the QuerySimulation based on the DFR model.