The term frequency normalisation tuning estimates the free-parameter of a TermFrequencyNormalisation method. It is a crucial issue in InformationRetrieval that significantly affects robustness and effectiveness of retrieval performance.
Term frequency normalisation tuning by measuring normalisation effect
The tuning method by measuring normalisation effect estimates the free-parameters of BM25'sNormalisation and the Normalisation2 of the DivergenceFromRandomness models (He & Ounis, ECIR2005). The notion of normalisation effect refers to the variation of the term frequency with respect to the Document Length distribution, as defined in (He & Ounis, ECIR2005). The optimal normalisation effect stands for the normalisation effect measure that corresponding to the parameter setting that gives the highest mean average precision, which is a collection-independent constant. The tuning methodology can be summarised as follows:
Obtain the optimal normalisation effect on a training collection using relevance assessment.
On a given new collection, apply the parameter setting such that it gives the optimal normalisation effect.