These examples are connected to testing independence using the Gini rank statistic and Spearman’s footrule.
An extremal problem with applications to testing multivariate independence
As mentioned above, Spearman’s footrule is calculated on the re-ranked list of overlapping elements, and ignores the actual rank of the overlapping elements.
Methods for comparing rankings of search engine results
In the case of list similarity, we have a contribution by providing a weighted generalization of Spearman’s footrule and Kendall’s tau and prove their equivalence for permutations and partial lists but we presented separately .
On the Weakenesses of Correlation Measures used for Search Engines' Results (Unsupervised Comparison of Search Engine Rankings)
Among them, Spearman’s footrule and Kendall’s tau are commonly used.
On the Weakenesses of Correlation Measures used for Search Engines' Results (Unsupervised Comparison of Search Engine Rankings)
In Figure 1, we present the concordance measure results using Spearman’s footrule and Kendal Tau.
On the Weakenesses of Correlation Measures used for Search Engines' Results (Unsupervised Comparison of Search Engine Rankings)
We have three observations: First, most of the queries have low overlap as measured by the Jaccard ratio; second, most of the queries fall into the narrow interval [-0.2, 0.2] as measured by the normalized footrule; third, for most of the queries the DCG value is orthogonal to both measures.
On the Weakenesses of Correlation Measures used for Search Engines' Results (Unsupervised Comparison of Search Engine Rankings)
There is no correlation (not upon intended) between DCG and footrule.
On the Weakenesses of Correlation Measures used for Search Engines' Results (Unsupervised Comparison of Search Engine Rankings)
URL-based Jaccard ratio vs. normalized footrule In practice, here we present how weak lists based correlation measures are and we show in a plain cross product (scatter plot) that little correlation (or lack of correlation thereof ) is because the lists have really small intersection.
On the Weakenesses of Correlation Measures used for Search Engines' Results (Unsupervised Comparison of Search Engine Rankings)
In Fig. 6, we show the relationship between the overlap of the URLs and the normalized footrule for two markets US and JP.
On the Weakenesses of Correlation Measures used for Search Engines' Results (Unsupervised Comparison of Search Engine Rankings)
As soon as the overlap of the lists decreases, the range of the normalized footrule also shrinks.
On the Weakenesses of Correlation Measures used for Search Engines' Results (Unsupervised Comparison of Search Engine Rankings)
Content-based measures vs. normalized footrule Now, we ﬁnally present the comparison between contentbased measures such as Jterm,k and φterm,k versus the most common correlation measures.
On the Weakenesses of Correlation Measures used for Search Engines' Results (Unsupervised Comparison of Search Engine Rankings)
In Fig. 8, we present the relationship between the contentbased measures and the normalized footrule for the US market —for which we have the largest overlap in our experiments.
On the Weakenesses of Correlation Measures used for Search Engines' Results (Unsupervised Comparison of Search Engine Rankings)
This is to show how small the range of the normalized footrule is and, in contrast, how the range of the contentbased measure offers more variety and insights.
On the Weakenesses of Correlation Measures used for Search Engines' Results (Unsupervised Comparison of Search Engine Rankings)
On one side, this will provide the same experience to the user, we see intuitively that the engines are highly correlated for the query; on the other side, the footrule measure does not provide any information, despite our best efforts to ﬁnd common items in the lists.
On the Weakenesses of Correlation Measures used for Search Engines' Results (Unsupervised Comparison of Search Engine Rankings)
In such a case, having a 10 URLs lists (20 total) is large enough that if only 3-4 URLs are really common and high in the result list, the footrule is dominated by the denominator and the contribution in the numerator is mixed.
On the Weakenesses of Correlation Measures used for Search Engines' Results (Unsupervised Comparison of Search Engine Rankings)
***