Please enable JavaScript.
Coggle requires JavaScript to display documents.
TOP-DOWN SPEARMAN CORR (Order statistic (then, U_1, U_2, ..., U_n be order…
TOP-DOWN SPEARMAN CORR
Usage Case
-
Useful for SKEWED ranked-values such as ctr, clicks, ...
-
How
-
E.g such as reciprocal of index, discount factor of NDCG
-
With the notation of lower index for more IMPORTANT rank, S[k] = summation{iii from k to n}(1.0/iii)
-
Hypothesis Testing
-
if the marginal distribution is skewed, then this statistic (top-down spearman corr) is powerful
if the marg dist is normal, Pearson correl is a better choice
If the marg dist is uniform, normal Spearman or Kendall's Tau are better choice
If the marg dist is at two extremes, not the center, Van Der Waerden is a better statistic
-
Order statistic
then, U_1, U_2, ..., U_n be order statistics of Uniform(0,1)
U_k ~ Beta(k, n+1-k) distribution
-
-
let X_1, ..., X_n be any continuous distribution and ASCENDING
E[U_(k+1) - U_k] = 1/(n+1) for all k. So come the (100* k/(n+1)) - percentile of the underlying distribution is X_k