On the T-test
S.Y. Novak
(Submitted on 28 Dec 2020)
https://export.arxiv.org/abs/2012.14530
    The T-test is probably the most popular statistical test; it is routinely recommended by the textbooks. The applicability of the test relies upon the validity of normal or Student's approximation to the distribution of Student's statistic tn. However, the latter assumption is not valid as often as assumed. We show that normal or Student's approximation to $\,\L(t_n)\,$ does not hold uniformly even in the class P_n of samples from zero-mean unit-variance bounded distributions. We present lower bounds to the corresponding error. The fact that a non-parametric test is not applicable uniformly to samples from the class Pn seems to be established for the first time. It means the T-test can be misleading, and should not be recommended in its present form. We suggest a generalisation of the test that allows for variability of possible limiting/approximating distributions to $\,\L(t_n)$.