"Maximally possible" value of Qmax
Suppose
that we make 5000 experiments with a chosen level of significance a0
= 0.01, and we are ready to tolerate 5 percent false discoveries. We obtain,
say, 2000 discoveries in which p< 0.01 = a0 (Note: a
is used instead of alpha).
m = 5000
; S = 2000
Qmax = [(m/S) -1]
/ [(1/a) -1] = 0.0152
If all the null
hypotheses were true, we would have m = m0 = 5000 and we would expect: 0.01×5000 = 50 false discoveries = F. However, randomly, F can be larger, but practically not larger
than F=93 (because the probability to obtain more than 93 false discoveries from 5000 experiments
with true null hypotheses and with a = 0.01 is about 0.000000001 =10^-9).
So, S-F = 2000 -93 = 1907 true discoveries, which means that at the beginning
we must have had at the most 5000 - 1907 = 3093
true null hypotheses (instead of 5000 as we had wrongly supposed). So, we now
take m0 =3093 and we expect 0.01×3093 = 30.93 or 31 false
discoveries. Randomly we can obtain more, but practically not more than 65 (because the probability
to obtain more is again 0.000000001). So,
F < 65.
Now we know that
in fact S-F = 2000 -65 = 1935 true alternative
hypotheses;
5000 - 1935 = 3065 true null hypotheses = m0
Now we expect 0.01×3065
= 30.65 , but practically there cannot be more than 64 false discoveries. Hence, the maximal true proportion of false discoveries is 64/2000 = 0.032 which is more than the expected 0.0152, but this is still tolerable, because it is less than 5 percent.
- Here are the above calculations
once again:
m
= 5000
___ S= 2000 _
m0= 5000 5000×0,01
= 50 ---> 93 2000-65 = 1907
m0= 5000 -
1907 = 3093 3093×0.01 = 30.9 ---> 65
2000-65 = 1935
m0= 5000 - 1935 = 3065 3065×0.01 = 30.7 ---> 64 2000-64 = 1936
m0= 5000 - 1936 = 3064 3064×0.01 = 30.6 ---> 64 2000-64
= 1936
64/2000 = 0.0320 >
0.0152
If, say, m = 20.000 or 50,000
and S = 8000 or 20,000, respectively,
the true proportion of false discoveries would be considerably closer to 0.0152
= Qmax.
For example:
_m = 20,000 S=
8000 _
m0= 20,000 20,000×0,01= 200--> 285
8000-285= 7715
m0= 20,000-7715= 12,285
12,285×0.01=123-->
190 8000-190=
7810
m0= 20,000-7810= 12,190
12,190×0.01=122-->
189 8000-189=
7811
m0= 20,000-7811= 12,189
12,189×0.01=122-->
189 8000-189=
7811
189/8000= 0.0236 > 0.0152
................................................................................................................................................................
................................................................................................................................................................
