A generalized Sidak-Holm procedure and control of generalized error rates under independence

Wenge Guo, Joseph Romano

Research output: Contribution to journalArticlepeer-review

51 Scopus citations


For testing multiple null hypotheses, the classical approach to dealing with the multiplicity problem is to restrict attention to procedures that control the familywise error rate (FWER), the probability of even one false rejection. In many applications, one might be willing to tolerate more than one false rejection provided the number of such cases is controlled, thereby increasing the ability of the procedure to detect false null hypotheses. This suggests replacing control of the FWER by controlling the probability of k or more false rejections, which is called the k-FWER. In Hommel and Hoffmann (1987) and Lehmann and Romano (2005a), single step and stepdown procedures are derived that control the k-FWER, without making any assumptions concerning the dependence structure of the p-values of the individual tests. However, if the p-values are mutually independent, one can improve the procedures. In fact, Sarkar (2005) provided such an improvement. However, we show other improvements are possible which appear to be generally much better, and are sometimes unimprovable. When k=1, the procedure reduces to the classical method of Sidak, and the stepdown procedure is unimprovable and strictly dominates that of Sarkar. Under a monotonicity condition, an unimprovable procedure is obtained. In the case k=2, the monotonicity condition is satisfied, and the condition can be checked numerically in general. We then develop a stepdown method that controls the false discovery proportion. Except for the case of k-FWER control with k=1, the gains are surprisingly dramatic, and theoretical and numerical evidence is given.

Original languageEnglish (US)
Article number3
JournalStatistical applications in genetics and molecular biology
Issue number1
StatePublished - Jan 25 2007
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Molecular Biology
  • Genetics
  • Computational Mathematics


  • False discovery proportion
  • Generalized familywise error rate
  • Multiple testing
  • P-value
  • Sidak procedure
  • Stepdown procedure


Dive into the research topics of 'A generalized Sidak-Holm procedure and control of generalized error rates under independence'. Together they form a unique fingerprint.

Cite this