The last two questions correspond broadly to outcomes of statistical analyses using confidence intervals and examining the statistical power of a test, but careful interpretation is needed. Other statistical approches to these questions are available.
When a study involves multiple statistical tests, some laymen assume that the confidence associated with individual tests is the confidence one should have in the results of the study itself. In fact, the results of all the statistical tests conducted during a study must be judged as a whole in determining what confidence one may place in the positive links it produces. If researchers conducting a study perform 40 independent statistical tests of the existence of an effect at a 5% significance level, they can expect about two of the tests to return false positives. If they in fact find 3 tests where the result of the test is "effect detected", the confidence associated with the conclusion, 'as the result of the survey', that the effect exists is actually about 32%; it's what one should expect to see two-thirds of the time even if the effect does not exist.
Prime Minister Vladimir Putin Meets with Vyacheslav Volodin, Appointed Deputy Prime Minister and Chief of the Government Executive Office by a Presidential Decree Today
Oct 21, 2010; Prime Minister Vladimir Putin expressed the hope that in his new position Vyacheslav Volodin will use the positive management...