When evaluating a study, statisticians consider conclusion validity, internal validity, construct validity and external validity along with inter-observer reliability, test-retest reliability, alternate form reliability and internal consistency. Statistical validity describes whether the results of the research are accurate. Reliability describes whether the results are repeatable.
Conclusion validity describes whether the variables being studied are actually related. A study might not have good conclusion validity if the researcher has a very small sample size because the relationship might be missed due to individual differences.
Internal validity considers whether the study identifies a cause-effect relationship. If a group of researchers want to determine whether a tutorial session has an impact on grades, they need to confirm that the effect is not because the students who participate are more motivated to work or put in extra time rather than the tutorial itself.
Construct validity concerns whether the research is studying what it claims to study. If a researcher wants to study people who are shy, he should be sure that he is studying people who are shy and not just quiet observers who are comfortable with people.
External validity describes how well the results can be generalized to situations outside of the study. For example, the researcher would want to know that the successful tutoring sessions work in a different school district.
Different researchers obtain the same results if a study has high inter-observer reliability. Test-retest reliability describes whether the researcher finds the same results when conducting the test again at a later time. Alternate form reliability evaluates whether two assessments of the same information lead to the same results. Internal consistency looks at whether the results in one data set are reliable by dividing the data into different sets and comparing them.