Has the Sample Been Appropriately Selected?

The selection of the participants for an investigation is a vital research process, since the degree to which research findings can be generalised is largely dependent on the appropriateness of the sample. The target population is the entire group to which the results of the experiment can be generalised, such as all dental hygienists or all individuals with cerebral palsy. Since it is usually not feasible or even necessary to collect data on the entire population, a representative portion or sample is chosen. The best method for choosing a sample is random sampling, where every member of the population has an equal chance of being selected for the sample. Often access to the entire population is not available to the researcher, and a convenience sample of those members who are available is used. In this case, the researcher will try to identify participants characteristics that may have influence on the study and either verify statistically the groups are equal in regard to these characteristics or adjust for these differences during the analysis of the data with statistical tests such as analysis of covariance (ANCOVA)4 and logistic regression.

The size of the sample must be large enough to (1) accommodate the expected loss of participants during the investigation and (2) demonstrate differences between groups by statistical logic. If the groups are too small, a real difference may not be detected with statistics. Possible misinterpretations are that the experimental treatment or product is not effective (even though it is) or the treatments or products used for both groups are equally effective (when one is actually superior).

A formal method for determining sample size is called sample size calculation, which provides more assurance that real differences or effects, if present, will be identified. Computer programs such as G*Power5 that perform sample size calculation are now readily available. They allow both an investigator to choose an appropriate sample size and a reader to decide whether a conclusion of "no difference" or "no effect" is warranted, given the actual results of the study. Investigators who perform proper power analyses when designing their studies are much more likely to produce (and publish) clear and unambiguous results, from which conclusions of “effect present” or “effect absent” can be drawn. Studies in which “power” was not addressed directly are more likely to yield inconclusive results.6-7

A research report should clearly describe the characteristics of the sample so the reader can evaluate whether or not the sample is representative of the population and appropriate for the experiment. This is particularly true if the sample was not randomly selected. Included should be such characteristics as age, sex, race or ethnic group, relevant medical conditions, oral hygiene habits, etc. If the reader is not provided with this information, the extent to which the conclusions apply to any other group of individuals or situations remains ambiguous. For instance, a researcher might generalise about the incidence of congenital heart disease in Europe based on an experimental sample. However, if a description of the sample indicated it was composed of institutionalized patients (who have a much higher incidence of the disease), the reader would know such a generalisation was inappropriate. The paper should also describe the size of the sample, both at the beginning and the end of the investigation.

Cookie Consent