Validity, Reliability, & Ethics

Here is the last leg of my Research class. As always, feedback is more than welcome!


VALIDITY

Validity refers to “the extent to which a measure reflects only the desired construct without contamination from other systematically varying constructs” (Hoyle, Harris & Judd, 2002, p. 83). The fewer the number of systematic errors, those errors that have the potential to distort the study’s conclusions, the higher the validity of the study (p. 81). There are three components to validity when looking at a research project: construct validity, internal validity, and external validity. The following sections will discuss them with respect to the research study.

Construct Validity

Construct validity refers to the extent of which “both the independent and dependent variables accurately reflect or measure the constructs of interest (Hoyle, Harris & Judd, 2002, p. 32). In this case, the objective of the research study is to assess the impact of American media on young Latin American women ages 16-24. While the survey designed for this research study appears to be an effective way to measure the research questions at hand, it may be impossible to say if this is truly valid. Potentially there may be constructs of disinterest, or things we would rather not measure. There is also potential for random errors of measurement. For instance, there is a possibility of a mis-click on the survey, or the participant may not be feeling well that day.

Internal Validity

Internal validity refers to “the extent to which we can infer causal connections from an association between two variables” (Hoyle, Harris & Judd, 2002, p. 35-36). Such causal inference is possible only by conducting a randomized experiment, with random assignment of the independent variable, to prevent threats to internal validity (Hoyle, Harris & Judd, 2002, p. 40). Hoyle, Harris, and Judd (2002) state, “The chief strength of randomized experiments is their internal validity, which is accomplished through the researcher’s assumption of control over the independent variables in the design” (p. 38). The experimental design of this research study will deal with individual difference variables. Thus, casual connections will be a difficult to infer, and that is why a random experiment will be used so as not to threaten the internal validity.

External Validity

External validity refers to “the extent to which one can generalize the results of the research to the populations and settings of interest in the hypothesis” (Hoyle, Harris & Judd, 2002, p. 33). In this research, using a sample of populations from Mexico, Nicaragua, and Chile has external validity strengths. Because the three countries span across all the Latin American countries on the North and South American continents it will be easier to generalize the results of the research. However, the external validity is limited to the capital cities of those countries, and therefore may not represent those who live in significantly poorer cities in those countries.

RELIABILITY

Reliability refers to the extent to which “a measure is defined as the extent to which it is free from random error” (Hoyle, Harris & Judd, 2002, p. 83). As previously stated, validity refers to “the extent to which a measure reflects only the desired construct without contamination from other systematically varying constructs” (p. 83). Therefore, reliability is a prerequisite to validity.
Hoyle, Harris & Judd (2002) note three important influences on reliability. First, longer measures are more reliable than shorter ones. Second, a larger range of variation among participants in the survey leads to higher reliability. This will add to reliable distinctions to be made in the research study. Lastly, freedom from distractions, misunderstandings, etc. will help decrease a person’s tendencies to make random errors (Hoyle, Harris & Judd, 2002, p. 85).

ETHICS

The concluding consideration of the design of this research study is a discussion on ethics, since when conducting research with human participants, it is essential to protect their rights, safety, and dignity (Hoyle, Harris & Judd, 2002, p. 70). This project will involve persons ranging in age from 16 to 24, and therefore may cause some concerns for those who are under the age of 18. The main ethical issue addressed in this research design includes confidentiality of the persons who are participating.

Confidentiality of testing results will be communicated to the students in the introduction, with testing results being secured and viewed only by the researcher. The results, however, will potentially be shared with the larger public. Because the measurement procedure involves an online survey, anonymity will not be difficult to establish. By using this method the researcher will not be able to identify the persons involved.

The three ethical principles underlying federal regulations are respect for persons, beneficence, and justice (Hoyle, Harris & Judd, 2002, p. 48), which are discussed in the subsequent sections within the context of organizational research.

Respect

“Respect for persons incorporates at least two ethical convictions: first, that individuals should be treated as autonomous agents, and second, that persons with diminished autonomy are entitled to protection” (Hoyle, Harris & Judd, 2002, p. 48). Thus, all research participants should have the right to choose whether they want to participate in the study or not, without any adverse consequences for non-participation.

This research project addresses these two issues by inviting those who are eligible to participate, and it is clearly stated in the introduction to all participants that their involvement is completely voluntary. In addition, to ensure that people in the study have participated of their own free will and choice, an informed consent form can be created to accompany each online survey. One draw back to this process is that in order to get true “informed” consent, the full detail of the purpose of the study would need to be disclosed to participants prior to gathering feedback (Hoyle, Harris & Judd, 2002, p. 48). This could have a negative impact on the reliability of the data collected from the study. The risks versus benefits of obtaining informed consent would need to be weighed before that level of detail was shared with study participants.

Ideally, participants will be given information about the general nature of the research in the introduction, and avoiding active deception, they will not be given precise behaviors or the hypothesis before the research study to prevent demand characteristics or expectancy effects from contaminating the validity of the study (Hoyle, Harris & Judd, 2002, p. 56). Thus, participants will be subjected to some degree of deception by omission, though it seems unlikely that it would lead them to behave in ways that they would regret doing, and hence probably does not pose a significant ethical problem (Hoyle, Harris & Judd, 2002, p. 57).

Beneficence

In a research context, “the ethical principle of beneficence means that one should net harms one’s participants and that benefits to participants should be maximized while possible harms are minimized” (Hoyle, Harris & Judd, 2002 p. 51). This typically involves a risk-benefit analysis of the research study with respect to potential harm incurred by participants versus the potential benefits accrued by conducting the research (Hoyle, Harris & Judd, 2002, p. 52).

In evaluating this research project, it is clear that minimal harm can come to participants as a result of their participation. However, safeguards have been integrated into the research design to address any sort of potential risk. First, their participation is completely voluntary. Second, their identities are protected because there will be no way to connect specific questionnaire responses with an individual respondent. The topic of the study does not ask them to disclose significant personal information, nor does it ask them to divulge extremely specific information relating to their experiences that will expose them to a certain level of pain or anxiety related to their experience.

While there are some potential risks associated with the design of this project, the design could also lead to participants realizing certain benefits from the data that is gathered. The information that is shared by participants will be used to evaluate the overall affects of American media on a growing population of young Latin American women. Given that there is no current, specific information about this, participants in the research project will have an opportunity to contribute to how American media affects an international audience.

Justice

The study group that is chosen to participate in this research project will be randomly chosen from a group of students in three demographic areas (Mexico City, Mexico, Managua, Nicagragua, and Santiago, Chile). The idea of justice in social science research means that the researcher should not take advantage of overburdened populations simply because they are convenient and available (Hoyle, Harris & Judd, 2002, p. 71). Overburdened populations are populations at a disadvantage, such as the poor, the very sick, or the institutionalized (Hoyle, Harris & Judd, 2002, p. 54). While this study does use a group of students who live in third-world countries, they are not at a significant disadvantage.