This session focuses on developing the thinking skills associated with becoming critical consumers of research literature. Through these learning experiences, you will be able to identify key research concepts and utilize these research concepts to begin evaluating research articles more critically. These critical thinking competencies are directly applicable to your coursework and to each aspect of the research process that you will continue to encounter as you advance in your program of studies.
These sessions directly support the following Track 1 competencies:
Throughout the session, you will be using two research articles. They will be referred to as Article 1 and Article 2. Article 1 is an example of a quantitative research study and Article 2 is an example of a qualitative research study. You will find links to the articles in the courseroom activities, Dissecting a Quantitative Article and Dissecting a Qualitative Article.
Upon completion of these sessions, you will be able to:
Identifying a Peer-reviewed Journal
When evaluating research articles, there may be varying opinions on the credibility of the evidence presented. A beginning step for graduate learners is to determine if the article under scrutiny is a peer-reviewed research article.
Although one could make a case for only using peer-reviewed journals, there are many important published studies that have not gone through the rigorous process of peer review. Therefore, a graduate learner needs to understand how to judge the credibility of research studies.
Knowing how to evaluate research does not happen naturally. Throughout life, individuals make decisions about truth based on nonscientific methods. Examples of nonscientific evidence are opinions, value judgments, personal experiences, unsupported assertions, and second-hand reports. Therefore, learning to distinguish between nonscientific evidence and scientific evidence is a learning process that requires self awareness and practice. As a graduate student, recognizing nonscientific methods and learning how to apply critical thinking skills to the dissection and evaluation of research articles is an essential tool for your educational success.
In this session, obtaining a comprehensive understanding of quantitative and qualitative research methods and data analysis is not the objective. The focus is on helping you achieve a basic understanding of key research concepts that form the foundation of the research process. This session will help you to develop the foundation of knowledge and understanding that you will continue to build upon as you complete coursework in your doctoral program.
You will apply your knowledge of the research process when reading research articles for your courses and comprehensive exam papers, and when designing your dissertation proposal. As you continue in the transformational process, you will become increasingly sophisticated in critically evaluating the claims of authors in research articles, and be able to separate out scientifically verified claims from unsubstantiated claims. "Research can be regarded as a process of asking a question (or a related series of questions) and then initiating a systematic process to obtain valid answers to that question" (Meltzoff, 2001, p. 13). A research article describes the research design and the results of the study conducted.
In activities Dissecting a Quantitative Article and Dissecting a Qualitative Article, you will evaluate each of these components of a research article. The activity documents will identify the evaluation questions you will ask. For now, remember that the key things to look for when evaluating research are:
The first step to dissecting and evaluating a research article is the review for clarity and consistency. The abstract, the introduction, the problem statement and the research hypothesis or research question should not only be clearly stated, they should be consistent. The reader should be able to understand the relationship between each component. These components set the stage for the rest of the document.
The method section describes in detail how the study was conducted. The evaluation of the methodology consists of reviewing the research design, how the research participants were recruited, and the data collection procedures. Different research methodologies have different reporting requirements. The evaluation of a research article will vary somewhat depending on whether the research article is quantitative or qualitative. Article 1 is a quantitative study.
According to Creswell (2003, p. 18), "A quantitative approach is one in which the investigator primarily uses positivistic claims for developing knowledge (i.e., cause and effect thinking, reduction to specific variables and hypotheses and questions, use of measurement and observation, and the test of theories), employs strategies of inquiry such as experiments and surveys, and collects data on predetermined instruments that yield statistical data. The goal of quantitative research is to support or disprove the hypotheses."
Quantitative research designs attempt to find relationships between and among variables. The essence of quantitative research studies involves numbers and measurement. So the data collected involves numbers and, ultimately, statistics are used to analyze those numbers. Quantitative research is aimed at verification. Examples of quantitative designs are:
In order to dissect and evaluate research articles, there is a need to have knowledge of research methodology. Being able to critique a research study is not based on your personal beliefs or ideas, but on the scientific soundness of the study. As you progress through your graduate program, you will develop the skills necessary to critique a research article. As you continue to read research articles, the following questions will guide you in developing your skills for critically reading and evaluating research articles.
Successful evaluation of a research article, in the case of quantitative studies, requires a basic understanding of variables—what they are and how they are described. Variables are specific concepts that are being studied. They vary along a continuum in terms of its characteristics (continuous variables) or they have at least two categories (categorical variables). For example, let us say the study is to examine gender differences in levels of morality. Gender is one variable in the study, and the other variable is frequency of exercise.
Gender has two categories: (1) male and (2) female.
Frequency of exercise can vary along a continuum. In this case, let us say frequency scores can range from zero to 10 times per week, with higher scores reflecting higher frequency of exercise.
In some studies, the variables can be divided into two categories: independent and dependent variables. The independent variable causes or influences some change in the dependent variable. The dependent variable is the outcome variable and must be measurable. You can use this formula to help you differentiate between the independent and dependent variable:
X → Y
X influences Y where X is the independent variable (IV) and Y is the dependent variable (DV). For example, a researcher might ask whether a particular intervention causes a desired result. The intervention would be the IV and the desired result the DV.
In the case above, gender might be considered the independent variable and frequency the dependent variable, but since gender cannot really be a cause of something like one's exercise regime, most methodologists do notuse the term IV and DV in such cases, preferring terms such as criterion variable and outcome variable.
An important class of variables is named extraneous variables. An extraneous variable can create problems for the researcher. Extraneous variables are undesired variables that can influence the dependent variable and change or invalidate the results of an experiment. For example, suppose a researcher is investigating whether a particular training experience improves performance on some task. She assigns participants from an organization randomly to either the experimental group or the control group. Training is then provided to the experimental group, but not the control group. But the fact that some of the participants may have more experience with the task than others could be an extraneous variable—it could confuse the results because the more experienced participants score higher, not as a result of the training, but as a result of their having more experience.
Extraneous variables are the chief reason for controlled studies—studies designed to reduce or eliminate the influence of extraneous variables.
Creswell (2003, p.18) describes qualitative research as follows:
"…a qualitative approach is one in which the inquirer often makes knowledge claims based primarily on constructivist perspectives (i.e., the multiple meanings of individual experiences, meanings socially and historically constructed, with an intent of developing a theory or a pattern)…The researcher collects open-ended, emerging data with the primary intent of developing themes from the data."
Essentially, qualitative research provides descriptions of experiences in the participants' own words, which are then analyzed and interpreted by the researcher. Put the simplest way, where quantitative data are numbers, qualitative data are words.
The goal of qualitative research is to gain an understanding of the experiences of the individuals who participate in the research study. Qualitative research is aimed at discovery, understanding, meaning, and developing knowledge from the perspective of first person accounts of the experience under inquiry. Examples of qualitative research methodologies are:
When presenting the results of a qualitative research study, there needs to a clear and detailed description that includes direct quotes from data collected from the participants. One way in which the quality of the research results can be judged is by the reaction of the reader. Are the results believable? The reader should be able to say, "I understand," or "That makes perfect sense." The presentation of the data should convince the reader of the credibility of the data analysis.
The results section of a qualitative study is a written presentation that includes the voices of the participants, the reflections of the researcher and an interpretation of the answer to the research question and problem (Creswell, 2007).
"A weak or unconvincing analysis can also stem from a failure to provide adequate examples from the data... The researcher needs to make sure that their interpretations and analytic points are consistent with the data extracts" (Braun & Clarke, 2006, pp. 94–95).
When evaluating the research findings, first look for the scientific research evidence. Examples of nonscientific evidence are opinions, value judgments, personal experiences, unsupported assertions, and second-hand reports.
When you are dissecting the findings of the study or what the researcher wants you to believe, how do you decide what evidence to believe? The following questions are helpful:
Researchers are susceptible to confirmation bias, meaning that there can be a tendency to find one's personal beliefs confirmed by the data. This is true of both quantitative and qualitative research. When evaluating a research article it is important to be aware of possible confirmation bias, and of your own personal beliefs, so that you can objectively decide whether the results have been reasonably presented and interpreted.
The results section must thoroughly explain how the data was collected and what statistics were used to analyze the data. The results must be described in enough detail to justify the conclusions.
Ethics in research is of particular importance. For a thorough presentation of ethics in research visit the Web site of your discipline's professional association and review their research ethics. You can find these Web sites listed on the iGuide Research at Capella: Ethical Foundations page. From this page, click the appropriate link under "Professional Associations and Organizations."
For the purpose of this session, some fundamental concerns are listed:
In the two activities in this unit, you will be using two research articles. They will be referred to as Article 1 and Article 2. Article 1 is an example of a Quantitative research study and Article 2 is an example of a Qualitative research study. In the activity, Dissecting a Quantitative Article, you will read your school's Article 1 and answer a series of questions. In the activity, Dissecting a Quantitative Article, you will read your school's Article 2 and answer another series of questions.
Please answer the questions posed in each section. If you do not know the answer, circle the Neutral answer. Each section will be followed by a discussion, at which time any remaining questions will be addressed.
When you are finished with each activity, please review your answers. If you answered Neutral to more than a couple of questions, consider your knowledge of the evaluation of research articles to be below standard for Track 2, and discuss with your faculty instructor ways to improve your understanding.
American Psychological Association. (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: Author. ISBN: 9781433805622.
Beebe, L. (1993). Professional writing for the human services. Washington, DC: NASW Press.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.
Caelli, K., Ray, L., & Mill, J. (2003). 'Clear as mud': Toward greater clarity in generic qualitative research. International Journal of Qualitative Methods, 2(2), 1–24.
Charmaz, K. (2006). Constructing ground theory: A practical guide though qualitative analysis. Thousand Oaks, CA: Sage Publications.
Craig, E. (l978). The heart of the teacher. Ann Arbor, Michigan: University Microfilms International.
Creswell, J. (2003). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: Sage Publications, Inc.
Creswell, J. W. (2007). Qualitative inquiry and research design: Choosing among five traditions (2nd ed.). Thousand Oaks, CA: Sage.
Giorgi, A. (2009). The descriptive phenomenological method in psychology: A modified Husserlian approach. Pittsburgh, PA: Duquesne University Press.
Leedy, P. D., & Ormrod, J. E. (2005). Practical research: Planning and design (8th ed.). Upper Saddle River, NJ: Prentice-Hall. ISBN 0131108956.
Meltzoff, M. (1998). Critical thinking about research. Washington, DC: American Psychological Association.
Moustakas, C. (1990). Heuristic research. Newbury Park, CA: Sage.
Moustakas, C. (1994). Phenomenological research. Thousand Oaks, CA: Sage.
Stake, R. (1995). The art of case study research. Thousand Oaks, CA: Sage.
Doc. reference: phd_t1_u05s1_h01_dissect.html