Learning From The Literature

There are two broad classes of statistical tests, which are parametric and nonparametric tests. Parametric tests involve the estimation of a parameter, require measurements on an interval scale, and involve several assumptions. Nonparametric, by contrast, do not estimate parameters and are usually applied when data has been measured on the nominal or ordinal scale (Salkind, 2014). The one-way analysis of variance [ANOVA] is a parametric test typically used to determine whether there are any significant differences between the means of three or more independent or unrelated groups but can be used with two groups. The one-way ANOVA compares the means between groups and determines whether any of those means are significantly different from each other. The one-way ANOVA is a compilation test statistic and cannot determine which specific groups were significantly different from each other, only that at least two groups were (Salkind, 2014). This paper will report how an expert researcher reports and interprets the results of data analysis in a published quantitative study from scholarly literature that utilized statistical analysis of one-way ANOVA.

Article – Reference citation:

Rezaei, A. R. (2015). Frequent collaborative quiz taking and conceptual learning. Active Learning in Higher Education, 16(3), 187-196. doi:10.1177/146978741558962

   
Research design The research design for Rezaei’s (2015) study was a quasi-experimental.
Research purpose The Rezaei (2015) study was conducted to determine:

1.      Whether there were any improvements in students’ conceptual learning when a weekly quiz is used for grading purposes compared to using midterm and final examinations only.

2.      To ascertain whether students perform better when they are allowed to work collaboratively on quizzes compared to students taking quizzes individually.

Research Questions RQ 1: It is not known whether there are differences in students’ conceptual learning when a frequent quiz is used for grading purposes compared to using midterm and final examinations only.

RQ 2: It is not known whether students’ perform better when they are allowed to work collaboratively on quizzes compared to students taking quizzes individually.

Sample description and sample size The target population included university students who took a quantitative research methods course in 12 different sections during fall 2009 to fall 2014. There were a total of 288 students.
Instrument Quizzes
Variables/Level of Measurement Conceptual learning is the independent variable. Dependent variables were no quiz, frequency quiz, and collaborative quiz all at the interval level of measurement.
Data analysis/Statistical test The one-way ANOVA was used to compare students’ learning in the three phases of teaching, no quiz, frequent quiz, and collaborative quiz. Schefft’s post hoc test result clarified there were significant differences between no quiz and frequent quiz.
Results
  1. Students performed better when the quizzes are open-book, and when they have a chance to collaborate or discuss in pairs how to answer the quiz questions.
  2. Students perform significantly higher in their final examinations and their final projects, an indicator of conceptual learning.

Article Summary

Rezaei (2015) conducted a quasi-experimental study to determine if there would be improvement in a student’s conceptual learning when a weekly quiz is used for grading purposes, compared to using midterm and final examinations only, and to ascertain whether students perform better when they are allowed to work collaboratively on quizzes compared to students taking quizzes individually. The study focused on two research questions. First, it is unknown whether there are differences in students’ conceptual learning when frequent quizzes are used for grading purposes compared to using midterm and final examines only. Second, it is unknown whether students’ perform better when they are allowed to work collaboratively on quizzes compared to students taking quizzes individually. The sample did not represent random sampling. The target population included university students who took a quantitative research methods course in 12 different sections during fall 2009 to fall 2014. There were a total of 288 students. The research instrument used was quizzes, conceptual learning was the independent variable, and the dependent variables were no quiz, frequency quiz, and collaborative quiz at the interval level of measurement. The one-way ANOVA was used to compare students’ learning in the three phases of teaching, no quiz, frequent quiz, and collaborative quiz. Schefft’s post hoc test result clarified there were significant differences between no quiz and frequent quiz.

Rezaei’s (2015) study sounded a note of warning in the discussion of limitations of the study, in that the study lacks random sampling and random assignment. A closer examination of the study revealed Rezaei (2015) failed to establish two groups, the experimental group, and control group, and therefore could not make an assumption that the experimental group and the control group are not really different after the manipulation. Failing to establish two groups is a concern of internal validity of the study. Random sampling is a probability sampling method which means it relies on the laws of probability to select a sample which can be used to make inference to the population; this is the basis of statistical tests of significance (Salkind, 2014). Random selection is thus essential to external validity or the extent to which the researcher can use the results of the study to generalize to the larger population. Furthermore, by neglecting to use random assignment, the researcher forfeited assessing causal hypothesis in which participants would be randomly assigned either to receive the treatment or intervention or to act as a control in the study in one of the two groups. Random assignment is central to internal validity, which allows the researcher to make causal claims about the effect of the treatment. Rezaei (2015) can only speculate that frequent testing improves deeper and more sustainable understanding in higher order and critical thinking. However, Rezaei (2015) could not make an inference about whether the effect would likely be found in the population.

Another concern in Rezaei’s (2015) study is that the research instruments were quizzes; however, the literature review presented by Rezaei (2015) did not support the reliability and validity of these quizzes. Biases were not identified or how any would be addressed. It is not known what the risk and benefits were to the participants who participated in the study. It was disappointing that Rezaei’s (2015) study failed to deliver more conclusive results based on sound quantitative principles in conducting a quasi-experimental study. What was interesting about Rezaei’s (2015) study was that the data analysis looked sound; however, upon closer examination of the study, reading with an understanding of inferential statistics and the importance of establishing validity and reliability in quantitative research, helped identify that there were indeed issues in the study.

Comparison in Reporting

The data analysis of Rezaei’s (2015) research study is congruent with the findings and conclusions drawn by the study’s author. Formulating a hypothesis is questionable because comparison and control groups were not identified in the study. One-way analysis of variance [ANOVA] was used to compare students’ learning in the three phases of teaching to their final performance, on the final exam, and final project. The result showed differences in the means between no quiz (M = 83.12), frequent quiz (M = 86. 92), and collaborative quiz (M = 90.21) results. Students’ final performance in the third group showed significantly higher performance on their final performance in comparison with the first and the second group. A statistically significant F value is evidenced by looking at the p-level that is zero, which is less than 0.05 (p < 0.05), which would mean the null hypothesis was rejected.

Schefft’s post hoc comparison was run due to the overall significant difference in group means and also to confirm where the differences occurred between groups. Looking at table 3, group 1 and group 2 indicate the mean difference is significant at the 0.05 level between no quiz (frequent quiz M = -3.80; collaborative M = -7.09) and frequent quiz (no quiz M = 3.80; collaborative M = 7.09).

Scientific Contribution

Overall, Rezaei’s (2015) study is not sound, and would not be used as evidence to support an argument; however, a primary scientific contribution to the body of knowledge that informs the discipline of education is highlighting that there could be benefits of frequent testing and collaborative quiz as assessment strategies. Frequent testing and collaborative quiz taking are innovative evidence-based assessment strategies that align with the new shift in higher education moving from a focus on teaching to a focus on learning.

References

Rezaei, A. R. (2015). Frequent collaborative quiz taking and conceptual learning. Active Learning in Higher Education, 16(3), 187-196. doi:10.1177/1469787415589627

Salkind, N. J. (2014). Statistics for people who (think they) hate statistics (5th ed.). Thousand Oaks, CA: SAGE Publications.

Leave a Reply