Reliability assessment of a peer evaluation instrument in a team-based learning course

  • Joy Wahawisan
  • Miguel Salazar
  • Robin Walters
  • Fadi M. Alkhateeb
  • Omar F. Attarabeen
Keywords: Students, Pharmacy, Education, Educational Measurement, Curriculum, Cooperative Behavior, United States

Abstract

Objective: To evaluate the reliability of a peer evaluation instrument in a longitudinal team-based learning setting.

Methods: Student pharmacists were instructed to evaluate the contributions of their peers. Evaluations were analyzed for the variance of the scores by identifying low, medium, and high scores. Agreement between performance ratings within each group of students was assessed via intra-class correlation coefficient (ICC).

Results: We found little variation in the standard deviation (SD) based on the score means among the high, medium, and low scores within each group. The lack of variation in SD of results between groups suggests that the peer evaluation instrument produces precise results. The ICC showed strong concordance among raters.

Conclusions: Findings suggest that our student peer evaluation instrument provides a reliable method for peer assessment in team-based learning settings.

Downloads

Download data is not yet available.

References

1. Garland D. Assessment issues in group work. In Foot HC, Howe CJ, Anderson A, Tolmie A, Warden D, Eds. Group and Interactive Learning. Southampton: Computational Mechanics Publications;1994. p.417-422.

2. Allen RE, Copeland J, Franks AS, Karimi R, McCollum M, Riese DJ 2nd, Lin AY. Team-based learning in US colleges and schools of pharmacy. Am J Pharm Educ. 2013;77(6):115. doi: 10.5688/ajpe776115

3. Ofstad W, Brunner LJ. Team-based learning in pharmacy education. Am J Pharm Educ. 2013;77(4):70. doi: 10.5688/ajpe77470

4. Chung EK, Rhee JA, Baik YH, A OS. The effect of team-based learning in medical ethics education. Med Teach. 2009;31(11):1013-1017. doi: 10.3109/01421590802590553

5. Wiener H, Plass H, Marz R. Team-based learning intensive course format for first-year medical students. Croat Med J. 2009;50(1):69-76.

6. Vasan NS, DeFouw DO, Compton S. A survey of student perceptions of team-based learning in anatomy curriculum: Favorable views unrelated to grades. Anat Sci Educ. 2009;2(4):150-155. doi: 10.1002/ase.91

7. Koles PG, Stolfi A, Borges NJ, Nelson S, Parmelee DX. The impact of team-based learning on medical students’ academic performance. Acad Med. 2010;85(11):1739-1745. doi: 10.1097/ACM.0b013e3181f52bed

8. Redwanski J. Incorporating team-based learning in a drug information course covering tertiary literary. Curr Pharm Teach Learn. 2012;4(3):202-206. doi: 10.1016/j.cptl.2012.04.004

9. Carmichael J. Team-based learning enhances performance in introductory biology. J Coll Sci Teach. 2009;38(4):54-61.

10. Fatmi M, Hartling L, Hillier T, Campbell S, Oswald AE. The effectiveness of team-based learning on learning outcomes in health professions education: BEME Guide No. 30. Med Teach. 2013;35(12):e1608-e1624. doi: 10.3109/0142159X.2013.849802

11. Accreditation Council for Pharmacy Education. Accreditation standards and guidelines for the professional program in pharmacy leading to the Doctor of Pharmacy Degree. Guidelines Version 2.0. 2011. Available at: https://www.acpe-accredit.org/pdf/CPE_Standards_Final.pdf (accessed April 1, 2015).

12. Accreditation Council for Pharmacy Education. Accreditation standards and key elements for the professional program in pharmacy leading to the doctor of pharmacy degree, standards 2016. Available at: https://www.acpe-accredit.org/pdf/Standards2016FINAL.pdf (accessed Jan 14, 2016).

13. Nelson M, Allison SD, McCollum M. Luckey SW, Clark DR, Paulsen SM, Malhotra J, Brunner LJ. The Regis Model for pharmacy education: A highly integrated curriculum delivered by Team-Based Learning (TBL). Curr Pharm Teach Learn. 2013;5(6):555-563. doi:10.1016/j.cptl.2013.07.002

14. Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standard assessment instruments in psychology. Psychol Assess. 1994;6(4):284-290.

15. Farland MZ, Sicat BL, Franks AS, Pater KS, Medina MS, Persky AM. Best practices for implementing team-based learning in pharmacy education. Am J Pharm Educ. 2013;77(8):177. doi: 10.5688/ajpe778177

16. Wahawisan J, Salazar M, Bremick MA. Assessment of a Peer Evaluation Instrument in a Team-Based Learning Course. 115th Annual Meeting of the American Associaton of Colleges of Pharmacy, Grapevine, TX, July 26-30, 2014. Available at: http://www.ajpe.org/doi/pdf/10.5688/ajpe785111 (accessed Jan 14, 2016).
Published
2016-03-12
Section
Original Research