Document Type : Original Article
Authors
- . Ali Kazemi Karyan
- . Satar Rezaei
- . Shokooh Etesami 1
- . Leyla Pezhman 1
- . Behzad Karami Matin
- . Sajad Delavari 2
1 Students Researches Committee, Kermanshah University of Medical Sciences, Kermanshah, Iran,
2 Heath Human Resources Research Center, School of Management and Information Sciences, Shiraz University of Medical Sciences, Shiraz, Iran
Abstract
BACKGROUND: Evaluation has become an inseparable part of education process which gives
feedback to students and professors to improve education quality. This study aimed to elicit
preferences of professors and students about attributes of evaluation methods in theoretical courses
in Kermanshah University of Medical Sciences, Iran, in 2018.
MATERIALS AND METHODS: Discrete choice experiment (DCE) method used for eliciting preferences
of participants of the study. A narrative literature review and interview with eight professors and ten
students conducted to determine attributes and levels of evaluation methods in the university. Furthermore,
experimental design used for making final choice sets of the evaluation methods. We included 213
students and 30 professors in the study. Conditional logistic regression model performed to data analysis.
RESULTS: Most of the professors (36.67%) preferred to allocate up to 30% of evolution scores to
midterm examination. However, the most percentage of students (30.45%) were agree to include
midterm examination up to 15% of total scores. The majority of students prefer to examination
questions compromise just presented materials, while 70% of professors prefer to include additional
texts for evaluation examinations. In case of quiz examination, professors in comparison with students
prefer that quiz should have higher proportion of total scores. DCE analysis indicated that professors
and students preferred a mix of questions in examinations. In addition, additional resources beyond
what is taught in class made utility for professors and disutility for students. Quiz, also, increased
the utility of an evaluation package in professors.
CONCLUSION: The findings showed that there is a gap between preferences of professors and
students regarding some attributes of evaluation methods such as student’s discipline, examination
materials, and quiz. Further studies are needed to examining other attributes of evaluation methods
in theatrical and practical courses in Iran and other contexts.
Keywords
Sichuan Province: Problems and Solutions; 2013. p. 3663‑6.
2. Chen X. Harmonizing ecological sustainability and higher
education development: Wisdom from Chinese ancient education
philosophy. Educ Philos Theor 2019;51:1080‑90.
3. Guo L, Huang J, Zhang Y. Education development in China:
Education return, quality, and equity. Sustainability 2019;11: 13,
10.3390/su11133750.
4. Shin JC. Higher education development in Korea: Western
university ideas, Confucian tradition, and economic development.
High Educ 2012;64:59‑72.
5. McGrath S. Education and development: Thirty years of continuity
and change. Int J Educ Dev 2010;30:537‑43.
6. Darling‑Hammond L, Flook L, Cook‑Harvey C, Barron B,
Osher DJ. Implications for ducational practice of the science of
learning and development. Applied Developmental Sciences,
2020;24:97‑140. https://doi.org/10.1080/10888691.2018.153779
1.
7. Zhao C, Zhao Y, Luo L, Li Y. Fractional modeling method research
on education evaluation. J Softw 2011;6:901‑7.
8. Zhao P. Study on Higher Education Quality Evaluation System
of Local Universities; 2013. p. 2113.
9. Lewallen LP. Practical strategies for nursing education program
evaluation. J Prof Nurs 2015;31:133‑40.
10. Botek M. Comparison of education evaluation models. Qual
Access Success 2018;19:63‑7.
11. Pearson T, Garrett L, Hossler S, McConnell P, Walls J.
A progressive nurse practitioner student evaluation tool. J Am
Acad Nurse Pract 2012;24:352‑7.
12. Sun X, Fang X. Construction and operation of analytic hierarchy
process about moral education evaluation in colleges and
universities. Adv Inf Sci Serv Sci 2011;3: DOI: 10.4156/aiss.vol3.
issue11.15.
13. Wang K, editor. Ideological and Political Education Evaluation of
Universities by a Novel AHP. 2014 Sixth International Conference
on Measuring Technology and Mechatronics Automation,
Zhangjiajie, 2014, pp. 309‑312, doi: 10.1109/ICMTMA.2014.77.
14. Bao Q, Gu X, Shen Y, editors. An Improved Neural Network
Model for Graduate Education Evaluation. 2007 International Conference on Computational Intelligence and Security (CIS
2007), Harbin, 2007, pp. 550‑554, doi: 10.1109/CIS.2007.190.
15. Xu J, Zhang G, editors. The Construction of Enterpreneurship
Education Evaluation Index System Based on Dynamic Fuzzy
Logic Theory. 2012 DOI: 10.1109/ISRA.2012.6219149.
16. Cheng Y, Wang Y. Design of Network Education Evaluation
System Based on Data Mining Technology. Applied Mechanics
and Materials 241‑244:2849‑2852و DOI: 10.4028/www.scientific.
net/AMM.241‑244.2849.
17. Shams A, Yarmohammadian MH, Abbarik HH. Determination
of rate of customer focus in educational programs at Isfahan
University of Medical Sciences(1) based on students’ viewpoints.
J Educ Health Promot 2012;1:24.
18. Danaei SM, Mazareie E, Hosseininezhad S, Nili M. Evaluating the
clinical quality of departments as viewed by juniors and seniors
of Shiraz dental school. J Educ Health Promot 2015;4:75.
19. Kennelly B, Flannery D, Considine J, Doherty E, Hynes S.
Modelling the preferences of students for alternative assignment
designs using the discrete choice experiment methodology. Pract
Assess Res Eval 2014;19:1‑13.
20. Cunningham CE, Deal K, NevilleA, Rimas H, Lohfeld L. Modeling
the problem‑based learning preferences of McMaster University
undergraduate medical students using a discrete choice conjoint
experiment. Adv Health Sci Educ Theory Pract 2006;11:245‑66.
21. Sadeghi T, Loripoor M. Usefulness of 360 degree evaluation
in evaluating nursing students in Iran. Korean J Med Educ
2016;28:195‑200.
22. Abbasi S, Einollahi N, Gharib M, Nabatchian F, Dashti N,
Zarebavani MJ. Evaluation methods of theoretical and practical
courses of paramedical faculty laboratory sciences undergraduate
students at tehran university of medical sciences in the academic
Year 2009‑2010. Payavard 2013;6:342‑53.
23. Bastani P, Amini M, Taher Nejad K. Shaarbafchi Zadeh NJJoAiME,
professionalism. Faculty members’ viewpoints about the present
and the ideal teacher evaluation system in Tehran. Univ Med Sci
2013;1:140‑7.
24. Sarabi‑Asiabar A, Jafari M, Sadeghifar J, Tofighi S, Zaboli R,
Peyman H, et al. The relationship between learning style
preferences and gender, educational major and status in first year
medical students: A survey study from iran. Iran Red Crescent
Med J 2015;17:e18250.
25. Ryan M, Kolstad JR, Rockers PC, Dolea C. How to Conduct a
Discrete Choice Experiment for Health Workforce Recruitment
and Retention in Remote and Rural Areas: A User Guide with
Case Studies. The World Bank; 2012.
26. Karyani AK, Rashidian A, Sari AA, Sefiddashti SEJMjotIRoI.
Developing attributes and levels for a discrete choice experiment
on basic health insurance in Iran. Med J Islam Repub Iran
2018;32:26. doi: 10.14196/mjiri.32.26.
27. Johnson FR, Lancsar E, Marshall D, Kilambi V ,
Mühlbacher A, Regier DA, et al. Constructing experimental
designs for discrete‑choice experiments: Report of the ISPOR
conjoint analysis experimental design good research practices
task force. Value Health 2013;16:3‑13.
28. Hauber AB, González JM, Groothuis‑Oudshoorn CG, Prior T,
Marshall DA, Cunningham C, et al. Statistical methods for the
analysis of discrete choice experiments: A report of the ISPOR
conjoint analysis good research practices task force. Value Health
2016;19:300‑15.
29. Cushing B. Conditional logit, IIA, and Alternatives for Estimating
Models of Interstate Migration; 2007. Available from: https://
researchrepository.wvu.edu/rri_pubs/65.[Last accessed on 2020
Jun 24].
30. Orme BK. Getting started with conjoint analysis: Strategies for
product design and pricing research: Research Publishers, LLC;
2006.
31. Shigli K, Nayak SS, Gali S, Sankeshwari B, Fulari D,
Shyam Kishore K, et al. Are multiple choice questions for post
graduate dental entrance examinations spot on?‑Item analysis of
MCQs in prosthodontics in India. J Natl Med Assoc 2018;110:455‑8.
32. JusticeP, MarshmanE, Singh C. Improving student understanding
of quantum mechanics underlying the Stern‑Gerlach experiment
using a research‑validated multiple‑choice question sequence. Eur
J Phys 2019;40:1‑15. DOI: 10.1088/1361‑6404/ab2135.
33. Mkrtchyan A, Abrahamyan A, editors. Combined Multiple Choice
Questions: An objective and efficient assessment of knowledge.
Proceedings of the IADIS International Conference e‑Learning
2011, Part of the IADIS Multi Conference on Computer Science
and Information Systems 2011, MCCSIS 2011; 2011.
34. Villafranca JJ, Ash DE, Wedler FC. Manganese(II) and substrate
interaction with unadenylylated glutamine synthetase (Escherichia
coli w). II. Electron paramagnetic resonance and nuclear magnetic
resonance studies of enzyme‑bound manganese(II) with
substrates and a potential transition‑state analogue, methionine
sulfoximine. Biochemistry 1976;15:544‑53.
35. Chen M, editor. Detect Multiple Choice Exam Cheating
Pattern by Applying Multivariate Statistics. Proceedings of
the International Conference on Industrial Engineering and
Operations Management, Bogota, Colombia, October 25‑26, 2017.
36. McLeod AI, Zhang Y, Yu H. Multiple‑choice randomization. J Stat
Educ 2003;11:https://doi.org/10.1080/10691898.2003.11910695.
37. Tang JK, Yeung HC, Man YF, Li KK, Wong TL, Pang WM,
et al. A Web‑Based Computer‑Aided Assessment Creation and
Invigilation System; 2015. p. 74‑85.
38. Wang JY, Chen FG, Tai DW, Chen JS, editors. An Automatic Test
Generator for Enhancing the Technolgy Profciency of Senior
High School Students. 2016, 5th IIAI International Congress on
Advanced Applied Informatics (IIAI‑AAI).
39. Kojury J, Rivaz S, Amini M, Rivaz MJBJoMEEDCBUoMS.
Assessment of educational group’s status based on types of
evaluation methods of medical students at the Shiraz University
of Medical Sciences 2014. Journal of Medical Development
Center (EDC), 2017;5:7‑13.
40. Tangianu F, Mazzone A, Berti F, Pinna G, Bortolotti I, Colombo F,
et al. Are multiple‑choice questions a good tool for the assessment
of clinical competence in Internal Medicine? Ital J Med
2018;12:88‑96.
41. Pearce G, Lee G. Viva voce (oral examination) as an assessment
method: Insights from marketing students. J Marketing Educ
2009;31:120‑30.
42. Lakhal S, Sévigny S, Frenette É. Personality and preference for
evaluation methods: A study among business administration
students. Stud Educ Evaluat 2013;39:103‑15.
43. Sheard M. Hardiness commitment, gender, and age differentiate
university academic performance. Br J Educ Psychol
2009;79:189‑204.
44. Abolfaz Khademi. Preference for students to learn English.
Daneshvar Raftar 2003;11:29‑42.
45. Aragon SR. Facilitating Learning in Online Environments: New
Directions for Adult and Continuing Education, Number 100:
John Wiley & Sons; 2010.
)