Journal of Education and Health Promotion

Abstract BACKGROUND: Kane’s validity framework examines the validity of the interpretation of a test at the
four levels of scoring, generalization, extrapolation, and implications. No model has been yet proposed
to use this framework particularly for a system of assessment. This study provided a model for the
validation of the internal medicine residents’ assessment system, based on the Kane’s framework.
MATERIALS AND METHODS: Through a five stages study, first, by reviewing the literature, the
methods used, and the study challenges, in using Kane’s framework, were extracted. Then, possible
assumptions about the design and implementation of residents’ tests and the proposed methods for
their validation at each of their four inferences of Kane’s validity were made in the form of two tables.
Subsequently, in a focus group session, the assumptions and proposed validation methods were
reviewed. In the fourth stage, the opinions of seven internal medicine professors were asked about
the results of the focus group. Finally, the assumptions and the final validation model were prepared.
RESULTS: The proposed tables were modified in the focus group. The validation table was developed
consisting of tests, used at each Miller’s pyramid level. The results were approved by five professors
of the internal medicine. The final table has five rows, respectively, as the levels of Knows and Knows
How, Shows How, Shows, Does, and the fifth one for the final scores of residents. The columns
of the table demonstrate the necessary measures for validation at the four levels of inferences of
Kane’s framework.
CONCLUSION: The proposed model ensures the validity of the internal medicine specialty residency
assessment system based on Kane’s framework, especially at the implication level.