Investigating Musical Aptitude Examination with a Many-Facet Rasch Model

Year 2021, Volume: 8 Issue: 4, 334 - 345, 01.12.2021

Neşe Öztürk Gübeş

https://doi.org/10.17275/per.21.93.8.4

Cited By: 1

Abstract

The aim of this study is to show how a many-facet Rasch measurement model (MFRM) can be used for quality control whilst monitoring a musical aptitude examination. The data used in this study was gathered from a musical aptitude examination which was applied in 2019-2020 academic year for selecting teacher candidates to a music education department in one public university in Turkey. In this study, the total scores of musical singing and playing exams were used. The study group of this research is consisted of 164 candidates and five specialists who rated the musical performance of candidates. A three-facet Rasch model was used including student (n=164), rater (n=5), and task (n=2). Data was gathered with fully crossed design. MFRM analysis showed good fit the data. The reliability of separation index for students was very high and it indicated that the musical aptitude examination differentiate among students in terms of their musical performance. The reliability of the rater separation index was found as 0.00 and it suggested that raters rated students’ musical performance with very similar levels of severity/leniency and they were interchangeable. The results of task measure showed that musical singing task is harder than musical playing task. The results of bias analyses showed that there is no bias based on rater by task and rater by student interactions. However, student by task interaction has some bias measures.

Keywords

Many-facet Rasch Model, rater reliability, musical aptitude examination

References

• Akın, Ö. & Baştürk, R. (2012). The evaluation of the basic skills in violin training by many facet Rasch model. Pamukkale University Journal of Education, 31(1), 175-187.
• Atak Yayla, A. (2004, April). Müziksel performansın ölçülmesi [Measuring musical performance]. Paper presented at the 1924-2004 Musiki Muallim Mektebinden Günümüze Müzik Öğretmeni Yetiştirme Sempozyumu [The Symposium of Training Music Teachers from Music Teaching School to the Present Day], Isparta.
• Atılgan, H. (2005). Analysis of special ability selection examination for music education department using many-facet Rasch measurement (İnönü University Case). Eurasian Journal of Educational Research, 20, 62-73.
• Baker, F. B. (2001). The basics of item response theory (2nd edition). College Park: ERIC Clearinghouse on Assessment and Evaluation.
• Barkaoui, K. (2013). Multi-facet Rasch analysis for test evaluation. In Kunnan, A. J. (Ed.), The companion to language assessment (pp. 60-4-1079). US: John Wiley & Sons.
• Birel, A. S. & Albuz, A. (2014). Evaluation and test of graded scoring key (rubric) prepared for performance assessment in teaching violoncello. Atatürk University Journal of Social Sciences Institute, 18(3), 207-281.
• Bond, T. G., & Fox, C. M. (2001). Applying the Rasch model: Fundamental measurement in the human sciences. Lawrence Erlbaum Associates Publishers.
• Dalkıran, E. (2008). Measurement of performance in violin education. Journal of Yüzüncü Yıl University Education Faculty, 5(2), 116-136.
• de Ayala, R. J. (2009). The theory and practice of item response theory. New York: The Guilford.
• Dineen, M. (2015). Re: Which requires more skill in music, singing or playing an instrument [Web log comment]. Retrieved from https://thesession.org/discussions/35070.
• Ece, A. S. & Kaplan, S. (2008). Müzik özel yetenek seçme sınavının puanlayıcılar arası güvenirlik çalışması [The study of inter rater reliability of music special ability examination]. Milli Eğitim [National Education], 177, 36-49.
• Eckes, T. (2009). Many-facet Rasch measurement. In S. Takala (Ed.), Reference supplement to the manual for relating language examinations to the Common European Framework of Reference for Languages: Learning, teaching, assessment (Section H). Strasbourg, France: Council of Europe/Language Policy Division.
• Engelhard, G. (1994). Examining rater errors in the assessment of written composition with a Many-Faceted Rasch Model. Journal of Educational Measurement, 31, 93-112.
• Engelhard, G. & Myford, C. M. (2003). Monitoring faculty consultant performance in the advanced placement English literature and composition program with a many-faceted Rasch model (College Board Research Report No. 2003-1 ETS RR-03-01).
• Engur, A., Çeliktaş, H., & Demirbatir, R. E. (2015). A study on testing reliability of 2013 musical aptitude test scores conducted by music department in Uludağ University. Procedia Socail and Behavioral Sciences, 197, 821-825.
• Fan, X. (1998). Item response theory and classical test theory: An emprical comparison of their item/person statistics. Educational and Psychological Measurement, 58(3), 357-381.
• Girgin, D. (2020). An investigation of the songs created by student-teachers in music via an interdisciplinary approach based on the RASCH measurement model and MAXQDA analysis program. International Online Journal of Education and Teaching (IOJET), 7(4). 1661-1687.
• Gün, E. & Demirtaş, H. O. (2015). International Journal of Social Science, 40, 157-165.
• Haiyang, S. (2010). An application of classical test theory and many-facet Rasch measurement in analyzing the reliability of an English test for non-English majör graduates. Chinese Journal of Applied Linguistics, 33(2), 87-102.
• Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Sage Publications, Inc.
• Köse, İ. A., Acay Sözbir, S. & Kalender, C. (2016). Examination of the violin playing skills by means of Rasch model. Journal of Abant İzzet Baysal University Faculty of Education, 16 (İpekyolu Özel Sayısı), 2339-2349.
• Kurtuldu, M. K. (2010).Validity and reliability of evaluation scale in piano education directed for the grades. Electronic Journal of Social Sciences, 9(31), 224-232.
• Linacre, J. M. (2004). Rasch model estimation: Further topics. Journal of Applied Measurement, 5(1), 95–110.
• Linacre, J. M. (2020). A user’s guide to FACETS Rasch-Model computer programs. Available online www.winsteps.com.
• Myford, C. M., &Wolfe, E.W. (2003). Detecting and measuring rater effects using many-facet Rasch measurement: Part I. Journal of Applied Measurement, 4(4), 386–422.
• Myfold, C. M. & Wolfe, E. W. 2004. Detecting and measuring rater effects using many-facet Rasch measurement: Part II. Journal of Applied Measurement, 5, 189-227.
• Öztürk, D. & Güdek, B. (2016).An assay concerning improvinh the graded scoring method (rubric) for rating the violoncello performance. Journal of Academic Music Research, 2(3), 1-20.
• Thompson, S., Williamon, A., & Valentine, E. (2007). Time-dependent characteristic of performance evaluation. Music Perception: An interdiciplinary Journal, 25(1), 13-29.
• Wesolowski, B. C., Wind, S. A., & Engelhard, G. (2016). Rater analyses in music performance assessement: Application of the many facet Rasch model. Paper presented at the 5th International Symposium on assessment in music education, Williamsburg, VA.
• Weir, C. 2005. Language Testing and Validation: An Evidence-based Approach. Basingstoke: Palgrave Macmillan.
• Wolfe, E. W., & Dobria, L. (2008). Applications of the multifaceted Rasch model. In J. W. Osborne (Ed.), Best practices in quantitative methods (pp. 71–85). Los Angeles: Sage.