The effect of polytomous item ratio on ability estimation in multistage tests

Hasibe Yahsi Sari; Hülya Kelecioğlu

Makale detayları

The effect of polytomous item ratio on ability estimation in multistage tests

Yazarlar:

Hasibe Yahsi Sari | Hülya Kelecioğlu

Yayıncı:

İzzet KARA

Ana Başlıklar:

Computer Based Exam Applications

Dergi:

International Journal of Assessment Tools in Education Vol.12 Issue 2, pp.414-429

Belge Tipi:

Makale

Yayın Tarihi:

1/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001341204

pISSN:

2148-7456

Anahtar kelimeler:

Özet:

 The aim of the study is to examine the effect of polytomous item ratio on ability estimation in different conditions in multistage tests MST using mixed tests. The study is simulation-based research. In the PISA 2018 application, the ability parameters of the individuals and the item pool were created by using the item parameters estimated from the dichotomous and polytomous items obtained in the field of reading skills. MST conditions; panel design, test lengths, routing methods, and polytomous item ratio. Simulation data, MST pattern and analysis were obtained with the help of WinGen, CPLEX, and the mstR package in the R Studio program. A total of 108 conditions and 100 replications were examined in the study. As a result of the simulations, RMSE, mean absolute bias and correlation values were calculated. As a result of the research, it is seen that when the ratio of polytomous items in the tests increases from 10 to 50, the mean absolute bias and RMSE values decrease while the correlation values increase. As the test length increases, RMSE and mean absolute bias values decrease while correlation values increase. In terms of routing methods, MFI performed better than the NC routing method. In general, three-stage panel designs gave significantly better results than two-stage panel designs. In 1-2 and 1-4 panel designs, it does not matter which routing method is used.

 The aim of the study is to examine the effect of polytomous item ratio on ability estimation in different conditions in multistage tests MST using mixed tests. The study is simulation-based research. In the PISA 2018 application, the ability parameters of the individuals and the item pool were created by using the item parameters estimated from the dichotomous and polytomous items obtained in the field of reading skills. MST conditions; panel design, test lengths, routing methods, and polytomous item ratio. Simulation data, MST pattern and analysis were obtained with the help of WinGen, CPLEX, and the mstR package in the R Studio program. A total of 108 conditions and 100 replications were examined in the study. As a result of the simulations, RMSE, mean absolute bias and correlation values were calculated. As a result of the research, it is seen that when the ratio of polytomous items in the tests increases from 10 to 50, the mean absolute bias and RMSE values decrease while the correlation values increase. As the test length increases, RMSE and mean absolute bias values decrease while correlation values increase. In terms of routing methods, MFI performed better than the NC routing method. In general, three-stage panel designs gave significantly better results than two-stage panel designs. In 1-2 and 1-4 panel designs, it does not matter which routing method is used.

 The aim of the study is to examine the effect of polytomous item ratio on ability estimation in different conditions in multistage tests MST using mixed tests. The study is simulation-based research. In the PISA 2018 application, the ability parameters of the individuals and the item pool were created by using the item parameters estimated from the dichotomous and polytomous items obtained in the field of reading skills. MST conditions; panel design, test lengths, routing methods, and polytomous item ratio. Simulation data, MST pattern and analysis were obtained with the help of WinGen, CPLEX, and the mstR package in the R Studio program. A total of 108 conditions and 100 replications were examined in the study. As a result of the simulations, RMSE, mean absolute bias and correlation values were calculated. As a result of the research, it is seen that when the ratio of polytomous items in the tests increases from 10 to 50, the mean absolute bias and RMSE values decrease while the correlation values increase. As the test length increases, RMSE and mean absolute bias values decrease while correlation values increase. In terms of routing methods, MFI performed better than the NC routing method. In general, three-stage panel designs gave significantly better results than two-stage panel designs. In 1-2 and 1-4 panel designs, it does not matter which routing method is used.

 The aim of the study is to examine the effect of polytomous item ratio on ability estimation in different conditions in multistage tests MST using mixed tests. The study is simulation-based research. In the PISA 2018 application, the ability parameters of the individuals and the item pool were created by using the item parameters estimated from the dichotomous and polytomous items obtained in the field of reading skills. MST conditions; panel design, test lengths, routing methods, and polytomous item ratio. Simulation data, MST pattern and analysis were obtained with the help of WinGen, CPLEX, and the mstR package in the R Studio program. A total of 108 conditions and 100 replications were examined in the study. As a result of the simulations, RMSE, mean absolute bias and correlation values were calculated. As a result of the research, it is seen that when the ratio of polytomous items in the tests increases from 10 to 50, the mean absolute bias and RMSE values decrease while the correlation values increase. As the test length increases, RMSE and mean absolute bias values decrease while correlation values increase. In terms of routing methods, MFI performed better than the NC routing method. In general, three-stage panel designs gave significantly better results than two-stage panel designs. In 1-2 and 1-4 panel designs, it does not matter which routing method is used.

Makale detayları

The effect of polytomous item ratio on ability estimation in multistage tests

Yazarlar:

Hasibe Yahsi Sari | Hülya Kelecioğlu

Yayıncı:

İzzet KARA

Ana Başlıklar:

Computer Based Exam Applications

Dergi:

International Journal of Assessment Tools in Education Vol.12 Issue 2, pp.414-429

Belge Tipi:

Makale

Yayın Tarihi:

1/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001341204

pISSN:

2148-7456

Multistage testing

Polytomous item

Mixed test

Ability estimation

Pisa 2018

Sayı

Makale detayları

The effect of polytomous item ratio on ability estimation in multistage tests

Yazarlar: Hasibe Yahsi Sari | Hülya Kelecioğlu

Yayıncı: İzzet KARA

Ana Başlıklar: Computer Based Exam Applications

Dergi: International Journal of Assessment Tools in Education Vol.12 Issue 2, pp.414-429

Belge Tipi: Makale

Yayın Tarihi: 1/2025

Belge Dili: İngilizce

Document ID (Org No.): 2001341204

pISSN: 2148-7456

Multistage testing

Polytomous item

Mixed test

Ability estimation

Pisa 2018

Sayı

Yazarlar:

Hasibe Yahsi Sari | Hülya Kelecioğlu

Yayıncı:

İzzet KARA

Ana Başlıklar:

Computer Based Exam Applications

Dergi:

International Journal of Assessment Tools in Education Vol.12 Issue 2, pp.414-429

Belge Tipi:

Makale

Yayın Tarihi:

1/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001341204

pISSN:

2148-7456