Automated Essay Scoring Effect on Test Equating Errors in Mixed-format Test

İbrahim UYSAL; Nuri DOĞAN

doi:https://doi.org/10.21449/ijate.815961

Makale detayları

Automated Essay Scoring Effect on Test Equating Errors in Mixed-format Test

Yazarlar:

İbrahim UYSAL | Nuri DOĞAN

Yayıncı:

İzzet KARA

Ana Başlıklar:

Eğitim

Dergi:

International Journal of Assessment Tools in Education 2021 Vol.8 Issue 2, pp.222-238

Belge Tipi:

Makale

Yayın Tarihi:

6/2021

Belge Dili:

İngilizce

Document ID (Org No.):

2000718575

Sıklık:

Üç Aylık

eISSN:

2148-7456

DOI:

https://doi.org/10.21449/ijate.815961

Özet:

 


Scoring constructed-response items can be highly difficult, time-consuming, and costly in practice. Improvements in computer technology have enabled automated scoring of constructed-response items. However, the application of automated scoring without an investigation of test equating can lead to serious problems. The goal of this study was to score the constructed-response items in mixed-format tests automatically with different test/training data rates and to investigate the indirect effect of these scores on test equating compared with human raters. Bidirectional long-short term memory (BLSTM) was selected as the automated scoring method for the best performance. During the test equating process, methods based on classical test theory and item response theory were utilized. In most of the equating methods, errors of the equating resulting from automated scoring were close to the errors occurring in equating processes conducted by human raters. It was concluded that automated scoring can be applied because it is convenient in terms of equating.

.

 


Scoring constructed-response items can be highly difficult, time-consuming, and costly in practice. Improvements in computer technology have enabled automated scoring of constructed-response items. However, the application of automated scoring without an investigation of test equating can lead to serious problems. The goal of this study was to score the constructed-response items in mixed-format tests automatically with different test/training data rates and to investigate the indirect effect of these scores on test equating compared with human raters. Bidirectional long-short term memory (BLSTM) was selected as the automated scoring method for the best performance. During the test equating process, methods based on classical test theory and item response theory were utilized. In most of the equating methods, errors of the equating resulting from automated scoring were close to the errors occurring in equating processes conducted by human raters. It was concluded that automated scoring can be applied because it is convenient in terms of equating.

.

Makale detayları

Automated Essay Scoring Effect on Test Equating Errors in Mixed-format Test

Yazarlar: İbrahim UYSAL | Nuri DOĞAN

Yayıncı: İzzet KARA

Ana Başlıklar: Eğitim

Dergi: International Journal of Assessment Tools in Education 2021 Vol.8 Issue 2, pp.222-238

Belge Tipi: Makale

Yayın Tarihi: 6/2021

Belge Dili: İngilizce

Document ID (Org No.): 2000718575

Sıklık: Üç Aylık

eISSN: 2148-7456

DOI: https://doi.org/10.21449/ijate.815961

Sayı

Yazarlar:

İbrahim UYSAL | Nuri DOĞAN

Yayıncı:

İzzet KARA

Ana Başlıklar:

Eğitim

Dergi:

International Journal of Assessment Tools in Education 2021 Vol.8 Issue 2, pp.222-238

Belge Tipi:

Makale

Yayın Tarihi:

6/2021

Belge Dili:

İngilizce

Document ID (Org No.):

2000718575

Sıklık:

Üç Aylık

eISSN:

2148-7456

DOI:

https://doi.org/10.21449/ijate.815961