The Effect of Rubric Type On Raters’ Behaviors in Scoring Non-Routine Mathematics Problems

Esra Onkun Özgür; İsmail Karakaya

Makale detayları

The Effect of Rubric Type On Raters’ Behaviors in Scoring Non-Routine Mathematics Problems

Yazarlar:

Esra Onkun Özgür | İsmail Karakaya

Yayıncı:

Bursa Uludağ University

Ana Başlıklar:

Eğitimde Ölçme ve Değerlendirme (Diğer)

Dergi:

Uludağ Üniversitesi Eğitim Fakültesi Dergisi Vol.37 Issue 3, pp.1037-1068

Belge Tipi:

Makale

Yayın Tarihi:

12/2024

Belge Dili:

Türkçe

Document ID (Org No.):

2001329654

pISSN:

2667-6788

Anahtar kelimeler:

Özet:

 In this study, the effects of scoring the mathematics achievement test, which consists of non-routine mathematics problems, with analytical and holistic rubrics on rater behavior were examined with the many-facet Rasch measurement model. Survey model, one of the descriptive research methods, was used. An achievement test consisting of 15 different non-routine mathematics problems prepared by the researcher was administered to the students in two different sessions in two days. According to research, it was seen that model data fit was achieved on the rater, individual and item surfaces in all scoring. Additionally, it was observed that individuals were differentiated according to their ability levels and the difficulty levels of the items were different. Scoring with analytical rubric gives relatively more reliable results in terms of rater surface than scoring with holistic rubric; when the rater surfaces were compared, it was determined that the levels of rater severityleniency in scoring using analytical rubric were higher than in scoring using holistic rubric. When rater behaviors were examined, rater strictness and generosity behaviors were higher in raters using holistic rubrics than raters using analytical rubrics; It was determined that bias behaviors were less.

 In this study, the effects of scoring the mathematics achievement test, which consists of non-routine mathematics problems, with analytical and holistic rubrics on rater behavior were examined with the many-facet Rasch measurement model. Survey model, one of the descriptive research methods, was used. An achievement test consisting of 15 different non-routine mathematics problems prepared by the researcher was administered to the students in two different sessions in two days. According to research, it was seen that model data fit was achieved on the rater, individual and item surfaces in all scoring. Additionally, it was observed that individuals were differentiated according to their ability levels and the difficulty levels of the items were different. Scoring with analytical rubric gives relatively more reliable results in terms of rater surface than scoring with holistic rubric; when the rater surfaces were compared, it was determined that the levels of rater severityleniency in scoring using analytical rubric were higher than in scoring using holistic rubric. When rater behaviors were examined, rater strictness and generosity behaviors were higher in raters using holistic rubrics than raters using analytical rubrics; It was determined that bias behaviors were less.

 In this study, the effects of scoring the mathematics achievement test, which consists of non-routine mathematics problems, with analytical and holistic rubrics on rater behavior were examined with the many-facet Rasch measurement model. Survey model, one of the descriptive research methods, was used. An achievement test consisting of 15 different non-routine mathematics problems prepared by the researcher was administered to the students in two different sessions in two days. According to research, it was seen that model data fit was achieved on the rater, individual and item surfaces in all scoring. Additionally, it was observed that individuals were differentiated according to their ability levels and the difficulty levels of the items were different. Scoring with analytical rubric gives relatively more reliable results in terms of rater surface than scoring with holistic rubric; when the rater surfaces were compared, it was determined that the levels of rater severityleniency in scoring using analytical rubric were higher than in scoring using holistic rubric. When rater behaviors were examined, rater strictness and generosity behaviors were higher in raters using holistic rubrics than raters using analytical rubrics; It was determined that bias behaviors were less.

 Bu araştırmada, rutin olmayan matematik problemlerinden oluşan matematik başarı testinin analitik ve bütünsel dereceli puanlama anahtarları DPA ile puanlanmasının puanlayıcı davranışları üzerine etkileri Çok Yüzeyli Rasch Ölçme Modeli ile incelenmiştir. Çalışma grubu, rutin olmayan matematik problemlerinden oluşan başarı testinin uygulandığı öğrenci grubu ve cevaplanan başarı testini değerlendiren puanlayıcılar olmak üzere iki farklı kısımdan oluşmaktadır. Bu çalışmada, betimsel araştırma yöntemlerinden tarama modeli kullanılmıştır. Bu çalışmada, araştırmacı tarafından hazırlanmış, 15 farklı rutin olmayan matematik probleminden oluşan başarı testi, iki farklı oturum şeklinde, iki günde öğrencilere uygulanmıştır. Çalışmanın bulguları incelendiğinde, yapılan tüm puanlamalarda, puanlayıcı, birey ve madde yüzeylerinde model veri uyumunun sağlandığı görülmüştür. Ayrıca, bireylerin yetenek düzeylerine göre ayrıştığı ve maddelerin güçlük düzeylerinin farklı olduğu görülmüştür. Analitik DPA ile yapılan puanlamanın bütünsel DPA ile yapılan puanlamaya göre, puanlayıcı yüzeyi bakımından göreceli olarak daha güvenilir sonuçlar verdiği; puanlayıcı yüzeyleri karşılaştırıldığında ise, analitik DPA kullanılan puanlamalarda puanlayıcı katılıkcömertlik düzeylerinin, bütünsel DPA kullanılan puanlamalardan daha fazla olduğu belirlenmiştir. Ayrıca, analitik DPA kullanılan puanlamalar arasındaki uyumun, bütünsel DPA kullanılan puanlamalardan daha düşük olduğu sonucuna varılmıştır. Puanlayıcı davranışları incelendiğinde, bütünsel dereceli puanlama anahtarı kullanan puanlayıcılardan, analitik dereceli puanlama anahtarı kullanan puanlayıcılara göre puanlayıcı katılık ve cömertlik davranışları gösteren puanlayıcı sayısının daha fazla olduğu; yanlılık davranışlarının ise daha az olduğu belirlenmiştir.

Makale detayları

The Effect of Rubric Type On Raters’ Behaviors in Scoring Non-Routine Mathematics Problems

Yazarlar:

Esra Onkun Özgür | İsmail Karakaya

Yayıncı:

Bursa Uludağ University

Ana Başlıklar:

Eğitimde Ölçme ve Değerlendirme (Diğer)

Dergi:

Uludağ Üniversitesi Eğitim Fakültesi Dergisi Vol.37 Issue 3, pp.1037-1068

Belge Tipi:

Makale

Yayın Tarihi:

12/2024

Belge Dili:

Türkçe

Document ID (Org No.):

2001329654

pISSN:

2667-6788

Analytical rubric

Holistic rubric

Many-facet Rasch model

Non-routine math problem

Rater behaviorAnalitik dereceli puanlama anahtarı

Bütünsel dereceli puanlama anahtarı

Çok yüzeyli Rasch modeli

Puanlayıcı davranışları

Sayı

Makale detayları

Yazarlar: Esra Onkun Özgür | İsmail Karakaya

Yayıncı: Bursa Uludağ University

Ana Başlıklar: Eğitimde Ölçme ve Değerlendirme (Diğer)

Dergi: Uludağ Üniversitesi Eğitim Fakültesi Dergisi Vol.37 Issue 3, pp.1037-1068

Belge Tipi: Makale

Yayın Tarihi: 12/2024

Belge Dili: Türkçe

Document ID (Org No.): 2001329654

pISSN: 2667-6788

Sayı

Yazarlar:

Esra Onkun Özgür | İsmail Karakaya

Yayıncı:

Bursa Uludağ University

Ana Başlıklar:

Eğitimde Ölçme ve Değerlendirme (Diğer)

Dergi:

Uludağ Üniversitesi Eğitim Fakültesi Dergisi Vol.37 Issue 3, pp.1037-1068

Belge Tipi:

Makale

Yayın Tarihi:

12/2024

Belge Dili:

Türkçe

Document ID (Org No.):

2001329654

pISSN:

2667-6788