Evaluating the Psychometric Characteristics of Generated Visual Reading Comprehension Items

Ayfer Sayın

Makale detayları

Evaluating the Psychometric Characteristics of Generated Visual Reading Comprehension Items

Yazarlar:

Ayfer Sayın

Yayıncı:

Bartin University

Ana Başlıklar:

Measurement Theories and Applications in Education and Psychology | Turkish Education

Dergi:

Bartın University Journal of Faculty of Education Vol.13 Issue 2, pp.380-395

Belge Tipi:

Makale

Yayın Tarihi:

4/2024

Belge Dili:

İngilizce

Document ID (Org No.):

2001313686

pISSN:

1308-7177

Anahtar kelimeler:

Özet:

 Reading comprehension, a crucial skill in todays information-rich environment, extends beyond text to include visual elements. Manual creation of visual reading comprehension items poses challenges, necessitating an innovative approach. This situation has led to the exploration of Automatic Item Generation AIG as a solution. This study aims to demonstrate the use of AIG for the creation of visual reading comprehension items. By developing cognitive and item models through expert input and utilizing computer algorithms for item generation, the study seeks to provide a time-efficient and reliable alternative for item writers. The field test involved 1,380 8th-grade students to evaluate the psychometric properties of the generated visual reading comprehension items. The AIG process starts with expert insights to develop cognitive and item models. Computer algorithms are then employed for AIG. The study utilizes a diverse sample of 8th-grade students for field testing, assessing the psychometric properties of the generated items. Field test results indicate the potential of AIG in efficiently producing a substantial item pool for visual reading comprehension. The generated items exhibit consistent difficulty levels 0.58 to 0.66, ensuring an appropriate challenge for students. High item discrimination 0.48 to 0.69 effectively distinguishes between students with varying visual reading comprehension skills. Item-total correlations 0.40 to 0.57 further validate the quality and validity of the generated items. The automated process yields efficient results in terms of item difficulty and discrimination, emphasizing the potential of AIG for high-quality assessment of visual reading comprehension items.

 Reading comprehension, a crucial skill in todays information-rich environment, extends beyond text to include visual elements. Manual creation of visual reading comprehension items poses challenges, necessitating an innovative approach. This situation has led to the exploration of Automatic Item Generation AIG as a solution. This study aims to demonstrate the use of AIG for the creation of visual reading comprehension items. By developing cognitive and item models through expert input and utilizing computer algorithms for item generation, the study seeks to provide a time-efficient and reliable alternative for item writers. The field test involved 1,380 8th-grade students to evaluate the psychometric properties of the generated visual reading comprehension items. The AIG process starts with expert insights to develop cognitive and item models. Computer algorithms are then employed for AIG. The study utilizes a diverse sample of 8th-grade students for field testing, assessing the psychometric properties of the generated items. Field test results indicate the potential of AIG in efficiently producing a substantial item pool for visual reading comprehension. The generated items exhibit consistent difficulty levels 0.58 to 0.66, ensuring an appropriate challenge for students. High item discrimination 0.48 to 0.69 effectively distinguishes between students with varying visual reading comprehension skills. Item-total correlations 0.40 to 0.57 further validate the quality and validity of the generated items. The automated process yields efficient results in terms of item difficulty and discrimination, emphasizing the potential of AIG for high-quality assessment of visual reading comprehension items.

 Reading comprehension, a crucial skill in todays information-rich environment, extends beyond text to include visual elements. Manual creation of visual reading comprehension items poses challenges, necessitating an innovative approach. This situation has led to the exploration of Automatic Item Generation AIG as a solution. This study aims to demonstrate the use of AIG for the creation of visual reading comprehension items. By developing cognitive and item models through expert input and utilizing computer algorithms for item generation, the study seeks to provide a time-efficient and reliable alternative for item writers. The field test involved 1,380 8th-grade students to evaluate the psychometric properties of the generated visual reading comprehension items. The AIG process starts with expert insights to develop cognitive and item models. Computer algorithms are then employed for AIG. The study utilizes a diverse sample of 8th-grade students for field testing, assessing the psychometric properties of the generated items. Field test results indicate the potential of AIG in efficiently producing a substantial item pool for visual reading comprehension. The generated items exhibit consistent difficulty levels 0.58 to 0.66, ensuring an appropriate challenge for students. High item discrimination 0.48 to 0.69 effectively distinguishes between students with varying visual reading comprehension skills. Item-total correlations 0.40 to 0.57 further validate the quality and validity of the generated items. The automated process yields efficient results in terms of item difficulty and discrimination, emphasizing the potential of AIG for high-quality assessment of visual reading comprehension items.

 Reading comprehension, a crucial skill in todays information-rich environment, extends beyond text to include visual elements. Manual creation of visual reading comprehension items poses challenges, necessitating an innovative approach. This situation has led to the exploration of Automatic Item Generation AIG as a solution. This study aims to demonstrate the use of AIG for the creation of visual reading comprehension items. By developing cognitive and item models through expert input and utilizing computer algorithms for item generation, the study seeks to provide a time-efficient and reliable alternative for item writers. The field test involved 1,380 8th-grade students to evaluate the psychometric properties of the generated visual reading comprehension items. The AIG process starts with expert insights to develop cognitive and item models. Computer algorithms are then employed for AIG. The study utilizes a diverse sample of 8th-grade students for field testing, assessing the psychometric properties of the generated items. Field test results indicate the potential of AIG in efficiently producing a substantial item pool for visual reading comprehension. The generated items exhibit consistent difficulty levels 0.58 to 0.66, ensuring an appropriate challenge for students. High item discrimination 0.48 to 0.69 effectively distinguishes between students with varying visual reading comprehension skills. Item-total correlations 0.40 to 0.57 further validate the quality and validity of the generated items. The automated process yields efficient results in terms of item difficulty and discrimination, emphasizing the potential of AIG for high-quality assessment of visual reading comprehension items.

Makale detayları

Evaluating the Psychometric Characteristics of Generated Visual Reading Comprehension Items

Yazarlar:

Ayfer Sayın

Yayıncı:

Bartin University

Ana Başlıklar:

Measurement Theories and Applications in Education and Psychology | Turkish Education

Dergi:

Bartın University Journal of Faculty of Education Vol.13 Issue 2, pp.380-395

Belge Tipi:

Makale

Yayın Tarihi:

4/2024

Belge Dili:

İngilizce

Document ID (Org No.):

2001313686

pISSN:

1308-7177

visual reading comprehension

automatic item generation

high-stake tests

parallel test

Sayı

Makale detayları

Evaluating the Psychometric Characteristics of Generated Visual Reading Comprehension Items

Yazarlar: Ayfer Sayın

Yayıncı: Bartin University

Ana Başlıklar: Measurement Theories and Applications in Education and Psychology | Turkish Education

Dergi: Bartın University Journal of Faculty of Education Vol.13 Issue 2, pp.380-395

Belge Tipi: Makale

Yayın Tarihi: 4/2024

Belge Dili: İngilizce

Document ID (Org No.): 2001313686

pISSN: 1308-7177

visual reading comprehension

automatic item generation

high-stake tests

parallel test

Sayı

Yazarlar:

Ayfer Sayın

Yayıncı:

Bartin University

Ana Başlıklar:

Measurement Theories and Applications in Education and Psychology | Turkish Education

Dergi:

Bartın University Journal of Faculty of Education Vol.13 Issue 2, pp.380-395

Belge Tipi:

Makale

Yayın Tarihi:

4/2024

Belge Dili:

İngilizce

Document ID (Org No.):

2001313686

pISSN:

1308-7177