Effects of diacritics on Turkish information retrieval

Adil ALPKOÇAK; Meltem CEYLAN

Makale detayları

Effects of diacritics on Turkish information retrieval

Yazarlar:

Adil ALPKOÇAK | Meltem CEYLAN

Yayıncı:

TUBITAK

Dergi:

Turkish Journal of Electrical Engineering and Computer Science 2012 Vol.20 Issue 5, pp.787-804

Belge Tipi:

Makale

Yayın Tarihi:

4/2012

Belge Dili:

Türkçe

Document ID (Org No.):

2000743544

Sıklık:

İki Aylık

eISSN:

1303-6203

pISSN:

1300-0632

Özet:

 

We investigate the effects of improper use of diacritics in  the Turkish alphabet on information retrieval.  A diacritic is simply  a supplementary sign added to a letter to change the sound value of  the letter, and the Turkish alphabet has 5 special letters derived  from Latin by adding different diacritics.  The statistical analysis  performed in this study shows that retrieval performance significantly decreases when documents and queries contain letters with different forms, such that documents consist of letters with diacritics while queries consist of standard Latin letters and vice versa.  In order to tackle this challenge, we propose 3 approaches: token normalization by equivalence classes, document expansion, and query expansion.  The experimental evaluations carried on the Bilkent Turkish information retrieval test collection suggests that the proposed approaches are promising as a remedy in this line of research.

 

We investigate the effects of improper use of diacritics in  the Turkish alphabet on information retrieval.  A diacritic is simply  a supplementary sign added to a letter to change the sound value of  the letter, and the Turkish alphabet has 5 special letters derived  from Latin by adding different diacritics.  The statistical analysis  performed in this study shows that retrieval performance significantly decreases when documents and queries contain letters with different forms, such that documents consist of letters with diacritics while queries consist of standard Latin letters and vice versa.  In order to tackle this challenge, we propose 3 approaches: token normalization by equivalence classes, document expansion, and query expansion.  The experimental evaluations carried on the Bilkent Turkish information retrieval test collection suggests that the proposed approaches are promising as a remedy in this line of research.

Makale detayları

Effects of diacritics on Turkish information retrieval

Yazarlar: Adil ALPKOÇAK | Meltem CEYLAN

Yayıncı: TUBITAK

Dergi: Turkish Journal of Electrical Engineering and Computer Science 2012 Vol.20 Issue 5, pp.787-804

Belge Tipi: Makale

Yayın Tarihi: 4/2012

Belge Dili: Türkçe

Document ID (Org No.): 2000743544

Sıklık: İki Aylık

eISSN: 1303-6203

pISSN: 1300-0632

Sayı

Yazarlar:

Adil ALPKOÇAK | Meltem CEYLAN

Yayıncı:

TUBITAK

Dergi:

Turkish Journal of Electrical Engineering and Computer Science 2012 Vol.20 Issue 5, pp.787-804

Belge Tipi:

Makale

Yayın Tarihi:

4/2012

Belge Dili:

Türkçe

Document ID (Org No.):

2000743544

Sıklık:

İki Aylık

eISSN:

1303-6203

pISSN:

1300-0632