CNN-Based Approaches for Automatic Recognition of Turkish Sign Language

Ishak Pacal; Melek Alaftekin

Makale detayları

CNN-Based Approaches for Automatic Recognition of Turkish Sign Language

Yazarlar:

Ishak Pacal | Melek Alaftekin

Yayıncı:

Igdir University

Ana Başlıklar:

Bilgisayar Yazılımı

Dergi:

Iğdır Üniversitesi Fen Bilimleri Enstitüsü Dergisi Vol.13 Issue 2, pp.760-777

Belge Tipi:

Makale

Yayın Tarihi:

1/2023

Belge Dili:

Türkçe

Document ID (Org No.):

2001289839

pISSN:

2536-4618

Anahtar kelimeler:

Özet:

 Sign language is a nonverbal communication tool used by deaf and dumb individuals to convey their feelings, thoughts and social identities to their environment. Sign language has a key role in communication between deaf and dumb individuals and the rest of the society. Many sign language recognition systems have been developed with the increase in human-computer interaction and the fact that sign language is not widely known among normal people. In this study, a new number-based data set for Turkish sign language is proposed for the first time in the literature. The most up-to-date deep learning approaches have been applied to the proposed data set in order to classify Turkish sign language autonomously and to enable computer-based communication of people who have difficulties in this regard. The most up-to-date and popular architectures such as CNN-based VGG, ResNet, MobileNet, DenseNet and EfficientNet were used in the study. In experimental studies, it has been observed that the ResNet152 model performs better than other models with 98.76 accuracy, 98.85 precision, 98.81 sensitivity and 98.80 F1-score. Additionally, the other models used in experimental studies all show a success rate above 90, supporting the effectiveness of the proposed data set. This shows that CNN models can successfully detect Turkish sign language.

 Sign language is a nonverbal communication tool used by deaf and dumb individuals to convey their feelings, thoughts and social identities to their environment. Sign language has a key role in communication between deaf and dumb individuals and the rest of the society. Many sign language recognition systems have been developed with the increase in human-computer interaction and the fact that sign language is not widely known among normal people. In this study, a new number-based data set for Turkish sign language is proposed for the first time in the literature. The most up-to-date deep learning approaches have been applied to the proposed data set in order to classify Turkish sign language autonomously and to enable computer-based communication of people who have difficulties in this regard. The most up-to-date and popular architectures such as CNN-based VGG, ResNet, MobileNet, DenseNet and EfficientNet were used in the study. In experimental studies, it has been observed that the ResNet152 model performs better than other models with 98.76 accuracy, 98.85 precision, 98.81 sensitivity and 98.80 F1-score. Additionally, the other models used in experimental studies all show a success rate above 90, supporting the effectiveness of the proposed data set. This shows that CNN models can successfully detect Turkish sign language.

 Sign language is a nonverbal communication tool used by deaf and dumb individuals to convey their feelings, thoughts and social identities to their environment. Sign language has a key role in communication between deaf and dumb individuals and the rest of the society. Many sign language recognition systems have been developed with the increase in human-computer interaction and the fact that sign language is not widely known among normal people. In this study, a new number-based data set for Turkish sign language is proposed for the first time in the literature. The most up-to-date deep learning approaches have been applied to the proposed data set in order to classify Turkish sign language autonomously and to enable computer-based communication of people who have difficulties in this regard. The most up-to-date and popular architectures such as CNN-based VGG, ResNet, MobileNet, DenseNet and EfficientNet were used in the study. In experimental studies, it has been observed that the ResNet152 model performs better than other models with 98.76 accuracy, 98.85 precision, 98.81 sensitivity and 98.80 F1-score. Additionally, the other models used in experimental studies all show a success rate above 90, supporting the effectiveness of the proposed data set. This shows that CNN models can successfully detect Turkish sign language.

 İşaret dili, sağır ve dilsiz bireylerin duygularını, düşüncelerini ve sosyal kimliklerini çevrelerine aktarabilmek için kullandıkları sözsüz bir iletişim aracıdır. İşaret dili, sağır ve dilsiz bireyler ile toplumun geri kalan bireyleri arasındaki iletişimde kilit bir role sahiptir. Normal insanlar arasında işaret dilinin çok yaygın bilinmemesi ve insan-bilgisayar etkileşiminin artmasıyla birlikte birçok işaret dili tanıma sistemleri geliştirilmiştir. Bu çalışmada, Türk işaret dili için literatürde ilk kez rakam temelli yeni bir veri seti önerilmiştir. Türk işaret dilinin otonom bir şekilde sınıflandırılması ve bu konuda sıkıntı yaşayan insanların iletişimini bilgisayar temelli yapabilmesi için en güncel derin öğrenme yaklaşımları önerilen veri setine uygulanmıştır. Çalışmada özellikle CNN tabanlı VGG, ResNet, MobileNet, DenseNet ve EfficientNet gibi en güncel ve popüler mimariler kullanılmıştır. Deneysel çalışmalarda ResNet152 modeli, 98.76 doğruluk, 98.85 kesinlik, 98.81 duyarlılık ve 98.80 F1-skoru ile diğer modellere göre daha iyi performans gösterdiği gözlemlenmiştir. Ayrıca, deneysel çalışmalarda kullanılan diğer modellerin hepsi 90ın üzerinde bir başarım oranı göstererek önerilen veri setinin etkililiğini desteklemektedir. Bu, CNN modellerinin Türk işaret dilini tanımayı başarılı bir şekilde tespit yapabildiğini göstermektedir.

Makale detayları

CNN-Based Approaches for Automatic Recognition of Turkish Sign Language

Yazarlar:

Ishak Pacal | Melek Alaftekin

Yayıncı:

Igdir University

Ana Başlıklar:

Bilgisayar Yazılımı

Dergi:

Iğdır Üniversitesi Fen Bilimleri Enstitüsü Dergisi Vol.13 Issue 2, pp.760-777

Belge Tipi:

Makale

Yayın Tarihi:

1/2023

Belge Dili:

Türkçe

Document ID (Org No.):

2001289839

pISSN:

2536-4618

Turkish Sign Language

CNN

ResNet

ClassificationTürk İşaret Dili

CNN

ResNet

Sınıflandırma

Sayı

Makale detayları

Yazarlar: Ishak Pacal | Melek Alaftekin

Yayıncı: Igdir University

Ana Başlıklar: Bilgisayar Yazılımı

Dergi: Iğdır Üniversitesi Fen Bilimleri Enstitüsü Dergisi Vol.13 Issue 2, pp.760-777

Belge Tipi: Makale

Yayın Tarihi: 1/2023

Belge Dili: Türkçe

Document ID (Org No.): 2001289839

pISSN: 2536-4618

Sayı

Yazarlar:

Ishak Pacal | Melek Alaftekin

Yayıncı:

Igdir University

Ana Başlıklar:

Bilgisayar Yazılımı

Dergi:

Iğdır Üniversitesi Fen Bilimleri Enstitüsü Dergisi Vol.13 Issue 2, pp.760-777

Belge Tipi:

Makale

Yayın Tarihi:

1/2023

Belge Dili:

Türkçe

Document ID (Org No.):

2001289839

pISSN:

2536-4618