Comparison of Some Performance Metrics Used in Multiple Classification Problems

Ali Vasfi Ağlarcı; Cengiz Bal

Makale detayları

Comparison of Some Performance Metrics Used in Multiple Classification Problems

Yazarlar:

Ali Vasfi Ağlarcı | Cengiz Bal

Yayıncı:

Kastamonu University

Ana Başlıklar:

Ekonometrik ve İstatistiksel Yöntemler

Dergi:

Kastamonu Üniversitesi İktisadi ve İdari Bilimler Fakültesi Dergisi Vol.27 Issue 1, pp.22-39

Belge Tipi:

Makale

Yayın Tarihi:

6/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001353802

pISSN:

2147-6012

Anahtar kelimeler:

Özet:

 The purpose of this research is to compare the performance metrics used in multiple classification problems in machine learning. For this purpose, simulation study was carried out under different scenarios by using 4 different classification methods and the performance metrics obtained were compared in this direction. While comparing the performance metrics in the study, the data to be used for classification purposes were derived under different scenarios, taking into account the effect of 4 factors. 90 different scenarios were created by considering the number of 3 different categories of the response variable, 5 different sample sizes, 3 different correlation structures, and the balanced and unbalanced distribution of the response variable. Accuray, Kappa and CramerV metrics used in multiple classification problems were used as performance measures. Changes in performance metrics in the determined scenarios are summarized in tables and compared. As a result of the comparisons made with the simulation study, it has been seen that Kappa performance measure is a more accurate performance metric than the other two metrics in multi-class classification problems, and the method gives more reliable information about the classification success.

 The purpose of this research is to compare the performance metrics used in multiple classification problems in machine learning. For this purpose, simulation study was carried out under different scenarios by using 4 different classification methods and the performance metrics obtained were compared in this direction. While comparing the performance metrics in the study, the data to be used for classification purposes were derived under different scenarios, taking into account the effect of 4 factors. 90 different scenarios were created by considering the number of 3 different categories of the response variable, 5 different sample sizes, 3 different correlation structures, and the balanced and unbalanced distribution of the response variable. Accuray, Kappa and CramerV metrics used in multiple classification problems were used as performance measures. Changes in performance metrics in the determined scenarios are summarized in tables and compared. As a result of the comparisons made with the simulation study, it has been seen that Kappa performance measure is a more accurate performance metric than the other two metrics in multi-class classification problems, and the method gives more reliable information about the classification success.

 The purpose of this research is to compare the performance metrics used in multiple classification problems in machine learning. For this purpose, simulation study was carried out under different scenarios by using 4 different classification methods and the performance metrics obtained were compared in this direction. While comparing the performance metrics in the study, the data to be used for classification purposes were derived under different scenarios, taking into account the effect of 4 factors. 90 different scenarios were created by considering the number of 3 different categories of the response variable, 5 different sample sizes, 3 different correlation structures, and the balanced and unbalanced distribution of the response variable. Accuray, Kappa and CramerV metrics used in multiple classification problems were used as performance measures. Changes in performance metrics in the determined scenarios are summarized in tables and compared. As a result of the comparisons made with the simulation study, it has been seen that Kappa performance measure is a more accurate performance metric than the other two metrics in multi-class classification problems, and the method gives more reliable information about the classification success.

 Bu araştırmanın amacı, makine öğrenmesinde birden fazla sınıflandırma probleminde kullanılan performans metriklerini karşılaştırmaktır. Bu amaçla 4 farklı sınıflandırma yöntemi kullanılarak farklı senaryolar altında simülasyon çalışması yapılmış ve elde edilen performans metrikleri bu doğrultuda karşılaştırılmıştır. Çalışmada performans metrikleri karşılaştırılırken, sınıflandırma amacıyla kullanılacak veriler 4 faktörün etkisi dikkate alınarak farklı senaryolar altında türetilmiştir. Yanıt değişkeninin 3 farklı kategori sayısı, 5 farklı örneklem büyüklüğü, 3 farklı korelasyon yapısı ve yanıt değişkeninin dengeli ve dengesiz dağılımı dikkate alınarak 90 farklı senaryo oluşturulmuştur. Çoklu sınıflandırma problemlerinde kullanılan Accuray, Kappa ve CramerV metrikleri performans ölçüsü olarak kullanılmıştır. Belirlenen senaryolardaki performans metriklerindeki değişimler tablolar halinde özetlenmiş ve karşılaştırılmıştır. Simülasyon çalışması ile yapılan karşılaştırmalar sonucunda, Kappa performans ölçütünün çok sınıflı sınıflandırma problemlerinde diğer iki metriğe göre daha doğru bir performans metriği olduğu ve yöntemin sınıflandırma başarısı hakkında daha güvenilir bilgi verdiği görülmüştür.

Makale detayları

Comparison of Some Performance Metrics Used in Multiple Classification Problems

Yazarlar:

Ali Vasfi Ağlarcı | Cengiz Bal

Yayıncı:

Kastamonu University

Ana Başlıklar:

Ekonometrik ve İstatistiksel Yöntemler

Dergi:

Kastamonu Üniversitesi İktisadi ve İdari Bilimler Fakültesi Dergisi Vol.27 Issue 1, pp.22-39

Belge Tipi:

Makale

Yayın Tarihi:

6/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001353802

pISSN:

2147-6012

Classification success

classification performance

machine learning

simulation

performance metricsSınıflandırma başarısı

sınıflandırma performansı

makine öğrenimi

simülasyon

performans ölçümleri

Sayı

Makale detayları

Yazarlar: Ali Vasfi Ağlarcı | Cengiz Bal

Yayıncı: Kastamonu University

Ana Başlıklar: Ekonometrik ve İstatistiksel Yöntemler

Dergi: Kastamonu Üniversitesi İktisadi ve İdari Bilimler Fakültesi Dergisi Vol.27 Issue 1, pp.22-39

Belge Tipi: Makale

Yayın Tarihi: 6/2025

Belge Dili: İngilizce

Document ID (Org No.): 2001353802

pISSN: 2147-6012

Sayı

Yazarlar:

Ali Vasfi Ağlarcı | Cengiz Bal

Yayıncı:

Kastamonu University

Ana Başlıklar:

Ekonometrik ve İstatistiksel Yöntemler

Dergi:

Kastamonu Üniversitesi İktisadi ve İdari Bilimler Fakültesi Dergisi Vol.27 Issue 1, pp.22-39

Belge Tipi:

Makale

Yayın Tarihi:

6/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001353802

pISSN:

2147-6012