A Novel Data Imputation Method (M-CBRI) for Industrial Analytic Applications

Mehmet Alper Şahin; Uğur Üresin

Makale detayları

A Novel Data Imputation Method (M-CBRI) for Industrial Analytic Applications

Yazarlar:

Mehmet Alper Şahin | Uğur Üresin

Yayıncı:

Gazi University

Ana Başlıklar:

Mühendislik

Dergi:

Politeknik Dergisi Vol.27 Issue 6, pp.2225-2229

Belge Tipi:

Makale

Yayın Tarihi:

12/2024

Belge Dili:

İngilizce

Document ID (Org No.):

2001333502

pISSN:

2147-9429

Anahtar kelimeler:

Özet:

 Data analysis is mainly based on understanding and preprocessing the data coming from various sources for various applications. Missing values might play a critical role to reflect to characteristic of datasets; thus, imputation of missing values is a valuable process to not only handle reducing deviation but also avoid loss of data. There are different approaches to filling missing values. One of them is correlation-based imputation method. This approach is based on the high correlation between the parameters, these parameters are variables of linear equation, the linear equation enables to predict missing values. In this study, improvements were made to the correlation-based imputation method to predict missing values. The proposed method was performed on three various datasets which are related to the automotive industry. Missing values are handled in a manual process, and these values are picked randomly from the real data. After generating missing values, missing values are predicted using the correlation-based imputation method; furthermore, the margin of error between the estimated value and actual value was calculated. The results were compared to different methods which are arithmetic mean assignment, median value assignment, k- nearest neighbor assignment, and multivariate imputation by chained equations; consequently, much more successful results were obtained with the proposed method for three datasets.

 Data analysis is mainly based on understanding and preprocessing the data coming from various sources for various applications. Missing values might play a critical role to reflect to characteristic of datasets; thus, imputation of missing values is a valuable process to not only handle reducing deviation but also avoid loss of data. There are different approaches to filling missing values. One of them is correlation-based imputation method. This approach is based on the high correlation between the parameters, these parameters are variables of linear equation, the linear equation enables to predict missing values. In this study, improvements were made to the correlation-based imputation method to predict missing values. The proposed method was performed on three various datasets which are related to the automotive industry. Missing values are handled in a manual process, and these values are picked randomly from the real data. After generating missing values, missing values are predicted using the correlation-based imputation method; furthermore, the margin of error between the estimated value and actual value was calculated. The results were compared to different methods which are arithmetic mean assignment, median value assignment, k- nearest neighbor assignment, and multivariate imputation by chained equations; consequently, much more successful results were obtained with the proposed method for three datasets.

 Data analysis is mainly based on understanding and preprocessing the data coming from various sources for various applications. Missing values might play a critical role to reflect to characteristic of datasets; thus, imputation of missing values is a valuable process to not only handle reducing deviation but also avoid loss of data. There are different approaches to filling missing values. One of them is correlation-based imputation method. This approach is based on the high correlation between the parameters, these parameters are variables of linear equation, the linear equation enables to predict missing values. In this study, improvements were made to the correlation-based imputation method to predict missing values. The proposed method was performed on three various datasets which are related to the automotive industry. Missing values are handled in a manual process, and these values are picked randomly from the real data. After generating missing values, missing values are predicted using the correlation-based imputation method; furthermore, the margin of error between the estimated value and actual value was calculated. The results were compared to different methods which are arithmetic mean assignment, median value assignment, k- nearest neighbor assignment, and multivariate imputation by chained equations; consequently, much more successful results were obtained with the proposed method for three datasets.

 Veri analitiği çalışmalarının ilk aşamaları, veriyi toplama, veriyi analiz etme ve veriyi temizleme şeklindedir. Toplanan verilerin, farklı kaynaklardan elde edilmesi ve veri kaynaklarındaki kesilmeler, veriseti içerisinde eksik değerlerin oluşmasına sebep olabilmektedir. Bununla birlikte, veriyi temizleme çalışmalarında bazı aykırı değerlerin verisetinden çıkarılması da yine eksik değerlerin oluşmasına yol açmaktadır. Veride yer alan eksik değerler, analitik uygulamalarda elde edilmek istenen çıktılarda sapmalara sebep olabilir. Hem bu sapmayı azaltmak hem de toplanan veride kayıp yaşamamak adına eksik verilerin giderilmesi önemli bir süreçtir. Literatürde, eksik verilerin yerine değer atanması konusunda pek çok yöntem yer almaktadır ama söz konusu yöntemlerden uygun olanın seçilmesi tecrübe ve uzmanlık gerektirmektedir. Bu çalışmada, eksik verileri tahminlemek adına doğrusal korelasyona bağlı değer atama algoritması üzerinden geliştirmeler yapılmıştır. Bu algoritma, bir otomotiv üretecisinin farklı proseslerinden elde edilen üç farklı gerçek veriseti üzerinde test edilmiştir. Verisetlerinden rastgele silinen veriler, geliştirilen yöntemler yardımıyla tahminlenmiştir ve tahminlenen değer ile gerçek değer arasındaki hata payı hesaplanmıştır. Geliştirilen algoritmanın sonuçları, ortalama değer atama, medyan değer atama, en yakın komşuya göre değer atama ve zincir denklemlerle çok değişkenli değer atama yöntemleriyle karşılaştırılmıştır. Üç veriseti için de, geliştirilen yöntemin diğer yöntemlere göre daha başarılı tahminde bulunduğu gözlemlenmiştir.

Makale detayları

A Novel Data Imputation Method (M-CBRI) for Industrial Analytic Applications

Yazarlar:

Mehmet Alper Şahin | Uğur Üresin

Yayıncı:

Gazi University

Ana Başlıklar:

Mühendislik

Dergi:

Politeknik Dergisi Vol.27 Issue 6, pp.2225-2229

Belge Tipi:

Makale

Yayın Tarihi:

12/2024

Belge Dili:

İngilizce

Document ID (Org No.):

2001333502

pISSN:

2147-9429

missing data imputation

data preprocessing

missing value

data imputation

industrial data processingeksik verilere değer atama

veri önişleme

eksik veri

değer atama

endüstriyel veri işleme

Sayı

Makale detayları

Yazarlar: Mehmet Alper Şahin | Uğur Üresin

Yayıncı: Gazi University

Ana Başlıklar: Mühendislik

Dergi: Politeknik Dergisi Vol.27 Issue 6, pp.2225-2229

Belge Tipi: Makale

Yayın Tarihi: 12/2024

Belge Dili: İngilizce

Document ID (Org No.): 2001333502

pISSN: 2147-9429

Sayı

Yazarlar:

Mehmet Alper Şahin | Uğur Üresin

Yayıncı:

Gazi University

Ana Başlıklar:

Mühendislik

Dergi:

Politeknik Dergisi Vol.27 Issue 6, pp.2225-2229

Belge Tipi:

Makale

Yayın Tarihi:

12/2024

Belge Dili:

İngilizce

Document ID (Org No.):

2001333502

pISSN:

2147-9429