A Novel Reviewer Evaluation Process and Decision Support System Proposal Based on Entropy and Clustering

Seda Nur Budak; Kumru Didem Atalay; Mehmet Kabak; Tacettin Sercan Pekin

Makale detayları

A Novel Reviewer Evaluation Process and Decision Support System Proposal Based on Entropy and Clustering

Yazarlar:

Seda Nur Budak | Kumru Didem Atalay | Mehmet Kabak | Tacettin Sercan Pekin

Yayıncı:

Karyay Karadeniz Yayımcılık

Ana Başlıklar:

Karar Desteği ve Grup Destek Sistemleri | Çok Ölçütlü Karar Verme | Endüstri Mühendisliği

Dergi:

Black Sea Journal of Engineering and Science Vol.8 Issue 4, pp.1034-1049

Belge Tipi:

Makale

Yayın Tarihi:

7/2025

Belge Dili:

Türkçe

Document ID (Org No.):

2001346245

pISSN:

2619-8991

Anahtar kelimeler:

Özet:

 The complexity and shortcomings of the evaluation processes of research and development RD projects lead to problems of consistency and traceability in the evaluation of the projects. This study aims to develop an objective, consistent, and highly applicable decision support system by addressing the main challenges encountered in the evaluation processes of RD projects in Türkiye. In this context, the literature on evaluation criteria was thoroughly reviewed, expert opinions were collected, the criteria were ranked according to their importance using the entropy method, and the most effective criteria for evaluation were identified. Consequently, the number of criteria was reduced from 247 to 96, thereby facilitating the evaluators task and forming a more effective evaluation set. The developed decision support system enables comprehensive analysis of projects and allows all stakeholdersfrom reviewers to executive board membersto conduct evaluations on a more transparent platform. Based on the reviewer scores obtained through the system, projects were classified into four main groups using the k-means clustering algorithm. This platform has accelerated evaluation processes and reduced the workload of decision-makers by 31 to 39. The results demonstrate that more comprehensive and objective evaluations can be made in dimensions such as the innovative aspects of RD projects, their economic and national contributions, project planning, and technical infrastructure. The newly developed system addresses the shortcomings of existing evaluation procedures and offers a more structured and holistic approach. It is envisioned that the system can be integrated into various RD programs in the future to maximize the countrys RD potential and innovation capacity.

 The complexity and shortcomings of the evaluation processes of research and development RD projects lead to problems of consistency and traceability in the evaluation of the projects. This study aims to develop an objective, consistent, and highly applicable decision support system by addressing the main challenges encountered in the evaluation processes of RD projects in Türkiye. In this context, the literature on evaluation criteria was thoroughly reviewed, expert opinions were collected, the criteria were ranked according to their importance using the entropy method, and the most effective criteria for evaluation were identified. Consequently, the number of criteria was reduced from 247 to 96, thereby facilitating the evaluators task and forming a more effective evaluation set. The developed decision support system enables comprehensive analysis of projects and allows all stakeholdersfrom reviewers to executive board membersto conduct evaluations on a more transparent platform. Based on the reviewer scores obtained through the system, projects were classified into four main groups using the k-means clustering algorithm. This platform has accelerated evaluation processes and reduced the workload of decision-makers by 31 to 39. The results demonstrate that more comprehensive and objective evaluations can be made in dimensions such as the innovative aspects of RD projects, their economic and national contributions, project planning, and technical infrastructure. The newly developed system addresses the shortcomings of existing evaluation procedures and offers a more structured and holistic approach. It is envisioned that the system can be integrated into various RD programs in the future to maximize the countrys RD potential and innovation capacity.

 The complexity and shortcomings of the evaluation processes of research and development RD projects lead to problems of consistency and traceability in the evaluation of the projects. This study aims to develop an objective, consistent, and highly applicable decision support system by addressing the main challenges encountered in the evaluation processes of RD projects in Türkiye. In this context, the literature on evaluation criteria was thoroughly reviewed, expert opinions were collected, the criteria were ranked according to their importance using the entropy method, and the most effective criteria for evaluation were identified. Consequently, the number of criteria was reduced from 247 to 96, thereby facilitating the evaluators task and forming a more effective evaluation set. The developed decision support system enables comprehensive analysis of projects and allows all stakeholdersfrom reviewers to executive board membersto conduct evaluations on a more transparent platform. Based on the reviewer scores obtained through the system, projects were classified into four main groups using the k-means clustering algorithm. This platform has accelerated evaluation processes and reduced the workload of decision-makers by 31 to 39. The results demonstrate that more comprehensive and objective evaluations can be made in dimensions such as the innovative aspects of RD projects, their economic and national contributions, project planning, and technical infrastructure. The newly developed system addresses the shortcomings of existing evaluation procedures and offers a more structured and holistic approach. It is envisioned that the system can be integrated into various RD programs in the future to maximize the countrys RD potential and innovation capacity.

 Araştırma ve geliştirme Ar-Ge projelerinin değerlendirme süreçlerinin karmaşıklığı ve eksiklikleri, projelerin değerlendirilmesinde tutarlılık ve izlenebilirlik sorunlarına yol açmaktadır. Bu çalışma, Türkiyedeki Ar-Ge projelerinin değerlendirme süreçlerinde karşılaşılan temel sorunları ele alarak, nesnel, tutarlı ve uygulanabilirliği yüksek bir karar destek sistemi geliştirmeyi amaçlamaktadır. Bu bağlamda, projelerin değerlendirilmesinde kullanılan kriterler için literatür detaylı bir şekilde incelenmiş, uzmanların görüşleri alınmış, entropi yöntemi ile önem derecelerine göre sıralanmış ve değerlendirmede daha etkili olduğu görülen kriterler belirlenmiştir. Böylece, kriter sayısı 247den 96ya indirilmiş ve değerlendiriciye kolaylık sağlanarak daha etkili bir değerlendirme seti oluşturulmuştur. Geliştirilen karar destek sistemi, projelerin kapsamlı bir analizine olanak tanımakta ve hakemlerden yürütme kurulu üyelerine kadar tüm paydaşların daha şeffaf bir platform üzerinde değerlendirme yapmasını sağlamaktadır. Karar destek sistemi ile elde edilen hakem puanlarına bağlı olarak projeler, K-ortalama kümeleme algoritması ile dört ana gruba ayrılmıştır. Bu platform, değerlendirme süreçlerinin hızlandırılmasını ve karar vericilerin iş yükünün 31 ila 39 oranında azaltılmasını sağlamıştır. Elde edilen sonuçlar, Ar-Ge projelerinin yenilikçi yönleri, ekonomik ve ulusal katkıları, proje planları ve teknik altyapıları gibi boyutlarda daha kapsamlı ve objektif bir değerlendirme yapılmasını mümkün kılmıştır. Yeni sistem, mevcut değerlendirme süreçlerinin eksikliklerini gidererek daha yapılandırılmış ve bütüncül bir yaklaşım sunmaktadır. Geliştirilen bu sistem, gelecekte farklı Ar-Ge programlarına entegre edilerek ülkenin Ar-Ge potansiyelini ve inovasyon kapasitesini en üst seviyeye çıkarmayı hedeflemektedir.

Makale detayları

A Novel Reviewer Evaluation Process and Decision Support System Proposal Based on Entropy and Clustering

Yazarlar:

Seda Nur Budak | Kumru Didem Atalay | Mehmet Kabak | Tacettin Sercan Pekin

Yayıncı:

Karyay Karadeniz Yayımcılık

Ana Başlıklar:

Karar Desteği ve Grup Destek Sistemleri | Çok Ölçütlü Karar Verme | Endüstri Mühendisliği

Dergi:

Black Sea Journal of Engineering and Science Vol.8 Issue 4, pp.1034-1049

Belge Tipi:

Makale

Yayın Tarihi:

7/2025

Belge Dili:

Türkçe

Document ID (Org No.):

2001346245

pISSN:

2619-8991

R&D project evaluation process

Multi-criteria decision making

Entropy

K-means clustering

Decision support systemAr-Ge projeleri değerlendirme süreci

Çok kriterli karar verme

Entropi

K-ortalama kümeleme

Karar destek sistemi

Sayı

Makale detayları

Yazarlar: Seda Nur Budak | Kumru Didem Atalay | Mehmet Kabak | Tacettin Sercan Pekin

Yayıncı: Karyay Karadeniz Yayımcılık

Ana Başlıklar: Karar Desteği ve Grup Destek Sistemleri | Çok Ölçütlü Karar Verme | Endüstri Mühendisliği

Dergi: Black Sea Journal of Engineering and Science Vol.8 Issue 4, pp.1034-1049

Belge Tipi: Makale

Yayın Tarihi: 7/2025

Belge Dili: Türkçe

Document ID (Org No.): 2001346245

pISSN: 2619-8991

Sayı

Yazarlar:

Seda Nur Budak | Kumru Didem Atalay | Mehmet Kabak | Tacettin Sercan Pekin

Yayıncı:

Karyay Karadeniz Yayımcılık

Ana Başlıklar:

Karar Desteği ve Grup Destek Sistemleri | Çok Ölçütlü Karar Verme | Endüstri Mühendisliği

Dergi:

Black Sea Journal of Engineering and Science Vol.8 Issue 4, pp.1034-1049

Belge Tipi:

Makale

Yayın Tarihi:

7/2025

Belge Dili:

Türkçe

Document ID (Org No.):

2001346245

pISSN:

2619-8991