Improving LLM Reliability with RAG in Religious Question-Answering: MufassirQAS

Ahmet Yusuf Alan; Enis Karaarslan; Ömer Aydın

Makale detayları

Improving LLM Reliability with RAG in Religious Question-Answering: MufassirQAS

Yazarlar:

Ahmet Yusuf Alan | Enis Karaarslan | Ömer Aydın

Yayıncı:

Murat YAKAR

Ana Başlıklar:

Information Systems User Experience Design and Development | Management Information Systems | Computer Software | Software Engineering (Other)

Dergi:

Turkish Journal of Engineering Vol.9 Issue 3, pp.544-559

Belge Tipi:

Makale

Yayın Tarihi:

1/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001339467

pISSN:

2587-1366

Anahtar kelimeler:

Özet:

 Religious teachings can sometimes be complex and challenging to grasp, but chatbots can serve as effective assistants in this domain. Large Language Model LLM based chatbots, powered by Natural Language Processing NLP, can connect related topics and provide well-supported responses to intricate questions, making them valuable tools for religious education. However, LLMs are prone to hallucinations as they can generate inaccurate or irrelevant information, and these can include sensitive content that could be offensive, inappropriate, or controversial. Addressing such topics without inadvertently promoting hate speech or disrespecting certain beliefs remains a significant challenge. As a solution to these issues, we introduce MufassirQAS, a system that enhances LLM accuracy and transparency using a vector database-driven Retrieval-Augmented Generation RAG approach. We built a dataset comprising fundamental books containing Turkish translations and interpretations of Islamic texts. This database is leveraged to answer religious inquiries while ensuring that responses remain reliable and contextually grounded. Our system also presents the relevant dataset sections alongside the LLM-generated answers, reinforcing transparency. We carefully designed system prompts to prevent harmful, offensive, or disrespectful outputs, ensuring that responses align with ethical and respectful discourse. Moreover, MufassirQAS provides supplementary details, such as source page numbers and referenced articles, to enhance credibility. To evaluate its effectiveness, we tested MufassirQAS against ChatGPT with sensitive questions, and our system demonstrated superior performance in maintaining accuracy and reliability. Future work will focus on improving accuracy and refining prompt engineering techniques to further minimize biases and ensure even more reliable responses.

 Religious teachings can sometimes be complex and challenging to grasp, but chatbots can serve as effective assistants in this domain. Large Language Model LLM based chatbots, powered by Natural Language Processing NLP, can connect related topics and provide well-supported responses to intricate questions, making them valuable tools for religious education. However, LLMs are prone to hallucinations as they can generate inaccurate or irrelevant information, and these can include sensitive content that could be offensive, inappropriate, or controversial. Addressing such topics without inadvertently promoting hate speech or disrespecting certain beliefs remains a significant challenge. As a solution to these issues, we introduce MufassirQAS, a system that enhances LLM accuracy and transparency using a vector database-driven Retrieval-Augmented Generation RAG approach. We built a dataset comprising fundamental books containing Turkish translations and interpretations of Islamic texts. This database is leveraged to answer religious inquiries while ensuring that responses remain reliable and contextually grounded. Our system also presents the relevant dataset sections alongside the LLM-generated answers, reinforcing transparency. We carefully designed system prompts to prevent harmful, offensive, or disrespectful outputs, ensuring that responses align with ethical and respectful discourse. Moreover, MufassirQAS provides supplementary details, such as source page numbers and referenced articles, to enhance credibility. To evaluate its effectiveness, we tested MufassirQAS against ChatGPT with sensitive questions, and our system demonstrated superior performance in maintaining accuracy and reliability. Future work will focus on improving accuracy and refining prompt engineering techniques to further minimize biases and ensure even more reliable responses.

 Religious teachings can sometimes be complex and challenging to grasp, but chatbots can serve as effective assistants in this domain. Large Language Model LLM based chatbots, powered by Natural Language Processing NLP, can connect related topics and provide well-supported responses to intricate questions, making them valuable tools for religious education. However, LLMs are prone to hallucinations as they can generate inaccurate or irrelevant information, and these can include sensitive content that could be offensive, inappropriate, or controversial. Addressing such topics without inadvertently promoting hate speech or disrespecting certain beliefs remains a significant challenge. As a solution to these issues, we introduce MufassirQAS, a system that enhances LLM accuracy and transparency using a vector database-driven Retrieval-Augmented Generation RAG approach. We built a dataset comprising fundamental books containing Turkish translations and interpretations of Islamic texts. This database is leveraged to answer religious inquiries while ensuring that responses remain reliable and contextually grounded. Our system also presents the relevant dataset sections alongside the LLM-generated answers, reinforcing transparency. We carefully designed system prompts to prevent harmful, offensive, or disrespectful outputs, ensuring that responses align with ethical and respectful discourse. Moreover, MufassirQAS provides supplementary details, such as source page numbers and referenced articles, to enhance credibility. To evaluate its effectiveness, we tested MufassirQAS against ChatGPT with sensitive questions, and our system demonstrated superior performance in maintaining accuracy and reliability. Future work will focus on improving accuracy and refining prompt engineering techniques to further minimize biases and ensure even more reliable responses.

 Religious teachings can sometimes be complex and challenging to grasp, but chatbots can serve as effective assistants in this domain. Large Language Model LLM based chatbots, powered by Natural Language Processing NLP, can connect related topics and provide well-supported responses to intricate questions, making them valuable tools for religious education. However, LLMs are prone to hallucinations as they can generate inaccurate or irrelevant information, and these can include sensitive content that could be offensive, inappropriate, or controversial. Addressing such topics without inadvertently promoting hate speech or disrespecting certain beliefs remains a significant challenge. As a solution to these issues, we introduce MufassirQAS, a system that enhances LLM accuracy and transparency using a vector database-driven Retrieval-Augmented Generation RAG approach. We built a dataset comprising fundamental books containing Turkish translations and interpretations of Islamic texts. This database is leveraged to answer religious inquiries while ensuring that responses remain reliable and contextually grounded. Our system also presents the relevant dataset sections alongside the LLM-generated answers, reinforcing transparency. We carefully designed system prompts to prevent harmful, offensive, or disrespectful outputs, ensuring that responses align with ethical and respectful discourse. Moreover, MufassirQAS provides supplementary details, such as source page numbers and referenced articles, to enhance credibility. To evaluate its effectiveness, we tested MufassirQAS against ChatGPT with sensitive questions, and our system demonstrated superior performance in maintaining accuracy and reliability. Future work will focus on improving accuracy and refining prompt engineering techniques to further minimize biases and ensure even more reliable responses.

Makale detayları

Improving LLM Reliability with RAG in Religious Question-Answering: MufassirQAS

Yazarlar:

Ahmet Yusuf Alan | Enis Karaarslan | Ömer Aydın

Yayıncı:

Murat YAKAR

Ana Başlıklar:

Information Systems User Experience Design and Development | Management Information Systems | Computer Software | Software Engineering (Other)

Dergi:

Turkish Journal of Engineering Vol.9 Issue 3, pp.544-559

Belge Tipi:

Makale

Yayın Tarihi:

1/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001339467

pISSN:

2587-1366

Question Answering Systems

Chatbots

Generative Artificial Intelligence

Large Language Model

Retrieval Augmented Generation

Sayı

Makale detayları

Improving LLM Reliability with RAG in Religious Question-Answering: MufassirQAS

Yazarlar: Ahmet Yusuf Alan | Enis Karaarslan | Ömer Aydın

Yayıncı: Murat YAKAR

Ana Başlıklar: Information Systems User Experience Design and Development | Management Information Systems | Computer Software | Software Engineering (Other)

Dergi: Turkish Journal of Engineering Vol.9 Issue 3, pp.544-559

Belge Tipi: Makale

Yayın Tarihi: 1/2025

Belge Dili: İngilizce

Document ID (Org No.): 2001339467

pISSN: 2587-1366

Question Answering Systems

Chatbots

Generative Artificial Intelligence

Large Language Model

Retrieval Augmented Generation

Sayı

Yazarlar:

Ahmet Yusuf Alan | Enis Karaarslan | Ömer Aydın

Yayıncı:

Murat YAKAR

Ana Başlıklar:

Information Systems User Experience Design and Development | Management Information Systems | Computer Software | Software Engineering (Other)

Dergi:

Turkish Journal of Engineering Vol.9 Issue 3, pp.544-559

Belge Tipi:

Makale

Yayın Tarihi:

1/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001339467

pISSN:

2587-1366