Abstract

Large Language Models (LLMs) have revolutionized Natural Language Processing (NLP) tasks, enabling critical capabilities in fields such as business and finance with their cybersecurity analysis. In particular, LLMs are becoming vital in cybersecurity in providing more accurate solutions to different types of threats, vulnerabilities, and security challenges through pattern recognition, intelligent response generation, and automating threat detection. Although LLMs are pre-trained with a large amount of knowledge, their application in any specific domain, such as cybersecurity, is often limited by the domain knowledge constraints of their pretraining data. To solve this problem, incorporating domain-specific knowledge is more often accomplished through fine-tuning or Retrieval-Augmented Generation (RAG). RAG enriches the model’s responses at inference time by retrieving relevant external information, while fine-tuning embeds new knowledge directly into the model’s parameters. In this position paper, we describe fine-tuning and RAG from the perspective of cybersecurity applications. Moreover, we also discuss the hybrid approach of finetuning and RAG and highlight proper best cases for when to select fine-tuning, RAG, or a hybrid approach based on the desired cybersecurity use case and operational constraints. Overall, this position paper aims to contribute to the ongoing discussion of LLM adaptation processes and best practices for the purpose of applying RAG and or fine-tuning.