Maximizing SIEM Efficiency: A Practical Guide to Implementing LLM-Powered AI Agents

The complexity of the modern digital landscape presents significant challenges for enterprise security teams. Cloud migration, the adoption of container technologies, and increased utilization of SaaS have blurred traditional security perimeters, leading to a quantitative explosion of logs and alerts flowing into Security Information and Event Management (SIEM) systems. Against this backdrop, security teams in large-scale financial environments are continuously exploring methods to effectively respond to advanced threats and maximize operational efficiency with limited resources.

While past practices heavily relied on human intelligence and manual analysis, the critical factor now is the rapid and accurate processing of vast amounts of data. Specifically, detecting actual threats and reducing false positives amidst a multitude of real-time events imposes a considerable burden even on skilled analysts. In this environment, LLM (Large Language Model)-powered AI agents are emerging as a practical alternative that can overcome the limitations of traditional SIEM operations and innovatively enhance security monitoring capabilities. This post aims to present concrete strategies and practical application cases for maximizing SIEM efficiency using LLM-powered AI agents.

Scenario Introduction: Challenges Faced by a Financial SOC

Recently, a large-scale Security Operations Center (SOC) within the financial sector was grappling with increasing threat complexity and declining operational efficiency. This SOC has various domestic and international regulatory compliance obligations and collects tens of terabytes (TB) of log data daily into Seekurity SIEM from thousands of servers, cloud workloads (AWS, Azure), numerous endpoints, and network devices. While this vast amount of data is a critical resource for detecting potential threats, it simultaneously contributes to the increased workload of analysts.

The primary objectives included reducing the false positive rate, shortening the Mean Time To Respond (MTTR) for actual threats, and enabling experienced analysts to focus on advanced threat analysis instead of repetitive tasks. Specifically, enhancing rapid detection and response capabilities for complex threats, such as current attack trends like Supply Chain Attacks, Advanced Persistent Threats (APT), and cloud environment misconfigurations and identity theft, emerged as urgent priorities. Within this environment, it became evident that the existing SIEM operational model had reached its limitations.

Challenge: Alert Flood and Analyst Fatigue

This SOC faced several technical and operational challenges in its existing SIEM operations. The most significant issue was 'Alert Fatigue.' A substantial portion of daily alerts detected by Seekurity SIEM were identified as false positives rather than actual threats, consuming considerable time for analysts to manually review and classify each alert. Furthermore, with the added complexity of cloud environments, alerts related to cloud misconfigurations or anomalous activities detected via FRIIM CNAPP often proved difficult to analyze in conjunction with existing SIEM alerts.

Previous attempts included refining SIEM detection rules and implementing some automation through SOAR playbooks. However, rule-based detection struggled to adapt flexibly to new threat patterns, and SOAR playbooks were applicable only to predefined scenarios. Therefore, the capabilities for 'Situational Awareness' and 'judgment' regarding dynamically evolving threat situations largely remained with the analysts. A frequently overlooked aspect was the strong tendency for experienced analysts' knowledge and experience to remain undocumented and reliant on individuals, posing a constant risk of critical know-how loss when personnel depart. Consequently, standardizing and enhancing the overall detection and response capabilities of the SOC emerged as a core requirement.

Technology Selection Process: The Importance of Flexibility and Intelligent Judgment

The SOC evaluated various technological approaches to address the challenges faced. Initially, extending SOAR playbooks and implementing machine learning-based anomaly detection solutions were considered. While SOAR playbooks are effective in automating repetitive response procedures, they clearly demonstrated limitations in understanding the context of alerts and making 'judgments' in complex situations. While traditional methods remained confined to rule execution based on specific conditions, LLM-powered agents showed a difference in their ability to analyze unstructured data and perform natural language-based reasoning.

Machine learning-based anomaly detection demonstrated strengths in detecting specific patterns of anomalies but lacked 'explainability' for detected anomalous behaviors, making it difficult for analysts to trust the results and decide on subsequent actions. In contrast, LLM-powered AI agents garnered attention for their ability to perform 'intelligent judgment,' such as interpreting alert content in natural language, explaining the context of a threat using the MITRE ATT&CK framework or internal knowledge bases, and suggesting next steps. The ability to evaluate the potential risks of AI models and ensure stability through KYRA AI Sandbox also played a positive role. Ultimately, the SOC selected LLM-powered AI agents as the core technology, recognizing their provision of flexible scalability and situation-aware judgment capabilities. This was determined to be the most efficient approach to maximize synergy with the existing Seekurity SIEM/SOAR environment.

Implementation Process

LLM-Powered AI Agent Architecture Design

The first step in implementing AI agents was designing a robust architecture. The core objective was to establish a framework centered around the LLM, integrated with various security tools and data sources, enabling the agent to autonomously 'think' and 'act.' The architecture was designed with three main components. First, the Orchestrator manages the agent's overall workflow, interpreting user requests and directing the appropriate tool usage. Second, the LLM (Large Language Model) engine is responsible for core intelligent reasoning, including alert analysis, threat information summarization, and response strategy suggestions. Third, the Tools encompass various resources that the agent can leverage, such as the Seekurity SIEM API, Seekurity SOAR API, internal Threat Intelligence (TI) databases, and external public TI platforms.

The agent analyzes alerts received from Seekurity SIEM and, if necessary, utilizes SIEM's log querying capabilities to obtain additional context. For instance, it may query past events related to a specific IP address or examine recent login records for a user account. During this process, the RAG (Retrieval Augmented Generation) pattern was applied to provide the LLM with a knowledge base including internal SOAR playbooks, threat information, and incident response procedures, thereby enhancing the accuracy and reliability of its responses. The overall architecture operates with the following logical flow:

• Alert Collection: Alerts detected by Seekurity SIEM are transmitted to the agent orchestrator in real time.
• Initial Analysis: The LLM analyzes the alert content, identifying the type of threat, severity, and suspicious indicators.
• Contextualization: Upon instruction from the LLM, the agent invokes the SIEM API to query additional log data and retrieves relevant IOCs (Indicators of Compromise) from the TI database.
• Comprehensive Judgment and Recommendation: Based on all acquired information, the LLM makes a comprehensive judgment and proposes the validity of the threat, its potential impact, and response strategies utilizing Seekurity SOAR playbooks.
• Response Execution (Optional): Upon analyst approval, the agent invokes the Seekurity SOAR API to execute automated responses such as isolation, blocking, or password reset.

Prompt Engineering and Knowledge Base Construction

The performance of LLM-powered AI agents is highly dependent on the quality of prompts. Therefore, systematic prompt engineering combined with the construction of a reliable knowledge base is essential. Prompts were structured to clearly define the agent's role, objectives, the format of input data provided, and the expected output format. For instance, when requesting an in-depth analysis of a specific SIEM alert, the following prompt configuration was utilized:

role:
  - task: "SIEM 경고 분석 및 초기 분류"
  - goal: "오탐을 최소화하고, 실제 위협에 대한 맥락과 대응 방안을 제안"
input_data:
  - alert_id: "{{alert.id}}"
  - alert_name: "{{alert.name}}"
  - event_time: "{{alert.timestamp}}"
  - source_ip: "{{alert.source.ip}}"
  - destination_ip: "{{alert.destination.ip}}"
  - username: "{{alert.user.name}}"
  - description: "{{alert.description}}"
  - raw_logs: "{{alert.raw_logs}}"
context_guidelines:
  - "MITRE ATT&CK 기술 및 전술을 기반으로 분석"
  - "내부 SOAR 플레이북 'phishing_response.yaml' 참조"
  - "최근 탐지된 클라우드 관련 취약점(FRIIM CNAPP 정보) 고려"
output_format:
  - summary: "경고 요약 및 의심 정도"
  - threat_analysis: "MITRE ATT&CK 매핑, 잠재적 공격 시나리오"
  - evidence: "관련 로그 및 TI 정보 요약"
  - recommendations: "SOAR 플레이북 실행 제안, 추가 조사 필요 여부"

The knowledge base was constructed primarily in two ways. First, structured knowledge includes internal incident response procedures, SOAR playbook definitions, past analysis cases, MITRE ATT&CK mapping information, and cloud security policies and compliance standards collected from FRIIM CNAPP. Second, unstructured knowledge, such as security experts' analysis notes, past threat reports, and technical blog posts, was stored in text form in a vector database, enabling the RAG system to retrieve it. This approach allowed the LLM to go beyond relying solely on learned knowledge, leveraging real-time updated internal SOC know-how and the latest threat intelligence to perform more sophisticated analyses.

Automation Through SIEM/SOAR Integration

The practical value of LLM-powered AI agents arises from their organic integration with existing Seekurity SIEM/SOAR platforms. The agent utilizes Seekurity SIEM's API to query necessary log data in real-time and performs the function of reflecting analysis results on the SIEM dashboard. For example, when analyzing an alert about an abnormal logon attempt from a specific IP address, the agent automatically checks whether that IP has been involved in other attacks in the past or is blacklisted in internal TI. Furthermore, if an abnormal network access attempt to a specific VM in a cloud environment is detected by FRIIM CNAPP, it integrates with SIEM logs to ascertain the comprehensive threat situation.

Upon completion of analysis, the agent invokes Seekurity SOAR's API to trigger a response playbook or automatically transmits the necessary parameters for playbook execution. This establishes a foundation for rapid initial response without manual analyst intervention. The following is an example of the agent's workflow for blocking a suspicious IP in conjunction with a Seekurity SOAR playbook:

# Python 코드 예시 (에이전트 스크립트의 일부)
import requests
import json
SOAR_API_URL = "https://soar.seekerslab.com/api/v1/playbooks"
SOAR_API_KEY = "YOUR_SOAR_API_KEY"
def trigger_block_ip_playbook(ip_address, reason):
    headers = {
        "Authorization": f"Bearer {SOAR_API_KEY}",
        "Content-Type": "application/json"
    }
    payload = {
        "playbook_name": "block_malicious_ip",
        "parameters": {
            "ip_to_block": ip_address,
            "block_reason": reason,
            "agent_analysis_id": "{{analysis_id}}" # 에이전트 분석 ID 연동
        }
    }
    try:
        response = requests.post(SOAR_API_URL, headers=headers, json=payload)
        response.raise_for_status() # HTTP 오류 발생 시 예외 발생
        print(f"SOAR 플레이북 'block_malicious_ip' 실행 성공: {response.json()}")
        return True
    except requests.exceptions.RequestException as e:
        print(f"SOAR 플레이북 실행 오류: {e}")
        return False
# LLM이 특정 IP를 차단해야 한다고 판단했을 때 호출
malicious_ip = "192.168.1.100"
analysis_reason = "Repeated failed login attempts from unknown location"
if trigger_block_ip_playbook(malicious_ip, analysis_reason):
    print(f"{malicious_ip} 차단 요청이 SOAR에 성공적으로 전달되었습니다.")
else:
    print(f"{malicious_ip} 차단 요청 실패. 수동 개입이 필요합니다.")

Through such integration, AI agents gain the autonomy to not only provide information but also to undertake 'actions' against actual threats. Naturally, high-priority automated responses were always designed to include a 'Human-in-the-Loop' approval stage, focusing on minimizing potential risks due to malfunction.

Results and Achievements: Enhanced Efficiency and Analyst Empowerment

Following the implementation of LLM-powered AI agents, notable achievements have emerged in SOC operations. Specifically, significant improvements in operational efficiency were observed during the alert classification and initial analysis phases. Quantitative metrics and qualitative improvements are summarized as follows:

Beyond quantitative achievements, qualitative improvements in operational efficiency were also substantial. Analysts can now dedicate more time to advanced tasks such as forensic analysis or threat hunting for actual threats, utilizing the in-depth analysis information and response recommendations provided by the AI agent, rather than engaging in simple, repetitive alert classification or false positive handling. This has led to an overall strengthening of the SOC team's capabilities. Moreover, the alert analysis process handled by the AI agent maintains consistent quality, contributing to reducing discrepancies among analysts and achieving operational standardization. Notably, the establishment of a foundation for rapidly testing and applying new AI-based detection models through KYRA AI Sandbox in the event of new threats or zero-day attacks is also considered a significant achievement.

Lessons Learned and Retrospection: Gradual Adoption and Human-Centric Approach

During the implementation of LLM-powered AI agents, unexpected challenges included the complexity of initial prompt engineering and the need for meticulous control over LLM 'hallucination' phenomena. Merely connecting the LLM to existing systems proved insufficient; continuous prompt optimization and validation of the RAG knowledge base were identified as essential to ensure the agent provides accurate and reliable information. Specifically, the risk of LLMs generating incorrect information, especially concerning sensitive security data, must not be overlooked.

Retrospectively, a stronger application of the 'Human-in-the-Loop AI' principle would have been implemented from the outset. The lesson learned is the importance of designing all automated responses to be limited to alerts of minimal criticality, and actions involving critical judgments or actual system modifications must invariably undergo explicit analyst approval. Furthermore, efforts should focus on increasing the trustworthiness of LLM judgments by transparently documenting the agent's decision-making process and presenting it to analysts. An unexpected ancillary benefit was that the agent implementation process served as an impetus for the systematic organization and documentation of the SOC's internal knowledge base and SOAR playbooks. This has significantly contributed to strengthening the SOC's operational efficiency and knowledge management capabilities in the long term.

Application Guide: A Phased Adoption Roadmap

For organizations seeking to implement LLM-powered AI agents for security monitoring in similar environments, the following phased roadmap is proposed. First, clear goal setting is crucial. Instead of an unrealistic objective to replace all SIEM operations with AI, it is more effective to set specific, measurable goals such as 'reducing false positive rates by 30%' or 'shortening Tier 1 alert triage time by 50%.' Second, data quality assurance is an essential prerequisite, as the consistency of logs flowing into SIEM and rich context determine the agent's analytical accuracy. Seekurity SIEM's log normalization and parsing capabilities, along with the standardization of cloud environment logs through FRIIM CNAPP, will enhance the learning and analysis quality of AI agents.

Third, gradual adoption through pilot projects is recommended. Rather than immediately applying AI agents to all alerts, starting with specific alert types or low-criticality tasks and gradually expanding the scope of application is a method to minimize risks. Fourth, continuous monitoring and optimization are necessary. The performance of AI agents should be periodically evaluated, and prompts and knowledge bases continuously updated to adapt to evolving threat landscapes. Finally, maintaining a 'Human-in-the-Loop AI' through collaboration with security experts is key. Organizations must remember that AI serves as a supplementary tool for analysts, and ultimate judgment and responsibility always rest with humans, focusing on establishing implementation strategies with this principle in mind.