Graph-Augmented LLMs for Scalable Malware Detection: Integrating GFT and RAG in Portable Executable Malware Analysis

Authors

• Matthew Jackson, University of Nevada, Las VegasFollow

Files

Description

The proliferation of generative AI has exponentially accelerated the generation of novel malicious software. Las Vegas’ gaming industry is a high-value target for these attacks. In 2023, a massive ransomware incapacitated gaming operations, leading to over $100 million dollars of damages and Nevada state services disrupted by cyberattacks in 2025, underscores the critical necessity for capable automated detection frameworks. Traditionally, malware analysis relies on manual, resource-intensive signature extraction by experienced analysts. While Large Language Models (LLMs) demonstrate utility in source code analysis, applying them to compiled Portable Executable (PE) Windows files is bottlenecked by LLM context window limitations and difficulty in analyzing full reverse-engineered assembly instruction sequences. To resolve this, we implemented a fully automated reverse engineering tool for PE Windows files to extract CFGs, assembly instruction, and Pseudo C code from these compact binaries. We represent compiled malware as Control Flow Graphs (CFGs), encoding basic assembly blocks as nodes and the transition of blocks as edges. For detection we use a Graph Foundation Model (GFT) utilizing a Transferable Tree Vocabulary to process these large-scale topologies. GFT structurally encodes the CFG, isolating high-value malicious subgraphs, pinpointing only important areas of the code for LLM detection. This condensed structural representation interfaces directly with LLMs for precise classification. By employing Retrieval-Augmented Generation (RAG) to anchor the system with known malicious structural patterns and relevant areas of code, this methodology equips researchers with a highly scalable, automated, and context-aware framework for analyzing and detecting complex malware variants.

Publisher Location

Las Vegas (Nev.)

Publication Date

Spring 4-17-2026

Publisher

University of Nevada, Las Vegas

Language

English

Keywords

Malware Analysis; Large Language Models (LLMs); Retrieval-Augmented Generation (RAG); Graph Analysis; Reverse Engineering

Disciplines

Artificial Intelligence and Robotics | Computer Sciences | Gaming and Casino Operations Management

File Format

PDF

File Size

451 KB

Comments

Mentor: Junggab Son

Recommended Citation

Jackson, Matthew, "Graph-Augmented LLMs for Scalable Malware Detection: Integrating GFT and RAG in Portable Executable Malware Analysis" (2026). Undergraduate Research Symposium Podium Presentations. 79.
https://oasis.library.unlv.edu/durep_podium/79

Rights

IN COPYRIGHT. For more information about this rights statement, please visit http://rightsstatements.org/vocab/InC/1.0/