SAST and CodeQL with AI: use cases and limits
Static code analysis finds vulnerabilities before the code runs. In practice, the bottleneck is rarely detection. It is the volume of findings. Large language models (LLMs) target exactly that point: specifications, triage and fix suggestions. This article explains how SAST and CodeQL work, what combining them with AI demonstrably improves, and where licensing, false positives and model risks set limits.
What SAST checks and how CodeQL differs
SAST (Static Application Security Testing) analyses source code or intermediate representations without running the program. Simple tools search for patterns, such as a call to an unsafe function. They cannot tell whether untrusted input ever reaches that point.
Taint tracking, a form of data-flow analysis, closes that gap. It follows values from sources (external input) to sinks (security-relevant operations such as executing SQL). A barrier or sanitiser on the way breaks the flow.
CodeQL works differently from a pattern-matching tool. When it builds the database, it extracts the syntax tree, control flow and type information into one relational database per language. You write queries in QL, a declarative, object-oriented logic language (Avgustinov et al., ECOOP 2016). The same database answers any number of queries.
According to the CodeQL documentation, CodeQL covers C/C++, C#, Go, Java, Kotlin, JavaScript, TypeScript, Python, Ruby, Rust, Swift and GitHub Actions. For compiled languages, extraction needs access to the build. The current CodeQL documentation lists which languages work without one.
Taint analysis by example
The following example is generic and does not come from a client project. The first variant builds the SQL query by string concatenation. The second passes the value as a parameter.
# vulnerable: input ends up in the SQL stringdef get_order(request, cursor): order_id = request.args["id"] # source sql = f"SELECT * FROM orders WHERE id = {order_id}" # concatenation cursor.execute(sql) # sink # fixed: the value is bound, not inserteddef get_order(request, cursor): order_id = request.args["id"] cursor.execute("SELECT * FROM orders WHERE id = %s", (order_id,))The placeholder %s depends on the database driver. Other drivers use ? or named parameters.
In QL, a query describes sources and sinks and lets the library compute the flow. The query below is shortened for illustration. The standard suites already contain this query. Custom queries come into question for internal frameworks whose sources and sinks the standard models do not know.
/** * @name SQL query built from user input (example) * @kind path-problem * @id example/sql-injection */import pythonimport semmle.python.Conceptsimport semmle.python.dataflow.new.DataFlowimport semmle.python.dataflow.new.TaintTrackingimport semmle.python.dataflow.new.RemoteFlowSources module SqlConfig implements DataFlow::ConfigSig { predicate isSource(DataFlow::Node source) { source instanceof RemoteFlowSource } predicate isSink(DataFlow::Node sink) { sink = any(SqlExecution e).getSql() }} module SqlFlow = TaintTracking::Global<SqlConfig>;import SqlFlow::PathGraph from SqlFlow::PathNode source, SqlFlow::PathNode sinkwhere SqlFlow::flowPath(source, sink)select sink.getNode(), source, sink, "SQL query built from $@.", source.getNode(), "user input"Why classic SAST slows teams down
The IRIS study (ICLR 2025) names two causes that also affect CodeQL. Both can be traced to concrete mechanisms.
Missing specifications for libraries
Taint analysis needs models: which method delivers input, which is a sink, which cleans values? For third-party libraries, this information is often missing. The result is false negatives, meaning vulnerabilities without an alert.
Missing context
Even with complete specifications, the analysis lacks an understanding of conditions, validation and intent. That produces false positives. Developers must work through many potentially false findings and lose time doing so.
Limits of the method
Static analysis approximates program behaviour. Computing it exactly is undecidable in general (Rice's theorem). Reflection, dynamic resolution and runtime configuration make flows hard to follow. Business-logic flaws, such as missing authorisation checks, can hardly be expressed as source-to-sink patterns either.
What LLMs change
LLMs do not replace the analysis. They complement it where rules are missing or context is needed. Four points of use are documented.
Inferring specifications
IRIS has a model infer sources and sinks from library APIs and passes them to CodeQL. On the CWE-Bench-Java benchmark, IRIS with GPT-4 found 55 vulnerabilities, CodeQL alone 27.
Assessing alerts
The model receives the code path with its surroundings and judges whether the alert is real.
Suggesting fixes
Copilot Autofix generates fixes for CodeQL alerts. For the beta phase, GitHub cites coverage of more than 90 % of alert types in JavaScript, TypeScript, Java and Python. More than two thirds of findings could be fixed with little or no editing. This is a vendor claim.
Working agentically
Since July 2026, Agentic Autofix has been available as a public preview. It explores the codebase and validates the fix by running CodeQL again. It consumes AI Credits and Actions minutes. The announcement does not say whether tests also run.
How reliable are the figures?
| Statement | Source | Classification |
|---|---|---|
| 55 instead of 27 vulnerabilities found on CWE-Bench-Java | IRIS, ICLR 2025 | peer-reviewed |
| False discovery rate 84.82 %, 5.21 percentage points below CodeQL | IRIS, ICLR 2025 | peer-reviewed |
| More than 90 % of alert types, about two thirds with little or no editing | GitHub Blog | vendor claim |
| Better false-positive detection using memory of earlier alerts (Memoir) | arXiv preprint | preprint |
| 91 % fewer false positives (225 to 20) compared with Semgrep alone (SAST-Genius) | arXiv preprint | preprint |
The IRIS results come from a Java benchmark with known vulnerabilities. They say nothing about arbitrary enterprise applications. The preprints have not been independently verified.
Use in the pipeline
CodeQL typically runs in CI and reports results in SARIF format as code scanning alerts. On pull requests, findings appear directly on the diff. The workflow below is a generic example.
name: codeqlon: pull_request: push: branches: [main] schedule: - cron: "17 3 * * 1" # weekly full run jobs: analyze: runs-on: ubuntu-latest permissions: contents: read security-events: write strategy: matrix: language: [python, javascript-typescript] steps: - uses: actions/checkout@v7 - uses: github/codeql-action/init@v4 with: languages: ${{ matrix.language }} queries: security-extended - uses: github/codeql-action/analyze@v4 with: category: "/language:${{ matrix.language }}"Three settings shape the result. The query suite sets how broadly the analysis searches: default, security-extended or security-and-quality. The category per language keeps results separate. The trigger decides whether findings surface before the merge or only afterwards.
Where AI sits in the pipeline
Two architectures are conceivable. The comparison below is an architecture example, not a measurement.
| Aspect | Vendor feature (Autofix) | Own triage stage over SARIF |
|---|---|---|
| Design | Integrated in GitHub, triggered on the alert | Separate job reads SARIF, assesses, writes a comment or label |
| Model choice | Set by the vendor | Free, a self-hosted model is possible |
| Code leaving the environment | Code goes to the vendor's service | Depends on the chosen model and its operation |
| Maintenance | With the vendor | Prompts, thresholds and evaluation are run in-house |
| Cost | Licence; Agentic Autofix additionally AI Credits and Actions minutes | Model operation and pipeline run time |
Limits
Licence
GitHub's CodeQL terms allow analysis of open-source codebases under an OSI-approved licence. If such code is hosted on GitHub.com, creating databases in CI and CD is also permitted. For other codebases, such as private repositories, these rights do not apply. There, a paid GitHub Code Security or GitHub Advanced Security licence applies.
Detection limits
| Cause | Effect | Does an LLM help? |
|---|---|---|
| Missing models for libraries | False negatives | Partly, through inferred sources and sinks (IRIS) |
| Missing context in the data flow | False positives | Partly, through triage with surrounding code |
| Missing authorisation, logic flaws | Hard to express as source-to-sink patterns | Not demonstrated |
| Dynamic behaviour at runtime | Flows cannot be followed completely | Not demonstrated |
False positives remain
Even in the best result of the IRIS study, the false discovery rate is 84.82 %. Most reported findings were still wrong. The gain lies in the distance to CodeQL alone, not in a result free of false alarms.
Risks of using LLMs
- Non-determinism: The same input can lead to different verdicts. Triage results are therefore only partly reproducible.
- Manipulation through code: Comments and strings in the analysed code reach the prompt. A model can be influenced by them.
- Code leaving the environment: With cloud models, code leaves your own environment. Contract terms and data protection questions are for your data protection and legal advisers.
- Faulty fixes: A vanished alert does not prove that behaviour was preserved. That takes tests and review.
- Cost model: Agentic runs consume credits and pipeline minutes. Costs grow with the number of alerts assigned.
Operations
Database creation and run time grow with the codebase. Depending on the language and build mode, compiled languages need a working build in CI. Alerts need a process: who assesses them, who closes them and on what grounds, and how long a finding may stay open.
Summary
SAST remains a method with clear limits. LLMs move these limits at two points: missing specifications and triage. The figures come mainly from benchmarks and vendors. How well the combination works on a particular codebase only a run on that code will show.
Sources
- GitHub Blog: Found means fixed: Introducing code scanning autofix, powered by GitHub Copilot and CodeQL (opens in a new tab)
- GitHub Changelog: Agentic autofix for code scanning alerts in public preview (opens in a new tab), July 2026
- Microsoft DevOps Blog: Copilot Autofix for GitHub Advanced Security for Azure DevOps (opens in a new tab), 11 June 2026
- CodeQL documentation: Supported languages and frameworks (opens in a new tab)
- GitHub: GitHub CodeQL Terms and Conditions (opens in a new tab)
- Li et al.: IRIS: LLM-Assisted Static Analysis for Detecting Security Vulnerabilities (opens in a new tab), ICLR 2025
- arXiv: Memoir: Learning, Verifying, and Evolving False-Positive Memories for Static Application Security Testing Tools (opens in a new tab) (preprint)
- arXiv: LLM-Driven SAST-Genius: A Hybrid Static Analysis Framework for Comprehensive and Actionable Security (opens in a new tab) (preprint)
- Avgustinov, de Moor, Jones, Schäfer: QL: Object-oriented Queries on Relational Data (opens in a new tab). ECOOP 2016, Schloss Dagstuhl.
- Rice, H. G.: Classes of Recursively Enumerable Sets and Their Decision Problems (opens in a new tab). Transactions of the American Mathematical Society, 1953.
Security review at Seovantis
Seovantis carries out security reviews when they are agreed in the technical specification. How the review before handover works is described on the working method page.
A similar project?
Briefly describe your situation. I will get back to you within one business day.