October | 2026 | Smarter Forensics

When I started in digital forensics in 2002 (I know, I just dated myself), I was told my job was to find the needle in the haystack. AI can throw a lot more hay on that pile, and if you don’t have a strong foundation as an examiner, you may never find the needle.

I said that while moderating the AI panel on day two of the SANS DFIR Summit & Training 2026 in Arlington, with three panelists I knew wouldn’t agree on everything. Ovie Carroll is Director of the DOJ Cybercrime Lab and has taught for SANS since 2009 (he was clear his comments were his own, not his employer’s). Ovie is also co-author of FOR500: Windows Forensic Analysis. Mari DeGrazia is Director at IDX Cyber Response, a Principal Instructor at SANS Institute and the author of FOR563: Applied AI for Digital Forensics and Incident Response. Raymond DePalma is authoring FOR581: Investigating and Responding to Compromised AI Systems, a new SANS course on what to do when the AI itself is the victim.

Lee Crognale, our Curriculum Lead for DFIR, warmed up the room with a few polls, including whose boss has banned AI at work and whose boss requires it. Panels aren’t my favorite format because the best answers get rushed at the end, so I started with closing thoughts first.

Never trust, always verify

Mari went first. Our industry has always said “trust but verify,” and she thinks AI calls for “never trust, always verify.” Ovie was just as blunt: AI will fail, because all tools fail, and it will hallucinate, lie to you, guess at what you want, and try to please you. Everyone agreed on that much. Where they split was on what verifying looks like, and whether most of us do it at all.

Ovie, who has 30 years in this field, doesn’t think most examiners validate the tools they already use. They push the button and move on, often because leadership doesn’t understand forensics well enough to give them time to do more. He also said there has never been such a thing as a full forensic analysis. Hand your best case to the weakest analyst in your office and they’ll find an artifact you missed, because nobody has enough time.

Ovie worries AI becomes the super easy button. If we aren’t validating now, he asked, what makes us think AI will suddenly make us start? And if it hands you a mountain of output to check, are you really saving time?

He also has a rule of five: the first time you use AI you’re seeing what it’s about, the second time it’s not bad, the third time it’s wow, the fourth time you’re a genius, and from the fifth time on you’ve stopped reading what it puts out. Last month he tried to stage an intervention for a friend of 30 years who now sends him AI-written emails.

Ray agreed and disagreed, which he said was to make it spicy. Training is fundamental, he said, because the judgment layer is the one thing we still have that AI doesn’t. But when a model hallucinates, he wants you to ask why. Did it run out of context window, or miss the right knowledge base? Could a script do that step the same way every time? A planned workflow that calls deterministic tools, with an evaluation harness testing the results, holds up better than a one-shot prompt with no examples and no stopping point.

Treat AI like an intern, at best

Mari’s answer to the validation problem was one of the more practical things I heard at the Summit. She treats AI like an intern. It takes a first pass at the data while she works on other outputs, flags four or five things worth a look, and she validates those. You wouldn’t let an intern hand you a final answer on an important case without checking it, especially with your name going on the report. Use your AI where you would an intern. Keep in mind that not all cases are the same and not all niches of DFIR are the same. Act wisely!

Mari also doesn’t ask AI to do a tool’s job. Her agent doesn’t convert timestamps; it calls a conversion tool she has already validated, so the only thing left to check is whether it picked the right tool. Ray learned the same lesson decoding web shells. The AI’s output looked convincing until CyberChef proved it wrong, and now his workflow has the AI call CyberChef instead. Without that check, he said, he could have handed a client incorrect evidence.

That only works because you were trained on those tools yourself. Ovie made the same argument from the other side: without a deep understanding of the artifacts, an analyst can’t tell when AI missed something or got it wrong. He named where that depth comes from: FOR500 and FOR508 for Windows, FOR585 for smartphones, FOR577 for Linux, and FOR518 for Mac. Yes, we’re a training company and those are our courses. His point stands anyway. If you don’t know what the artifact should look like, you won’t catch the mistake.

Then Ovie put me on the spot and asked how often, in my last multi-murder investigation, I got that gut feeling that something in the data wasn’t right. The honest answer is daily. AI can follow rules, but it doesn’t have the instinct that sends you back to look again.

In court, the artifact is what matters

One audience question asked whether non-experts can testify about AI findings. Ovie, reminding us he isn’t a lawyer, said no. Testimony rests on reliability under the Daubert standard, and he pointed to the debate over proposed Federal Rule of Evidence 707, which would hold machine-generated evidence offered without an expert to the same bar as expert testimony. It’s still a proposal; the federal Advisory Committee on Evidence Rules decided this year to gather more input first.

Ray and Ovie then did a decent chatbot impression (“I apologize for the confusion. I made an error in my previous response.”) to make a serious point. That exchange is discoverable, and so are your prompts, the model, how it was trained, and every mistake it made. Ovie added that models will change an answer just to agree with you. He quoted a friend who told him credibility is your most valuable commodity, so spend it wisely. One unchecked AI report in court can burn yours and your agency’s.

Mari asked the follow-up I suspect a lot of the room was thinking: what if AI is just a brainstorming partner and you find everything again independently in your forensic tools? Ovie’s answer applies with or without AI. Your report and testimony should be about the artifact itself, the SQLite database or the log record, and tools belong on the last page of the report. Use AI to get to the artifacts faster, like when you’re facing a unified audit log with 20 million records, then validate and write about what you found. I was glad to hear it, since my keynote on the Idaho case the day before showed only the artifacts.

Ovie also borrowed a line from Eric Zimmerman for tool developers: when your tool fails, fail spectacularly, not silently in an audit log nobody looks at. A recent tool with AI built on top told his team five times, with high confidence, that the data they wanted wasn’t there. It was sitting in an unencrypted SQLite database the tool just didn’t parse. Ray’s advice is to question AI providers the way we question every other vendor, and he pointed to the EDR Telemetry Project, which shows what endpoint products collect and where the gaps are.

Local or cloud: pick the right vehicle

Mari admits she’s biased toward local models, since that’s what FOR563 teaches. Her argument is control. About a year ago a major provider pulled a model overnight, and anyone who had tested its tool calling and built a workflow around it lost that workflow the same night. A model you’ve downloaded stays on your drive and your network, and your evidence never goes to the cloud, which matters for a lot of law enforcement agencies.

Frontier models are smarter, and she uses them too. But you don’t always need a Porsche when a minivan will get you to the grocery store, and a small local model can often do the job without burning money and tokens. Ray favors a hybrid approach: decide what you’re trying to accomplish, keep evidence separated from anything going to a provider, and know your tools’ limits. Mari is also planning a second day for FOR563 that brings in MCP servers and community-built agents. Ray’s course is more focused on when an agent gets hijacked or a RAG pipeline gets poisoned, some of the evidence sits on the host. The rest lives in prompt histories, tool-call logs, and agent memory that most responders were never trained to read. FOR581 will teach responders to collect that evidence soundly, rebuild the attack, and use AI to amplify their expertise, not replace it.

When AI is the one doing the work

Ovie has published two research papers on AI’s footprint of digital activity and one on AI generated Email via autonomous agents that are worth your time. In one, he had AI find a Word document on his computer, open it, add an executive summary, save it, and leave it open for him. Then he looked for the artifacts we use to prove a user knew about a file and found no LNK file, no Jump List entry, and nothing in RecentDocs, OpenSaveMRU, or the Office File MRU. He admitted it freaked him out. His other test had AI answering email on its own, and he sees “I didn’t do it, my AI did” as where defenses are headed.

That’s the problem the FOR500 authors are working through. Ovie describes the course as proving what happened on a PC and, where possible, who was responsible, and now “who” might be an AI agent. The current run doesn’t really include AI yet, and he’s against throwing the topic du jour into a course just because it’s popular. The question is what an examiner needs to prove when an agent took the action.

In FOR585, we start with AI used for harm, like AI-generated CSAM and sextortion content, and how to detect it. Then we use AI for good. If a query or parsing script broke because Facebook Messenger updated, you already know the app, the database, and which table matters, so you ask AI to help get that table into a CSV without handing it a single piece of evidence.

Ray is developing material on the flip side: investigating AI when it’s the victim, which calls for different procedures than most of us trained on. For prompt injection, he looks for instructions to disregard the system prompt, earlier prompts, or agent memory, and pulls AI gateway logs, prompt and completion logs, and MCP configurations. Then he baselines normal behavior so outliers stand out. That part is hard, because a model can’t tell the difference between me saying “disregard my colleague’s comments” and an attacker saying “disregard all prior instructions.”

Our frameworks, the hackathon, and AI in SANS courses

At RSAC this year, I stepped on a landmine by saying our greatest threat in DFIR is the improper or irresponsible use of AI. Our PR team then told me I had to offer suggestions for responsible use, and I did not want to. I was that far on the “no” side. Eventually I softened a little, and that’s how we ended up with two frameworks, one for digital forensics mapped to the trusted SWGDE best practices and one for incident response mapped to the NIST Cybersecurity Framework 2.0. They cover how to apply AI safely at each step, like triage, and how to validate what it gives you. We didn’t team with SWGDE and NIST on these frameworks; we simply used their suggested steps for handing incident response and digital forensic cases.

Phase two is a community effort with 13 volunteers from across the field. So far it’s all about AI helping a trained examiner, and none of it covers AI as the victim yet. I’ve also been asked to mark which uses are for experienced examiners only, which is fair. These frameworks are dynamic and will change. We are working to adjust the IR +AI framework we map to the updated DAIR method, by Joshua Wright and other brilliant contributors (Megan Roddie, Ryan Chapman, and Dean Parsons).

At the social hour the night before, an attendee asked what SANS is doing about AI in our courses and whether the tools from our Find Evil! hackathon are coming into class. On the hackathon, not yet. The winners will present their tools at the SANS AI Cybersecurity Summit on November 2 and 3 in Arlington and Live Online, but the tools won’t drop into our course VMs as they are. We’re deciding whether to adapt them, have the winners hand them over for integration into our VMs, or build a short online course of our own in the new CyberStudio by SANS. Meanwhile, FOR563 is available today, FOR585 already teaches AI-assisted scripting, the FOR500 have AI in our day 6 CTF and are deciding what to add in our next update, and Ray’s material is in development.

Be honest about using it

The last thing we talked about is the one I care about most. I want AI usage labels on training content, like a nutrition label, right under the course title on the SANS website. “The author used AI to build the SQLite database in this lab” is fine. Just flat out say it. I borrowed the idea from two speakers at DFIR Summit Europe in Prague, and I’m glad I did.

Ovie already runs his team this way. He tells his staff to lean into AI, and the first thing they tell him when they bring him something should be how they used it. Just before the Summit, he shared a Python script with me and Mattia that organizes output from the Zimmerman tools and a few others the way he likes to see it, and when someone complimented it, he told them up front that AI wrote it from his examples and he validated it. When tested, this will be shared!

When it comes to AI, I think a lot of people are faking it right now, either about how much they know or about how much they use it. We all just need to be honest.

Key takeaways for examiners

Treat AI like an intern. Let it take the first pass at triage, timelines, and big data sets, and validate every finding before you trust it. Remember, the case type matters here. If you wouldn’t let the intern do it, you better gut check why AI should.

Give AI validated tools instead of asking it to do the work itself. If your agent calls a timestamp converter or CyberChef that you’ve already tested, the thing left to check is whether it called the right tool.

Write and testify about the artifact itself and save tool names for the last page. Assume your prompts, the model, its outputs, and every correction made are discoverable.

When an AI agent did the work, the usual file-knowledge artifacts may not be there. Ovie’s test turned up no LNK file, Jump List entry, or MRU record for a document AI opened, edited, and saved.

Ask vendors, AI vendors included, what their tools don’t parse. A tool can tell you with high confidence that something isn’t there when it is. Vendors and AI must do better when their AI tool relies upon data parsed by another tool.

Key takeaways for team leads and managers

Budget time for validation. If AI produces more output than your team can check, it hasn’t saved you anything.

Plan for Ovie’s rule of five. Somewhere around the fifth use, people stop reading what AI gives them, so build review into the workflow before that happens.

Deadlines are when unchecked AI work leaves the building: billable hours, a client waiting, court tomorrow, and an intern’s report that sounds really convincing. One AI-written report that falls apart in court can cost your organization its credibility.

Make it normal for your team to say up front when and how they used AI.

Ovie’s view is that non-technical leaders can’t lead technical teams through this. You don’t have to be the smartest person in the lab, but you do have to be technical.

Key takeaways if you’re just getting started

Start where you already know the right answer. Mari suggests picking an artifact, researching it, and having AI help write a parser, since you’ll know right away whether the output is correct.

Don’t give AI evidence while you’re learning. In FOR585 we describe the app, the database, and the table that matters, and ask for help with the query or script.

Have AI write a prompt for your task, then edit it line by line. Ovie says that’s how he found out how horrible he is, as a human, at prompting.

Be careful with skills from public repos, which Ray warned can be compromised. Ovie has AI read one and write a new skill that’s entirely his own.

Learn each provider’s tools on their own terms. Ray recommends Anthropic Academy, and his open-source AI for the Win labs walk from Python and Colab notebooks through prompting.