---
title: "How Agentic AI Fails—and Which Controls Actually Stop It"
url: https://appsecpodcast.com/how-agentic-ai-fails-and-which-controls-actually-stop-it/
date: 2026-09-15
duration_seconds: 2193
season: 13
episode: 12
guests: ["Petra Vukmirovic"]
topics: ["Threat Modeling", "Security Testing", "AI and LLM Security", "Privacy and Compliance"]
audio: https://www.buzzsprout.com/1730684/episodes/19804673-how-agentic-ai-fails-and-which-controls-actually-stop-it.mp3
video: https://www.youtube.com/watch?v=_kiXMlgLX6E
transcript: true
---

# How Agentic AI Fails—and Which Controls Actually Stop It

*September 15, 2026 · 37 min · Season 13, episode 12*

with [Petra Vukmirovic](https://appsecpodcast.com/guests/petra-vukmirovic/)

on [Threat Modeling](https://appsecpodcast.com/topics/threat-modeling/), [Security Testing](https://appsecpodcast.com/topics/security-testing/), [AI and LLM Security](https://appsecpodcast.com/topics/ai-security/), [Privacy and Compliance](https://appsecpodcast.com/topics/privacy-and-compliance/)

[Audio](https://www.buzzsprout.com/1730684/episodes/19804673-how-agentic-ai-fails-and-which-controls-actually-stop-it.mp3) · [Video](https://www.youtube.com/watch?v=_kiXMlgLX6E)

## Show notes

Most fault trees get built on gut feeling. Petra Vukmirovic did something rarer: she borrowed the actual math from aviation and nuclear-plant safety engineering and pointed it at AI agents. Petra traded emergency medicine for application security and now heads information security at Numan — and she joins Chris Romeo and Robert Hurlbut to make the case for fault tree analysis (FTA), the deductive method that picks up exactly where threat modeling stops. Petra walks through a "wrong customer refund" AI agent scenario step by step, showing how AND/OR gates and minimal cut sets turn vague worry into ranked, data backed probabilities. They dig into where AI helps build a tree, and where garbage in, garbage out still applies, why "comprehensive test coverage" is a myth, and how attaching real dollar figures to failure paths makes it easier to sell security controls to leadership.

This episode is sponsored by [Corgea](https://appsecpodcast.com/corgea). Design it. Build it. Ship it. Corgea secures it.

About Corgea  
Corgea is an AI-native application security platform that secures software from design to production. It brings together security design reviews, AI SAST, dependency and IaC scanning, code quality checks, and autonomous pentesting—helping security and engineering teams find risk earlier, fix what matters, and ship securely.  
→ [Learn more about Corgea](https://appsecpodcast.com/corgea)

Connect with Petra Vukmirovic:  
→ [Petra Vukmirovic on LinkedIn](https://www.linkedin.com/in/petravukmirovic/)  
→ [OWASP Threat Model Library](https://owasp.org/www-project-threat-model-library/)

Mentioned in this episode:  
→ [Adam Shostack: "Stop Trying to 'Manage Risk'" (keynote)](https://www.youtube.com/watch?v=THSfJIlPGPk)  
→ [OWASP Global AppSec USA 2026](https://owasp.glueup.com/event/owasp-global-appsec-usa-2026-167174/) (San Francisco, Nov 5–6)

Chapters:  
00:00 Cold open — the math behind where to put your controls  
01:09 Meet Petra Vukmirovic  
01:28 Petra's origin story: from ER doctor to AppSec  
02:50 Career path: engineer to Head of InfoSec at Numan  
04:18 What is fault tree analysis, and where threat modeling ends  
06:22 Can AI actually do fault tree analysis?  
08:12 Walking the "wrong customer refund" agent example  
12:33 Storing your trees: JSON vs. Markdown  
16:08 Why conjunctive failures trip up narrow thinking  
17:27 Top 3 failure modes when agents touch downstream systems  
19:29 Real story: an agent pushed code to main without approval  
21:27 Testing: why "comprehensive coverage" is a myth  
23:54 How rough is rough? Assigning probabilities  
28:08 Getting started without a six week science project  
31:35 Using FTA to sell controls and build credibility  
33:43 The epiphany: FTA is about controls, not faults  
34:48 The one thing every agentic team should add today  
35:48 Closing thoughts and OWASP Global AppSec USA preview

## Transcript

*6,658 words · assemblyai*

**0:00 Petra Vukmirovic:** And that's where you figure out that, you know, instead of being led by feeling and you think, oh, you know what, this could lead to a disaster, you realize, yes, it could lead to a disaster, but its probability is super low. So, I'm not going to invest my controls there. I'm actually going to invest my controls and my efforts into testing into where I have or probabilities and where I have higher actual probabilities of failure. And where if I introduce a control, it will actually break that path, that failure path.

**0:31 Chris Romeo:** Petra is a technology leader and former emergency medicine doctor who transitioned into cybersecurity. With expertise in threat modeling, AI, and security engineering, she has built and led global teams, developed patented malware detection technology, and driven security innovation at companies including Job and Talent and DevArmor. She currently serves as Head of Information Security at Newman. Welcome to the Application Security Podcast. And with us, we have Petra Vukmirovic. And certainly, we're honored to have you here, Petra. And we always like to start right off the bat with this simple question. If your security career were a comic book, what would I find in episode 1?

**1:13 Petra Vukmirovic:** I think, because in my past life, I was an emergency medicine doctor. Probably the first episode would be, you know, me in one day being in the emergency room department thinking what can go wrong, and then the next day doing a threat model, also thinking what can go wrong. So, that's kind of the scene that you would probably have to see in the comic book.

**1:35 Chris Romeo:** We've always said that threat modeling applies to everything. Yeah. It's every job role. And so, how'd you get from being a medical doctor. This is the first time I've heard this story of somebody went from being a medical doctor to being in cybersecurity. So, I must know, what was the path that took you down there?

**1:55 Petra Vukmirovic:** Well, it was definitely the obsession of always trying to figure out what can go wrong. But jokes aside, I was always into tech, and I felt like medicine wasn't providing enough room for creativity to me. You know, obviously you can tamper with computers, but you can't really tamper with humans. So, I decided to go to tech, which was my initial kind of first love, went back to uni, did a cyber degree, and was lucky enough to get an internship. So, here I am.

**2:29** Okay.

**2:31 Chris Romeo:** And you've been, what type of roles have you had in your background in cyber?

**2:35 Petra Vukmirovic:** Yeah, I started as an intern and then cybersecurity engineer, quite generalist, capturing everything from AppSec, cloud sec to SecOps. And similarly, I went to senior engineer, and then there I jumped into management. So, now I'm head of InfoSec at Newman. I held director of cyber engineering roles. So, the, you know, my path was from engineering to management.

**3:02 Chris Romeo:** Okay. Well, I mean, you have hit a first. We've been doing this podcast for over 10 years and over 300 episodes, and this is the first time somebody said, I was a medical doctor and now I'm in cybersecurity. I love it. I love it. I love it. I'm going to probably tease that out as we go through here because I'm curious about some of the similarities there as well. This episode's sponsored by Corgia. Design it, build it, ship it. Corgia secures it. Corgia is an AI-native app security platform covering your whole software lifecycle from the first architecture diagram to code in production. It brings design reviews, AI-powered code scanning, and autonomous pen testing into one platform so your team spends less time chasing noise and more time fixing what matters. Visit corgea.com. C-O-R-G-E-A.com. Corgea, one security platform for the entire SDLC. But Robert, why don't you take us into the primary topic that we've got for today?

**4:05** Sure, sure. Yeah, so, Petra, we're talking about fault tree analysis. So, what is fault tree analysis, and where does it pick up once the threat model ends?

**4:14 Petra Vukmirovic:** So, fault tree analysis is kind of like a deductive method. And again, probably the reason why I went into it is because I had like a little rabbit hole I went into last year after I saw Adam Schoszak's keynote on risk and him challenging us managing risk. So, I was thinking, so, what are the alternatives? Are there any better kind of data-driven methods? And I stumbled into safety engineering in aviation or, you know, in big nuclear plants and in big industries. They do like to prevent, you know, big accidents that can kill a lot of people. So, they do this exercise and it's essentially a root cause analysis to get you the most important failure paths and to get you the probabilities so that you can actually focus on introducing the most impactful controls. And I was thinking, why not introduce that in security? Why are we not doing this? And I realized we are, but in a way that we're calling it attack trees. So, there is a slight difference between attack trees and fault tree analysis, but it's only in this AND/OR gating system. And to answer the question of where does the threat model stop and where does FTA begin, there is a small overlap. essentially, you do a threat model and you go broad and you find all the threats. And then you think about which threat is your biggest one that you want to prevent. What worries you at night out of all of these? And you take that one and then you can break it down into the fault analysis tree. And that helps you find not only, you know, what controls to focus on, what tests to focus on, but also some data-driven probabilities that you can report back to management and leadership. And trust me, when they see these graphs and, you know, data-driven probabilities, they really love it. So, it really helps you push the case, especially if you need to introduce some more costly controls.

**6:08 Chris Romeo:** So, before we get to this agent example that you're going to walk us through, I'm just curious, is fault tree analysis something that AI can do now? Because when we think about threat modeling, there's a lot, and Robert's teaching a class on this, like there's a lot of movement to say AI's actually pretty good at threat modeling if it's long as you scope it and set it right and give it the right constraints. Like, can I go to Claude or to Codex right now and say, do a fault tree analysis for me of this threat?

**6:37 Petra Vukmirovic:** Yeah, I think I've tried to do it with AI a couple of times. My first few attempts were whiteboarding, but then I went to AI because it becomes, you know, time extensive. So, absolutely, you can use AI to do it. And it's kind of like what you said, you know, garbage in, garbage out. You need to make sure you give it the right information in the right structure with the right constraints and the right guidelines. And you also need to make sure to leverage reusable elements because when you build a tree, you get all the way down to your bottom leaves and minimal cut sets. And those are actually reusable. So once you kind of build them once, you can try and make it into a system where you can reuse it. So then you have a little bit of elements of deterministic plus the AI reasoning.

**7:26 Chris Romeo:** And AI loves to write MD files these days, I've found. So it sounds like if I told it to store the artifacts from this, I'm going to get an MD file that has some of that context, but I think that would work in the short term. I mean, things will get better in the future, but for now.

**7:41 Petra Vukmirovic:** Yeah.

**7:42** Yeah.

**7:42 Petra Vukmirovic:** I, I stored it in JSON. I'm, I'm just a big fan of JSON. So I stored it in JSON and then I had multiple JSONs for all the cut sets, But absolutely, you can store it in Markdown too. When I did it with Claude, you're absolutely right. He was coming back with a lot of Markdown files.

**7:57 Chris Romeo:** So, let's walk through an example of this. And I know you've got a working example of this idea of a refund agent. And how does this idea of wrong customer refund become a tree? Kind of walk us through an example, because I feel like I've got an idea now for what fault tree analysis is at a high level, but I don't have the connection for how it kind of plays out in a real-world example.

**8:19 Petra Vukmirovic:** Yeah, a lot of times. So when you think about how you go, you know, with a fault tree, you start from your top event. What is what you want to avoid? And I think refund is probably not a catastrophe you want to avoid, but it's maybe one of the things that you kind of lose money on. So it's a good example. You have to always think about, okay, what, what are the things that need to be true for this to happen? And then you decide whether it's, do all these things need to be true all at once, or either one of them can be true? So, all of these things, if they can be true at, you know, they have to be true at once, then usually you have to think about, okay, well, is there a trigger? Is there a capability and maybe a lack of control? So, in your customer refund example, so what needs to be true for the refund to be given to the wrong customer? Well, you need somehow the agent to have the, you know, get the customer name wrong and the agent needs to have a capability that allows it to do that refund. And then there has to be a lack of controls, like some kind of deterministic checking at the background that allows that to happen as well. So those are maybe your top 3 elements and that's where you have your AND gate, right? And then when you go into each one of these, you kind of start going down the rabbit hole. So how can an agent get the wrong customer name? Well, many things can happen, right? It could be due to some kind of prompt injection. It could be someone sending a malicious ticket. It could even be someone from the inside sending, let's say, a request from the inside or creating, you know, some kind of, you know, inaccurate transaction. So if we dig down, let's say, you know, a malicious insider, what does need to be true for that to happen? Well, either someone's machine needs to be compromised or, you know, someone has to be compromised themselves, like they have to be malicious. And then if you think about, oh, well, if someone's machine is compromised, how does that happen? Well, I have my EDR and my antivirus. Will they have to click on a link? my antivirus has to fail and the malware has to execute. So that's where you kind of probably are quite far enough in the rabbit hole where you decide, well, I've reached my minimal, what we call minimal cut set. And you reach a point where you can actually reason very well about the probabilities and, you know, do testing as well. Probably the tests are good in some other situations where the probabilities are difficult to reason about, then you can choose to do the test instead. Like, for example, you can inject faults into systems, like do a bit of chaos engineering. But in your example, you know, EDR is a really good example. Like, for it to fail, it has— a lot of EDRs publish their kind of failure rates, right? You can trust them, but that's your rough probability that you can derive from that. And then once you have all your minimum cut sets, when you go all the way down to the rabbit hole in all your elements, then you can actually bubble up those probabilities.

**11:29 Chris Romeo:** Hmm.

**11:29 Petra Vukmirovic:** So, what happens is you have a mathematical formula that is publicly available for FTA. And essentially you realize that if there is a failure that has to happen and has to be multiple elements that need to be true at once, the probability suddenly decreases drastically. And that's where you figure out that, you know, instead of being led by feeling and you think, Oh, you know what? This could lead to a disaster. You realize, yes, it could lead to a disaster, but its probability is super low. So, I'm not going to invest my controls there. I'm actually going to invest my controls and my efforts into testing into where I have or probabilities and where I have higher actual probabilities of failure and where if I introduce a control, it will actually break that path. that failure path.

**12:19 Chris Romeo:** So, when, where do you store all the things you just said while you're going through the process? Because obviously you've done this a bunch of times before. And like, when I think about the example, Robert and I have done a lot of threat modeling over our multi-decade careers. And so, when I'm threat modeling, I don't write anything down because I've kind of got the playbook in my head. But like, if I was, if I'm somebody who's new to doing fault tree analysis, like, is there a, is there like a spec? template, or is there something that I can kind of go through what you just did, but I don't know what the next question's gonna be. So, I was like, is there a template that exists? Let's, like, it's a standard or something that, that you recommend?

**12:56 Petra Vukmirovic:** So, I, I actually tried to find if there's software for this, 'cause I was thinking surely it's done in safety engineering quite often. I found some really, really old school software for this that was like, you know, looked like it was from the '90s and You know, you can store it in JSON. That's how I did it. So my first kind of iteration was doing it on a whiteboard and then going for the storing the probabilities and the cutsets, I went to JSON.

**13:24** Okay.

**13:25 Petra Vukmirovic:** The other option is you can just, you know, store it, like you said, in Markdown. You know, ultimately this is all structured data, so you can decide how you want to structure it. You can make it into Markdown to be a bit more free text. You can put it in JSON and then try to put some Python scripts and automation on it to get like deterministic probability values. I'm trying to build a skill for it as well. And I've been trying to create, because I'm going to talk about this in OWASP San Francisco, and I'm trying to, before I do that, I'm going to try to create a UI for that kind of You know, with Claude, just a simple UI where you can actually do the fault tree analysis and then store your findings somewhere. So, and you can also store it in your JSONs in a repo, you know, just to have like a tree, because especially you can reuse a lot of them, those minimal cut sets, which are your, you know, bottom few leaves. And you can keep reusing that as you go along.

**14:30 Chris Romeo:** And you can reuse those because like in your EDR example, one of those cut sets is the data from CrowdStrike or whoever about how trustworthy or how, what's their failure rate. And so you basically are feeding that in and those are things that you'd want to refresh at some interval because they probably, they probably come out with new numbers probably all the time now.

**14:51 Petra Vukmirovic:** But now, yes.

**14:52 Chris Romeo:** But now, but yeah, so like, so those things do have to be refreshed, but you're saying I can bring some of those statistics at the bottom layer and they can feed up through everything. And then at some interval I refresh the, those baseline things to ensure that my probabilities are feeding correctly based on the current state of threat intelligence, for example.

**15:12 Petra Vukmirovic:** Yeah. And a lot of those lower cuts is I didn't just get from, you know, stats from the industry, you get from logs. Like, for example, if you think about application attacks, you can look at, you know, how many attacks have got hit your WAF, have any of them been successful, and then you can you know, add those probabilities based on logs. You can look at your outages. If you're just trying to figure out the probability of an outage, you can see if there's like the minor outages, you can probably, you know, work with your SRE team to determine how many outages of single things. So yeah, all of this needs to be refreshed. You can probably decide your own interval rate at which you want to do those refreshings.

**15:53** So there's also this concept of catastrophes are conjunctive. So capability and trigger and missing control. What is that catch that finding-by-finding thinking misses?

**16:06 Petra Vukmirovic:** Yeah, it's a good question because a lot of times when you think about, you know, when you do a threat model or it happens a lot in compliance, you know, individual non-compliances get raised as risks, but they're not really risks per se. They're more elements that contribute to a bigger risk or a bigger failure. So, it's good to have a system of how to put those together. It's, that's where threat modeling is great to identify all of those. It's great to figure out what are all the things that can go wrong. But once you know all the things that can go wrong, you need to see how they come together to actually, you know, create disaster or chaos inside your application. So sometimes it's not just one thing that goes wrong. It's a lot of things that need to go wrong. in a chain for the disaster to happen. And you don't really want to be investing your time and efforts, which are costly, to fix some things that need a huge chain of things to happen, to materialize, even though, you know, emotionally you might want to be driven to do it.

**17:12 Chris Romeo:** All right, let's go into the world and the issue that's on everybody's minds these days. Let's talk about agents picking up their own tools. and then touching downstream systems. What failure modes should we be worried about in that type of a scenario? Like, what are the top 3 that, that we should be focused on?

**17:33 Petra Vukmirovic:** Yeah, I think the main things that, and this is where failure tree analysis is really helpful because it helps you understand your top 3, because in every system, depending on your controls, you might not have, you might have different top 3s. But what I've noticed by doing, especially with you know, doing FTA on multi-agent systems, the biggest failure rate is actually where your agent's output determines the action of the next agent. So when some kind of LLM's output becomes like a direct action on your system, unless you have some kind of deterministic checking, and this comes back to your refund example. So if, you know, an agent based on a ticket decides that that customer needs a refund, and unless there's some kind of check in the database that that customer hasn't already received a refund, then, you know, there is a huge potential for failure here. So I would say in terms of, you know, downstream failures, most of the time it's all about actions and The other one, I think we all know, it's when you give agents excessive agency. So if they can do a lot of things and when the same agent that produces the output is the one that creates action, that's even worse. So excessive agency is huge because it can have massive impacts and downstream effects. And we need to be careful here how we chain tools together and what are all the permissions that these tools are allowed to, you know, to have?

**19:15 Chris Romeo:** Yeah, I've got an example that I can share of that real time, just happened this morning. So, I've been playing with AI coding agents, like a lot of people, quite aggressively over the last, for me, it's been the last 6 to 8 weeks or so that I've really, it's really, I've really caught fire. And I had an agent doing some work for me while I was sleeping. And I come back to the console this morning and it says, code's been pushed. deployed, merged and deployed to main. And I went, oh, who told you to do that? And it said, oh yeah, you're right. You didn't tell me to do that.

**19:50 Petra Vukmirovic:** Whoops.

**19:51 Chris Romeo:** Now people are making fun of me on LinkedIn because I didn't have merge gates and all the other checks and things happening. And that's fine. They can make fun of me for that. But it was more of a, I'm not setting those things right now because I want to know what it can do. I want to understand what, how, how the machine is going to work. And it pushed them to and deployed to main. Without my approval. Now, did it pull that from some previous instruction, a goal, and I accidentally told it something further back up in the prompt that it took as this is something that Chris wants to get pushed out to production? I don't know, but I just thought the answer was funny. You're right. I didn't, I didn't ask you to do that. You didn't ask me to do that.

**20:30 Petra Vukmirovic:** Yeah, it's funny. It always admits it's wrong and you can always gaslight it. It's wrong too.

**20:37 Chris Romeo:** It's true. That's true. I did, I realized that it didn't have emotions. So like I, you know, yelling at it wasn't gonna do anything. I've noticed like typing in all caps doesn't have any impact either, but you know, it used to in the '90s.

**20:49 Petra Vukmirovic:** I think that that's like when a, when an agent has a goal, we've all seen lately how dangerous that can be. Like agents are so persistent in achieving that goal no matter what it takes. So maybe you're right, you've given it some kind of goal and it just did everything that it could in their possible power to achieve that goal. And one of those goals was, you know, needs to push some code no matter what.

**21:13 Chris Romeo:** Yeah.

**21:15** Let's talk about testing. So, you call comprehensive test coverage a myth. How does a tree become your test backlog in that case?

**21:23 Petra Vukmirovic:** Yeah, like, especially with AI, we know that there's so many ways that things can go wrong, right? And we don't want to test for everything. we want to be able to do some testing and we don't want to just sit around and try 100 prompts. We probably want to do some deterministic tests, even, you know, via the API, have proper test cases, right? We all know that focus testing is always more accurate, even when you do pen tests, right? When pen testers just go out and do a vulnerability scan and then, you know, whatever they find, they find. It's so way less effective than if you do a threat model and then you give them that threat model and they can just test the threats from your threat model. So similarly for FDA, if you do your threat model and then you do an FDA, you're, you know, you can find your most problematic failure paths and you can probably find places where you don't have controls that could be adequate for it. So those are great places to test. So you know exactly what could go wrong, what you need to inject to do, to execute that test. And obviously the whole tree, It can be a set of tests, but because you've done the work and you've assigned the probabilities and you have your AND and OR gates, you know exactly which parts of the tree are the most productive to test because you don't want to be spending all day. It's time-consuming to do different types of tests, especially if you combine some kind of prompting plus some API calls and tool calls and you try to set up a test suite. It's really time-consuming. So you want to have your ideal test cases and you can even then later on put that into some kind of CI/CD. You know, you can automate it, but if you know what are your most common failure rates and the ones that you maybe where the controls are lacking and you want to test it, maybe the place is actually the opposite where you're going to invest like a lot of money into patching your, you know, biggest failure paths and therefore you're happy with your controls. But then there is some other places where, you know, you want to be on the safe side. You might not want to put all the mitigations in, but maybe you want to just do the test to make sure that you're covered. So, that's kind of the approach that I tried to take.

**23:40 Chris Romeo:** I guess one of the things that a skeptic could say about fault tree analysis is that you're asking me to make up probabilities. And so, I guess the question in regards to fault tree analysis is really how rough is rough when we're making these things up and how tied to the expertise of the practitioner are those probabilities? Like if you've been in this for 10 years, 20 years, do you do better probabilities than somebody who's a junior person who's just joined the security team?

**24:14 Petra Vukmirovic:** It's a great question. I think with fault tree analysis, my advice that I gave when I even did the workshop is To go down that rabbit hole, go down the failure tree analysis, and only start assigning probabilities where you have a clear idea of what those would be. So, you want to try, like, the further you are up the tree, the further you are assigning probabilities based on a hunch, and the further you go down the tree, the more data-driven you can assign them. So, at some point, I think you need to kind of reason about Is this probability easily attributable? So, as an example, the EDR, you know, I came all the way down to my EDR point and I was thinking, well, can I actually see in the industry, is there any published statistics on this? And I realized there is. So that's where I stopped. And then I said, okay, now I have a good probability rate. So, I would say you don't want to be estimating a lot. you want to try and go down the tree as much as you can till you hit a point where you're pretty certain about your probabilities, or at least you can make kind of like an educated assumption. And you're asking about experience. I think this is something that anyone can reason about. When I've given this workshop, I've given it to practitioners of various different experiences. Some of them were juniors, some of them were senior, and all it takes is just someone with critical thinking, because when I explain the way you get to it, you know, you can look at logs and, you know, come up with statistical assumptions. You can look at industry data, you can look at threat reports. So once you explain the reasoning, how you assign those probabilities, everyone kind of gets it and tries to do it by themselves. So I don't think you actually need a lot of experience. You need, someone who is at least acquainted with some principles of cybersecurity and can reason well enough about probabilities and, you know, some mathematical statistics that they can then go out and try and figure that out.

**26:23 Chris Romeo:** So let me, let me read that back to you and make sure I understand. So I think what you're saying is that the closer you get to the bottom of the tree where it, with solid facts and data, the less estimated guesses and experience of the practitioner matters. So, as long as you've got that solid source of data on the lower end, I'm not making those estimated guesses where I would tie into my experience of building applications and products and things for many, many years. Is that kind of where you're going?

**26:56 Petra Vukmirovic:** Yes, exactly. I think you pretty much summed it up because if you're talking about, oh, you know, what is the estimate of an outage of one of my critical deployments? Like you said, this is where maybe practitioner experience comes along and you're like, you know, based on my experience and all the things I've seen in this setup, I think this is going to be the probability. But if you go all the way down and you, you know, go through all the tree and you say, oh, well, you know, database can fail. And then what can cause a database to fail? Oh, maybe ransomware. What can cause ransomware? Again, like an antivirus. Failure on a computer. Oh, okay. What are the failure rates? Boom, we're there. You don't need anyone to really have all of that experience and understand the intricacies of the complex environment. You just are looking at one individual thing that you need to assign probabilities to, and then you just go one by one, and then the math adds up.

**27:53** Let's think about the practical way to apply? A team wants to try this in their next design review without that 6-week science project. Where would they start?

**28:04 Petra Vukmirovic:** Good question. So, I think the best place to start is always threat modeling, of course, thinking of, you know, where can things go wrong? And when you figure that out, when you decide what are all the things that can go wrong, find out which one are you the most worried about. When you do have that, then Just start building your tree. And as I said, you could use AI, you could try and build a skill for it. You can even use Claude to try and build a UI for it so you can store it in JSON. But all you do is you go from your top event, think about what are the things that need to be true for this to materialize? Do all of these things need to be true together? Yes. Then you have your AND gate. Is there another separate thing that needs to be true? that is, you know, independent of all the others? Yes. Okay. You have your OR gate. So, and then look at each individual thing that you have at the bottom of your tree and then go all the way down until you get, like we said, to clear-cut probabilities where you can make some really good judgments about probabilities. That's where you stop. You don't want to go all the way down because it's too deep and too time-consuming. Use any helpers that you can get from AI, industry information, logs. You can use AI to query the logs. So we're so lucky these days. Everything is so much easier. I think if I was to set out to do an FTA deterministically with, you know, just some scripts, it would have taken me so much longer than I, than right now. So you can always use AI to help find reports because reports are not always that easy to find. So. You can find those industry reports and help you get that data that is data-driven.

**29:50 Chris Romeo:** So, yeah, this seems like a practice that's made, almost made for AI.

**29:56 Petra Vukmirovic:** Yes.

**29:56 Chris Romeo:** To be able to help you go through this because a lot of it is data collection from internal or external sources. And then kind of, you know, making your way down the tree. It just seems like something AI should be really good at, at least giving you a draft. that then you can look at and go, nah, I don't like that. That's wrong right there. You shouldn't have jumped. You jumped too far on that, you know, on the tree instead of, you should, there should be something in between. But it seems like something that should be teachable to AI to be able to deliver upon.

**30:29 Petra Vukmirovic:** Yeah, I agree. I agree. I think whenever I did it with AI, it came up with you know, some really good outcomes. So.

**30:38 Chris Romeo:** So I'm going to see a lot more, a lot more of this analysis happening now because it's going to be, it's going to be, and you're helping to make it easier too, just by somebody explaining it is, is a good, is a great first step for a new discipline in our practice because often people just don't have the perspective to be able to just pick this up and start doing it. But when they hear, okay, here's the, here's the the benefits and here's the return on investment. Now it's, okay, this is something as an AppSec person I should be taking a look at, pairing it with my threat modeling, maybe adapting my process to bring them together, I think is a good connection. Like, is that how you approach it too? Is it threat modeling and do you see these 2 as like partners side by side?

**31:21 Petra Vukmirovic:** Yes, definitely partners. I think it's also something that helps you sell your controls. Like, One of the things that I mentioned to the cohort was with, when you do a threat model, a lot of times you find some difficult controls that need to be implemented, changes to the design, and the engineering is already like rolling their eyes. They're like, oh gosh, there they go again. You know, first of all, a lot of times we in security struggle with credibility. They see us as people who catastrophize everything. And then on the other hand, engineering maybe wants to implement a design change, but it takes so much work and effort to do it. They just don't see the ROI. So, a lot of times, if you pair that with a bit of risk quantification as well, it's so easy to sell it to leadership. I used it internally and I used Cole to generate like nice HTML reports to show exactly which would be the most powerful controls. And it it clearly shows it because if you attribute also the financial number to your top event, so if your top event is an outage, a lot of times, you know, how much revenue you can lose from that outage, right? So, if you assign it probability and then you have your impact is, you know, how much money you're going to lose, it's so much easier to come up with, you know, a yearly kind of cost if you don't fix this, because it's gonna, there's gonna be outages that will make you lose revenue. So when you put it that way, it really helps you first of all sound more credible, and then it helps you also drive discussions about ROI. And I think, like you said, it's something that we haven't, like a lot of times we just base it on our experience and maybe on our gut feeling, which is great. But I think safety engineering has really kind of managed to get this into a way more deductive and data-driven analytical exercise. So, why not us use it as well? Let's just use what they kind of learned.

**33:28 Chris Romeo:** I think I just had an epiphany, but I have to bounce it off you, Petra, to make sure that this is, that I'm onto something here. I'm sure I'm not onto something. I just think in my head, I just made a connection and I want to see if this is Correct. So, when we do threat modeling, it's not about the threats, it's about the mitigations. Is it a fair statement to say in FTA, it's not about the fault, it's about the security controls that come as a result of it?

**33:53 Petra Vukmirovic:** 100%. Yes. Because, you know, ultimately, what you want— what I did is as well in my JSONs is I assigned each cutset a control. So, then based on my control kind of efficiency calculations, I could actually see which control contributes the most. And a lot of times you would see that a control shows up in multiple places, and that way you're like, oh, wow, well, this control is way more powerful than some other ones, so I'm going to invest my time there. So yeah, 100%, same like threat modeling, FTA is all about understanding what controls to put in and potentially do some tests if you don't want to put controls, if you want to just kind of understand your exposure.

**34:34 Chris Romeo:** So, one more question. What's the one thing from your viewpoint and your perspective that you think every team building an agentic system should add to its threat model today?

**34:45 Petra Vukmirovic:** Well, FDA. I believe, you know, when you do agentic threat modeling, especially if it's multi-agents, there is so many things that can go wrong, right? And you can get onto this spiral of trying to figure out all the threats. Ultimately, you at the end of the day, you have to prioritize. So, you know, pick a few things that are the most costly to you that will cause you to lose revenue or will cause, you know, reputational damage or will put your customers off and choose that and then do your fault tree analysis and test for that. But definitely start with a threat model, start with what can go wrong. It doesn't matter what framework you use. Ultimately, if you're asking, you know, the 4 questions of threat modeling, you're going to get into a good place.

**35:33 Chris Romeo:** Petra, thank you for joining the Application Security Podcast to share this idea of FTA and help us to understand it better. I'm sure it helped our audience. I know it helped me personally to, to just have a better perspective on what this, this concept is and then how I can, how I can use it, how I can pair it with threat modeling as partners. I like that idea of FTA and Threat Model as partners. That's a t-shirt idea as well for those people that make t-shirts around the world. But yeah, so we, uh, we definitely enjoyed having this conversation and, uh, look forward to your talk at OWASP San Francisco. It's coming up in a couple of months in November. So that'll be a great, great talk to see. We hope to see you there at the event. Thanks for being a part of the show.

**36:15 Petra Vukmirovic:** Amazing. Thank you very much for having me and looking forward to sharing some new skills and UIs that I create for FTA.

**36:22 Chris Romeo:** Thanks for listening to the Application Security Podcast. If you enjoyed this episode, subscribe, leave us a review, and share it with a friend or colleague who would enjoy the conversation. We'll see you next time.

---

Source: https://appsecpodcast.com/how-agentic-ai-fails-and-which-controls-actually-stop-it/