'Whatever you try to fix breaks things even more': How to avoid the hidden dangers of AI code

Date:
Fri, 18 Sep 2026 15:55:00 +0000

Description:
Gabriela Moreira, CEO of Quint, explains how catching flaws early can limit
AI code nightmares

FULL STORY ======================================================================Copy link Facebook X Whatsapp Reddit Pinterest Flipboard Threads Email Share this article 0 Join the conversation Follow us Add us as a preferred source on Google Newsletter Subscribe to our newsletter You no longer need knowledge of advanced coding languages to build apps. AI has torn down the barriers to coding, making it as simple as describing what you want in plain English.

Or at least, it can appear that way. But coding isn't just about writing syntax. It's about systems architecture, understanding how people will really use a platform, and ensuring the right measures are in place to limit vulnerabilities. I spoke with Gabriela Moreira, CEO of Quint, about how
people using AI tools to write code can better understand the associated
risks and put in place early checks and processes to limit unmaintainable
code and catastrophic bugs. Interview with: Interview with: Gabriela Moreira Social Links Navigation CEO of Quint Interview by: Interview by: Owain Williams Social Links Navigation SMB Editor at TechRadar Pro Can AI be
trusted to write production-ready code without human oversight? If not, what makes AI-generated bugs distinct from human error? Oversight is definitely needed, because AI exists to serve humans, and therefore humans need to tell AI what to do and evaluate its results. Now, which type of evaluation is the most adequate? You can read its code line by line or ship to production and monitor for problems, to talk about two extremes. Both of these are already techniques used to oversee code that might contain human error. The main differences that AI brings to the table are the amount of potentially wrong code that gets produced and the cognitive disconnection of any human to the elements of what gets produced. Code review and monitoring are both harder at this load scale and cognitive disconnection. Which core business systems are most vulnerable to AI coding flaws? The ones that are harder to test, like concurrent and distributed systems, or any system with a high level of non-determinism where there are too many paths to consider. AI is a
generation tool, which is only as good as our ability to decide if a
generated solution is correct or not. If I have good tests, or some other
kind of verification harness I trust, I can accept AI solutions with less risk. If a system is hard to test or verify, there might be too many blind spots where flaws can sneak in. What are the hidden operational and security risks of deploying AI-generated code without formal verification?

Theres no magical solution, and combining different strategies is the wisest thing to do. Perhaps the best way to think about risk in 2026 is by considering the asymmetry: defenders must protect all parts of their code, while attackers may only find one vulnerability to cause huge damage. AI companies are already taking this asymmetry into account when they decided to give a model like Mythos only to companies in a program to secure themselves, before they land in the hands of potential attackers. Formal verification is probably the strongest tool we know that can fight this asymmetry, but there are still many challenges in the area that make it infeasible to verify all the code surface, and vulnerabilities can still be found in unverified components. Theres no magical solution, and combining different strategies is the wisest thing to do. You may like The visibility gap that's smuggling risk into AI code Humans in the loop: how software teams are learning to trust AI AI monocultures: the code review problem nobody's talking about Why is catching a design-level bug early in the process so important?

Im extremely concerned about the design process involving mostly markdown files that cannot be compiled or executed. The answer to this question before AI wrote the code was quite different: because rewriting code is very expensive and error-prone if you catch the bug after months of development instead. I dont think this is a strong argument anymore. So, the new answer is: you are way more likely to find this problem in the scope of a design you can still reason about than after AI writes thousands of lines of code with the bug hiding in one of them. You could maybe notice the problem as you
typed the code, thinking wait, we didnt consider this scenario! But this is less likely to happen if AI wrote the code for you, so more careful design is important.

Developers are used to reasoning about possibilities and bugs while compiling and executing different versions of code, so Im extremely concerned about the design process involving mostly markdown files that cannot be compiled or executed, or interacted with in any way other than reading. Are you a pro? Subscribe to our newsletter Sign up to the TechRadar Pro newsletter to get
all the top news, opinion, features and guidance your business needs to succeed! Contact me with news and offers from other Future brands Receive email from us on behalf of our trusted partners or sponsors By submitting
your information you agree to the Terms & Conditions and Privacy Policy and are aged 16 or over. What simple checks, tools, and safety nets can dev teams put in place to verify AI code before it hits production? First of all,
tests! But tests that actually help with trust, not just any AI-generated
test that makes some arbitrary assertions and has to be updated all the time.

I like to think about tests in terms of which behaviors they are exercising and which kinds of regressions they are preventing. If controlling the
quality of tests becomes a challenge, formal models can help by either being
a generator of scenarios to test or by validating that existing tests enforce interesting behavior.

Additionally, mutation testing is very helpful in measuring the quality of tests by how many different regressions they could catch. What level of human oversight do you recommend to balance development speed with code quality and reliability? This depends so much on context and risks. The best I can recommend is to do whatever allows you to ship with enough confidence and to feel enabled to make software changes without fear of breaking what is
already working.

The worst scenario is always to get hacked or leak other peoples data. The second worst is to get to a state where there are a lot of things wrong with the software, but whatever you try to fix results in breaking things even more, so it gets stuck in a bad state.

I think its quite easy for AI-generated code to end up in this state. Interestingly, I think the way to escape it is very similar to how we handled legacy code that became too hard to maintain: first add enough tests to be confident your oversight is enough, and then change it.



======================================================================
Link to news story: https://www.techradar.com/pro/whatever-you-try-to-fix-breaks-things-even-more- how-to-avoid-the-hidden-dangers-of-ai-code


--- Mystic BBS v1.12 A49 (Linux/64)
* Origin: tqwNet Technology News (1337:1/100)