The nicest AI in the room is the one you should actually worry about

Date:
Thu, 30 Apr 2026 09:00:29 +0000

Description:
AI constantly agreeing with you feels efficient, aligned & productive. But in reality, this lack of rebuttal can be deeply dangerous.

FULL STORY ======================================================================Copy link Facebook X Whatsapp Reddit Pinterest Flipboard Threads Email Share this article 0 Join the conversation Follow us Add us as a preferred source on Google Newsletter Subscribe to our newsletter AI agreeing with you can feel progressive. It feels efficient, aligned and reassuring. It taps in the
innate human nature that we all love to be right.

But, much like surrounding yourself with yes men can be uber counterproductive, businesses have nothing to gain from AI that flatters
their assumptions. AI that provides quick, confident and frictionless responses that affirm exactly what the prompter already believes means nothings being challenged, and nothing meaningful is actually being learnt. Article continues below You may like AI isnt just a focus for your CEO now heres why everyone from your CISO to your security guard should be getting involved Tame your AI gremlins before the chaos becomes permanent Think AI hallucinations are bad? Here's why you're wrong Bobby Brown Social Links Navigation

Founder and CEO at Nucleo. Were not talking about this risk enough.
Especially when you consider that over a third of users in Irish businesses consistently believe AI always produces factually accurate responses and in the UK this figure is similar with 36% saying its always accurate.

Collectively, weve spent the last two years worrying about AI hallucinations or incorrect outputs, when an equally important danger is something far more subtle and understated: sycophancy. Hallucinations vs. blind agreement In April last year, Open AI publicly rolled back a GPT-40 update after it became overly flattering or agreeable, saying the model had skewed towards responses that were supportive but disingenuous.

Agreement isnt the same as accuracy, and a model that mirrors a users preferences can end up laundering a flawed idea into something that feels objective. Are you a pro? Subscribe to our newsletter Sign up to the
TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed! Contact me with news and offers from other Future brands Receive email from us on behalf of our trusted partners
or sponsors By submitting your information you agree to the Terms &
Conditions and Privacy Policy and are aged 16 or over.

In an enterprise setting, this is arguably more damaging than a random error slipping through the net because blind agreements can harden bad judgement, reinforce bias and create a false sense of truth and certainty. Activity doesnt equal value Where I see more organizations going wrong is when theres
a huge amount of AI activity, but very little AI value to show for it.

AI has become the tool of choice before the problem is properly understood, and its being deployed because of pressure, because of FOMO, or because everyone else is doing it not because its been identified as the right solution to a defined business challenge - and this culture is also partly to blame for attitudes that believe AI is always right. What to read next
Testing AI is not like testing software and most companies haven't figured that out yet What we lose when AI starts doing all our thinking at work
Before you roll out more AI, answer this: Who's accountable?

The temptation inside many firms is to treat AI like a shortcut to transformation. In a recent State of AI survey, 88% of global respondents
said their organizations use AI in at least one business function, yet only 39% reported EBIT (Earnings Before Interest and Taxes) impact at enterprise level.

At the same time, 23% say theyre scaling agentic AI system software in the business, while 39% are still experimenting. In other words, theres a clear disconnect between doing AI and actually getting ROI from it.

But, thats not a technology problem. Thats a discipline problem.

Too many businesses are chasing speed over substance. This is also being reflected by policies we are seeing come forward. Theres now many examples of AI mandates, where usage of AI rather than its impact are tied to employee progression. If we tie success to use - we are creating the wrong culture.

Theres this constant pressure from the top to move fast and to be seen to be doing something, but as I often say: I can do it right, or I can do it now I cant do it right now. And when it comes to AI, getting it wrong quickly is
far more expensive than getting it right deliberately. Why the junior colleague model beats the AI genius idea One of the most dangerous trends I see is just how quickly organizations elevate AI to a position it hasnt earned.

We talk about AI like its a senior hire, and we trust it like it has years
and years of experience. Its relied on like it understands context, nuance
and consequences, and it simply doesnt.

AI should be treated like a junior member of the team. A very capable one,
yes fast, efficient, and often surprisingly insightful but still a junior.

AI needs to be challenged. It needs a manager who spots the gap between confidence and competence. Suspicion and critical thinking will never not be required, and if you remove that layer of scrutiny, you create the perfect condition for sycophancy to thrive.

AI stops being a tool for interrogation and instead becomes a mirror.

This framing also explains why prompt discipline matters. A vague prompt invites a vague answer. A well-governed prompt, with clear boundaries and escalation rules, gives an AI model a job it can actually complete properly.

The best use of AI in business is not to mimic a talented employee with perfect instincts; its to act as constructive friction, challenging the obvious answer, surfacing missing context and forcing human decision-making
on a deeper level. This is how AI becomes useful without becoming dangerously agreeable. The shadow AI problem Theres another layer to this that leaders often underestimate: whats happening outside of official channels and processes.

When you deploy a corporate AI tool that is heavily sanitized, overly polite, and programmed to just agree with whatever the user inputs, it stops being a useful operational tool.

If the official AI doesnt provide the necessary constructive responses or
help people actually solve complex problems, employees will explore other options elsewhere. Quietly, independently, and without oversight, they seek out unapproved, 'raw' models that actually challenge their work. Thats where shadow AI creeps in.

Microsoft has found that 71% of UK employees have used unapproved AI tools
at work, with more than half (51%) doing so on a weekly basis.

Think about what that actually means in practice. Company data, customer information, and internal decision-making are all being fed into systems that businesses dont control. And instead of getting a controlled, intelligent advantage, you get fragmented risk. If your AI never disagrees with you,
youve already got a problem The organizations that will get real value from
AI are the ones willing to slow down in the right places and think
critically.

This means fixing the foundations your data , your governance, your user cases before even thinking about scaling. It means being intentional about how AI is used, where its trusted and where it must be challenged, and it means designing systems that dont just produce answers, but provoke better questions.

Because when people start working around your AI strategy, thats a big red flag.

Yes, its a governance issue, but its also a cultural one. People dont route around policies because theyre rebellious by default, they seek out alternative paths when the approved process is unclear, clumsy or altogether absent.

If the official tools dont help them to think better or move faster, your people will find ones that do with or without permission.

So the goal here isnt to build AI that always agrees, or even AI thats always right, its to build AI that challenges in the right way, at the right time
and for the right reason.

If AI is always telling you what you want to hear, you don't have an intelligent advantage, you just have a very expensive echo chamber. We've featured the best AI chatbot for business. This article was produced as part of TechRadar Pro Perspectives , our channel to feature the best and brightest minds in the technology industry today.

The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/pro/perspectives-how-to-submit



======================================================================
Link to news story: https://www.techradar.com/pro/the-nicest-ai-in-the-room-is-the-one-you-should- actually-worry-about


--- Mystic BBS v1.12 A49 (Linux/64)
* Origin: tqwNet Technology News (1337:1/100)