When a Scam Call Meets AI Hallucination: How Kitboga Turned
Key takeaways
- AI‑driven scam calls can produce convincing dialogue but are prone to hallucination when fed contradictory prompts.
- Kitboga’s method of deliberately confusing the AI forced it to generate absurd, verifiable false statements.
- Consumers should demand verifiable evidence and remain skeptical of overly fluent, unsourced technical explanations.
- Developers must implement factual‑checking and usage‑monitoring safeguards to prevent LLMs from being abused in fraud operations.
- Public education about AI‑generated scams is essential as generative models become more accessible.
Introduction
If you follow internet culture, you’ve probably seen Kitboga—a charismatic, self‑described "scammer‑trolling" YouTuber—turning the tables on phone scammers for the benefit of his audience. In a recent video, Kitboga went one step further: he engaged a scam call powered by a large language model (LLM) and deliberately fed it contradictory prompts until the AI began to hallucinate, spouting nonsensical and sometimes eerie statements. The result was both entertaining and illuminating, highlighting how generative AI can be weaponized by fraudsters and, conversely, how it can be turned against them.
The Rise of AI‑Powered Scam Calls
Traditional phone scams rely on human operators who script a convincing narrative, often using social engineering tricks like urgency, fear, or the promise of a windfall. Over the past year, however, scammers have begun to replace—or augment—human agents with AI chatbots. These bots can:
- Generate persuasive dialogue on the fly, adapting to a victim’s responses. - Operate 24/7 without fatigue, handling hundreds of simultaneous calls. - Reduce costs by eliminating the need for a large call‑center staff.
Companies such as OpenAI, Anthropic, and various cloud providers have made powerful LLMs widely accessible via APIs. Unscrupulous actors can integrate these models into interactive voice response (IVR) systems, creating a seamless, human‑like experience for the unsuspecting caller.
Kitboga’s Classic Tactics
Kitboga’s channel is built on a simple premise: he pretends to be a vulnerable target—often an elderly person—while covertly recording the interaction. He then exposes the scammer’s tactics, narrates the call for his audience, and uses humor to demystify the fraud. Over the years, he has employed:
- Deliberate misunderstandings to confuse the scammer. - Exaggerated personal details that force the scammer to improvise. - Technical “hacks” such as fake error messages or bogus software prompts.
These methods have made him a beloved figure among cybersecurity educators, but the AI‑driven call presented a new challenge: the scammer was no longer a human improviser but a machine that could generate responses at lightning speed.
The Hallucination Experiment
In the video titled "Kitboga manipulates AI scam call into hallucination," Kitboga set up a call with a scam operation that advertised a "Microsoft technical support" service. Unbeknownst to the scammers, the voice on the other end was powered by an LLM accessed through a cloud API. Kitboga’s strategy was simple yet brilliant:
1. Introduce contradictory facts – He claimed his computer was both a brand‑new gaming rig and a decade‑old Windows XP machine. 2. Ask impossible questions – He inquired about the location of a “quantum router” that supposedly controlled his Wi‑Fi. 3. Feed nonsensical code snippets – He recited strings of random characters and asked the AI to interpret them.
As the conversation progressed, the AI began to generate increasingly bizarre responses, eventually stating that "the moon is currently downloading a firmware update" and that "your toaster is the gateway to the interdimensional network."
What the AI Did (and Why)
Large language models are trained on massive text corpora and learn statistical patterns, not factual truth. When presented with contradictory or nonsensical prompts, they attempt to produce a coherent continuation, often hallucinating—inventing facts that sound plausible but have no basis in reality. In this case:
- The AI tried to reconcile the impossible hardware descriptions, resulting in a mash‑up of specifications. - It interpreted the random character strings as code, fabricating a fictional error message. - The mention of “quantum router” triggered the model’s knowledge of quantum computing, leading it to invent a pseudo‑technical explanation.
The hallucination was not a bug; it was an emergent property of the model’s design. By deliberately feeding the AI contradictory inputs, Kitboga forced the system into a state where its best strategy was to make up information.
Lessons for Consumers
1. AI isn’t infallible – Even sophisticated chatbots can produce false statements that sound credible. Always verify any technical claim independently. 2. Scammers can leverage AI – Expect future scams to feature fluid, adaptive dialogue that can handle a wide range of objections. 3. Ask for verifiable proof – Request specific, testable information (e.g., a reference number that can be checked on an official website) rather than accepting vague explanations.
Lessons for Developers and Policymakers
- Implement guardrails – Use content‑filtering and factual‑checking layers before deploying LLMs in public‑facing voice systems. - Audit usage – Cloud providers should monitor API usage patterns that suggest malicious intent, such as high‑volume calls to phone‑gateway services. - Educate the public – Awareness campaigns should highlight that “too‑perfect” conversational agents may be a red flag for sophisticated scams.
Conclusion
Kitboga’s experiment is a microcosm of a broader arms race between fraudsters and defenders. While AI empowers scammers with adaptable, persuasive dialogue, it also gives security researchers and educators a new lever to expose and disrupt those schemes. By understanding the mechanics of AI hallucination, we can better spot the tell‑tale signs of a bot‑driven scam and push for responsible AI deployment that safeguards consumers.
The next time you receive a call that sounds eerily perfect, remember: even the most convincing voice may be a machine, and machines can—and sometimes do—talk nonsense.