How Do You Know Who Is Real Online?

In early 2024, a finance worker at the engineering giant Arup joined a video call with what appeared to be the company’s chief financial officer and several other colleagues. The meeting looked ordinary. The faces were familiar. The voices were right. Over the course of the call, the worker was persuaded to transfer HK$200 million—roughly $25.6 million—across 15 transactions. Every other participant on that call, including the CFO, was an AI-generated deepfake.

That was the moment the world woke up to a new kind of fraud. But in the two years since, the problem has not been solved. It has industrialised. Deepfake-enabled financial fraud has grown into an estimated $40 billion annual market. Video synthesis now accounts for 42.7% of observed cases, audio-based voice cloning for 21.9%, and identity fraud for 17.6%. The United States is the largest target and source of incidents at 34.1%, followed by India at 7.7% and the UK at 6.6%. Document deepfakes—forged passports, IDs, and certificates—are projected to surge nearly 3,900% year over year.

The question is no longer whether AI can impersonate someone convincingly. It can. The question is whether the systems we rely on to tell real from fake—our eyes, our liveness checks, our identity documents—can still be trusted. And if not, what is being built to replace them.

The Collapse of the “Unforgeable” Signal

For years, the defence against impersonation rested on a simple premise: some things are too hard to fake. A live video call, where the person blinks and responds in real time. A voice that matches a known speaker’s biometric profile. A government-issued ID with holograms and microprinting.

Each of those signals is now forgeable. AI tools can replicate holograms, microprinting, ultraviolet imagery, and scannable barcodes. They can generate facial photographs convincing enough to defeat biometric verification checks. Virtual camera injection software can feed a manipulated video stream directly into a verification system, bypassing the camera entirely. And voice cloning can replicate speech patterns from just a few seconds of audio.

The financial industry is on the front line. In one case, attackers used identity morphing and deepfake liveness bypasses to open 46 fraudulent accounts at ABN AMRO before detection. In another, fraudsters allegedly used a cloned version of Ferrari’s CEO’s voice in WhatsApp audio messages to request an unauthorised currency hedge transaction. Deloitte estimates that AI-enabled fraud losses in the US alone could reach **$40 billion by 2027**, up from an estimated $12.3 billion in 2023.

The Detection Arms Race

The first instinct of the security industry was to build better detectors. And they have. New systems are emerging that can spot deepfakes in real time, at the edge of the network, without the latency of GPU-based post-processing.

SENTRY, a system published in October 2026, is a streaming trust engine designed for inline audio deepfake detection at the network edge. It uses integer lifting-wavelet front ends and multiplier-free transforms to analyse speech without floating-point hardware, achieving 98.75% accuracy on the WaveFake dataset and 94.75% on ASVspoof 2019 telephony attacks. Its feature front end runs on a single Arm Cortex-A53 core, returning a decision 220 nanoseconds after a feature vector is ready—fast enough to screen a call in real time.

DeepSFU, presented at the ACM SIGCOMM 2026 conference, is a video conferencing architecture that builds deepfake detection natively into the media server, reducing per-frame processing latency by 143.7× compared to conventional approaches. Instead of routing video through a separate detection service, DeepSFU integrates detection into the forwarding path, making it scalable across large meetings.

And a growing body of research is moving beyond single-modality detection. A unified CNN-Transformer framework published in June 2026 combines spoofed speech, manipulated video, and AI-written phishing text into a single attack probability score, recognising that modern fraud rarely relies on one signal alone. Multimodal enterprise authentication systems now combine facial anti-spoofing with multilingual deepfake speech detection, designed specifically to resist AI-based identity manipulation.

But there is a fundamental problem with the detection-first approach, and it is the reason the industry is pivoting to a different strategy altogether. Detection is an arms race. Every detector is trained on yesterday’s fakes. Today’s fakes are built to evade yesterday’s detectors. And tomorrow’s fakes will be built to evade today’s. The asymmetry favours the attacker: they only need to succeed once, while the defender needs to succeed every time.

The Pivot: Proving You’re Human, Not Proving the Fake Is Fake

On October 19, 2026, World—the identity verification company co-founded by OpenAI CEO Sam Altman—launched World ID for banking and finance, a suite of tools designed to help financial institutions prevent deepfake fraud and AI impersonation across customer accounts, employee workflows, and AI agents.

The flagship product is World ID for Zoom, which is now generally available. Here is how it works: a user visits a physical Orb device, which scans their iris and creates a cryptographically secure biometric reference. A selfie through the World ID app then verifies the human participant against that reference during a Zoom call. The system compares three signals—the original Orb enrolment, a fresh smartphone selfie, and the live video feed. All three must cryptographically match, producing a “Verified Human” badge on the participant’s tile.

Meeting hosts can require verification before anyone joins, or invoke mid-call checks for sensitive moments such as wire authorisations. World has also launched World ID for Agents, which lets an AI agent present proofs on behalf of a verified human, and a World ID app that stores and presents identity credentials.

The scale is significant. More than 18 million people across 160 countries currently hold a World Proof of Human. Zoom is adding a native integration with World’s Deep Face technology, and DocuSign is integrating World ID as an additional proof-of-human layer inside its agreement workflows. Tinder, Okta, Vercel, Browserbase, and Exa have also integrated the credential.

The logic is compelling: if you cannot reliably tell what is fake, build a system where the default is “unverified” and trust is granted only through proof. It is an inversion of the internet’s original architecture, where anonymity was the default and identity was optional.

The Other Layer: Content Provenance

Parallel to the “proof of human” movement is a quieter but equally important effort to label content at the point of creation.

C2PA Content Credentials are a metadata standard that records how a piece of media was made or edited—whether AI tools were used, and what changes were applied. SynthID, developed by Google DeepMind, is an invisible watermark embedded in AI-generated content that can survive compression and metadata stripping.

In May 2026, Google made both verification systems accessible to ordinary users. You can now right-click or circle an image in Chrome or Search and ask, “Is this made with AI?” The tool checks for both SynthID watermarks and C2PA metadata in a single interface. OpenAI has begun embedding SynthID into images generated by ChatGPT, Codex, and its API. Google says SynthID has already watermarked 1 billion images and videos and 60 years of audio.

This is meaningful progress. But it has a glaring limitation: it only works if the label is there. Content without a watermark or credential is not necessarily real—it is just unlabeled. And the most malicious actors are precisely the ones who will strip metadata, remove watermarks, and re-encode their fakes before distributing them. As The Verge put it, SynthID and C2PA are getting “their biggest expansion to date, and with it, the opportunity to turn the tide”—but the outcome is not guaranteed. OpenAI itself has acknowledged that C2PA metadata is “often stripped out” when content is posted to other platforms.

Provenance tells you what a file claims to be. It does not tell you what it is.

The Regulatory Backstop (and Its Limits)

Lawmakers are beginning to respond. The EU AI Act’s Article 50 transparency obligations took full effect on August 2, 2026. AI-generated or AI-manipulated imagery, video, audio, and text must be identifiable as artificial, and deepfakes must be disclosed to viewers. In the United States, the TAKE IT DOWN Act criminalises the publication of non-consensual intimate deepfakes, and the AI Fraud Accountability Act would create a criminal prohibition on using digital impersonations to commit fraud. New York has amended its business law to require disclosure when an advertisement features a “synthetic performer”.

But regulation has a structural disadvantage: it moves at the speed of legislation, while the technology moves at the speed of a software update. The EU AI Act’s transparency rules apply to “providers and deployers” of AI systems—but they cannot force a criminal in a jurisdiction beyond reach to label their deepfake. Regulation sets the floor for legitimate actors. It does not stop the illegitimate ones.

The Human Problem: Trust Is Not a Technical Setting

The deepest challenge is not technological. It’s psychological.

A 2025 Deloitte study found that nearly 70% of respondents worried synthetic content would be used to deceive them, nearly 60% struggled to distinguish AI content from human-made material, and 84% believed synthetic content should be clearly labelled. People know the problem exists. They know they can be fooled. And they know that the old signals—seeing, hearing, recognising—are no longer reliable.

But knowing is not the same as being protected. The scams that succeed are not the ones that fool sophisticated technologists. They’re the ones that fool a finance worker on a video call where everyone looks and sounds exactly right. They’re the ones that fool a company into wiring $25 million because the CFO’s face was on the screen.

There is also the “liar’s dividend” : the mere prevalence of deepfakes enables bad actors to dismiss authentic evidence as fake, compounding the challenge for institutions that rely on digital media. Trust is not just being eroded by fakes. It is being eroded by the possibility of fakes.

So How Do You Know Who Is Real?

There is no single answer. But a new architecture of trust is emerging, built on four layers that work together—and none of which is sufficient on its own.

Layer 1: Cryptographic proof of personhood. Systems like World ID that bind a physical human to a digital credential through biometric enrolment. This doesn’t tell you who someone is; it tells you that a unique human is present. For high-stakes interactions—banking, corporate video calls, government services—this may become the default requirement.

Layer 2: Content provenance. C2PA and SynthID that label media at the point of creation. This won’t catch everything, but it raises the cost of undetected forgery and gives platforms and users a signal to work with.

Layer 3: Real-time, multimodal detection. Systems like SENTRY and DeepSFU that analyse voice, video, and behavioural signals inline, at the edge, without adding latency. The best tools now flag manipulated media in under a second, and the more elaborate the deepfake, the more likely it trips one of the other checks.

Layer 4: Institutional and social verification. Out-of-band confirmation. A phone call to a known number. A code word agreed upon in advance. A policy that no transfer above a certain amount is authorised on the basis of a single video call. The mundane, low-tech practices that security professionals have recommended for years—and that the Arup finance worker, tragically, did not follow.

The World ID approach is not a panacea. It requires physical hardware, it raises privacy questions about biometric enrolment, and it creates a new kind of dependency—on a single company, a single standard, a single point of failure. But it represents a philosophical shift that is likely to define the next era of digital trust: stop trying to prove that the fake is fake. Start proving that the real is real.

AI can now pretend

to be almost anyone. It can wear your face, speak in your voice, and sit across from your colleagues on a video call. The old signals—the ones we trusted without thinking—have been compromised. Seeing is no longer believing. Hearing is no longer believing. Even a live conversation is no longer proof that the person on the other end is who they claim to be.

What replaces those signals is not a single technology or a single law. It is a layered system of proofs and checks, some cryptographic, some institutional, some stubbornly low-tech. It is less convenient than the world we had before. It requires more friction, more verification, more moments of “prove it” that feel intrusive and exhausting.

But that friction is the price of trust in an age when anything can be synthesised. The question is not whether we will pay it. The question is whether we will build systems that make the payment worth it—systems that protect the vulnerable, hold the impersonators accountable, and preserve the possibility of genuine human connection in a world where the fake is indistinguishable from the real.

The Arup worker transferred $25 million because the faces on the call looked real. The next time, the face might be verified, the voice might be signed, and the transfer might be blocked by a policy that says: a video call is not enough. That won’t solve the problem. But it’s a start.