you cannot fake a shared memory

April 2026 – Abi Wiguna

Next.jsReactTypeScriptGemini 2.5Firebase GenkitVertex AIFirestoreCloud Run

Group project (3 people) Team Lead and Sole Developer Finalist, MyAI Future Hackathon 2026

Live ↗

A voice that sounded exactly like him

There is a scam going around Malaysia where someone uses AI to clone a person’s voice and sends a WhatsApp voice note asking for money. The voice sounds real. It sounds exactly like your boss, your child, your mother. In one case a worker transferred RM5,000 to someone impersonating their boss, asking for Touch ‘n Go PINs over the phone. By the time she understood what had happened, the money was already gone. That was a real incident, reported by Malaysian police. CCID had investigated more than 450 similar deepfake cases since early 2024, with losses reaching RM2.72 million. In Q1 2025 alone, 12,110 scam cases were recorded totaling RM573.7 million across Malaysia.

Woman loses RM5,000 to AI voice scam, May 2025. Source: The Rakyat Post

My team was three people at the MyAI Future Hackathon, organized by GDG On Campus UTM in Malaysia. I was the team lead and the only developer. We knew we wanted to work on something that actually mattered, and this problem kept coming back. The people most targeted are older, less familiar with how AI works, and least likely to pause and verify before acting. We wanted to give them a reason to pause.

dengardulu overview

We stopped trying to detect deepfakes

The obvious first move is to build a deepfake detector. You submit the audio, the model says real or fake, problem solved.

We spent about a day on that direction before we gave it up. A deepfake classifier trained today is already falling behind the models coming out tomorrow. The fakes keep getting better, and a detection race is one you can never win for long. Even a 95% accurate detector is dangerously wrong one in twenty times, and in this context that means someone loses their savings. The harder we looked at deepfake detection, the more it felt like the wrong problem.

So we reframed everything. What if instead of judging the audio, we gave the user a way to verify the person?

The thing a scammer cannot answer

That question led us to the actual idea. A scammer can clone a voice. They can match the tone, the accent, how someone laughs. What they cannot do is answer a question that depends on a private shared memory.

If someone calls pretending to be your son, and you ask them about the time you both got lost driving home from your cousin’s wedding and ended up eating nasi lemak at three in the morning, a scammer has no answer for that. That memory belongs to two people and no one else. No amount of voice cloning gets past it.

DengarDulu does not try to catch the fake. It hands you a personal counter-question to send straight back to the caller, in Bahasa Malaysia and in English, so you can verify for yourself whether this is really who they say they are. A copilot, not a verdict.

dengardulu results and counter-question

Building the pipeline

The app runs four AI agents orchestrated with Firebase Genkit, all on Google Gemini 2.5 Flash via Vertex AI. Each agent has a specific job and only sees what it needs.

The first agent transcribes the audio and listens for voice characteristics: intonation, pauses, breathing, pitch stability, anything that sounds synthetic. It deliberately avoids making a binary claim about whether the voice is real or fake, because that is not its job. The second agent reads the transcript and checks it against a library of 138 known scam phrases in Firestore, looking for urgency signals, impersonation cues, and secrecy markers. While that second agent runs, a phone-lookup tool fires in parallel, checking the caller’s number against Malaysia’s official police scam registry at PDRM. The third agent generates two counter-questions. It never sees the original audio, only the transcript and the analysis, so its output is grounded in what was actually said. The fourth agent synthesizes everything into a final suspicion assessment and a safety plan pointing to the right Malaysian authorities.

The whole thing runs in about fifteen seconds and installs as a PWA so you can share a voice note into it directly from WhatsApp.

flowchart TB
    U[User uploads voice note] --> API[/api/analyze SSE route/]
    API --> Flow[analyzeVoiceNote async generator]

    Flow --> A1[Agent 1: Transcribe & Characterize<br/>Gemini 2.5 Flash]
    A1 --> Par[Parallel]
    Par --> A2[Agent 2: Content Analyzer<br/>Flash + RAG 138 phrases]
    Par --> T1[Tool: PhoneLookup<br/>Semakmule PDRM → Firestore fallback]
    A2 --> A3[Agent 3: Challenge Generator<br/>Flash]
    T1 --> A3
    A3 --> A4[Agent 4: Safety Plan<br/>Flash - Pro on conflict]
    A4 --> Stream[SSE stage events]
    Stream --> UI[React stepper UI]

    subgraph Google Cloud asia-southeast1
      VAI[Vertex AI]
      FS[(Firestore)]
      CR[Cloud Run]
    end

    A1 -.-> VAI
    A2 -.-> VAI
    A3 -.-> VAI
    A4 -.-> VAI
    T1 -.-> FS
    API -.-> CR

    subgraph Community Reporting
      CU[User reports scam number] --> RL[Rate limit 5/hr/IP]
      RL --> FU[Firestore upsert scam_numbers]
    end

    subgraph PWA Share Target
      WA[WhatsApp share audio] --> ST[/share-target POST/]
      ST --> TMP[(Firestore _shared_audio)]
      TMP --> AN[Redirect → /analyze]
    end

Let the model think, but not the math

One of the most useful things I learned came from a bug. The model would give detailed, well-reasoned analysis of a voice note, and then produce a suspicion score that did not match the evidence it had just laid out. You could read the reasoning and see clearly that this was a high-risk call, and the score would say forty out of a hundred.

The fix was to take the scoring out of the model entirely. I wrote a deterministic function that reads the structured outputs from all four agents and computes the final score in code. High-risk requests, urgency above a threshold, impersonation signals, a number found in the registry: each contributes a fixed number of points and the total determines the verdict. The model does not do the math anymore. The code does. The comment I left in that function says: “LLMs can’t do math.”

Since that change, the verdict has always matched the evidence.

What it cannot do yet

DengarDulu gives you a suspicion score and a counter-question, but it does not tell you definitively whether a voice was generated by AI. That is honest, but it is also a real limitation. A person who receives a LOW verdict might still have been targeted by a careful scammer who avoided the usual signals.

A real classifier for voice synthesis would require training data and skills in audio machine learning that I have not developed yet. The roadmap includes integrating a model trained specifically to detect synthetic speech, but that was not something I could build in a hackathon. I want to learn more about audio and signal processing, because there is a real difference between “the content sounds like a scam” and “this audio was generated by a machine,” and right now DengarDulu can only tell you the first one.

The app also has no automated tests, and the rate limiting is in-memory per server instance, which means it resets on every cold start. Building something people could actually rely on for real decisions at scale would require reliability engineering I was only starting to understand while working on this.

The people we built it for

DengarDulu was selected as a finalist at the hackathon. But the number I think about most is not that. It is the statistic from Malaysia’s police data: most of the people losing money to these scams are over fifty. They are not naive. They are trusting voices they recognize, and the voices are not real anymore.

I built the entire application while my teammates handled research, presentation, and testing. I learned what it feels like to be the person responsible for whether something works or falls apart under pressure. And I learned that the best protection against a scam that exploits trust is not a tool that makes the decision for you. It is one that helps you make it yourself.

dengardulu easy mode for older users

Sources