Code-switching is when a speaker mixes two languages inside a single conversation — often inside a single sentence. A Hong Kong office worker moves between Cantonese and English several times per sentence. A Singaporean order alternates English, Malay, and Hokkien. A Manila support call runs Tagalog with English product terms embedded in every second clause.

This is not an edge case. It is the everyday speech of hundreds of millions of people across Asia and beyond — and it is precisely where speech AI still breaks.

Why models fail on mixed speech

Speech recognition and voice systems are trained mostly on monolingual data, because that is what exists at scale. Confronted with mixed speech, they fail in predictable ways:

• Language identification flips too slowly. The model commits to one language and mangles the embedded words from the other.
• Borrowed words get “corrected.” An English product name inside a Cantonese sentence comes out as a phonetically similar Cantonese word that means nothing.
• The switch points themselves are the hard part. Accuracy collapses exactly at the boundary — which is where names, numbers, and product terms tend to sit.

For any product serving multilingual markets — voice assistants, customer-service bots, in-car speech, dictation — these failures land on the most valuable tokens in the utterance.

Why the data can’t be scraped or faked

The obvious fixes don’t work:

• The open web doesn’t have it. Broadcast and podcast speech is edited toward monolingual standard forms; natural mixed speech lives in conversations that are not published.
• Synthesis collapses on the long tail. TTS-generated code-switching inherits the same boundary errors the models already make; training on it reinforces the failure. Research programs are working on TTS-augmented approaches precisely because real data is scarce — augmentation stretches real data, it does not replace it.
• It cannot be crowdsourced casually. Natural-sounding mixed speech requires genuinely bilingual speakers, briefed on register and scenario, recorded to technical spec — not gig workers reading prompts in a noisy room.

Which leaves one path: record it, to specification, with real bilingual speakers — controlled scenarios (service calls, ordering, navigation), balanced speaker composition, defined environments, and documented consent covering AI-training use.

What good code-switching data looks like

A production-grade batch specifies: the language pair and switching style (intra-sentence vs inter-sentence), scenario and register, speaker balance by gender, age band, and regional pronunciation, audio format (16–48 kHz, channel separation), environment mix, and per-item metadata down to speaker and batch IDs. Transcription follows a convention that marks the switch points — because a transcript that flattens the mixing destroys the very signal the model needs.

That specification discipline, not microphone access, is what separates usable code-switching corpora from noise. It is also why this data commands attention from the teams building the next generation of multilingual voice systems: the languages are hard to source, the speakers are hard to qualify, and the QA has to hold across thousands of recordings.

This is one of the core lines of our language data work for AI teams — Cantonese–English and Mandarin–English code-switching audio recorded and validated at production scale, with consent and provenance documented for AI-regulation requirements. For how the human production line behind speech AI actually runs, see the AI voice you talk to was trained by humans; for the quality layer on the text side, see how to evaluate AI translation quality.