Speaker diarization test: who said what in a 168-minute Chinese podcast

Summary

• Test source: one 168-minute Chinese multi-guest talk episode with 6 real speakers. The source is anonymised on purpose; only measurements are published.
• Model: NVIDIA Nemotron 3 Diarization (Sortformer, about 99M parameters, open weights, commercial-friendly license), run with no tuning.
• Result on a hand-labelled 20-minute window: it found 6 clusters for 6 speakers, got 30 of 33 speaker-change places right and 56 of 59 sentence pieces right.
• The speaker ids returned by the cloud ASR provider on the same episode were much less reliable: 48 ids for 6 people, which collapsed to 19 after merging.
• Status: we are integrating this model into our transcription backend. Today, speaker labels in Audio2SRT come from the ASR provider.

Why this test

A subtitle file for a multi-guest conversation is only useful if each line is attributed to the right person. For Chinese podcasts, fast turn-taking, short interjections and speakers who talk over each other make attribution hard. We wanted a number for how well a speaker diarization model does on this kind of audio, and compared it with the speaker ids that a cloud ASR provider returns.

Method

1. Source: a 168-minute Chinese multi-guest episode, 6 speakers.
2. Ground truth: a human labelled a 20-minute window by hand, marking every place where the speaker changes (33 places) and the speaker of each sentence piece (59 pieces, 6 true speakers).
3. Model: Nemotron 3 Diarization, default settings, no per-episode tuning, run on a 20-thread CPU.
4. Scoring: a speaker-change place counts as correct if the model puts a speaker boundary there and assigns the right cluster on each side. A sentence piece counts as correct if its dominant cluster maps to the right true speaker. We also report diarization error rate (DER).
5. Baselines: the cloud ASR provider's own speaker ids on the same episode, a voiceprint approach, and the model's low-latency streaming configuration.

Results

Per-speaker hits (sentence pieces, anonymised as Speaker A to F):

Comparisons on the same episode:

What went wrong

Both kinds of error were small in number and easy to describe:

• A speaker change in the middle of a sentence, which a sentence-level view cannot represent.
• Interjections shorter than one second, which the model tends to absorb into the neighbouring speaker.

Most of the 15.3% missed speech came from pauses inside sentences, not from lost speakers.

What this means for your SRT

Speaker labels in a subtitle file decide who gets credited for each line. With a diarization model that finds the right number of speakers, a transcript of a long multi-guest episode needs far less manual cleanup than with provider ids that split one person into many.

Our integration plan is:

• Use the diarization output as the speaker-segment source, and map it onto word timestamps by time overlap.
• Interjections under one second inherit the neighbouring speaker and are marked low-confidence, so you know which lines to check.
• Speakers are labelled Speaker 1, 2, 3 and so on; a person names them.

This is not live in Audio2SRT yet. Today, speaker labels come from the ASR provider and should be reviewed. To try the current pipeline, see MP3 to SRT and pricing.

Limits

One episode, one language, one 20-minute hand-labelled window, 6 speakers. These numbers describe this test, not a general guarantee. Other recordings, with more overlap or fewer microphones, will give different results.