ANTIMEME SYSTEMWORKS CREATIVE UTILITIES DIVISION RESEARCH // V1.1_BUILD

NEMOTRON 3.5

Eight synthetic voices argue about eggs for 45 seconds in eight languages, three of them speaking at once. Nemotron 3.5 Transcribe returns 75.88 to 85.09% speaker-attributed character error on it, depending on how the prompt is worded and which GPU runs it, and finds 6 of the 8 speakers. Scored against a provisional reference that no human has verified; 25 of 31 utterances still need review.

The audio is synthesized, so the generation script and the per-speaker voice stems exist. With them the flagged utterances become exact text, the overlap becomes exact spans, and the timestamp check becomes measurable. Until then every figure here is agreement with an unverified annotation.

DOC: AM-RS-2026 // NEMOTRON-3.5 C1 103 WORDS // C2 116 WORDS 1× H100 80GB // BF16 DATASET

Results

cpCER range

75.88–85.09%

11 RUNS // 2 GPUS // 10 WORDINGS // LOWER IS BETTER

Wording spread

9.21pts

WIDER THAN THE 4.88 PT PROMPT GAP

Diarization error

49.45%

COLLAR 0 // LOWER IS BETTER

JER

52.06%

DIARIZATION // LOWER IS BETTER

Overlap F1

0.5044

HIGHER IS BETTER

REPLICATED ON AN H200 THE ORDERING REVERSES: C1 81.57, C2 78.86. FOUR PARAPHRASES PER PROMPT SPREAD 5.7 TO 9.2 POINTS, WIDER THAN THE 4.88 GAP. ONE WORDING PER CONDITION DOES NOT SUPPORT A RANKING.

FIG.01Character error

cpCER // % // LOWER IS BETTER

EVERY VALID RUN ON ONE SCALE // YELLOW IS THE PUBLISHED H100 RUN, OUTLINED ARE PARAPHRASES // THE TWO FAMILIES OVERLAP // ONE PARAPHRASE IS ABSENT BECAUSE IT LOOPED INTO THE TOKEN LIMIT

FIG.02Voices over time

CONCURRENT SPEAKERS // 0-45 S

BAR HEIGHT IS HOW MANY VOICES OVERLAP // YELLOW = 3+ VOICES IN REFERENCE // RED = MODEL 3+ // MODEL EVEN CLAIMS 4 // TRUE 3-WAY 14.30-14.61 S NOT DETECTED

What the model did

Same utterance, both sides

C1 OUTPUT // RAW

Language	Reference	Nemotron	What happened
Hindi	मेरी भुर्जी कभी सूखी नहीं होती।	meri burji kabhi suki nahi hoti	Phonetically correct, written in the wrong script.
Mandarin	茶叶蛋。	Selene Chayote que el Rich-	Sounds survive as approximations. Characters never appear.
Japanese	でも、スクランブルエッグはちょっと乾いてることが……	gelb pero scramble eggs take wa- chotto kawaii teru Suki koto us ga to	Fragments of three languages, no kana or kanji.

ROWS ARE UTTERANCES FROM THE REFERENCE WITH THE MODEL'S OWN WORDS, TAKEN BY TIME OVERLAP, NOTHING CORRECTED

ALL THREE LANGUAGES ARE LISTED AS SUPPORTED FOR MULTISPEAKER ASR IN NVIDIA'S MATRIX // HINDI LEADING, JAPANESE PRODUCTION-READY, MANDARIN SUPPORTED // CARD REV 9D8CD6B7 // THIS IS A MODEL RESULT, NOT A COVERAGE GAP

Method

Model: nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B, BF16
Audio: 45.000 s, 44.1 kHz, 8 speakers, 8 languages
Conditions: C1 operator prompt // C2 model-card prompt
cpCER: speaker-attributed, UAX #29 graphemes, optimal identity assignment
Diarization: DER collar 0, skip_overlap false // pyannote 4.2.0
Reference: 31 utterances, provisional, 25 flagged

Reproduce

# from the dataset: evaluation/
$ python -m venv .venv
$ .venv/bin/python -m pip install -r requirements.txt
$ .venv/bin/python evaluate.py
$ .venv/bin/python verify.py

WHY SPEAKER-ATTRIBUTED: OVERLAPPING SPEECH HAS NO TOTAL ORDER, SO FLATTENING IT CHARGES A CORRECT TRANSCRIPT FOR WORD ORDER. cpCER CONCATENATES EACH SPEAKER'S OWN TOKENS, THEN ASSIGNS PREDICTED IDENTITIES OPTIMALLY.