Audio8-TTS-Realistic-Air: Edge0's 0.6B Multilingual Text-to-Speech Model Explained

OverviewAudio8-TTS-Realistic-Air is Edge0’s 0.6B multilingual text-to-speech model for speech generation with or without a reference voice. It supports zero-shot voice cloning from reference audio and its exact transcript, and lists 23 supported languages. Maintained by Edge0.Best use casesGenerate speech from text in the 23 listed languages, or clone a voice using a clean reference recording and matching transcript. It can also be deployed through an SGLang Omni adapter with an OpenAI-compatible speech API.LimitationsLanguage quality varies. Reported error-rate and speaker-similarity metrics assess intelligibility and similarity, not perceived realism; the supplied evaluation does not directly quantify realism. Runtime checks were performed on an NVIDIA H20 in BF16, but throughput and streaming were not benchmarked. The model uses custom Transformers code, so loading requires trust_remote_code=True; review repository files before enabling it.How it comparesCompared with the earlier Preview release, Realistic Air adds 12 supported languages (23 versus 11). On the reported Seed-TTS aggregate, its error rate is 1.25% versus Preview’s 1.23%, while SIM is 0.704 versus 0.682; the releases were evaluated separately. On the shared eight-language CV3 subset, Realistic Air reports 4.56% error versus Preview’s 4.80%. The supplied reference comparison cautions that different evaluation pipelines and text normalization prevent a strictly matched ranking. Other Audio8 versions include Audio8-TTS-Realistic-0.6b, Audio8-TTS-Preview-0.6b, and Audio8-TTS-Preview-0.1b; the supplied evidence does not establish detailed capability differences for those alternatives.Technical specificationsDualAR architecture: a slow autoregressive transformer predicts one semantic token per audio frame; a fast autoregressive transformer predicts codec codebooks conditioned on the slow hidden state and preceding codebooks. Main model: 601,159,424 parameters, excluding the separately bundled codec. Slow AR: 24 layers, width 896, 14 attention heads, 2 KV heads. Fast AR: 4 layers, width 896, 14 attention heads, 2 KV heads. Audio uses 10 codebooks with 4,096 entries each. The codec operates at 44.1 kHz and 2,048 samples per model frame (about 21.5 frames/s); context is up to 2,048 packed text/audio positions. The repository includes the checkpoint, codec, tokenizer, processor, and custom Transformers code. Python 3.10 or newer and a CUDA-capable GPU are recommended. Runtime smoke tests used PyTorch 2.8.0, Transformers 4.57.3, and an NVIDIA H20 with BF16. License metadata specifies Apache-2.0.Model inputs and outputsInputsText is passed to the processor as a list. For voice cloning, provide reference audio and a transcript that exactly matches the spoken content; a clean recording is recommended. Reference-free generation needs only text. The example uses max_new_tokens, temperature, top_p, top_k, and do_sample as generation controls. Supported languages: Cantonese, Chinese, Dutch, English, French, German, Italian, Japanese, Korean, Polish, Spanish, Arabic, Finnish, Hindi, Indonesian, Malay, Norwegian, Portuguese, Russian, Swedish, Thai, Turkish, and Vietnamese.OutputsGeneration returns audio codes that are decoded by the bundled codec into waveforms. The example writes a WAV file at model.config.codec_sample_rate. The SGLang Omni adapter supports a non-streaming OpenAI-compatible /v1/audio/speech API, reference-free generation, and zero-shot voice cloning.Getting startedimport soundfile as sf import torch from transformers import AutoModel, AutoProcessor model_id = "Edge0/Audio8-TTS-Realistic-Air" device = "cuda" if torch.cuda.is_available() else "cpu" dtype = torch.bfloat16 if device == "cuda" else torch.float32 processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True) model = AutoModel.from_pretrained( model_id, trust_remote_code=True, dtype=dtype ).eval().to(device) inputs = processor( text=["Welcome to Audio8 TTS."], reference_audio=["reference.wav"], reference_text=["Exact transcript of the reference recording."], return_tensors="pt", ) inputs = {name: value.to(device) for name, value in inputs.items()} with torch.inference_mode(): output = model.generate( **inputs, max_new_tokens=1024, temperature=0.8, top_p=0.95, top_k=50, do_sample=True, return_dict_in_generate=True, ) waveforms, lengths = model.decode_audio(output.codes) audio = waveforms[0, : int(lengths[0])].float().cpu().numpy() sf.write("output.wav", audio, model.config.codec_sample_rate) Frequently asked questionsQ: Can it generate speech without a reference recording?A: Yes. Pass text to the processor without reference-audio or reference-transcript fields, then use the same generation and decoding steps.Q: What is required for zero-shot voice cloning?A: Provide a clean reference recording and a transcript that exactly matches what is spoken in it.Q: How many languages are supported?A: The model lists 23 supported languages, including 12 additions over the earlier Preview release.Q: Does the 0.6B parameter count include the codec?A: No. The 601,159,424-parameter count is for the main model; the bundled codec is separate.Q: What software and hardware are recommended?A: Python 3.10 or newer and a CUDA-capable GPU are recommended. The documented smoke test used PyTorch 2.8.0, Transformers 4.57.3, and BF16 on an NVIDIA H20.Q: Does the evaluation prove improved perceived realism?A: No. The reported error-rate and speaker-similarity metrics measure intelligibility and similarity, not perceived realism directly.Q: Is streaming performance documented?A: No. The SGLang Omni runtime check covered a non-streaming speech API; throughput and streaming were not benchmarked.This is a simplified guide to an AI model called Audio8-TTS-Realistic-Air maintained by Edge0. If you like these kinds of analysis, join AIModels.fyi or follow us on Twitter.