---

Dubber EN→IT

Real-time local audio dubber from English to Italian. Captures system audio, transcribes it, translates it, and synthesizes it in Italian with a male or female voice depending on the speaker.

Fully local. No LLMs. No virtual cables. Open source.

---

Features

• System audio capture via WASAPI loopback (no virtual cable required)
• English transcription with Vosk large (vosk-model-en-us-0.22)
• Voice gender detection (male/female) with ECAPA-TDNN
• Automatic punctuation of the transcript with kredor/punctuate-all
• Offline EN→IT translation with Argos Translate
• Italian speech synthesis with Piper (Riccardo and Paola voices)
• Minimal GUI in Tkinter with device selection, widget locking during execution, and real-time log
• Output to a separate audio device (e.g. USB headphones), so the original stays muted
• 100% open source, usable in commercial contexts

---

Architecture

System audio (speakers)
   ↓ WASAPI loopback
VAD (webrtcvad) — segments speech
   ↓
Gender detection (ECAPA-TDNN) — on English audio
   ↓
ASR (Vosk large) — English → text
   ↓
Punctuation (transformers) — adds punctuation
   ↓
Translation (Argos) — English → Italian
   ↓
TTS (Piper) — text → Italian voice (M or F)
   ↓
Playback on USB headphones

---

Requirements

Hardware

• Windows 10/11 (tested on Windows 11)
• CPU: modern quad-core (tested on i5-10400F)
• RAM: at least 8 GB (16 GB recommended)
• GPU: optional, but recommended for ECAPA and punctuation (tested on RTX 3060)
• Two separate audio devices:
  • one for capture (e.g. speakers or main sound card)
  • one for Italian output (e.g. USB headphones)

Software

• Python 3.10+ (tested on 3.12.4)
• Up-to-date audio drivers (for WASAPI loopback)
• ~5 GB of disk space for models and dependencies

---

Installation

1. Clone or download the project

git clone https://github.com/<your-username>/dubber.git
cd dubber

Or download the ZIP and extract it to a folder of your choice.

2. Create the virtual environment

python -m venv venv
.\venv\Scripts\Activate.ps1

If PowerShell blocks activation:

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

Then retry:

.\venv\Scripts\Activate.ps1

3. Upgrade pip

python -m pip install --upgrade pip

4. Install PyTorch

If you have an NVIDIA GPU with CUDA 12.6 or newer:

pip install torch==2.14.0 torchaudio==2.11.0 --index-url https://download.pytorch.org/whl/cu126

If you don't have an NVIDIA GPU (or prefer the CPU version):

pip install torch==2.14.0 torchaudio==2.11.0 --index-url https://download.pytorch.org/whl/cpu

5. Install dependencies

pip install -r requirements.txt

6. Download the models

Vosk large (English, ~1.8 GB)

Invoke-WebRequest -Uri "https://alphacephei.com/vosk/models/vosk-model-en-us-0.22.zip" -OutFile "vosk-model-en-us-0.22.zip"
Expand-Archive -Path "vosk-model-en-us-0.22.zip" -DestinationPath "models/"
Remove-Item "vosk-model-en-us-0.22.zip"

Italian Piper voices (Riccardo M, Paola F)

New-Item -ItemType Directory -Force -Path "models/piper"
Invoke-WebRequest -Uri "https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/it/it_IT/riccardo/x_low/it_IT-riccardo-x_low.onnx" -OutFile "models/piper/it_IT-riccardo-x_low.onnx"
Invoke-WebRequest -Uri "https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/it/it_IT/riccardo/x_low/it_IT-riccardo-x_low.onnx.json" -OutFile "models/piper/it_IT-riccardo-x_low.onnx.json"
Invoke-WebRequest -Uri "https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/it/it_IT/paola/medium/it_IT-paola-medium.onnx" -OutFile "models/piper/it_IT-paola-medium.onnx"
Invoke-WebRequest -Uri "https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/it/it_IT/paola/medium/it_IT-paola-medium.onnx.json" -OutFile "models/piper/it_IT-paola-medium.onnx.json"

Argos EN→IT package (~100 MB)

python -c "import argostranslate.package; argostranslate.package.update_package_index(); avail = argostranslate.package.get_available_packages(); pkg = next(filter(lambda x: x.from_code=='en' and x.to_code=='it', avail)); argostranslate.package.install_from_path(pkg.download()); print('Installed EN->IT')"

HuggingFace models (ECAPA + punctuation): downloaded automatically on first launch. The first launch will take a few extra minutes.

---

First launch

python main.py

The program loads the models in the background. Wait for the message:

All models ready. You can start.

At that point the Start button becomes active.

---

Usage

1. Capture device (loopback): select the output endpoint you want to capture audio from (e.g. "Speakers (Realtek Audio) [Loopback]").
2. Output device (USB headphones): select the headphones where you want to hear the Italian voice.
3. Voice: Auto (from gender) for automatic selection based on the speaker, or force male/female manually.
4. Press Start.

Important: mute the capture device (the speakers), otherwise you will also hear the original English along with the Italian. WASAPI loopback keeps working even with the device muted.

During execution, the menus are locked. Press Stop to interrupt cleanly (no residual audio).

---

Video synchronization: final tip

The pipeline introduces a latency of 2-4 seconds between the original speech and the Italian voice. To sync video and dubbed audio for normal viewing, the cleanest approach is to delay the video with OBS.

Procedure

1. Open OBS and add a Source → Window Capture of the browser with YouTube.
2. On the source, click Filters.
3. Add one or more "Render Delay" filters (or the "Video Delay (Async)" filter if available for your source). Each "Render Delay" adds 500 ms; stacking several of them sums up (e.g. 10 filters = 5 seconds).
4. Project the preview fullscreen onto the monitor where you want to watch.
5. Mute the Windows speakers and do not add any audio sources in OBS. Audio is handled entirely by Dubber.
6. Start Dubber and press Start.
7. Adjust the video delay until the lip movement and the Italian voice are aligned. Start from 5000 ms and fine-tune by ear.

Note: the delay is not perfectly constant (it depends on segment length), so synchronization will be "good" but not always perfect. With a fixed delay of ~5 seconds, the misalignment is on the order of a few tenths of a second, which is tolerable.

If you need delays longer than 2-3 seconds, install the exeldro/obs-dynamic-delay plugin.

---

Configurable parameters

In config.py you can modify:

Parameter	Default	Description
VAD_MODE	2	VAD aggressiveness (0–3)
VAD_SILENCE_TAIL_MS	500	Silence to close a segment
VAD_MIN_SPEECH_MS	300	Minimum speech to start a segment
VAD_MAX_SEGMENT_MS	5000	Maximum segment duration
GENDER_MIN_SAMPLES	16000	Minimum samples to classify gender
LOG_GUI_LINES	500	Maximum lines in the GUI log

Practical tips:

• Segments too fragmented → increase VAD_SILENCE_TAIL_MS to 700–900.
• VAD not detecting weak speech → lower VAD_MODE to 1.
• VAD detecting too much noise → raise VAD_MODE to 3.
• Latency too high → lower VAD_MAX_SEGMENT_MS to 3000–4000.

---

Known limitations

• Latency 2-4 seconds between original speech and Italian voice. To synchronize, delay the video with OBS (see above).
• ASR errors on compressed audio or fast speech: Vosk large is robust but not perfect. Whisper.cpp would improve quality but is slower and requires a pipeline redesign.
• Literal translation of idiomatic constructions: Argos performs NMT without context, so some sentences sound rigid.
• Overlapping voices: if two people speak at the same time, the gender is assigned to the dominant speaker.
• Proper nouns and technical terms: may be mis-transcribed and then translated oddly.

---

Changelog

v2.2 — Clean stop

• Audio queue flush and self.running checks distributed across the pipeline
• No residual audio after Stop
• Fixed crash on log selection right after Stop

v2.1 — Thread safety and UX

• Thread-safe log via queue.Queue and root.after()
• GUI widgets locked during execution
• Start button disabled until models are ready
• Model warm-up after loading
• VAD_MAX_SEGMENT_MS reduced from 15000 to 5000

v2 — Punctuation

• Added kredor/punctuate-all model between ASR and translation
• Better English text structure → more natural Italian translation

v1 — Working base

• Full pipeline: capture → VAD → gender → ASR → translation → TTS → output
• Vosk large (vosk-model-en-us-0.22)
• ECAPA-TDNN (moorlee/gender-voice-classifier-ecapa)
• Argos Translate EN→IT
• Italian Piper voices Riccardo (M) and Paola (F)
• Tkinter GUI
• WASAPI loopback capture without virtual cables

---

Roadmap

• v3: replace Vosk with Whisper.cpp for more accurate ASR
• v3: translation with MarianMT / Opus-MT for more natural sentences
• v3: streaming ASR for latency < 2 seconds constant
• v4: PyInstaller packaging for distribution without Python

---

Component licenses

Component	License
Vosk	Apache 2.0
Argos Translate	LGPL
Piper	MIT
SpeechBrain	Apache 2.0
ECAPA-TDNN backbone	Apache 2.0
Gender classifier (moorlee)	MIT
Punctuator (kredor/punctuate-all)	MIT
Piper voices Riccardo and Paola	MIT

The project as a whole is usable in commercial contexts.

---

Credits

Project developed as an experiment in real-time local audio dubbing. Built on open source components from the Vosk, Piper, Argos, SpeechBrain, and HuggingFace communities.

---