Autodub
Table of Contents
AutoDub
An advanced AI-powered tool that automatically translates and dubs YouTube videos into different languages while dynamically adjusting video speed. This project combines state-of-the-art speech recognition, translation, and voice cloning technologies to create natural-sounding dubbed videos
This sounds interesting!!!
Prerequisites
- Python 3.8+
- CUDA-capable GPU (recommended for faster processing)
- I’ll start up my GPU engines
- FFmpeg installed and added to system PATH
- https://www.ffmpeg.org/download.html
- I guess I have installed it already
Installation
- Clone the repository:
git clone https://github.com/frrobledo/AutoDub.git
cd AutoDub
How about we create a python environment first:
python -m venv myenv && source myenv/bin/activate
There is no requirement.txt:
I have generated the file let’s hope it works.
Error:
(myenv) AtulyaRaaj@fedora:~/Desktop/Projects/Python/AutoDub/AutoDub$ pip install -r requirements.txt
Traceback (most recent call last):
File "/home/AtulyaRaaj/Desktop/Projects/Python/AutoDub/AutoDub/myenv/bin/pip", line 5, in <module>
from pip._internal.cli.main import main
ModuleNotFoundError: No module named 'pip'
Solution:
source myenv/bin/activate
python -m ensurepip --default-pip
python -m pip install --upgrade pip
pip install -r requirements.txt
- Install required packages:
pip install -r requirements.txt
There are many files, Torch seems big.
Error:
Building wheel for jieba (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for jieba (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [1 lines of output]
error: could not write to 'build/bdist.linux-x86_64/wheel/./jieba/posseg/prob_emit.py': No space left on device
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for jieba
Building wheel for docopt (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for docopt (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [23 lines of output]
running bdist_wheel
The [wheel] section is deprecated. Use [bdist_wheel] instead.
/tmp/pip-build-env-o32bcv44/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py:124: SetuptoolsDeprecationWarning: bdist_wheel.universal is deprecated
!!
********************************************************************************
With Python 2.7 end-of-life, support for building universal wheels
(i.e., wheels that support both Python 2 and Python 3)
is being obviated.
Please discontinue using this option, or if you still need it,
file an issue with pypa/setuptools describing your use case.
By 2025-Aug-30, you need to update your project and remove deprecated calls
or your builds will no longer be supported.
********************************************************************************
!!
self.finalize_options()
running build
running build_py
creating build/lib
copying docopt.py -> build/lib
error: could not write to 'build/lib/docopt.py': No space left on device
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for docopt
Building wheel for gruut-ipa (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for gruut-ipa (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [6 lines of output]
running bdist_wheel
running build
running build_py
creating build/lib/gruut_ipa
copying gruut_ipa/__init__.py -> build/lib/gruut_ipa
error: [Errno 28] No space left on device
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for gruut-ipa
Building wheel for gruut_lang_de (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for gruut_lang_de (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [6 lines of output]
running bdist_wheel
running build
running build_py
creating build/lib/gruut_lang_de
copying gruut_lang_de/__init__.py -> build/lib/gruut_lang_de
error: [Errno 28] No space left on device
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for gruut_lang_de
Building wheel for gruut_lang_en (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for gruut_lang_en (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [6 lines of output]
running bdist_wheel
running build
running build_py
creating build/lib/gruut_lang_en
copying gruut_lang_en/__init__.py -> build/lib/gruut_lang_en
error: [Errno 28] No space left on device
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for gruut_lang_en
Building wheel for gruut_lang_es (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for gruut_lang_es (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [6 lines of output]
running bdist_wheel
running build
running build_py
creating build/lib/gruut_lang_es
copying gruut_lang_es/__init__.py -> build/lib/gruut_lang_es
error: [Errno 28] No space left on device
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for gruut_lang_es
Building wheel for gruut_lang_fr (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for gruut_lang_fr (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [6 lines of output]
running bdist_wheel
running build
running build_py
creating build/lib/gruut_lang_fr
copying gruut_lang_fr/__init__.py -> build/lib/gruut_lang_fr
error: [Errno 28] No space left on device
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for gruut_lang_fr
Successfully built openai-whisper gruut encodec bnnumerizer
Failed to build jieba docopt gruut-ipa gruut_lang_de gruut_lang_en gruut_lang_es gruut_lang_fr
[notice] A new release of pip is available: 24.2 -> 25.0
[notice] To update, run: pip install --upgrade pip
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (jieba, docopt, gruut-ipa, gruut_lang_de, gruut_lang_en, gruut_lang_es, gruut_lang_fr)
Solution:
pip install --no-cache-dir jieba docopt gruut-ipa gruut_lang_de gruut_lang_en gruut_lang_es gruut_lang_fr
- Install additional dependencies:
apt-get install ffmpeg # for debian based systems
sudo dnf install ffmpeg-free
For other OS, refer to the ffmpeg installation guide
- Set up API keys:
- Create a DeepL API account and add your API key to the configuration
Project Structure
├── tools/
│ ├── audio_synthesis.py # Voice cloning and audio processing
│ ├── transcriber.py # Speech recognition and translation
│ ├── video_editing.py # Video speed adjustment and editing
│ ├── video_downloader.py # YouTube video downloading
│ ├── audio_splitter_ffmpeg.py # Audio separation
│ └── logger.py # Logging utilities
├── main.py # Main execution script
└── README.md
Usage
- Run the main script:
python main.py
Enter the YouTube URL when prompted.
The script will automatically:
- Download the video
- Extract and transcribe the audio
- Separate speech from background audio
- Translate the speech
- Clone the voice in the target language
- Adjust video speed for lip-sync
- Combine everything into the final video
Find the output video in the
final_outputdirectory.
Downgraded Python:
689 sudo dnf install python3.11 -y
690 sudo alternatives --set python3 /usr/bin/python3.11
691 sudo alternatives --install /usr/bin/python3 python3 /usr/bin/python3.11 1
692 sudo alternatives --set python3 /usr/bin/python3.11
693 python3 --version
I am having CUDA memory issues.
Traceback (most recent call last):
File "/home/AtulyaRaaj/Desktop/Projects/Python/AutoDub/AutoDub/main.py", line 36, in <module>
segments, detected_language = transcribe(audio)
^^^^^^^^^^^^^^^^^
File "/home/AtulyaRaaj/Desktop/Projects/Python/AutoDub/AutoDub/tools/transcriber.py", line 58, in transcribe
model = whisper.load_model('turbo')
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/AtulyaRaaj/Desktop/Projects/Python/AutoDub/AutoDub/myenv/lib64/python3.11/site-packages/whisper/__init__.py", line 160, in load_model
return model.to(device)
^^^^^^^^^^^^^^^^
File "/home/AtulyaRaaj/Desktop/Projects/Python/AutoDub/AutoDub/myenv/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1343, in to
return self._apply(convert)
^^^^^^^^^^^^^^^^^^^^
File "/home/AtulyaRaaj/Desktop/Projects/Python/AutoDub/AutoDub/myenv/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 903, in _apply
module._apply(fn)
File "/home/AtulyaRaaj/Desktop/Projects/Python/AutoDub/AutoDub/myenv/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 903, in _apply
module._apply(fn)
File "/home/AtulyaRaaj/Desktop/Projects/Python/AutoDub/AutoDub/myenv/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 903, in _apply
module._apply(fn)
[Previous line repeated 2 more times]
File "/home/AtulyaRaaj/Desktop/Projects/Python/AutoDub/AutoDub/myenv/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 930, in _apply
param_applied = fn(param)
^^^^^^^^^
File "/home/AtulyaRaaj/Desktop/Projects/Python/AutoDub/AutoDub/myenv/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1329, in convert
return t.to(
^^^^^
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 26.00 MiB. GPU 0 has a total capacity of 3.63 GiB of which 6.81 MiB is free. Including non-PyTorch memory, this process has 3.62 GiB memory in use. Of the allocated memory 3.38 GiB is allocated by PyTorch, and 183.95 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
