FFMPEG: m4a to mp3 conversion using multiple cores

Question

I use the following command to convert m4a format to mp3

ffmpeg -i audio.m4a -acodec libmp3lame audio.mp3

I've 32 x86 cores, however libmp3lame processes in a single thread. I know libmp3lame does not support multithreading, thus I'm open to other alternates that can be executed in ubuntu CLI.

If audio.m4a is 2hours long video, that usually takes > 3minutes and speed appears to be 45x to 50x.

My primary goal is to convert multiple youtube videos in mp3 format within seconds.

Update 1:

Since I'm using 32 cores CPU, when there's only one video conversion it utilizes just one core. So in that cases how to use multiple cores, to get output faster. I want to achieve the maximum from the CPU. Also if FFmpeg is not the answer, is there any other way.

Answer 1

GNU Parallel sounds exactly like what you are looking for.

Here is an example of how you could use GNU Parallel (assuming you already have a directory with all your *.m4a files named videos) using your original commands:

find /path/to/videos -name "*.m4a" | parallel 'ffmpeg -i {} -acodec libmp3lame {.}.mp3'

Two things to explain:

1: The output of the find command (i.e. the list of .m4a files to convert) will be piped into the parallel command, which will in turn (by default) execute as many jobs as there are cores. In essence it is splitting the output from the find command, and distributing it across the different jobs to work on.

2: Per the GNU Parallel Tutorial, {} is where your arguments will go, {.} will substitute in your arguments and remove the terminal extension (in this case the .m4a), and your output will be the original file name but with .mp3 extension.

NOTE: This example will write all your converted files to where your original files are. If this is an issue, then merely pass a directory to store the converted files in the argument string containing the ffmpeg command:

find /path/to/videos -name "*.m4a" | parallel 'ffmpeg -i {} -acodec libmp3lame /path/to/converted/vids/{.}.mp3'

Answer 2

Since the encoder is single-threaded only, in order to make the encoding the most efficient on a multi-core system, all you can do is launch several processes in parallel.

For example, if you're in a shell:

ffmpeg -i input1.mp4 -c:a libmp3lame output1.mp3 &
ffmpeg -i input2.mp4 -c:a libmp3lame output2.mp3 &
…

The & puts the command into the background. The commands will be run in parallel and the load should be distributed among your CPU cores.

Answer 3

There are multi-threaded implementations of lame mp3 encoder on the internet. I could track one here

https://github.com/dheller1/lame_pthreads

If you have large storage space, you could use ffmpeg to convert all m4a files into wav files and then point lame_pthreads to that folder.