FFmpeg: Applying effects to a video

Question

I'm trying to apply some effects to a video by adjusting colour, contrast, brightness, etc. Here are some of the effects I was trying. And following is the command I used to apply the "Kelvin" effect.

ffmpeg -i 3.mp4 -c:v libx264 -c:a libfaac -filter_complex "[0:v]eq=1.0:0:1.3:2.4:0.175686275:0.103529412:0.031372549:0.4[outv]" -map [outv] out.mp4

FFmpeg documentation helped me to figure out the boundaries for each value for eq filter. But still I'm not getting the expected output for "Kelvin" effect as in the link above. I calculated the rg, gg, bg values according to the values for sepia value used in css. (r=112, g=66, b=20 are the values I used for converting sepia to rgb) Following is the simple math I used to get the rg, gg, bg values.

rg = (112 / 255) * (gamma_r max - gamma_r min) * sepia value

Here gamma_r max is 10 and gamma_r min is 0.1 according to the documentation.

What am I doing wrong here? Is there any other alternative to get this done?

Answer 1

The eq filter adjusts the color channels relative to their current state, that is, increasing or decreasing their intensity (like a "volume" knob in sound). To achieve your target color temperature with eq you'd have to calculate the current color temperature of each region in the photo and then modify it - something for which you need a frame server such as AviSynth. But all this is not really needed, because you don't really want to fully control your white balance, you just want to apply a fixed filter that will result in a specific color tone - in this case sepia. For that you should use instead the colorchannelmixer filter which provides a way to manage the white balance of the pixels relative to each other. The filter documentation has a specific example for sepia:

colorchannelmixer=.393:.769:.189:0:.349:.686:.168:0:.272:.534:.131

How it works (very simplified explanation)

The colorchannelmixer filter describes the image as if it has 4 color channels - called Red, Green, Blue, and Alpha (the "mask" channel). By default, every channel represents the intensity of the color after which it is named as 1.0. So, the "Red" channel represents the portion of the image which is red times 1.0, the portion which is green times 0.0, the portion which is blue as 0.0., and the portion which is alpha times 0.0. Likewise, each other channel has a value of 1.0 for "its" color and 0.0 for all others. Now the filter enables you to "steal" a color from its channel and inject its "energy" to another channel. For example, you could increase the value of red in the "Green" channel to 1.0, and change the value of red in the "Red" channel to 0.0, so now the intensity of green will increase in every pixel by the original intensity of red, without leaving any red at all in the image, and without changing the relative intensity of blue and alpha. When applying this concept on all 16 color combinations you get a matrix that defines the resulting color intensity of each pixel as a product of all the original color intensities. The result is then normalized, and you get a simple way of expressing the color transformation curve.

The above is somewhat simplistic, and to understand how it applies to your specific question you can read more about Color Temperature theory. Note though that with the colorchannelmixer filter you can do also other stuff such as generate color negatives, reduce color depths to achieve comics-like effects, and much more.

Hope this works well for you!

Answer 2

Finally created the effects by chaining colorchannelmixer and eq filters. According to the effects mentioned here, I applied sepia effect using colorchannelmixer filter and adjusted rest of the properties using eq filter.

For sepia effect you can use sepia matrix by substituting amount with sepia value. All the other properties can be applied directly with out doing any math except brightness value. I substituted brightness value with,

(brightness - contrast) / 20

which resulted more likely output. Someone can correct this as this is not the exact calculation. Following is the FFmpeg command for "Willow" effect after doing these calculations.

ffmpeg -i input_image -filter_complex "
[0:v]colorchannelmixer=0.98786:0.01538:0.00378:0.0:0.00698:0.99372:0.00336:0.0:0.00544:0.01068:0.98262:0.0[colorchannelmixed];
[colorchannelmixed]eq=0.85:0.0175:0.02:1.0:1.0:1.0:1.0:1.0[color_effect]" -map [color_effect] -c:v png output_image

Hope this will help someone else. Cheers!