Here's the command that'd add pillar- or letterboxing for a fixed output width. It's a tad long, but you'll have to specify the padding some way.

First, in your shell define output width and height:

    width=700
    height=400

Then run the command:

    ffmpeg -i in.mp4 -filter:v "scale=iw*min($width/iw\,$height/ih):ih*min($width/iw\,$height/ih), pad=$width:$height:($width-iw*min($width/iw\,$height/ih))/2:($height-ih*min($width/iw\,$height/ih))/2" out.mp4

This is stripped down to the bare essentials needed to resize and pad—add your other video and audio options as you see fit. Note that the numbers for `width` and `height` have to be divisible by 2 in order to work for most codecs.

Here's the explanation of what's going on:

- Scaling:
 - First we need to figure out whether to scale by width or height. 
 - To do this, we divide the output width by the input width, and output height by input height. This will give us the scale factors for each dimension.
 - We then check which one is lower (with `min()`) and choose only that factor for resizing.
 - Finally, we multiply both input width and height by that factor (`iw*min(…):ih*min(…)`).
- Padding:
 - `$width:$height` is the output width and height
 - To figure out where to place the resulting video, we need to subtract the scaled width from the maximum output width, and the scaled height from the maximum output height. 
 - The scaled widths and heights are the expressions from the `scale` filter.
 - We divide the resulting offset by 2 to add borders at both sides.