Create a Video Wall With FFmpeg

FFmpeg can be used a to create a video wall or what might also be referred as a tiled mosaic. The technique can be used to create a mosaic with various dimensions but it best suited to scenarios where you want 4, 9 or 16 tiles.

FFmpeg is a free cross-platform solution to record, convert and stream audio and video. This technique uses the FFmpeg and its xstack filter.

Create a 2×2 Video Wall

For a 2×2 set of tiles you need 4 source video files. You can reuse files but in most cases if you are creating a video with 4 tiles you will be using 4 source files.

I’ll assume you are creating an output file with a resolution of 1920×1080. Each tile would then have a resolution of 960×540.

The first step is to create a text file to store the details of the filter.

Create a text file named “filter_2x2.txt” with the following contents:

[0:v]scale=960x540,setpts=PTS-STARTPTS[v0];
[1:v]scale=960x540,setpts=PTS-STARTPTS[v1];
[2:v]scale=960x540,setpts=PTS-STARTPTS[v2];
[3:v]scale=960x540,setpts=PTS-STARTPTS[v3];
[v0][v1][v2][v3]xstack=inputs=grid=2x2

The input files are scaled and given a label (e.g. v0). These labels are feed into the xstack filter. Placing the filter details in a separate file makes it much easier to read/update and the terminal command is much more manageable.

Here is the terminal command that will take your 4 source files and create the video mosaic.

ffmpeg -i vid1.mp4 -i vid2.mp4 -i vid3.mp4 -vid4.mp4 \
-filter_complex_script filter_2x2.txt VideoWall_2x2.mp4

The output file is VideoWall_2x2.mp4.

The command above has “\” characters at the end of each line which allows a one line command to be split across multiple lines. You should be able to cut and paste the whole block into your terminal. Or you can remove the “\” to form a long, one line command.

Create a 3×3 Video Wall

The technique for creating a 3×3 mosaic with 9 tiles is the same but requires more content in the filter file and a longer terminal command.

Each tile would be 640×360 pixels in size so the filter is slightly different to the previous 2×2 filter.

Create a file named “filter_3x3.txt” with the following contents:

[0:v]scale=640x360,setpts=PTS-STARTPTS[v0];
[1:v]scale=640x360,setpts=PTS-STARTPTS[v1];
[2:v]scale=640x360,setpts=PTS-STARTPTS[v2];
[3:v]scale=640x360,setpts=PTS-STARTPTS[v3];
[4:v]scale=640x360,setpts=PTS-STARTPTS[v4];
[5:v]scale=640x360,setpts=PTS-STARTPTS[v5];
[6:v]scale=640x360,setpts=PTS-STARTPTS[v6];
[7:v]scale=640x360,setpts=PTS-STARTPTS[v7];
[8:v]scale=640x360,setpts=PTS-STARTPTS[v8];
[v0][v1][v2][v3][v4][v5][v6][v7][v8]xstack=grid=3x3

It’s the same approach as the 2×2 grid but there are 9 source files and 9 sets of filters.

Here is the terminal command that will take your 9 source files and create the video mosaic.

ffmpeg -i vid1.mp4 -i vid2.mp4 -i vid3.mp4 \
-i vid4.mp4 -i vid5.mp4 -i vid6.mp4 \
-i vid7.mp4 -i vid8.mp4 -i vid9.mp4 \
-filter_complex_script filter_3x3.txt VideoWall_3x3.mp4

The output file is VideoWall_3x3.mp4.

Create a 4×4 Video Wall

Creating a 4×4 mosaic with 16 tiles is the same but once again requires more content in the filter file and an even longer terminal command.

Each tile needs to be 480×270.

Create the filter_4x4.txt with the following contents:

[0:v]scale=480x270,setpts=PTS-STARTPTS[v0];
[1:v]scale=480x270,setpts=PTS-STARTPTS[v1];
[2:v]scale=480x270,setpts=PTS-STARTPTS[v2];
[3:v]scale=480x270,setpts=PTS-STARTPTS[v3];
[4:v]scale=480x270,setpts=PTS-STARTPTS[v4];
[5:v]scale=480x270,setpts=PTS-STARTPTS[v5];
[6:v]scale=480x270,setpts=PTS-STARTPTS[v6];
[7:v]scale=480x270,setpts=PTS-STARTPTS[v7];
[8:v]scale=480x270,setpts=PTS-STARTPTS[v8];
[9:v]scale=480x270,setpts=PTS-STARTPTS[v9];
[10:v]scale=480x270,setpts=PTS-STARTPTS[v10];
[11:v]scale=480x270,setpts=PTS-STARTPTS[v11];
[12:v]scale=480x270,setpts=PTS-STARTPTS[v12];
[13:v]scale=480x270,setpts=PTS-STARTPTS[v13];
[14:v]scale=480x270,setpts=PTS-STARTPTS[v14];
[15:v]scale=480x270,setpts=PTS-STARTPTS[v15];
[v0][v1][v2][v3][v4][v5][v6][v7][v8][v9][v10][v11][v12][v13][v14][v15]xstack=grid=4x4

It’s the same approach as the 3×3 grid but there are 16 source files and 16 sets of filters.

Here is the terminal command that will take your 16 source files and create the video mosaic.

ffmpeg -i vid1.mp4 -i vid2.mp4 -i vid3.mp4 -i vid4.mp4 \
-i vid5.mp4 -i vid6.mp4 -i vid7.mp4 -i vid8.mp4 \
-i vid9.mp4 -i vid10.mp4 -i vid11.mp4 -i vid12.mp4 \
-i vid13.mp4 -i vid14.mp4 -i vid15.mp4 -i vid16.mp4 \
-filter_complex_script filter_4x4.txt VideoWall_4x4.mp4

The output file is VideoWall_4x4.mp4.

Correcting the Frame Rate and Excluding Audio

This technique works really well if all the clips are the same frame rate. In my project the frame rate varied and it caused some error messages.

You can force the inputs to comply with a set framerate by inserting the “-r” parameter just after the input parameters. In the example below I’ve added “-r 23.976” to force the output to use 23.976 frames per second.

ffmpeg -i vid1.mp4 -i vid2.mp4 -i vid3.mp4 -vid4.mp4 \
-r 23.976 \
-filter_complex_script filter_2x2.txt VideoWall_2x2.mp4

You can create the output without any audio by adding the “-an” parameter as shown below:

ffmpeg -i vid1.mp4 -i vid2.mp4 -i vid3.mp4 -vid4.mp4 \
-an \
-filter_complex_script filter_2x2.txt VideoWall_2x2.mp4

Of course you can also add them both:

ffmpeg -i vid1.mp4 -i vid2.mp4 -i vid3.mp4 -vid4.mp4 \
-r 23.976
-an \
-filter_complex_script filter_2x2.txt VideoWall_2x2.mp4

Triming the Input Files

If you need to adjust the start points

Tweaking Aspect Ratio of Inputs

In the filter files shown before we used this format to handle the incoming video clips:

[0:v]scale=480x270,setpts=PTS-STARTPTS[v0];

This scales input #1 to the tile size and restarts the timestamp for the video stream. By adding the crop filter we can grab a section with the correct aspect ratio. This is then scaled without distortion by the “scale” filter.

[0:v]crop=1600:900:150:15,scale=480x270,setpts=PTS-STARTPTS[v0];

Trimming the Input

If you need to adjust the in points of the input streams you can add the “trim” filter to each video stream as required:

[0:v]trim=10:30,scale=480x270,setpts=PTS-STARTPTS[v0];

In this example the first 10 seconds are ignored and the next 20 seconds are passed on to the xstack filter.

This could be combined with the crop filter if required.

Remember each video stream input can have different filters. Although for creating video walls your scale filter will always be the same because all the tiles should be the same size.

Creating a Custom Video Grid

You don’t have to stick with an equal number of rows and columns but you would need to make a number of changes.

• You need to appropriate number of input files. They would have to be included as “-i” references in the terminal command.
• The filter file would need to reference all those input files, scale them to the correct tile size, and assign a label.
• The labels would need to be included in the xstack line.
• The xstack “grid” parameter would need to be set to the appropriate value. (e.g. 2×4)

Be-aware that once you deviate from tiles that match the aspect ratio of your source files you may not get nice results.