Pad Video

This article explains how you can fit a 16:9 video into a 4:3 frame by sizing and padding the original picture.

Source Video

For a source video we use the MP4 file from the TED talk video What’s the next window into our universe? by Andrew Connolly. The original video format is Wide 480p or 16:9, 854 x 480.

Sample Code

This code takes a 16:9 480p (854x480) video, and squeezes it into a 4:3 640x480 frame, while maintaining the original 16:9 picture aspect ratio. The audio stream is copied from the source as is. The input and output file paths are hardcoded in the source code, no command line parsing is used.

The snippets in this section are from the simple_video_pad macOS sample.

Linux and Windows samples are also available:

• simple_video_pad Linux sample

• simple_video_pad Windows sample

Initialize AVBlocks

The first step in any AVBlocks application is to initialize the library. This must be done before using any other AVBlocks functionality. The Library::initialize() method sets up the internal state and loads necessary codecs. Always remember to call Library::shutdown() at the end of your program to properly clean up resources and release any allocated memory.

int main(int argc, const char *argv[]) {
    Library::initialize();

    auto inputFile = primo::ustring(L"AndrewConnolly_2014.mp4");
    auto outputFile = primo::ustring(L"AndrewConnolly_2014_640x480_Padded.mp4");

    // ... padding code ...

    Library::shutdown();
    return 0;
}

Configure Transcoder and Pad

We probe the input file with a MediaInfo object, and create an input socket from it. The output socket is created by cloning the input socket, so it starts out with an identical configuration, and then we set the output file path on it. The transcoder is the core component that performs the actual padding - it takes the 854x480 video from the input, squeezes it to fit in a 4:3 frame while maintaining the 16:9 aspect ratio, and adds padding on the top and bottom.

Probe Input and Create Sockets

First, we probe the input file and create the input socket. Then we clone it to create the output socket and set the output file path.

auto inputInfo = primo::make_ref(Library::createMediaInfo());
inputInfo->inputs()->at(0)->setFile(inputFile);

if (inputInfo->open()) {
    auto inputSocket = primo::make_ref(
        Library::createMediaSocket(inputInfo.get())
    );

    // Start with the same output as the input
    auto outputSocket = primo::make_ref(inputSocket->clone());
    outputSocket->setFile(outputFile);
} else {
    std::cerr << "inputInfo->open() failed: "
               << primo::ustring(inputInfo->error()->message())
               << std::endl;
}

Configure Output Video Settings

Next, we get the output video pin and configure the frame dimensions and display ratio. We set the new frame size to 640x480 and change the display ratio from 16:9 to 4:3.

auto outVideoPin = outputSocket->pins()->at(0);
auto outVideoStream = (VideoStreamInfo*)outVideoPin->streamInfo();

// Set the new frame width and height to 640 x 480
outVideoStream->setFrameWidth(640);
outVideoStream->setFrameHeight(480);

// Set the display ratio to 4:3
outVideoStream->setDisplayRatioWidth(4);
outVideoStream->setDisplayRatioHeight(3);

Apply Padding Parameters

To pad the video, we calculate how much padding to add on top and bottom. Since the input is 16:9 (854x480) and we want a 4:3 frame (640x480), the height of the 16:9 content in 640px width is 640 * 9 / 16 = 360. We pad (480 - 360) / 2 = 60 pixels on each side.

auto outVideoPinParams = primo::make_ref(Library::createParameterList());

// The input video is 16:9, 854 x 480,
// to squeeze it in 640 x 480 and keep the 16:9 display ratio,
// the height has to be: 640 * 9 / 16

// Pad by (480 - 640 * 9 / 16) / 2 pixels on each side.
auto padTop = primo::make_ref(Library::createIntParameter());
padTop->setName(Pad::Top);
padTop->setValue((480 - 640 * 9 / 16) / 2);
outVideoPinParams->add(padTop.get());

auto padBottom = primo::make_ref(Library::createIntParameter());
padBottom->setName(Pad::Bottom);
padBottom->setValue((480 - 640 * 9 / 16) / 2);
outVideoPinParams->add(padBottom.get());

// The padding will trigger downscaling,
// so we also set the interpolation method,
// for best result when downscaling, use InterpolationMethod::Super
auto interpolationMethod = primo::make_ref(Library::createIntParameter());
interpolationMethod->setName(Resize::InterpolationMethod);
interpolationMethod->setValue(InterpolationMethod::Super);
outVideoPinParams->add(interpolationMethod.get());

outVideoPin->setParams(outVideoPinParams.get());

Create and Run Transcoder

Finally, we create the transcoder, add the input and output sockets, and run the operation. The setAllowDemoMode(true) call allows the transcoder to work even without a valid license (useful for testing, but not recommended for production).

auto transcoder = primo::make_ref(Library::createTranscoder());
transcoder->inputs()->add(inputSocket.get());
transcoder->outputs()->add(outputSocket.get());

// Allow demo mode for the transcoder when
// using the demo version of the library
transcoder->setAllowDemoMode(true);

// Run the transcoder
if (transcoder->open()) {
    transcoder->run();
    transcoder->close();
} else {
    std::cerr << "transcoder->open() failed: "
              << primo::ustring(transcoder->error()->message())
              << std::endl;
}

Complete C++ Code

Here’s the complete working example that demonstrates video padding using AVBlocks. The input and output file paths are hardcoded, so no command line parsing is required.

#include <primo/avblocks/avb.h>
#include <primo/platform/reference++.h>
#include <primo/platform/ustring.h>

using namespace primo;
using namespace primo::codecs;
using namespace primo::avblocks;

int main(int argc, const char *argv[]) {
    Library::initialize();

    auto inputFile = primo::ustring(L"AndrewConnolly_2014.mp4");
    auto outputFile = primo::ustring(L"AndrewConnolly_2014_640x480_Padded.mp4");

    using namespace Param::Video;

    auto inputInfo = primo::make_ref(Library::createMediaInfo());
    inputInfo->inputs()->at(0)->setFile(inputFile);

    if (inputInfo->open()) {
        auto inputSocket = primo::make_ref(
            Library::createMediaSocket(inputInfo.get())
        );

        // Start with the same output as the input
        auto outputSocket = primo::make_ref(inputSocket->clone());
        outputSocket->setFile(outputFile);

        // Get the output video pin
        auto outVideoPin = outputSocket->pins()->at(0);

        auto outVideoStream = (VideoStreamInfo*)outVideoPin->streamInfo();

        // Set the new frame width and height to 640 x 480
        outVideoStream->setFrameWidth(640);
        outVideoStream->setFrameHeight(480);

        // Set the display ratio to 4:3
        outVideoStream->setDisplayRatioWidth(4);
        outVideoStream->setDisplayRatioHeight(3);

        // set Pad.Top and Pad.Bottom
        auto outVideoPinParams = primo::make_ref(Library::createParameterList());

        // The input video is 16:9, 854 x 480,
        // to squeeze it in 640 x 480 and keep the 16:9 display ratio,
        // the height has to be: 640 * 9 / 16

        // Pad by (480 - 640 * 9 / 16) / 2 pixels on each side.
        auto padTop = primo::make_ref(Library::createIntParameter());
        padTop->setName(Pad::Top);
        padTop->setValue((480 - 640 * 9 / 16) / 2);
        outVideoPinParams->add(padTop.get());

        auto padBottom = primo::make_ref(Library::createIntParameter());
        padBottom->setName(Pad::Bottom);
        padBottom->setValue((480 - 640 * 9 / 16) / 2);
        outVideoPinParams->add(padBottom.get());

        // The padding will trigger downscaling,
        // so we also set the interpolation method,
        // for best result when downscaling, use InterpolationMethod::Super
        auto interpolationMethod = primo::make_ref(Library::createIntParameter());
        interpolationMethod->setName(Resize::InterpolationMethod);
        interpolationMethod->setValue(InterpolationMethod::Super);
        outVideoPinParams->add(interpolationMethod.get());

        outVideoPin->setParams(outVideoPinParams.get());

        // Create Transcoder and configure it with
        // the input and output sockets
        auto transcoder = primo::make_ref(Library::createTranscoder());
        transcoder->inputs()->add(inputSocket.get());
        transcoder->outputs()->add(outputSocket.get());

        // Allow demo mode for the transcoder when
        // using the demo version of the library
        transcoder->setAllowDemoMode(true);

        // Run the transcoder
        if (transcoder->open()) {
            transcoder->run();
            transcoder->close();
        } else {
            std::cerr << "transcoder->open() failed: "
                      << primo::ustring(transcoder->error()->message())
                      << std::endl;
        }
    } else {
        std::cerr << "inputInfo->open() failed: "
                   << primo::ustring(inputInfo->error()->message())
                   << std::endl;
    }

    Library::shutdown();
    return 0;
}

How to Run

See the build instructions for macOS and the simple_video_pad example for details.

Download the sample video, the MP4 file from the TED talk video What’s the next window into our universe? by Andrew Connolly (Wide 480p, 854x480, 16:9):

cd samples/darwin/simple_video_pad
curl -L -o AndrewConnolly_2014.mp4 "https://archive.org/download/AndrewConnolly_2014/AndrewConnolly_2014.mp4"

Run the sample from the sample directory (the input/output file paths are relative to the working directory):

../../../bin/x64/simple_video_pad

The padded output file AndrewConnolly_2014_640x480_Padded.mp4 will be created in the samples/darwin/simple_video_pad directory.