Xabe.FFmpeg requirements

FFmpeg has to be installed on end user device. To use this library one of those conditions must be fulfilled:
  • FFmpeg directory added in PATH (contains ffprobe and ffmpeg executables)
  • FFmpeg.SetExecutablesPath method has to be set to FFmpeg directory path
  • FFmpeg executables have to be in the same directory with application executable
  • FFmpeg executables could be downloaded by using FFmpegDownloader.GetLatestVersion() - located in Xabe.FFmpeg.Downloader package

Default, the library is trying to find FFmpeg executables names containing "ffprobe", "ffmpeg". This function is case insensitive. Those names can be changed in FFmpeg.FFmpegExecutableName and FFmpeg.FFprobeExecutableName.
Install the Xabe.FFmpeg NuGet package via NuGet:

PM> Install-Package Xabe.FFmpeg

Documentation

C# FFmpeg wrapper conception

Xabe.FFmpeg uses only streams to operate on every media file. Most common conversions can be done with few simple steps:

  1. Extract streams from input file or create new streams with outside source (e.g. mp4 file)
  2. Manipulate with streams with embedded methods
  3. Add selected streams to conversion
  4. Set output
  5. Start conversion
Operating on streams (not on files) allows to work with multi-streams media files and single-stream files with the same way.

Getting streams from media file

Basic properties of media file can be read by MediaInfo class:


string filePath = Path.Combine("C:", "samples", "SampleVideo.mp4");
IMediaInfo mediaInfo = await FFmpeg.GetMediaInfo(filePath);

IMediaInfo contains basic properties about media:

Xabe.FFmpeg MediaInfo diagram

More properties can be found in specific stream.

Getting metadata from media file

It is possible to get additional metadata using FFprobe. To run FFprobe with custom arguments:


string result = await Probe.New()
    .Start($"-skip_frame nokey -select_streams v:0 -show_entries frame=pkt_pts_time {Resources.Mp4}");
Method returns console FFprobe output in format specified in input arguments.

Streams

Streams are basic structure in Xabe.FFmpeg so it is good to know their architecture:

Xabe.FFmpeg streams hierarchy

Diagram shows only most important properties and methods, for more information look at implementation.

.NET Video Conversion

Xabe.FFmpeg.Conversion is the main class to handle FFmpeg conversions. User can manipulate audio, video and subtitle through this class.

Sample below shows basic conversion video file from mkv to mp4 format:

string output = Path.Combine(Path.GetTempPath(), Guid.NewGuid() + ".mp4");
            var snippet = FFmpeg.Conversions.FromSnippet.Convert(Resources.MkvWithAudio, output);
            IConversionResult result = await snippet.Start();

This could be done also by:


string outputPath = Path.ChangeExtension(Path.GetTempFileName(), ".mp4");
IMediaInfo mediaInfo = await FFmpeg.GetMediaInfo(Resources.MkvWithAudio);

IStream videoStream = mediaInfo.VideoStreams.FirstOrDefault()
    ?.SetCodec(VideoCodec.H264);
IStream audioStream = mediaInfo.AudioStreams.FirstOrDefault()
    ?.SetCodec(AudioCodec.Aac);

FFmpeg.Conversions.New()
    .AddStream(audioStream, videoStream)
    .SetOutput(outputPath)
    .Start();

Almost all methods in streams return specific stream (IAudioStream, IVideoStream etc.). It allows to create chain of methods. Stream object could be use more than once.


string outputPath = Path.ChangeExtension(Path.GetTempFileName(), ".mp4");
IMediaInfo mediaInfo = await FFmpeg.GetMediaInfo(Resources.MkvWithAudio);

IStream videoStream = mediaInfo.VideoStreams.FirstOrDefault()
    ?.SetCodec(VideoCodec.H264)
    ?.Reverse()
    ?.SetSize(VideoSize.Hd480);

FFmpeg.Conversions.New()
    .AddStream(videoStream)
    .SetOutput(outputPath)
    .Start();

Method IConversions.Clear() sets IConversion to untouched state. All parameters passed to it are overrided by default values.

IConversion provides events to handle FFmpeg output. OnDataReceived and OnProgress events allow redirect FFmpeg output to user and inform him about progress.


conversion.OnProgress += (sender, args) =>
{
    var percent = (int)(Math.Round(args.Duration.TotalSeconds / args.TotalLength.TotalSeconds, 2) * 100);
    Debug.WriteLine($"[{args.Duration} / {args.TotalLength}] {percent}%");
};
await conversion.Start();

OnDataReceived:


conversion.OnDataReceived += (sender, args) =>
{
    Debug.WriteLine($"{args.Data}{Environment.NewLine}") ;
};
await conversion.Start();

Conversion result

Started conversion returns ConversionResult after it's completed. Returned object informs about conversion duration, output file, and more.

One of most useful value for debug may be "Arguments" property which have all arguments passed to FFmpeg process for conversion.

Stop conversion

Started conversion could be stopped. This requires passing CancellationToken to Start method.


var cancellationTokenSource = new CancellationTokenSource();
string outputPath = Path.ChangeExtension(Path.GetTempFileName(), ".mp4");
IMediaInfo mediaInfo = await FFmpeg.GetMediaInfo(Resources.MkvWithAudio);

IStream videoStream = mediaInfo.VideoStreams.FirstOrDefault()
    ?.SetCodec(VideoCodec.H264);

await FFmpeg.Conversions.New()
    .AddStream(videoStream)
    .SetOutput(outputPath)
    .Start(cancellationTokenSource.Token);

CancellationTokenSource can be cancelled manually...


cancellationTokenSource.Cancel();

or automatically after period of time:


cancellationTokenSource.CancelAfter(500);

Snippets

Conversions snippets are extensions of Conversion class with snippets. Most simple conversions will be done with built-in templates or with little modification. That was designed to simplify typical conversions and teach how to create own solutions.

FFmpeg.Conversions.FromSnippet is a good point to start using Xabe.FFmpeg library. Every method may be used as template for a more conversions. If you think that your new conversion method is really useful, do not worry about include them to Xabe.FFmpeg.FFmpeg.Conversions.FromSnippet.Helpers by pull request.

Every method is only a snippet and uses only IConversion with specific configuration. See the source to know how chosen conversion works.

Extracting audio

The simplest way to extract audio from media file is by ExtractAudio snippet:


string output = Path.ChangeExtension(Path.GetTempFileName(), "mp3");
var conversion = FFmpeg.Conversions.FromSnippet.ExtractAudio(Resources.Mp4WithAudio, output);
await conversion.Start();

Extracting video

The simplest way to extract video from media file is by ExtractVideo snippet:


string output = Path.ChangeExtension(Path.GetTempFileName(), Path.GetExtension(Resources.Mp4WithAudio));
var conversion = FFmpeg.Conversions.FromSnippet.ExtractVideo(Resources.Mp4WithAudio, output);
await conversion.Start();

Reversing media

Reverse is possible by operating on streams using Reverse() method:


IMediaInfo inputFile = await FFmpeg.GetMediaInfo(Resources.MkvWithAudio);
string outputPath = Path.ChangeExtension(Path.GetTempFileName(), ".mp4");

IVideoStream videoStream = inputFile.VideoStreams.First()
    .SetCodec(VideoCodec.H264)
    .Reverse();

IConversionResult conversionResult = await FFmpeg.Conversions.New()
    .AddStream(videoStream)
    .SetOutput(outputPath)
    .Start();

In given example output video file will have only one stream - reversed first video stream from source file.

Use Reverse() method is possible on IAudioStream and IVideoStream.

Add audio

The simplest way to add audio to video file is by AddAudio:


string output = Path.ChangeExtension(Path.GetTempFileName(), ".mp4");
var conversion = await FFmpeg.Conversions.FromSnippet.AddAudio(Resources.Mp4, Resources.Mp3, output);
IConversionResult result = await conversion
    .Start();

Add subtitles

There are two ways to add subtitles into video. The first one is to burn it into video. The next one is to add new stream with subtitles, as in .mkv format.

Burning subtitles

IVideoStream allows to burn subtitle into video:


IMediaInfo inputFile = await FFmpeg.GetMediaInfo(Resources.MkvWithAudio);
string outputPath = Path.ChangeExtension(Path.GetTempFileName(), ".mp4");

IVideoStream videoStream = inputFile.VideoStreams.First().AddSubtitles(Resources.SubtitleSrt);

IConversionResult conversionResult = await FFmpeg.Conversions.New()
    .AddStream(videoStream)
    .SetOutput(outputPath)
    .Start();

Add subtitles

Subtitles are streams too so could be added to conversion like other streams:


IMediaInfo mediaInfo = await FFmpeg.GetMediaInfo(inputPath);
IMediaInfo subtitleInfo = await FFmpeg.GetMediaInfo(subtitlePath);

ISubtitleStream subtitleStream = subtitleInfo.SubtitleStreams.First()
    .SetLanguage(language);

return New()
    .AddStream(mediaInfo.Streams)
    .AddStream(subtitleStream)
    .SetOutput(outputPath);

or easier using FFmpeg.Conversions.FromSnippet.AddSubtitle


string output = Path.ChangeExtension(Path.GetTempFileName(), ".mp4");

IConversion conversion = await FFmpeg.Conversions.FromSnippet.AddSubtitle(Resources.Mp4, output, Resources.SubtitleSrt);
IConversionResult result = await conversion.Start();

Changing speed

IVideoStream and IAudioStream allow to change media speed:


IMediaInfo inputFile = await FFmpeg.GetMediaInfo(Resources.MkvWithAudio);
string outputPath = Path.ChangeExtension(Path.GetTempFileName(), ".mp4");

IVideoStream videoStream = inputFile.VideoStreams.First().SetCodec(VideoCodec.H264).ChangeSpeed(1.5);

IConversionResult conversionResult = await FFmpeg.Conversions.New()
    .AddStream(videoStream)
    .SetOutput(outputPath)
    .Start();

ChangeSpeed() method accepting 1 argument - multiplayer. Multiplayer has to be between 0.5 and 2.0. If you want to speed up streams, use values greater than 1, if not, less than 1.

Changing size

The simplest way to change video size is by FFmpeg.Conversions.FromSnippet.ChangeSize:


string output = Path.ChangeExtension(Path.GetTempFileName(), ".mkv");
string input = Resources.MkvWithAudio;

var conversion = await FFmpeg.Conversions.FromSnippet.ChangeSize(input, output, VideoSize.Nhd);

IConversionResult result = await conversion.Start();

Changing video format

FFmpeg.Conversions.FromSnippet contains few predefined methods to change video format e.g.:


await FFmpeg.Conversions.FromSnippet.ToOgv(inputVideoPath, outputPathOgv);
await FFmpeg.Conversions.FromSnippet.ToTs(inputVideoPath, outputPathTs);
await FFmpeg.Conversions.FromSnippet.ToWebM(inputVideoPath, outputPathWebm);

More conversion types are possible by using new Conversion:


string inputVideoPath = Path.Combine("C:", "Temp", "input.mkv");
string outputPathMp4 = Path.Combine("C:", "Temp", "result.mp4");

IMediaInfo info = await FFmpeg.GetMediaInfo(inputVideoPath);

IStream videoStream = info.VideoStreams.FirstOrDefault()
    ?.SetCodec(VideoCodec.H264);
IStream audioStream = info.AudioStreams.FirstOrDefault()
    ?.SetCodec(AudioCodec.Aac);

return FFmpeg.Conversions.New()
    .AddStream(videoStream, audioStream)
    .SetOutput(outputPathMp4);

Concatenate videos

The simplest way to concatenate video files is by FFmpeg.Conversions.FromSnippet.Concatenate:


string output = Path.ChangeExtension(Path.GetTempFileName(), ".mp4");
var conversion = await FFmpeg.Conversions.FromSnippet.Concatenate(output,  Resources.MkvWithAudio, Resources.Mp4WithAudio);
await conversion.Start();

Files list is params so it is possible to concatenate more than two files.

Concatenate is a complicated operation so look at implementation to understand how it works.

Split

IVideoStream and IAudioStream allows split media but the fastest way is by FFmpeg.Conversions.FromSnippet.Split:


string output = Path.ChangeExtension(Path.GetTempFileName(), ".mp4");
IConversion conversion = await FFmpeg.Conversions.FromSnippet.Split(Resources.Mp4WithAudio, output, TimeSpan.FromSeconds(2), TimeSpan.FromSeconds(8));
IConversionResult result = await conversion.Start();

Helper splits all streams in media file and copies them (splitted) to output. In example on output will be media file with duration of 6 seconds contains both streams: audio and video.

Watermarks

Example of use watermarks:


string output = Path.ChangeExtension(Path.GetTempFileName(), ".mp4");

IConversion conversion = await FFmpeg.Conversions.FromSnippet.SetWatermark(Resources.Mp4WithAudio, output, Resources.PngSample, Position.Center);
IConversionResult result = await conversion.Start();

Watermark can be set in different position in a video:

  • Position.UpperRight
  • Position.BottomRight
  • Position.Right
  • Position.BottomLeft
  • Position.UpperLeft
  • Position.Left
  • Position.Center
  • Position.Bottom
  • Position.Up

Snapshot

The simplest way to get snapshot is by FFmpeg.Conversions.FromSnippet.Snapshot:


string output = Path.Combine(Path.GetTempPath(), Guid.NewGuid() + ".png");

IConversion conversion = await FFmpeg.Conversions.FromSnippet.Snapshot(Resources.Mp4WithAudio, output, TimeSpan.FromSeconds(0));
IConversionResult result = await conversion.Start();

Conversion always returns snapshot in png file format so outputPath should be with correct extension. Image has exactly the same size as a video.

Gifs

FFmpeg allows to create gif file from video. Number of loops (one to infinity) and delay between repeats can be specified in parameters. The easiest way to get gif from video is to use FFmpeg.Conversions.FromSnippet.ToGif() method:


string output = Path.Combine(Path.GetTempPath(), Guid.NewGuid() + ".gif");

IConversion conversion = await FFmpeg.Conversions.FromSnippet.ToGif(Resources.Mp4, output, 1, 1);
IConversionResult result = await conversion.Start();

Own arguments

It is impossible to wrap all functionality of FFmpeg in C#. To perform more complex tasks it will be necessary to pass your own arguments directly to FFmpeg.

If you want to add additional parameter to IConversion, use AddParameter method. All parameters added in this way will go near the end of ffmpeg arguments, just before output parameter. This code adds 3 parameters to conversion: -ss (start position), -t (duration) and -s (size).


bool conversionResult = FFmpeg.Conversions.New()
    .SetInput(Resources.MkvWithAudio)
    .AddParameter($"-ss {TimeSpan.FromSeconds(1)} -t {TimeSpan.FromSeconds(1)}")
    .AddParameter("-s 1920x1080")
    .SetOutput(outputPath)
    .Start();

//Output ffmpeg arguments should look like "ffmpeg.exe -i sample.mkv -ss 1 -t 1 -s 1920x1080 output.mp4"

Also user can pass only his own arguments, without using IConversion class. Simplest conversion, from one format to another, can be obtained in this way:


string inputFile = Path.Combine(Environment.CurrentDirectory, "Resources", "SampleVideo_360x240_1mb.mkv");
string outputPath = Path.ChangeExtension(Path.GetTempFileName(), Extensions.Mp4);
string arguments = $"-i "{inputFile}" "{outputPath}"";

bool conversionResult = await FFmpeg.Conversion.New().Start(arguments);

In a result, Arguments variable should look like this (depends on OS and directories):


-i "C:/Xabe.FFmpeg/test/Xabe.FFmpeg.Test/bin/Debug/netcoreapp2.0/Resources/SampleVideo_360x240_1mb.mkv" "C:/Temp/tmpA1AA.mp4"

That string will be passed directly to FFmpeg so final command running in console will look like:


ffmpeg.exe -i "C:/Xabe.FFmpeg/test/Xabe.FFmpeg.Test/bin/Debug/netcoreapp2.0/Resources/SampleVideo_360x240_1mb.mkv" "C:/Temp/tmpA1AA.mp4"

Conversions queue

ConversionQueue provides an easy to use interface to queue conversions. If parallel flag is set to true, Queue will process multiple conversions simultaneously. A number of parallel conversions depend on a number of cores, which give the best performance. With parallel flag, multithread in IConversion object should be disabled to gain performance. it is a lot more efficent in conversion small files under few megabytes.


var queue = new ConversionQueue(parallel: false);
IConversion conversion = await FFmpeg.Conversions.FromSnippet.ToMp4(Resources.MkvWithAudio, output);
IConversion conversion2 = await FFmpeg.Conversions.FromSnippet.ToMp4(Resources.MkvWithAudio, output2);
queue.Add(conversion);
queue.Add(conversion2);

queue.Start();

Events

OnException:


queue.OnException += (number, count, conversion) =>
{
    System.Console.Out.WriteLine($"Exception when converting file {number}/{count}");
};

OnConverted:


queue.OnConverted += (number, count, conversion) =>
{
    System.Console.Out.WriteLine($"File {number}/{count} converted into {FFmpeg.Conversions.FromSnippet.OutputFilePath}");
};

Download FFmpeg executables

Getting latest FFmpeg executables is possible on most common operating system or architectures. Currently there are lack support of Tizen and RaspberryPi. Start downloading it is quite easy. After installing package Xabe.FFmpeg.Downloader, just run this method:


FFmpegDownloader.GetLatestVersion(FFmpegVersion.Official)
This will acquire latest version for your operating system and saves it on your computer. Default directory is where your assemblies are. This can be changed by FFmpeg.ExecutablesPath. Above method create for version.json file in specified directory and if don't find it, download everything and save information about version. Next run will get version from file and check if new version is available.

Using hardware acceleration

Since version 3.1.0 there is a possibility to use hardware acceleration.


///<summary>
///Use hardware acceleration. This option set -threads to 1 for compatibility reasons. This should be use with proper codec (e.g. -c:v h264_nvencorh264_cuvid)
/// </summary>
/// <param name="hardwareAccelerator">Hardware accelerator. List of all accelerators available for your system - "ffmpeg -hwaccels"</param>
/// <param name="decoder">Codec using to decode input video.</param>
/// <param name="encoder">Codec using to encode output video.</param>
/// <param name="device">Number of device (0 = default video card) if more than one video card.</param>
/// <returns>IConversion object</returns>
IConversion UseHardwareAcceleration(HardwareAccelerator hardwareAccelerator, VideoCodec decoder, VideoCodec encoder, int device = 0);
gpu vs cpu conversion

You can read more at official ffmpeg wiki.

Working with images

Sometimes it is necessary to handle a lot of images and converting images to videos or vice versa. Changing video to images is possible thanks to ExtractEveryNthFrame:


Func outputFileNameBuilder = (number) => { return 'fileNameNo' + number + ".png"; };
IMediaInfo info = await FFmpeg.GetMediaInfo(Resources.MkvWithAudio).ConfigureAwait(false);
IVideoStream videoStream = info.VideoStreams.First()?.SetCodec(VideoCodec.Png);

IConversionResult conversionResult = await FFmpeg.Conversions.New()
    .AddStream(videoStream)
    .ExtractEveryNthFrame(10, outputFileNameBuilder)
    .Start();

Above code extracts every 10th frame from input video and saves it to the files: 'fileNameNo001.png', 'fileNameNo002.png' and so on until every 10th frame is saved. For example, if video has 26 frames, only 2 images is saved.

ExtractNthFrame can be used only if one frame is required:


Func outputFileNameBuilder = (number) => { return 'fileNameNo' + number + ".png"; };
IMediaInfo info = await FFmpeg.GetMediaInfo(Resources.MkvWithAudio).ConfigureAwait(false);
IVideoStream videoStream = info.VideoStreams.First()?.SetCodec(VideoCodec.Png);

IConversionResult conversionResult = await FFmpeg.Conversions.New()
    .AddStream(videoStream)
    .ExtractNthFrame(10, outputFileNameBuilder)
    .Start();

Above example creates file named 'fileNameNo001.png' containing 10th frame of a video.