FFmpeg by Example

I've enjoyed using ffmpeg 1000% more since I was able to stop doing manually the tedious task of Googling for Stack Overflow answers and cobbling them into a command and got Chat GPT to write me commands instead.

This reminds me I need to publish my write up on how I've been converting digitized home video tapes into clips using scene detection, but in case anyone is googling for it, here's a gist I landed on that does a good job of it [0] but sometimes it's fooled by e.g. camera flashes or camera shake so I need to give it a start and end file and have ffmpeg concatenate them back together [1]

Weird thing is I got better performance without "-c:v h264_videotoolbox" on latest Mac update, maybe some performance regression in Sequoia? I don't know. The equivalent flag for my windows machine with Nvidia GPU is "-c:v h264_nvenc" . I wonder why ffmpeg doesn't just auto detect this? I get about 8x performance boost from this. Probably the one time I actually earned my salary at work was when we were about to pay out the nose for more cloud servers with GPU to process video when I noticed the version of ffmpeg that came installed on the machines was compiled without GPU acceleration !

[0] https://gist.githubusercontent.com/nielsbom/c86c504fa5fd61ae...

[1] https://gist.githubusercontent.com/jazzyjackson/bf9282df0a40...

I've gotten pretty good at various bits of ffmpeg over time. Its CLI has a certain logic to it... it's order dependent (not all unix CLIs are).

Lately, I've been playing around with more esoteric functionality. For example, storing raw video straight off a video camera on a fairly slow machine. I built a microscope and it reads frames off the camera at 120FPS in raw video format (YUYV 1280x720) which is voluminous if you save it directly to disk (gigs per minute). Disks are cheap but that seemed wasteful, so I was curious about various close-to-lossless techniques to store the exact images, but compressed quickly. I've noticed that RGB24 conversion in ffmpeg is extremely slow, so instead after playing around with the command line I ended up with:

  ffmpeg -f rawvideo -pix_fmt yuyv422 -s 1280x720 -i test.raw  -vcodec libx264 -pix_fmt yuv420p  movie.mp4 -crf 13 -y

This reads in raw video- because raw video doesn't have a container, it lacks metadata like "pixel format" and "image size", so I have to provide those. It's order dependent- everything before "-i test.raw" is for decoding the input, and everythign after is for writing the output. I do one tiny pixel format conversion (that ffmpeg can do really fast) and then write the data out in a very, very close to lossless format with a container (I've found .mkv to be the best container in most cases).

Because I hate command lines, I ended up using ffmpeg-python which composes the command line from this:

  self.process = (
            ffmpeg.
            input(
                "pipe:",
                format="rawvideo",
                pix_fmt="yuyv422",
                s="{}x{}".format(1280, 720),
                threads=8
            )
            .output(
                fname, pix_fmt="yuv422p", vcodec="libx264", crf=13 
            )  
            .overwrite_output()
            .global_args("-threads", "8")
            .run_async(pipe_stdin=True)
            )

and then I literally write() my frames into the stdin of that process. I had to limit the number of threads because the machine has 12 cores and uses at least 2 at all times to run the microscope.

I'm still looking for better/faster lossless YUV encoding.

I thought this was going to be a website managed by an experienced user of FFmpeg sharing from their collection of accumulated knowledge, but then was immediately disappointed on the first example I clicked on.

https://www.ffmpegbyexample.com/examples/l1bilxyl/get_the_du...

Don’t call two extra tools to do string processing, that is insane. FFprobe is perfectly capable of giving you just the duration (or whatever) on its own:

  ffprobe -loglevel quiet -output_format csv=p=0 -show_entries format=duration video.mp4

Don’t simply stop at the first thing that works; once it does think to yourself if maybe there is a way to improve it.

Nice! This reminds me of my own ffmpeg cheatsheet; I would imagine that everyone who uses ffmpeg frequently has a similar set of notes

https://github.com/fastily/cheatsheet/blob/master/ffmpeg.md

FFmpeg is one of those tools I need to use so infrequently that he exact syntax never seems to stick. I've resorted to using an LLM to give me the command line I need. The only other tool that I ever had trouble with was 1990s-era MegaCLI from LSI Logic, also something I barely used from one year to the next (but one where you really need to get it right under pressure).

Don't forget that Gstreamer exists and its command line and documentation make a little bit more sense than ffmpeg because GStreamer is pipeline based and the composition is a little bit more sane. I stopped using ffmpeg entirely and only use GStreamer for intense video work.

The AMIA has a resource for this as well at https://amiaopensource.github.io/ffmprovisr/

It's not a great name and not very discoverable, but there's a lot of very useful ffmpeg-by-example snippets there with illustrated results, and an explanation of what each option in each example does.

Here is the GitHub repo for a ffmpeg book which may be a nice supplement to this site:

https://github.com/jdriselvato/FFmpeg-For-Beginners-Ebook

ffmpeg has always felt like a gui application crammed into tui format. I've had the displeasure of using the C api a few times, while it's straight forward in many respects, it makes invalid states extremely easy to represent. I would love a realtime AV1 encoding framework that "just works".

One thing on Linux systems I like to do is build ffmpeg statically.. as distro versions are sometimes too old or don't include modules I prefer.. this containerized version has done wonders for me https://github.com/wader/static-ffmpeg

I love using FFMpeg via Wasm for ... senseless ... mini projects i.e.: https://video-2-sprites.franzai.com/ Video 2 Sprites Converter - totally over-engineered

ah, ffmpeg, the tool that powers the entire online video industry, praised for its stellar code...yet no one still knows how to use it without getting a phd in sherlocking the internet for miniscule and obscure references to common problems.

I was excited to see this as just last night I was using FFmpeg to combine digital (legally owned) movies with multiple parts into a single MKV file. It worked great, the one thing (and this is undoubtedly just a knowledge problem on my part), is I could not get it to copy/include subtitles from both parts. I have a feeling I might need to extract the subtitle files and combine them and then re-include the combined version, but was hoping there would be a solution in this site for doing something like that. Unfortunately, I didn't even see the combine as one of the examples.

For reference:

One-liner:

> ffmpeg -loglevel info -f concat -safe 0 -i <(for f in *.mkv; do echo "file '$(pwd)/$f"; done) -c copy output.mkv

Or the method I ended up using, create a files.txt file with each file listed[0]

> ffmpeg -f concat -safe 0 -i files.txt -c copy output.mkv

files.txt

> file 'file 1.mkv' > file 'file 2.mkv' > # list any additional files

0: https://ma.ttias.be/use-ffmpeg-combine-multiple-videos/

I love "X by Example" sites! But if you don't work with a tool like ffmpeg imagemagick day in and out, there's no way you'll remember their unintuitive syntax or will want to spend the time to get your one-time job done. I'd still probably not use this site to scan a dozen of examples and try to put together the pieces of the puzzle; instead, I'd probably just use an LLM who already scanned the entire web corpus and can probably get me to a solution faster, right? At that point, I wonder what folks get out of this site?

One thing I have never been able to do reliably is get ffmpeg to split a file based on a fixed size - e.g. break this video into 100MB chunks.

The closest I seem to be able to get is to divide the file size by the file length, add some wiggle room and then split it based on time. Any pointers appreciated.

also worthwhile consulting this resource: https://www.youtube.com/watch?v=9kaIXkImCAM

I just use LLMs to help me with ffmpeg (and many other similarly complex tools) commands: https://github.com/dheera/scripts/blob/master/helpme

    $ helpme ffmpeg capture video from /dev/video0 every 1 second and write to .jpg files like img00000.jpg, img00001.jpg, ...
    $ helpme ffmpeg assemble all the .jpg files into an .mp4 timelapse video at 8fps
    $ helpme ffmpeg recompress myvideo.mp4 for HTML5-friendly use and save the result as myvideo_out.webm

I know there are full blown AI terminals like Warp but I didn't like the idea of a terminal app requiring a login, possibly sending all my commands to a server, etc. and just wanted a script that only calls the cloud AI when I ask it to.

"Print a text file to STDOUT using ffmpeg" ffmpeg -v quiet -f data -i input.txt -map 0:0 -c text -f data - I tried this in a directory with input.txt with some random text Nothing.

So changed the verbosity to trace ffmpeg -v trace -f data -i input.txt -map 0:0 -c text -f data -

---snip-- [dost#0:0 @ 0x625775f0ba80] Encoder 'text' specified, but only '-codec copy' supported for data streams [dost#0:0 @ 0x625775f0ba80] Error selecting an encoder Error opening output file -. Error opening output files: Function not implemented [AVIOContext @ 0x625775f09cc0] Statistics: 10 bytes read, 0 seeks

I was expecting text to be written to stdout? What did I miss?

While it does have a rather unusual/bespoke command syntax (strongly motivating TFA), lately ffmpeg works my webcam more reliably than Google chrome. Too bad my other-side conversations don't have things set up to negotiate a session with it!

I'm grateful for sites like this and these days LLMs too. ffmpeg is awesome, but I can never rememeber the right incantation. The sites/tools get me the right ffmpeg answer, but I still needed to remember the answer next time (or look things up again). Now I save them as reusable commands: https://lmno.lol/alvaro/how-i-batch-apply-and-save-one-liner...

See also:

* https://img.ly/blog/ultimate-guide-to-ffmpeg/

* https://ffmpeg.lav.io/

* https://github.com/leandromoreira/ffmpeg-libav-tutorial

* https://www.hadet.dev/ffmpeg-cheatsheet/

The great ffmpeg!, Even though you can generate commands using llm its still tedious to adjust things visually on cli so I ended up creating my own tool to generate ffmpeg commands.

https://newbeelearn.com/tools/videoeditor/

Ya know, it's websites like this that make me want to see a "best of HNN" list, so it can be easily found when I'm using ffmpeg and saying "geez, there was some cool ffmpeg site, but where the heck is it?...."

Can we have a best of HNN and put it on there, or vote on it, or whatever?

I think at this point docs should start to be written not for humans but for LLMs, i.e. package all of it up with the --help into one giant txt file for easy attachment to an LLM when asking the question you'd like. Imo it's a relatively good fit to the current capability.

The first video doesn't seem to work in Safari or Firefox, only Chrome. "Video can't be played because the file is corrupt." On Mac at least.

Somebody please pay people to create sites like this so LLMs can train on them and I never have to spend time thinking about arcane FFMPEG commands ever again. I remember losing whole days in the before times trying to figure out how to make FFMPEG do some highly specific set of things.

ffmpeg is so goated, I used it to merge video/audio from a mic on a camera I made ha

there was one time I didn't use pyaudio correctly so I was using this process where ffmpeg can stitch multiple audio files together into one passed in as an array cli argument, crazy

hey everyone, thanks for the "hunt".

Currently looking for an FFmpeg related job https://gariany.com/about

I was pretty happy because I was able to actually do something in ffmpeg recently. It is this amazingly powerfully tool, but every time I try to use it I get scared off by the inscrutable syntax. But this time as the mental miasma that usually kills my ffmpeg attempts was setting in I noticed something in the filter docs, a single throw away line about including files and the formatting of filters

Anyway long story short, instead of the usual terrifying inline ffmpeg filter tangle. the filter can be structured however you want and you can include it from a dedicated file. It sounds petty, but I really think it was the thing that finally let me "crack" ffmpeg

The secret sauce is the "/", "-/filter_complex file_name" will include the file as the filter.

As I am pretty happy with it I am going to inflect it on everyone here.

In motion_detect.filter

    [0:v]
    split
            [motion]
            [original];
    [motion]
    scale=
            w=iw/4:
            h=-1,
    format=
            gbrp,
    tmix=
            frames=2
            [camera];
    [1:v]
    [camera]
    blend=
            all_mode=darken,
    tblend=
            all_mode=difference,
    boxblur=
            lr=20,
    maskfun=
            low=3:
            high=3,
    negate,
    blackframe=
            amount=1,
    nullsink;
 
    [original]
    null

And then some python glue logic around the command

    ffmpeg -nostats -an -i ip_camera -i zone_mask.png -/filter_complex motion_display.filter -f mpegts udp://127.0.0.1:8888

And there you have it, motion detection while staying in a single ffmpeg process, the glue logic watches stdout for the blackframe messages and saves the video.

explanation:

"[]" are named inputs and outputs

"," are pipes

";" ends a pipeline

take input 0 split it into two streams "motion" and "original". the motion stream gets scaled down, converted to gbrp(later blends were not working on yuv data) then temporally mixed with the previous two frames(remove high frequency motion), and sent to stream "camera". Take the zone mask image provided as input 1 and the "camera" stream, mask the camera stream, find the difference with the previous frame to bring out motion, blur to expand the motion pixels and then mask to black/white, invert the image for correct blackframe analyses which will print messages on stdout when too many motion pixels are present. The "original" stream get sent to the output for capture.

One odd thing is the mpegts, I tried a few more modern formats but none "stream" as well as mpegts. I will have to investigate further.

I could, and probably should have, used opencv to do the same. But I wanted to see if ffmpeg could do it.

No one seems to be talking about the website itself.

While as a concept, I absolutely love "X by Example" websites, this one seems to make some strange decisions. First, the top highlighted example is just an overly complicated `cat`. I understand that it's meant to show the versatility of the tool, but it's basically useless.

Then below, there's 3 pages of commands, 10 per page. No ordering whatsoever in terms of usefulness. There looks like there's an upvote but it's actually just a bullet decoration.

There's also a big "try online" button for a feature that's not actually implemented.

All in all, this is a pretty disappointing website that I don't think anyone in this thread will actually use, even though everyone seems to be "praising" it.

Are ffmpeg user interested in a cloud base solution?

You push the input files, the command, and fetch the output when done.

Kind of OT: Does anyone know any video editing library in Python that adds fancy/animated text with background?

if you're the creator might I suggest an ai.txt like a robots.txt of all the content? so we can pass it to an LLM and use it to create custom commands we need based on real needs? that would be awesome!

anyone know how to use ffmpeg to get a video to cross-fade from the end to the start so it makes a nice loop? I cant seem to get the right response from ChatGPT.

Wow, I wish I had seen this literally yesterday!

I love ffmpeg, but yeah, some bits are a little ... obscure.

Right now, I am looking to normalize some audio without using ffmpeg-normalize, a popular Python package. Nothing against it on a personal level, I just ... want to know what is going on, and it's a lot of files and lines of code to do what is basically a two-pass process.

I have a growing interest in metadata and that's also a case which I do not find is often well-addressed.

wish there was a website like this for gstreamer

i believe this is where LLM will excel

Gotta leave this here for those that haven’t seen it :)

https://youtu.be/9kaIXkImCAM

Now I know what to read over the next weekend!