Turn a video to frames and then back again with ffmpeg

Here are a collection of bash commands and then a final script that I used this week to convert a video to frames. Edit those frames a bit, and then convert the edited frames back to video

This was for my rmbg tool, which I discussed in this weeks newsletter. The script at the end actually uses rembg which is the tool that inspired my rust port!

Disclaimer: It turns out this is NOT the "state of the art" background removal for videos! Keep your eyes peeled for a future TIL covering RobustVideoMatting

But the ffmpeg scripts for converting to frames are still useful!

ffmpeg -i input_video.mp4 'frames/%06d.png'

ffmpeg -framerate 24 -pattern_type glob -i 'frames/*.png' output_video_without_audio.mp4

ffmpeg -i output_video_without_audio.mp4 -i input_video.mp4 -map 0:v -map 1:a -c copy -shortest output.mp4

ffmpeg -i video.mp4 -i overlay.png -filter_complex "[0:v][1:v] overlay=0:0" -c:a copy output.mp4

input_video=$0

# Make PNGS from video
ffmpeg -i "$input_video" 'frames/%06d.png'

# Remove background from all frames
rembg p frames frames_out

# Make video from pngs
ffmpeg -framerate 24 -pattern_type glob -i 'frames_out/*.png' bg_removed.mp4

# Add back in the audio track
ffmpeg -i bg_removed.mp4 -i "$input_video" -map 0:v -map 1:a output.mp4