I'm currently using the command
ffmpeg -i mainvideo.mp4 -vcodec libvpx -i myoverlay.webm -filter_complex "overlay = 5:5" done.mp4
To overlay a transparent webm on top of my main mp4. But for some reason the output video is frozen for the first 1~ second but the audio is completely fine.
I've tried this command with other videos and it works sometimes but most of the time the first few frames seem frozen
Edit.
I have two main mp4 files, broken_source.mp4 and working_source.mp4. Putting the overlay on broken_source results in the output having the first 1~ second of video frozen (audio completely fine) but putting the same overlay on working_source.mp4 is perfect and has no issues
Logs for putting the overlay on working_source:
ffmpeg -i working_source.mp4 -vcodec libvpx -i chat.webm -filter_complex "overlay = 0:0" working_complete.mp4
ffmpeg version 4.3.1-2021-01-01-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 10.2.0 (Rev5, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
libavutil 56. 51.100 / 56. 51.100
libavcodec 58. 91.100 / 58. 91.100
libavformat 58. 45.100 / 58. 45.100
libavdevice 58. 10.100 / 58. 10.100
libavfilter 7. 85.100 / 7. 85.100
libswscale 5. 7.100 / 5. 7.100
libswresample 3. 7.100 / 3. 7.100
libpostproc 55. 7.100 / 55. 7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'working_source.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.51.101
Duration: 00:00:31.02, start: 0.000000, bitrate: 6176 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 6035 kb/s, 60 fps, 60 tbr, 90k tbn, 120 tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
handler_name : SoundHandler
[libvpx @ 0000016fb5074a80] v1.9.0-128-g3a38edea2
Last message repeated 1 times
Input #1, matroska,webm, from 'chat.webm':
Metadata:
ENCODER : Lavf58.51.101
Duration: 00:00:31.00, start: 0.000000, bitrate: 449 kb/s
Stream #1:0: Video: vp8, yuva420p(tv, progressive), 500x300, SAR 1:1 DAR 5:3, 60 fps, 60 tbr, 1k tbn, 1k tbc (default)
Metadata:
alpha_mode : 1
ENCODER : Lavc58.101.101 libvpx
DURATION : 00:00:31.000000000
[libvpx @ 0000016fb50761c0] v1.9.0-128-g3a38edea2
Stream mapping:
Stream #0:0 (h264) -> overlay:main (graph 0)
Stream #1:0 (libvpx) -> overlay:overlay (graph 0)
overlay (graph 0) -> Stream #0:0 (libx264)
Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[libvpx @ 0000016fb50761c0] v1.9.0-128-g3a38edea2
[libx264 @ 0000016fb5072040] using SAR=1/1
[libx264 @ 0000016fb5072040] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0000016fb5072040] profile High, level 4.2, 4:2:0, 8-bit
[libx264 @ 0000016fb5072040] 264 - core 161 r3027 4121277 - H.264/MPEG-4 AVC codec - Copyleft 2003-2020 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'working_complete.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.45.100
Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 60 fps, 15360 tbn, 60 tbc (default)
Metadata:
encoder : Lavc58.91.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
handler_name : SoundHandler
encoder : Lavc58.91.100 aac
frame= 1861 fps= 44 q=-1.0 Lsize= 20013kB time=00:00:31.01 bitrate=5285.3kbits/s speed=0.729x
video:19477kB audio:486kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.244893%
[libx264 @ 0000016fb5072040] frame I:8 Avg QP:19.04 size:240534
[libx264 @ 0000016fb5072040] frame P:538 Avg QP:23.76 size: 23836
[libx264 @ 0000016fb5072040] frame B:1315 Avg QP:31.66 size: 3951
[libx264 @ 0000016fb5072040] consecutive B-frames: 3.0% 3.5% 14.8% 78.7%
[libx264 @ 0000016fb5072040] mb I I16..4: 6.4% 47.4% 46.2%
[libx264 @ 0000016fb5072040] mb P I16..4: 1.2% 4.2% 1.9% P16..4: 21.4% 5.3% 2.9% 0.0% 0.0% skip:63.1%
[libx264 @ 0000016fb5072040] mb B I16..4: 0.1% 0.2% 0.1% B16..8: 12.6% 1.5% 0.5% direct: 0.4% skip:84.6% L0:35.9% L1:60.0% BI: 4.1%
[libx264 @ 0000016fb5072040] 8x8 transform intra:55.5% inter:60.0%
[libx264 @ 0000016fb5072040] coded y,uvDC,uvAC intra: 63.5% 77.4% 40.4% inter: 3.7% 4.6% 0.7%
[libx264 @ 0000016fb5072040] i16 v,h,dc,p: 19% 37% 15% 29%
[libx264 @ 0000016fb5072040] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 18% 23% 7% 7% 7% 7% 6% 8%
[libx264 @ 0000016fb5072040] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 22% 17% 7% 8% 7% 7% 6% 6%
[libx264 @ 0000016fb5072040] i8c dc,h,v,p: 45% 27% 18% 11%
[libx264 @ 0000016fb5072040] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0000016fb5072040] ref P L0: 74.5% 13.9% 9.1% 2.5%
[libx264 @ 0000016fb5072040] ref B L0: 91.9% 7.4% 0.7%
[libx264 @ 0000016fb5072040] ref B L1: 96.6% 3.4%
[libx264 @ 0000016fb5072040] kb/s:5144.05
[aac @ 0000016fb5026a80] Qavg: 193.230
Logs for putting the overlay on broken_source:
ffmpeg -i broken_source.mp4 -vcodec libvpx -i chat.webm -filter_complex "overlay = 0:0" broken_complete.mp4
ffmpeg version 4.3.1-2021-01-01-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 10.2.0 (Rev5, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
libavutil 56. 51.100 / 56. 51.100
libavcodec 58. 91.100 / 58. 91.100
libavformat 58. 45.100 / 58. 45.100
libavdevice 58. 10.100 / 58. 10.100
libavfilter 7. 85.100 / 7. 85.100
libswscale 5. 7.100 / 5. 7.100
libswresample 3. 7.100 / 3. 7.100
libpostproc 55. 7.100 / 55. 7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'broken_source.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.26.101
Duration: 00:00:45.02, start: 0.000000, bitrate: 5962 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 5952 kb/s, 60 fps, 60 tbr, 90k tbn, 120 tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
handler_name : SoundHandler
[libvpx @ 0000021071f90280] v1.9.0-128-g3a38edea2
Last message repeated 1 times
Input #1, matroska,webm, from 'chat.webm':
Metadata:
ENCODER : Lavf58.51.101
Duration: 00:00:31.00, start: 0.000000, bitrate: 449 kb/s
Stream #1:0: Video: vp8, yuva420p(tv, progressive), 500x300, SAR 1:1 DAR 5:3, 60 fps, 60 tbr, 1k tbn, 1k tbc (default)
Metadata:
alpha_mode : 1
ENCODER : Lavc58.101.101 libvpx
DURATION : 00:00:31.000000000
[libvpx @ 0000021072020dc0] v1.9.0-128-g3a38edea2
Stream #0:0 (h264) -> overlay:main (graph 0)
Stream #1:0 (libvpx) -> overlay:overlay (graph 0)
overlay (graph 0) -> Stream #0:0 (libx264)
Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[libvpx @ 0000021072020dc0] v1.9.0-128-g3a38edea2
[libx264 @ 0000021072022980] using SAR=1/1
[libx264 @ 0000021072022980] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0000021072022980] profile High, level 4.2, 4:2:0, 8-bit
[libx264 @ 0000021072022980] 264 - core 161 r3027 4121277 - H.264/MPEG-4 AVC codec - Copyleft 2003-2020 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'broken_complete.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.45.100
Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR
