问题
Related to Understanding/controlling MLT melt slideshow?, I am trying to understand how does melt's luma mixer work, especially in context of short (small frame number) durations.
For instance, if I use something like this specification tmppics/pic_01.jpg length=6 -mix 2 -mixer luma I would have expected 2 frames of fade of pic_01, then 2 frames full pic_01, then 2 frames of fade again - however, I've both experienced results that are like this, and those that are not.
To clarify this, I've developed a bash script, which uses ImageMagick convert to generate test images, then melt to make a "slideshow" with fades out of these images, ffmpeg to convert that to animated .gif, then ImageMagick convert and montage to obtain a film strip (sprite sheet) of the video .gif (Ubuntu 18.04, melt 6.6.0, ffmpeg version 3.4.4-0ubuntu0.18.04.1, ImageMagick 6.9.7-4 Q16 x86_64 20170114).
Here is the bash script, melt-test-strip.sh:
#!/usr/bin/env bash
FRAMERATE=25
echo "
description=DV PAL
frame_rate_num=$FRAMERATE
frame_rate_den=1
width=720
height=576
progressive=0
sample_aspect_num=59
sample_aspect_den=54
display_aspect_num=4
display_aspect_den=3
colorspace=601
" > my-melt.profile
mkdir tmppics
convert -background lightblue -fill blue -size 3840x2160 -pointsize 200 -gravity center label:"Test A" tmppics/pic_01.jpg
convert -background lightblue -fill blue -size 2160x3840 -pointsize 200 -gravity center label:"Test B" tmppics/pic_02.jpg
if [ -z "$IMGDURATIONF" ]; then
IMGDURATIONF=6 # picture duration, frames
fi
if [ -z "$FADEDURATIONF" ]; then
FADEDURATIONF=2 # single-end fade duration, frames
fi
melt -verbose -profile ./my-melt.profile \
tmppics/pic_01.jpg length=$IMGDURATIONF \
tmppics/pic_02.jpg length=$IMGDURATIONF -mix $FADEDURATIONF -mixer luma \
colour:black length=$IMGDURATIONF -mix $FADEDURATIONF -mixer luma \
-consumer avformat:meltout.mp4 vcodec=libx264 an=1
# auxiliary: just for creating sprite sheet/film strip:
melt -verbose -profile ./my-melt.profile tmppics/pic_01.jpg length=$IMGDURATIONF -consumer avformat:meltout-01.mp4 vcodec=libx264 an=1
melt -verbose -profile ./my-melt.profile tmppics/pic_02.jpg length=$IMGDURATIONF -consumer avformat:meltout-02.mp4 vcodec=libx264 an=1
melt -verbose -profile ./my-melt.profile colour:black length=$IMGDURATIONF -consumer avformat:meltout-b.mp4 vcodec=libx264 an=1
# convert to gif to obtain sprite sheet/film strip from:
ffmpeg \
-i meltout.mp4 \
-r 25 \
-vf scale=256:-1 \
-y meltout.gif
# auxiliary: just for creating sprite sheet/film strip:
ffmpeg -i meltout-01.mp4 -r $FRAMERATE -vf scale=256:-1 -y meltout-01.gif
ffmpeg -i meltout-02.mp4 -r $FRAMERATE -vf scale=256:-1 -y meltout-02.gif
ffmpeg -i meltout-b.mp4 -r $FRAMERATE -vf scale=256:-1 -y meltout-b.gif
convert meltout.gif -coalesce meltoutc.gif
convert meltout-01.gif -coalesce meltoutc-01.gif
convert meltout-02.gif -coalesce meltoutc-02.gif
convert meltout-b.gif -coalesce meltoutc-b.gif
FRAMETHICK=5
#~ montage temp.gif -tile x1 -geometry '1x1+0+0<' -border 5 -bordercolor "rgb(200, 200, 200)" -label 'Image' -quality 100 meltout.png
# "%p index of image in current image list" is "t=> index of current image (s) in list" in fx:
montage -label 'Frame %[fx:t+1]/%n' meltoutc.gif -tile x1 -geometry '1x1+0+0<' -frame $FRAMETHICK -bordercolor "rgb(200, 200, 200)" -quality 100 meltout.png
# here from the .gif - otherwise for replicating images: montage in.jpg +clone +clone +clone -tile x4 -geometry +0+0 out.jpg
montage -label 'Frame %[fx:t+1]/%n' meltoutc-01.gif -tile x1 -geometry '1x1+0+0<' -frame $FRAMETHICK -bordercolor "rgb(200, 200, 200)" -quality 100 meltout-01.png
montage -label 'Frame %[fx:t+1]/%n' meltoutc-02.gif -tile x1 -geometry '1x1+0+0<' -frame $FRAMETHICK -bordercolor "rgb(200, 200, 200)" -quality 100 meltout-02.png
montage -label 'Frame %[fx:t+1]/%n' meltoutc-b.gif -tile x1 -geometry '1x1+0+0<' -frame $FRAMETHICK -bordercolor "rgb(200, 200, 200)" -quality 100 meltout-b.png
# for offsetting:
# (gif) frame width/height
fw=$(convert meltoutc-01.gif[0] -format "%w" info:)
fh=$(convert meltoutc-01.gif[0] -format "%w" info:)
# strip width/height
sw=$(convert meltout.png -format "%w" info:)
sh=$(convert meltout.png -format "%h" info:)
echo fw $fw fh $fh sw $sw sh $sh
convert -size $(( (IMGDURATIONF-FADEDURATIONF)*(fw+2*FRAMETHICK) ))x$sh xc:white meltout-02.png +append meltout-02B.png
# IMGDURATIONF-FADEDURATIONF to get to the start of second clip; +IMGDURATIONF from there to get to end of second clip, and -FADEDURATIONF from there to get to start of third clip
convert -size $(( (IMGDURATIONF-FADEDURATIONF+IMGDURATIONF-FADEDURATIONF)*(fw+2*FRAMETHICK) ))x$sh xc:white meltout-b.png +append meltout-bB.png
montage -geometry '+0+0' meltout.png meltout-01.png meltout-02B.png meltout-bB.png -tile 1x meltout-all.png
eog meltout-all.png
So, if you call bash melt-test-strip.sh, you get the defaults, IMGDURATIONF=6 and FADEDURATIONF=2, for which the output is this (click for full size):
The only way the luma-mixed result (on top) makes sense to me, is if the starting frame of the new clip participates in the mix with 0% - which is why, for the second image clip, we observe 1 frame fade + 3 frames full + 1 frame fade (instead of 2 frames fade + 2 frames full + 2 frames fade, which I'd expect).
Is this correct?
Reading https://www.mltframework.org/docs/melt/#mixes I cannot really tell if this interpretation is correct.
If I run the script with other parameters, like IMGDURATIONF=8 FADEDURATIONF=3 bash melt-test-strip.sh, then the output is:
... in which case the interpretation holds (if every new clip's first frame participates in the mix with 0%, that explains why we're seeing 2 frames fade + 3 frames full + 2 frames fade for second clip = 7 frames in all, instead of the requested 8) - but now I'm not sure if this is just an artefact of my script (as opposed to the true behavior of melts mix).
Can anyone confirm if this is how melts mixer works - and if not, explain how can it be understood?
来源：https://stackoverflow.com/questions/54261066/understanding-mlt-melt-mixer-luma-mix-fade-duration