Corruption on seek with 444 h264 sample #59
Comments
|
AkarinVS/L-SMASH-Works has addition changes (info) that iirc cause other issues while this specific file is ok. I have to check it again. |
|
A bit more investigation shows what exactly is happening here: MediaInfo shows that this file has been encoded with
|
Opening and closing the codec in order to flush the buffer could lead to corrupted decoded frames (x264 version < 151). On the other side just flushing the buffer could lead to garbage decoded frames when SPS/PPS is available only in the header. This is based on AkarinVS/L-SMASH-Works@3d2f02e.
|
Hey, you can try the new version |
|
I ran another full seektest with this, the results are below. So, while this fixes the Another much more hacky solution I can think of is manually parsing the first packet of H264 streams and looking for type 5 SEI messages, which are then manually added to the first packed fed to reopened decoders. (Or, even worse, parsing the actual SEI and setting the found x264 version as an Seektest results (format is |
|
Did you use the release binary? I ran that seek test (1000 frames for every video) and the attached file is the result. |
|
Yes, I did use that binary. And when checking one of the files ( |
|
I used libvpx for VP9 in the latest release. So the issues you have are caused by libvpx. Interestingly I cannot see any issues with that Also another run for How did you run the test so I can try to reproduce the result? |
|
I use this VapourSynth script: # Originally from https://gist.github.com/HolyWu/c73b296c802f657be7b4b8f8a90b4cf2
#
# Usage:
# python3 seek-test.py "test files/*"
#
# Change the source filter as needed.
# Note: this script is slow because making the source filter decode frames randomly is slow.
import hashlib
import random
import sys
import os
import itertools
import glob
import vapoursynth as vs
from vapoursynth import core
def hash_frame(frame):
md5 = hashlib.md5()
for line in frame.readchunks():
md5.update(line)
return md5.hexdigest()
seen_files = []
for filename in itertools.chain(*(glob.glob(p) for p in sys.argv[1:])):
try:
with open("seektest.csv") as f:
seen_files = [l.split(",")[0] for l in f.read().splitlines()]
except:
pass
if filename in seen_files:
continue
print(f"Testing {filename}")
# clip = core.bs.VideoSource(filename, cachesize=0)
# clip = core.d2v.Source(filename, rff=False)
# clip = core.dgdecode.MPEG2Source(filename)
# clip = core.dgdecodenv.DGSource(filename)
cachefilename = "{}.lwindex".format(filename.replace("/", "_").replace("\\", "_"))
try:
os.remove(cachefilename)
except:
pass
try:
# clip = core.ffms2.Source(filename, cachefile="cache.ffindex")
clip = core.lsmas.LWLibavSource(filename, cachefile=cachefilename)
except vs.Error:
print("Failed to open file.")
with open("seektest.csv", "a") as logfile:
logfile.write(f"{filename},Error\n")
print(f"Clip has {clip.num_frames} frames.")
reference = []
for i in range(clip.num_frames):
reference.append(hash_frame(clip.get_frame(i)))
if i % 10 == 0:
print(f"Hashing: {i * 100 // (clip.num_frames - 1)}%", end="\r")
print("\nClip hashed.\n")
seek_tests = list(range(clip.num_frames))
random.shuffle(seek_tests)
seek_tests += list(range(clip.num_frames))[::-1]
failures = 0
for n, i in enumerate(seek_tests):
try:
result = hash_frame(clip.get_frame(i))
if result != reference[i]:
print(f"Requested frame {i}, ", end="")
failures += 1
try:
print(f"got frame {reference.index(result)}.")
except ValueError:
print(f"got new frame with hash {result}.")
if n % 10 == 0:
print(f"Seeking: {n * 100 // len(seek_tests)}%", end="\r")
except vs.Error as e:
failures += 1
print(f"requesting frame {i} broke source filter.")
raise e
print(f"Test complete. {failures} failures in total.")
print()
with open("seektest.csv", "a") as logfile:
logfile.write(f"{filename},{failures}\n")Here's an example of the corruption, the little yellow line on his forehead is not supposed to be there. |
|
Thanks. Can you share the same frame with the previous release |
|
Here. Note that the corruption on
|
I don't see corruption when using the script you provided |
|
@L4cache, can you test also |
|
I tested a few problematic files shown above. Could it be hardware related issue(s)? P.S. snow.MTS and seek-test.mkv can pass the test after change to single thread. But AkarinVS/L-SMASH-Works don't need single thread for seek-test.mkv to pass the test (but not for snow.MTS). |
|
Just noticed some vertical black lines in your screenshot, is this normal? I don't see them in mine. |
|
Thanks for testing. Those vertical black lines are from
@arch1t3cht, how many CPU threads do you have? If more than 16, can you test |
Thanks. I have 16 threads too. |
|
My CPU has 8 cores and 16 logical processors. I tried |
Instruction sets? |
|
It's a Ryzen 7 3700X, so:
|
|
@arch1t3cht, can you test this |
|
Still corrupts, I'm afraid, with the same issues as before. |
|
This could be somthing interesting, I made the script output the reference hashes, and some frames in the last seconds have same hash. But the sequential decode result of 1183 and 1170 is the same, so this is not defect. And if the topic is now about VP9 decoding: |
|
The default VP9 decoder is again the ffmpeg one (capped to 2 threads) and optionally You can test the attached version. |
|
Yes, with this version the vp9 files work for me again. |
|
What is the code that controls default decoder selection? |
|
|
You've said it. |
|
One way is to replace the latest line here with Other way is to build ffmpeg with |
Hello,
loading the file
Nagi no Asukara 2013 - EP01 [BD 1920x1080 23.976fps AVC-yuv444p10 FLACx3 Chap] - mawen1250.mkvfrom the classic doom9 collection of test files usingcore.lsmas.LWLibavSource()(with no other arguments except for acachefile) in VapourSynth with release 1170 causes major corruption in frames after a few seeks. I wasn't yet able to fully narrow down what property of this file causes this behavior, but I'd guess that it has to do with the 444 subsampling (but reencoding the file to another 444 file using ffmpeg/x264 doesn't cause the same corruption so this seems to not be the only reason). Interestingly enough, AkarinVS/L-SMASH-Works has no issues with this file.This is the only file in this doom9 collection that has seeking errors or corruption with this source filter (except for a couple ones which crash or return errors), so thanks a lot for your work on this source filter.
The text was updated successfully, but these errors were encountered: