[Question] Amd Graphics Card

[FeyrisTan]

Will it be possible to do this (in realtime) with an amd card in the near future?
I sold my 1070ti and I'm going to get a 6800xt + 13600k. Was also considering the rtx 4070 but 12gb vram and 192 bit made me look at amd.

[422415]

We utilize a specific NVidia function called a TensorRT Engine, which gives us the speed needed for realtime of Compact models. It is technically possible in realtime with the smallest model(SuperUltraCompact) with AMD however they are generations behind in speed for this sort of task.

Edit: there is japanese group who managed to get a RX7600 to run the SUC model but we have not been able to reproduce how they did it.

[hooke007]

9980hk+pro 5600m
720p x2 → 1440p
janai super ultra compact works fine

[animeojisan]

Is it possible to integrate some of the features of mpv-lazy-2023V5 into the mpv-upscale-2x_animejanai player? I think it would work that way.

[hooke007]

Both are mpv players. No barriers if you know how to use mpv.

[animeojisan]

What are the minimum required files and folders to copy from mpv-lazy-2023V5 to mpv-upscale-2x_animejanai?

[hooke007]

not sure. For generic gpu, mpv.exe + vsLite can allow you to use janai.

[animeojisan]

For example, will 2x_AnimeJaNai_V2_SuperUltraCompact_100k.onnx also work? I have seen some information that it only works with FP32. I don't know if the above ONNX is FP32 or not.

[hooke007]

No.
But fp32 janai models are bundled in vsLite.

[hooke007]

OFF topic:
Perhaps fp16 onnx is almost meaningless for nvidia TRT. Because trt beackend doesn’t use onnx directly, fp16 engine can be converted at runtime, which is actually used by this backend.

[animeojisan]

Thanks.
Can I assume that animejanaiV2L1.onnx and 2x_AnimeJaNai_V2_SuperUltraCompact_100k.onnx are identical? Is the image quality the same?

[hooke007]

In our use case, I don’t see any obvious diff.

[animeojisan]

Very helpful! Thank you.

[bananakid]

9980hk+pro 5600m 720p x2 → 1440p janai super ultra compact works fine
No. But fp32 janai models are bundled in vsLite.
not sure. For generic gpu, mpv.exe + vsLite can allow you to use janai.

Hey @hooke007, would you kindly clarify how to reference Janai SuperUltraCompact (that are bundled in vsLite) in conf file to actually use it for video playback? I'm running the rare AMD Radeon Pro 5600M with Windows 10 too. I've extracted vsLite to mvp-lazy location and I'm familiar with loading GLSL shaders and customizing conf. I just need a little help on where and what files should be loaded. BTW let me know if you're using your GPU in MBP or not.

P.S. I have tried injecting shaders and profiles from mpv-upscale-2x_animejanai to mvp-lazy+vsLib and specifying this in profiles.conf:

[default]
profile=upscale-on-ultracompact4x

[hooke007]

It isn't a shader. I never tried to mix this project with mine. In mpv-lazy with vsLite, the feature is linked to shift+5. For a clean mpv build with vsLite, create a shortcut in input.conf F1 vf set vapoursynth="~~/vs/SR_ESRGAN_DML.vpy" to allow you enable it at runtime.

[bananakid]

@hooke007, thanks so much for taking time to respond and for implementing the support in mpv-lazy, I succeeded with mpv-lazy+vsLite (via Shift+5). I'm sorry I didn't figure out the ESRGAN name in the UOSC menu was for Janai. The quality is incredible and definitely outperforms my typical fsrcnnx_x2_8-0-4-1_lineart.glsl+ssimsuperres.glsl shader combo when upscaling 720p ~3500 Kbps to 1440p. However the performance has a lot to be desired, frames are dropping and you can't really watch the result. GPU utilization is around 85–90%. Is there something else I can tweak for it to run better with AMD Radeon Pro 5600M, maybe a specific driver version is necessary?

[hooke007]

I have no idea what happened on your side. I did successfully enjoyed the full episode of the anime (no dropping frames) in the last test. And now I am using the new apple silicon macbook, so I cannot help more.

GPU utilization is around 85–90%.

Same here. It's much weaker than I thought.

[bananakid]

@hooke007 thank you for taking time to respond and for your article in Discussions of mpv-lazy. I figured out ONNX models are included in vs-mlt contrib-models.v14.test.7z. I will try older GPU driver and older mvp version, as well as different test file. I'll post an update in case I find a solution.

[hooke007]

Emm...The models you want are also included in vsLite.

[bananakid]

@hooke007 yeah, I just linked model source in case model ever gets updated.

I downloaded a couple of other files and they both play OK at ±60 fps:

2. h264 (H.264/AVC/MPEG-4 AVC/MPEG-4 part 10), 8-bit, 960x720, ~3450 Kbps, 23.976 fps (0 dropped frames for 6 min 30 sec)
3. h264 (H.264/AVC/MPEG-4 AVC/MPEG-4 part 10), 10-bit (Hi10P), 966x720, ~6000 Kbps, 23.976 fps (30 dropped frames for 6 min 30 sec)

It looks like the test file №1 I used is encoded with some issue. According to report it's encoded exactly the same as file №2, even bitrate is about 3500 Kbps, except one detail: files №2-3 (above, play OK at ±60 fps) report in mpv Statistics Video: h264 (H.264/AVC/MPEG-4 AVC/MPEG-4 part 10) (hwdec: d3d11va) and file №1 (unplayable at ±6–9 fps) reports Video: h264 (H.264/AVC/MPEG-4 AVC/MPEG-4 part 10). So the problematic file is missing (hwdec: d3d11va). The only other difference is that first file uses OGG Vorbis as audio codec and other two use AAC. The console output is exactly the same.

I have also obtained a different file from the same source and it has the same problem too (no (hwdec: d3d11va) in report and severe frames dropping). Here's my mpv.conf:

vo = gpu-next
gpu-context = d3d11
hwdec = d3d11va
hwdec-codecs = h264,hevc,vp8,vp9,av1,prores
vd-lavc-dr = no

Do you happen to have any clue what may be causing this?

P.S. I tried switching to gpu (from gpu-next), vulkan (for gpu-context and hwdec) and enabling vd-lavc-dr with forced 16 threads. Nothing helped.

[hooke007]

So the problematic file is missing (hwdec: d3d11va)

No issue here. Hi10p could not be decoded by most GPU.

10bit means more bandwidth will be used. Try to edit vpy file (enable feature Lk_Fmt), I hope it would help a bit.

[bananakid]

After some further testing I came to conclusion there's no problem for AMD Radeon Pro 5600M to use animejanaiV2L1.onnx via SR_ESRGAN_DML.vpy except the file resolution limitation: 960×720 upscales and plays flawlessly at ±60fps (h264, h265, 8-bit, 10-bit, any bitrate, any audio codec) while 1280×720 gets constant frame dropping (any codec, any color depth bits, any bitrate, any audio codec).

So thanks again @hooke007 for continuous support, you stated correctly in the very beginning regarding the weakness of GPU. But I'm still happy to know I can enjoy the next to worst quality being properly upscaled. After all newish releases can be downloaded in 1080p/2160p, but some 4:3 classic is often in 720p (960×720) at most.

I hope my observations will be a worthy contribution to data on AMD Radeon Pro 5600M capabilities.

[bananakid]

Just a quick update for AMD Radeon Pro 5600M users. I managed to overcome the resolution limitation of 960×720 described above by doing the following: open SR_ESRGAN_DML.vpy with Notepad and change line Gpu_T = 2 to Gpu_T = 4. Now 1280×720 and even 1920×1080 files are playing great and there're no frame drops. The only drawback is playback speed of 720p & 1080p anime of any bitrate and color is limited to ~24 fps (speed of ×1.05+ causes frame drops immediately). Playback speed of 4:3 720p (960×720) can be ×2 easily (60 fps) though.

For anyone googling around, here's my video profile configuration to use with SR_ESRGAN_DML.vpy:

[real-esrgan]
scale=lanczos
cscale=lanczos
dscale=hermite
deband=yes
deband-iterations=4
dither-depth=no
hdr-contrast-recovery=0.30
correct-downscaling=yes
sigmoid-upscaling=yes
autofit-larger=2560x1440

And here's my SR_ESRGAN_DML.vpy configuration:

H_Pre = 720
Lt_Hd = False
Model = 5005
Gpu = 0
Gpu_T = 4
H_Max = 1440
Lk_Fmt = True

@hooke007, I have trouble understanding what Lt_Hd I should use for 720p video and for 1080p video file for model to work correctly. Would you kindly take a minute to explain? I've examined the documentation and couldn't figure it out.

P.S. I have also switched Lk_Fmt = True from Lk_Fmt = False as it was advised by @hooke007 earlier, but it didn't improve performance in any noticeable manner.

[hooke007]

For DML, lt_hd only set the max res of 1280x720. It would terminate the process if the res of clip over it.

[bananakid]

@hooke007, I'm sorry to nag, but probably you could clarify the following. I noticed vs-mlrt v14.test3 release introduced FP16 I/O support for AMD GPUs via MIGraphX, which is faster than FP32, according to their benchmark. Can this be used to improve animejanaiV2L1.onnx/animejanaiV3-HD-L1.onnx of mpv-lazy performance somehow (for AMD GPU)? I didn't get that MIGraphX footnote in vs-mlrt's release.

[hooke007]

I never worked with Linux. It's out of plan.