[Issue]: Fire Stick 4K Max (2023) high latency #1276
Comments
|
It looks like they made a switch from Android 9 (Fire Stick 2021) to Android 11 (Fire Stick 2023). Could this be related? |
|
Yikes, looks like the android 11 and higher mediatek bug again. My hunch is this report is the same issue as: Where they've started using the "modern" c2 decoder which doesn't work with the magic "vdec-lowlatency" fix that got media tek decoders working fast for moonlight on older Android versions with the same chipset. Can you check your 2023 fire tv stick 4k max and see what decoder it's using? My guess is the new fire tv stick 4k max (2023), based on android 11, will show: C2.MTK.HEVC.DECODER Instead of the older android 9 sticks that use the omx decoder api. Let us know what you see, thanks! |
|
It's using OMX.MTK.VIDEO.DECODER.HEVC |
|
Oh weird, not what I expected! Could you perhaps sideload this codec info app onto your fire stick? and use that app to screenshot / dump as much as you can for the codecs that it lists related to HEVC? I know it will be a pain on a TV, but curious what extensions it claims to support. Also a logcat output from the ADB debug interface would be helpful too, I can compare that to my older 4k max (2021) and see if the decoder is loading the same way on the new 2023 max stick. Enable adb over network and connect from a computer: Run logcat over the adb shell: Usually just "adb logcat" will suffice after you've run the stream for a few seconds. Thanks! |
|
Just bought a fire stick 4k max (2023 version) and can confirm decoding latency sits around 15-16 ms no matter which settings you use. I will try to supply some of the information you requested above @peacepenguin tomorrow once I get some time! |
|
Just purchased this model of firestick and i am experiencing the same issue described. |
|
I switched yesterday from a 4k Max Gen 1 (2021) to a 4k Max Gen 2 (2023) and didn't change anything other in my setup. So PC settings (including sunshine settings) and network settings stayed exactly the same. Both Fire TV sticks are connected via ethernet dongle. In moonlight I chose HEVC as codec. Now I am experiencing the same issues as @LoserCard. On my gen 1 stick (2021) decoding latency was around 5 ms. Now on my gen 2 stick (2023) latency is between 15 ms and 20 ms. As @peacepenguin asked I am attaching the codec infos for HEVC from my 4K Max Gen 2 stick (2023) gathered with parseus codec info. There are 4 entries under VIDEO in parseus codec infos in the category video/hevc: |
|
@peacepenguin now for your question concerning the adb logcat logs: I started logcat over adb and then connected with moonlight to my pc running sunshine. I let the stream running a couple of secs and used an xbox controller to make some commands. Hope this suffices. If not please let me know. I am happy to help. |
|
@mrratherford The log cat looks good, there's no errors with the needed flags being applied to the decoder during the moonlight initialization like we see on some other newer mediatek devices. I see the correct 'vdec-lowlatency' flag for example being set. Can you run parsec and see if it performs better or worse or the same? On some other mediatek devices parsec works great, but moonlight is high latency, lets see if parsec has solved the issue and we can reverse engineer from there if there's optimization possible. |
|
@peacepenguin Strangely, in parsec the decoding time is very low, like 2 to 3ms. unfortunately I can't play like I did on Moonlight with 50 bitrate (Parsec doesn't seem to know how to manage the network very well on Fire TV and the network latency is very high most of the time, unlike Moonlight) |
|
@SnowJ7Z something is definitely up with networking in Moonlight too. I noted in the issue description that the threshold between 50-60mbps caused really bad latency in Moonlight. Hopefully both these issues can be addressed at the same time with a reverse engineering of Parsec. |
|
@SnowJ7Z can you set parsec to h265 (hevc) and share those results too? So far all we're looking at in depth is h265. Thanks! |
|
Apparently everything is fine |
|
Ok good to see parsec is working well, 3ms decode time is fantastic. It means that we should be able to make moonlight at LEAST as good. Can someone capture a logcat of parsec using h265 on the 2023 fire tv stick 4k max? Last parsec log I looked at was a little wonky, like they're bypassing the normal draw to screen functions and using their own. They seem to have found a trick to bypass some of the buffering that were seeing on mediatek decoders in android lately. |
|
@peacepenguin My version is the fire tv stick 4k 2023, basically the errors that are occurring are the same as in the max version. I'll send the logcat in an hour, maybe less. |
|
Yes makes sense the only difference I see in the spec sheet for the 2023 4k vs 4k max is the storage space and clock speed of the cpu and gpu. Looks like they have the same chipset. So yes, the issue I imagine will be identical on both. Thanks for collecting the data, it will make finding a fix much easier! |
|
I don't know if I did it right, but here it is. |
|
Thanks @SnowJ7Z and @mrratherford the logcats are really helpful Try this apk I just compiled, I just made a quick change to the decoder config to closer align it to parces. You'll need to uninstall your existing moonlight app before installing this modified one: https://github.com/peacepenguin/moonlight-android/releases Changes: |
|
@peacepenguin Just try this on my Mediatek tablet and it is still using the c2 decoder not the omx one. The soc is Helio G99. |
|
@peacepenguin Still unsuccessful, still within a range of 13 to 16ms |
|
@peacepenguin I'm seeing the same results as @SnowJ7Z . Moonlight (peacepenguin@7ff2561) runs with 10 -16 ms decoding latency. The only different to version 0.21 is that the screen is black for a couple of seconds after connecting to the sunshine server. but after this the stream runs stable. here are the logs for Moonlight (peacepenguin@7ff2561): logcat_moonlight_7ff2561_firetv_4k_max_gen2_2023.txt i installed parsec (v150-85c) on the fire tv 4k max (2023) and looked at its decoding latency. parsec seems to have some tricks up its sleeve, because it's decoding latency was around 3 ms:
I'm attaching the logs for parsec, too: (looking at the logs now i'm not sure if I captured the moment connecting to the host with parsec. If not please let me know and I will try again) |
|
@SnowJ7Z thanks for testing, good news is adaptive playback isn't causing trouble, that's a decoder feature that we can re-enable for these newer sticks that was previously causing problems. @mrratherford thanks for testing and sending more logs, it's very helpful. I'm now looking at these settings that are different and will make some further changes to my fork to hopefully get moonlight decoding sub 5ms on these sticks. I'll post test builds as I progress. parsec: vs moonlight with adaptive playback enabled (peacepenguin@7ff2561): |
|
After digging through lots of code and logs, I think what we need to do next is implement "Tunnel Mode" (or Multimedia Tunneling) for audio/video playback in moonlight-android. The codec info posted for the fire tv 4k max 2023 shows "Tunneled Playback" as a supported feature of the hardware hevc decoder. The notes for Tunneled mode are very promising from both google's android and amazon's fire os implementation notes for low latency video playback. I'm looking into how to convert my fork of moonlight-android over to use Tunneled Playback mode. It looks easy based on the docs linked below, but I'm not sure where to initialize the audiosessionid that is required to sync the video to the audio. Both the audio and video stream need to reference the same, pre-created audiosessionid when they are instantiated. Currently in moonlight-android the audio decoder gets created after the video decoder, I think we need to switch that around and setup the audio decoder first, with the audiosessionid, then pass that same audiosessionid to the video decoder too. Then just need to figure out how to remove the dequeueOutputBuffer and releaseOutputBuffer traditional rendering steps as noted in amazon's guidance. https://developer.amazon.com/docs/fire-tv/4k-tunnel-mode-playback.html
https://source.android.com/docs/devices/tv/multimedia-tunneling
|
|
@SnowJ7Z not sure but to test audio sync and latency issues I use this video on a loop: https://www.youtube.com/watch?v=TjAa0wOe5k4 First just make sure it's perfect when you sit in front of your computer, direct attached output to monitor and speakers, to ensure the base rendering is working as intended (no sunshine/moonlight involved here, just gotta get ensure even local playback is perfect first) Then open up moonlight, desktop session, and see if the audio and video is still in sync or not when viewing the same video over moonlight. So far in my experience moonlight has been perfect for AV synchronization, so it sounds like the issue you are having would be from the game itself not rendering the audio and video in sync. But you can deduce that down with the above test and see where the issue is coming from. |
|
Glad to see folks are already digging into this, just bought a Max mostly because I was excited about using moonlight with it. Let me know if there's anything I can do to help as far as logs or whatever. I can also potentially help with dev since I know Java really well, but my experience with these streaming protocols is pretty limited. |
|
I'm digging into this deep for academic reasons, and i just snagged a new max 2023 to test with. I could use your help, as java is not my main to say the least! I'm trying to find where the AudioTrack object "track" gets created, and move it to before the video decoder gets initialized. The goal being to implement Tunnel Mode decoding. Documented in very few places on the web, with even fewer examples. I need to get the audiosessionid from the "track" object, and pass it to the video decoder. Currently the video decoder is setup before the audio decoder, so I can't get the audiosessionid where I need it. Let me know if you can look at that, I'll post my progress and test builds in this thread so we can stay in touch. I should probly hop on the discord and see if any of the maintainers can weigh in on moving to Tunnel Mode decoding in general. |
|
@peacepenguin Honestly a bit confused by how exactly the method "Java_com_limelight_nvstream_jni_MoonBridge_init" works, but it seems like that's where they make all these statics calls to initialize each component. "bridgeArInit" makes the call inbound to create the AudioTrack. It seems that "bridgeDrSetup" is responsible for making the video decoder. You should just be able to swap those two to get the effect you want. You can find the calls in callbacks.c. |
|
thanks @BuiltInParris I've now got the audio session ID generated and passed down to the audio and video rendering startup as params, seems ok. As soon as i start the session things hang and crash though, which makes sense, as I've not removed the traditional rendering methods for the video, which amazon says explicitly needs removed in tunnel playback mode:
https://developer.amazon.com/docs/fire-tv/4k-tunnel-mode-playback.html I'm pretty much stuck here, I have no idea what to replace those calls with. dequeue and release output buffer is how the screen is drawn. I don't see any further documentation on how your supposed to draw the screen without those in existence. Non-working wip can be found here. At least it has the audio session id being passed into both audio and video now: That diff is from my 'tunnelmode' branch in my fork of moonlight-android here: Please chime in if anyone knows what to look at next. This all to enable "tunnel mode playback", which amazon says: "To get the best out of the hardware, use Tunnel Mode playback." Google says it reduces latency: https://source.android.com/docs/devices/tv/multimedia-tunneling And these sticks support it per their codec output. So should shave some decode latency off the top by switching to tunnel mode. So I think we're on a good path here, I'm just quickly reaching my limit on Java programming, especially with basically no implementation examples on how to use it from google nor amazon. Anyone with Java experience please take a look at the links above and my wip code and see if you can help. Thanks all! |
|
I don't know much about Java or video streaming in general, but, just in case you haven't seen this (although I'm sure you probably have) There is some information on the android documentation which might be helpful. The other source of example use is the ExoPlayer source which again you have probably seen. Thanks again for your help and efforts, it feels like a solution is close! |
I do not use any bluetooth device at all. My mouse is wired directly to my host PC, not to the stick for testing. FTV Stick (wired): So average rendering time is same on both devices, but the FTV Stick has sluggish mouse feel (MOUSE IS CONNETED TO HOST!). That makes no sense, since numbers are comparable. I also don´t think that decoding latency between 1ms and 3ms should be noticable. There´s still something messed up with the FTV stick wich I can´t explain. @LoserCard |
|
@Xedos9er sounds like you're not facing an input lag but a video lag (high decoding latency). Just like I mentioned in my previous message sometimes my video lag increasingly goes up that even when I continue using my controller, connected via Bluetooth on my stick and looking at my computer screen, the movements are instant but noticeably delayed on the fire TV's screen (tested on my dual monitor setup each the exact same monitor model). The question I may asked is do you feel any delay when you start playing the game? Or does it happen after a few minutes? |
|
@Xedos9er so I tested this and I would agree that 1ms decode/4ms network does NOT feel like 4ms decode/1ms network BUT this is not a problem with the Fire Stick specifically. I just ran the following tests while controlling mouse on host machine:
#1 and #2 feel equal while #3 feels like local gaming. I'm guessing that maybe the network latency number is actually "ping" which is round trip time. If this was the case, it would mean that real world number in terms of input responsiveness is only half the network latency while decoding is the full value; e.g.: encode+(network/2)+decode Also @constantindjonkam I did experience the increasing latency issue. This occurred at 20Mbps, but wasn't present at 10Mbps. At 20Mbps, it started out at 4ms then slowly climbed to 8ms and stayed there; I imagine this is worse at higher bitrates. There is definitely something still wrong with either Moonlight or the Fire Stick firmware. It's disappointing considering the 2021 Fire Stick 4K Max was so good. |
|
@LoserCard in my case it increasingly goes up to 200+ms decoding latency 1080@60 20Mbps. Yh seems like decoding matter most I was playing control on Luna for like 5 hours now it just feels local as I'm very close to the game server (~4ms says 3.716ms in settings today). The way they handle the stream is amazing. Hope we implement something similar 😔 |
|
Seems like lowering the bitrate is what works best on moonlight on the fire TV which also makes sense why Luna feels the way it is. |
|
Oh can someone that uses or has Luna capture a Log cat during a Luma stream? Thanks! |
|
@peacepenguin this is Luna Stream Logcat |
|
Hello, are there any devs trying to see how to improve performance on the fire tv? |
|
thanks @constantindjonkam for the logcat of luna. i see luna is using oboe for its audio output. It reminds me of this issue: where fire cube has 300ms extra audio latency. I want to try implementing the changes discussed in that thread. i also see high audio latency on the stick. Also still looking at the logcat, seeing some interesting params to test with. |
I think that fireOs is not stable and that it is the problem. I noticed that I had a lag no matter what. For information, the key is connected to my TV in HDMI, I have a home cinema connected in HDMI arc (so it recovers the sound from the TV and transcribes it) and if on this same socket I connect an Xbox I do not have the lag problem. |
|
Hi, i've read all the post, last version of the app from store is from November 23 and i dont think it is resolved the bug in that version, but in some post i have read about it, there are some version (maybe beta) to fix this problem? my situation is Firestick 4k Max on Wifi (Speedtest is 350Mbps) when start streaming i can see 2/4 ms and even if i start gaming, but when there are some speed movement and the frame are a lot different the game seems to stutter and latency jump over 10ms for a small time (the game tested is cyberpunk) with 10Mbps bandwith seem better but the image is uglyer thanks sorry for my bad english |
Hello, were you able to look at the potential solutions? |
|
So I've retested after the OS update on my Fire TV stick 4k max (2023). My decode times are in the 5-9 ms range now for HEVC, but only when I run low bitrate. If I go above ~30mbps, latency pretty much doubles into 10-20ms. This may be from the hardware bitrate limitation of the stick itself:
35 Mbps for HEVC, 20 Mbps for H264. 100Mbps for AV1. So maybe we're now running into hardware limitations of these Sticks? Perhaps the audio bug should be addressed next since the Fire OS update helped already with this issue related to video decode times: |
|
@peacepenguin I doubt it's a hardware limitation (at least from a raw performance standpoint; maybe a bottleneck somewhere though?). The 1st gen stick could do ~4ms 4k60 @ 60Mbps and the 2nd gen is almost twice as powerful yet chokes after 10Mbps. This issue seems like it may just be something that doesn't get solved (or can't get solved because of unknown bottlenecks). I appreciate your efforts but I have already written this in as an L in my book. |
|
Is anyone actively working to address this? 4k streaming on the Firestick 4K Max isn't viable without an input frame delay now. The older stick is viable. It's weird that new and more powerful hardware would perform worse. |
|
I've moved on for now, I tried a few things but I'm no java/android expert so I just can't keep poking at this. I'm using a linux build on an rk3588 with amazing results now for my needs. Feels better than any android device, delay bug or not. The hero of moonlight, @cgutman will have to help out unless someone else can join the good fight! A bug bounty is an option too. Throw some money out on bounty source and start gathering funds for the bounty via donations in reddit, discord, etc. People that use the app and love our maintainers might just call someone to put in the hours to fix this properly. |
What decoding time you have ? Thanks |
I am the author of the mentioned issue. I am wondering if you have measured the actual audio latency on the FireTV Stick ? I didn't find a reliable way to do it aside from recording my screen playing this video. It's been a long time, but I don't recall having any issue with the video latency, only the audio which was horribly off. I still want to release a proper Moonlight android with OBOE support but I have no time right now and I'm not sure it would help for your issues. |
|
@peacepenguin what Linux build do you use on your rk3588 device? Is 4k120 h265 possible with this build? If so, at which bitrate and how high is the decoding latency? |
|
Hi All, I'm happy to report that this issue is RESOLVED! (for the most part). The streams take anywhere from 1-5 minutes to even out and then their latency stabilizes (indefinitely -- tested for about an hour). There is no audio delay whatsoever (although this MAY vary depending on how your HDMI audio output is configured). Here are my most latest benchmarks on Fire OS 8.1.1.1 (RS8111/2273): 4K120 @ 113Mbps = ~7ms (not sure if this can actually output over HDMI but the stream will render this combo) *These two were an enigma for me... I'm not sure why these performed so much more poorly than higher resolution/refresh rate combos. I even tested to see if this was perhaps an overheating issue (used a portable fan) or something but even switching between 4K120 and back to 1080P120 the results were the same. I will edit the post later and add some screenshots with proof. EDIT: The above benchmarks were done on a 1080P144 screen. I retested on a 4K144 screen and here are the updated results: 4K120 @ 113Mbps = 8-9ms (I confirmed that the Fire Stick does NOT output at 4K120) What I surmised from these results is there is "some" overhead running the display at 4K despite the Moonlight game stream being capable of running these stream resolutions at a display resolution of 1080P at lower latencies. |
I had bought the new 4k when it launched (November 2023) and sent it back because it was performing considerably worse than my old 4k stick. I can confirm that now everything works as it should. I stream at 1440p60 40Mbit and 20Mbit and have 4-6ms with the cable adapter. The sound lag I had experienced (significant sound lag, around 200ms), is simply no more. Everything works perfectly well on the firetv 4k. |
Thinking of installing MoonLight on my Firestick 4k. Do you think the wired adapter is a must? How are you dealing with bluetooth controller connection? Also, what kind of input latency do you get with or without ethernet adapter? Thanks! |
|
For information this fire tv 4K max is a disaster. |
|
Hi, possibly unrelated but I am using the Fire TV 4k (1st generation, running Fire OS 6) and I am seeing around 17ms decoding latency with HEVC. Is there any fix for this? |
|
is this issue still happen? I noticed that in SoC-Benchmark is recommending the Fire TV Stick 4k MAX (2023 / 2nd Gen) with 3.70ms latency. |
|
|
|
@pr8x No, upgrade Moonlight clients. This hardware is underpowered 4K. Perhaps try 1080P on this device. @jairmedeiros Yes, this issue has been resolved. 4K60 @ 80Mbps is around 6ms decoding or 1080P60 around 3.5ms. Obviously not as good as the prior generation (I think this may have been Android 9 vs 11 discrepancy) but still decent enough to be the "best budget" option being at $30 when on sale. No other devices around this price point match it until Apple TV ($129), Shield TV ($149), or N100/N97 Mini PCs (~$120). @mb217x The Fire Cube (2019) uses an entirely different SOC (S922X / Mali G52-MP2) so this particular issue is not relevant to this device. I doubt there will be any more effort in to improving this particular platform (FireOS 7 / Android 9) so I would recommend upgrading to a different Moonlight client. See above recommendations. |
|
Closing issue since this was resolved via a FireOS update. |
and others
Describe the bug
The Fire Stick 4K Max (2023) has higher than expected latency given the hardware specs compared to the last generation (2021).
I observed the following benchmarks:
Enabling Game Mode on the Fire Stick did not make any difference. For comparison, the previous generation Fire Stick 4K Max (2021) achieves 4K60 80Mbps HEVC at 4ms average.
Setup:
One additional thing I noticed is that it seems to receive network bottlenecking after 50Mbps where my network latency jumps from 2-3ms to 8-12ms. This is tested using Wifi 6E on AXE6600 router (no other clients have network issues running 4K120 150Mbps). Using the built in Fire Stick internet speed test, I am seeing a max download speed of 419Mbps. This network issue may be hardware related but was not observed on the previous generation.
Steps to reproduce
Affected games
Tested using Desktop and opened Minecraft, Heaven Benchmark, and Timespy.
Other Moonlight clients
PC
Moonlight adjusted settings
Yes
Moonlight adjusted settings (please complete the following information)
I have tested using H.265 and H.264. Adjusted bitrate from 20-80Mbps (observed the bottlenecking past 50Mbps).
Moonlight default settings
Yes
Gamepad-related connection issue
No
Gamepad-related input issue
No
Gamepad-related streaming issue
No
Android version
Android 11
Device model
Fire Stick 4K Max (2023)
Server PC OS version
Windows 11
Server PC GeForce Experience version
0.21
Server PC Nvidia GPU driver version
546.17
Server PC antivirus and firewall software
Windows Defender
Screenshots
No response
Relevant log output
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: