Replay frame delimiters for EGL

[hevrard]

When export_replay creates a standalone APK to replay a trace, we want
to be able to trace the replay itself, and we want to detect frame
delimiters during this tracing. Hence the replay of EGL frame
delimiters.

[pmuetschard]

I am very worried that this will cause issues for regular replays. As we've not been replaying any EGL calls, we are not remapping the EGLDisplay, nor the EGLSurface. Simply using the pointers that the application had received at trace time during replay seems dangerous and I suspect would cause the replay to crash on some drivers.

I would recommend that instead we add a synthetic call that will do the eglSwapBuffers call on the replay surface. This way, we also would have the additional benefit of being able to only add the calls when we need them and not on every replay.

[hevrard]

OK, starting to replay some EGL calls may cause mayhem in the replay. I will have a go at writing synthetic calls later as I focus on testing now. Thanks for the feedback!

[hevrard]

@pmuetschard can you tell me if this is the right direction?

I'm not sure if you meant to have synthetics for the actual EGL calls, or for other made-up commands that would replace the EGL SwapBuffers calls?

[pmuetschard]

Yes, this is getting there. However, you still need to actually have the replayer call eglSwapBuffers, otherwhise this will not do anything. To do that add a swapBuffers method to the renderer and have the synthetic function call it.

Also, consider making the synthetic a separate API function and have the API call it. Similar to how the eglMakeCurrent calls the bindRenderer synthetic. I'm not certain, however, if that is necessary/the right thing, but please consider it and decide.

[hevrard]

Let me try to wrap up a chat with Ben:

• GAPIR needs to make a call to signal a frame delimiter
• GAPII needs to intercept that call as a frame delimiter
• This call may be eglSwapBuffers, but it can also be:
  i. an other API call (e.g. glFinish), such that it can be replayed by the driver with minimal impact and without having to deal with EGL
  ii. a new made-up API call implemented by GAPII, e.g. gapidFrameDelimiter, which acts as a delimiter when intercepted, but which is not passed onto the driver

(i) may be an issue as we basically modify the trace to replace an EGL call by a different GL call which may have impact on various things. Also, glFinish would need to be marked as a frame delimiter, which may create weird frames delimitations in applications that expect frame delimiters to be only at EGLSwapBuffers.
(ii) may be an issue for tools other than GAPID, which will not recognize gapidFrameDelimiter

A solution is to byte the bullet and do tracing and replay of all EGL-related things -- that will take some time.

In the meantime, shall we go with option (2)?

[ben-clayton]

A few other comments: a) While you *may* attempt to call eglSwapBuffers, we do not have all the info required to call it 'correctly' (surface, display are ?) This seems to me to be a Bad Idea. b) glFinish is *typically* a noop as drivers think they always know best. That said, I don't think it's a good idea to go emitting these all the time. This might change behavior / performance. c) the gapidFrameDelimiter suggestion is to expose this function via eglGetProcAddress d) once this function is called, there's a question of what GAPII should do with it. Do we encode a new synthetic API command to the stream? e) if we're making a new API command, it might be tempting to try to fold (c) and (d) together (as in pick up the new API function from eglGetProcAddress). However, this is unlikely to 'just work', and I'm not sure how/if it would work with the new gles layers.

…

On Thu, Apr 25, 2019, 2:31 PM Hugues Evrard ***@***.***> wrote: Let me try to wrap up a chat with Ben: - GAPIR needs to make a call to signal a frame delimiter - GAPII needs to intercept that call as a frame delimiter - This call *may* be eglSwapBuffers, but it can also be: i. an other API call (e.g. glFinish), such that it can be replayed by the driver with minimal impact and without having to deal with EGL ii. a new made-up API call implemented by GAPII, e.g. gapidFrameDelimiter, which acts as a delimiter when intercepted, but which is not passed onto the driver (i) may be an issue as we basically modify the trace to replace an EGL call by a different GL call which may have impact on various things. Also, glFinish would need to be marked as a frame delimiter, which may create weird frames delimitations in applications that expect frame delimiters to be only at EGLSwapBuffers. (ii) may be an issue for tools other than GAPID, which will not recognize gapidFrameDelimiter A solution is to byte the bullet and do tracing and replay of all EGL-related things -- that will take some time. In the meantime, shall we go with option (2)? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2708 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACXY4VDGKQLXL2RFG4B7HJDPSGXDXANCNFSM4HD6TYMQ> .

[AWoloszyn]

Following up on @ben-clayton's comment.
a is what we would do in an ideal world. This would mean we have to extract a significant amount of information from the original application in order to correctly replay. We would need to have surface/display information that we are currently lacking, and then re-create it correctly/exactly. Furthermore we would have to track this over time in order to handle cases like screen rotation/etc.

glFinish should cause a flush of the GPU, meaning we would be introducing stalls into the pipeline, so that is not an ideal option.

gapidFrameDelimiter may work, but has an the problem that we will still be changing the behavior of the program. Ignoring swap may have unintended side-effects. However this may work if we just want to be able to trace a replay for debug purposes (rather than performance purposes).

[pmuetschard]

I'd say we simplify it all and simply call swap buffers on the currently bound renderer (which has a display and a single surface). Isn't that what we are assuming already anyways? I mean, how are we currently associating an eglSwapBuffers call with a context? AFAICT, we just assume that it's called from the same thread that owns the context.

Maybe there will be that one app that'll swap buffers from a different thread, but it won't hurt to then just ignore it in the replay if there is no renderer bound on the current thread.

[hevrard]

@pmuetschard I think I implemented what you had in mind, and now tracing replay on GLES seems to work! Could you have a look and let me know what you think?

A few comments:

• how can we know which EGL context is currently bound? (Here I iterate over renderers until one of them has a valid surface to call swapBuffers on)
• if the current approach looks OK, we should we do something similar for eglSwapBuffersWithDamageKHR().
• only the Android platform actually replays the eglSwapBuffers, we can extend to other platforms
• do we want to not replay such frame delimiters unless we need to (e.g. generic replay does not need to, but a replay from a standalone APK needs to)?

[ben-clayton]

LGTM

[pmuetschard]

a few nits, and a simple change.

Please squash when you commit, as there is some back-and-forth in this PR.

[hevrard]

Thanks @pmuetschard for the review!

Could you also let me know what you think on these points?

• if the current approach looks OK, should we do something similar for eglSwapBuffersWithDamageKHR()?
• only the Android platform actually replays the eglSwapBuffers, should we extend to other platforms?
• do we want to not replay such frame delimiters unless we need to (e.g. generic replay does not need to, but a replay from a standalone APK needs to)?

[pmuetschard]

• if the current approach looks OK, should we do something similar for eglSwapBuffersWithDamageKHR()?

i don't think so.

• only the Android platform actually replays the eglSwapBuffers, should we extend to other platforms?

Only Android for now is fine with me.

• do we want to not replay such frame delimiters unless we need to (e.g. generic replay does not need to, but a replay from a standalone APK needs to)?

For now, unless we find a reason not to replay them, let's keep it as you have implemented it.