Update membership state of client user based on state events

[agraven]

Is your feature request related to a problem? Please describe.
Currently, the sdk updates what the client user's membership state of a given room is based on what timeline segment (invited, joined and left) in the sync response events for the given room appear in. However, this leads to edge cases where the sdk interprets the membership state of the client incorrectly. This is particularly an issue for appservices.

Describe the solution you'd like
Changes to the client's membership of a room should be based on which state events appear in a sync response, not which segment of the sync response events for a room appear in.

Describe alternatives you've considered
Library consumers could presumably do some of this handling based on state events themselves, however, the private state of the sdk cannot be affected by consumers, so this would be an incomplete approach.

Additional context
The specification's section on syncing: https://spec.matrix.org/latest/client-server-api/#syncing. As far as I can tell, it does not specify how clients should keep track of their own membership.

Some comments in #505 touch on the issue as well.

[agraven]

After discussing the matter in #matrix-dev:matrix.org, it seems that the only scenario where it can occur that there's a membership event in a sync response that indicates that a client should have a different membership that what segment a room appears in is when there's a DAG merge issue, or a server is ill-behaved. Link to the conversation: https://matrix.to/#/!jxlRxnrZCsjpjDubDX:matrix.org/$1ulwWgB3Zc9HAXl8ZRPWHWC_tKtVdf3zzfJtWu_Xg78?via=matrix.org&via=libera.chat&via=privacytools.io

On that note, I'm a little confused by the final state event in the mock sync response in matrix-sdk-test/src/test_json/sync.rs, as it includes a room_id field when events should be what the spec calls a ClientEventWithoutRoomID. I think the intent was to have the membership apply to a different room, but I'm relatively certain serde ignores the room_id field and just deserializes it as a AnySyncStateEvent, c.f. matrix-sdk-base/src/client.rs, line 466

Relevant part of the spec https://spec.matrix.org/v1.2/client-server-api/#get_matrixclientv3sync

[jplatte]

Given the tests are fixed and the conclusion overall seems to be that the current behavior is fine assuming the server is well-behaved (which you always have to assume, right??), can this be closed?

[agraven]

The membership handling for appservices still needs to be improved, but I guess that's best to have in a separate issue, rather than editing this one