Store fails to properly handle requests to create a new record which returns an id that belongs to another equivalent record on the store

[arthurmde]

I faced a tough bug in my App (Rails + Ember) that happens in an edge case of race condition between regular HTTP requests to the backend and async WebSocket messages updating the records on the store.

Summarized Description

Given the following piece of code:

store.createRecord('item').save();

Ember breaks if a request to create a new record on the backend returns a server-side generated id which has already been used by another (equivalent) record on the store, raising the following errors:

• Assertion Failed: 'item' was saved to the server, but the response returned the new id '5cf17edc141edc9828ac0039', which has already been used with another record.
• Assertion Failed: You can only unload a record which is not inFlight. <item:null>

Reproduction

Now, I'll describe how it is possible to achieve the above scenario in an App based on WebSocket.

Suppose the App has two models:

• Project - with a hasMany relationship with items
• Item - with a belongsTo relationship with project

Also, consider that the App has a WebSocket (WS) connection with the backend. Whenever a record is created or updated, the backend asynchronously streams a JSON-API message with the record to the frontend clients through the WS connection. The frontend simply adds/updates the store with the streamed resource through the store's pushPayload method.

Now, consider the following two concurrent flows indicated by sequential, italic numbers (HTTP-based flow) and bold letters (WS flow):

1. Ember runs let project = await store.createRecord('project').save();
2. Ember creates a POST request to the backend
3. Backend stores the new project and generates a new id for the project
4. Backend returns the resource as JSON-API payload and a 201 HTTP status code to Ember
5. Ember updates the store with the returned data and id
   a. The backend schedules a job to stream WS messages asynchronously with the created project
6. Ember runs let item = await store.createRecord('item', {project: project}).save();
7. Ember creates a POST request to the backend
8. Backend stores the new item with a new id and a relationship with the project
   b. The backend's WS worker get the project from the database and streams it as JSON-API to the frontend in response for step a. Since the new item has been saved to the database in step 8, it will be included in the items relationship of the project
   c. Ember WS handler receives the message with the updated project resource and updates the store with pushPayload. Consequently, not only the project entry is updated with the relationship to the new item, but also a new 'item' entry is created on the store corresponding to the same record created in step 6 with the server-side generated id.
9. Backend responds to the request started at step 7 with the item resource in a JSON-API payload and a 201 HTTP status code to Ember
10. Ember tries to update the store with the returned data and id but it raises the following error:

    Assertion Failed: 'item' was saved to the server, but the response returned the new id '5cf17edc141edc9828ac0039', which has already been used with another record.
    

11. Moreover, Ember does not change the state of the 'item' object declared in step 2 which remains in the inFlight state forever. Later, if you try to unload the item from the store, you'll get the following error:

    Assertion Failed: You can only unload a record which is not inFlight. `<item:null>`
    

Discussion

Notice that the above-described race condition problem could also happen with slow clients or if the backend streamed the WS events before responding to the HTTP request.

IMHO, for successful create requests, Ember should replace any existing mapped internalModel for the returned ID by a new internalModel with the resource returned by the backend. Or, it should at least change the state of the invalid entry to enable its unloading from the store.

Workaround

I've implemented the following workaround for my test environment since the described race condition happens more often in my acceptance tests. Please, tell me if you have a better idea/workaround.

I overrode the application adapter's createRecord method to remove any previously existing record in the store equivalent to the returned resource in app/adapters/application.js:

createRecord(store, type) {
    let ajaxRequest = this._super(...arguments);

    if (ENV.environment == 'test') {
      ajaxRequest.then((response) => {
        let id = response.data.id; // From JSON-API payload
        let map = store._internalModelsFor(type.modelName);
        let internalModel = map.get(id);
        map.remove(internalModel, id);
      });
    }

    return ajaxRequest;
  },

Related Issues

This issue may be related to #4972 and #4262

Versions

└── [email protected]
└── [email protected]
└── [email protected]

[jbk83]

I have the same issue, and your workaround seems to do the job!

[runspired]

Closing as a duplicate of #1829

For applications exhibiting this form of race condition the API needs to reflect back a local identifier. This is achievable today without changes to ember-data but will be easier to resolve in the near future with the implementation of Identifiers.

Additional reading and coming primitives to enable users to better resolve this can be found in the Identifiers RFC and in the proposal for the JSON:API spec

[AlexMayants]

the API needs to reflect back a local identifier

@runspired, could you please elaborate this bit?