Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'Index out of range' error using Ember.computed.sort with multiple sort properties #9278

Closed
slindberg opened this issue Oct 6, 2014 · 12 comments · Fixed by #9356
Closed

Comments

@slindberg
Copy link
Contributor

This is a nasty one. The issue occurs inside the removeItem array computed callback when finding the correct index for an item whose observed sort property changes. The binary search can return a bad index due to the internal order function not properly reporting the item's order. This can happen when 1) there are multiple sort properties, 2) only one of them changes, 3) the previous values of the changed sort property were equivalent, and 4) the change moves the item to the other side of the midpoint of the array. This is due to a change introduced in this commit, where the sort property is looked up on the item without Ember.get when it is a SearchProxy.

The problem is that the SearchProxy instance is created with the changedProperties meta object, which obviously does not contain unchanged sort properties. This means that when the unchanged property is looked up directly in this line:

result = compare(get(itemA, sortProperty), isProxy ? itemB[sortProperty] : get(itemB, sortProperty));

It uses undefined as the comparison instead of that actual sort property.

jsbin

The issue that the bugfix commit corrects is still valid, so I see a few ways to address this new one:

  1. Do a hasOwnProperty check for the sort property on the proxy object before accessing it directly. More overhead, but simple.
  2. Add all sort properties when creating the search proxy object in removeItem. Probably more overhead than the previous solution, but still simple.
  3. Address the original issue directly in SearchProxy somehow. Not sure if this is possible since it would likely require overriding get.
  4. Stop using a binary search altogether in removeItem and just use array.indexOf(...). I assume this was done originally for performance on large lists, so this is probably out.
  5. Maintain an index mapping for all objects so that the index never even needs to be found. I also assume this is out for memory reasons.

@mmpestorich, @hjdivad: thoughts?

Also possibly related to: #4831

@mmpestorich
Copy link
Contributor

@slindberg Thanks for this. Yes, it is a nasty one. Our previous commit 6c4079f should probably have never made it into beta. @mattraydub and I have been aware of this for a little while now and temporarily worked around it by doing exactly what you proposed in 4 above; eliminating the binarySearch in computed.sort altogether:

    addedItem: function (array, item, changeMeta, instanceMeta) {
      var index = instanceMeta.binarySearch(array, item);
      //array.insertAt(index, item);
      if (array.indexOf(item) === -1) array.insertAt(index, item);
      return array;
    },

    removedItem: function (array, item, changeMeta, instanceMeta) {
      //var proxyProperties, index, searchItem;
      //
      //if (changeMeta.previousValues) {
      //  proxyProperties = merge({ content: item }, changeMeta.previousValues);
      //
      //  searchItem = SearchProxy.create(proxyProperties);
      //} else {
      //  searchItem = item;
      //}
      //
      //index = instanceMeta.binarySearch(array, searchItem);
      //array.removeAt(index);
      array.removeObject(item);
      return array;
    }

However, I don't belive that that eliminating binarySearch is a good solution. It is, however, the only one that we have come up with that allows us to still use property paths in sortPropertyDefinitions. There are actually two related issues here:

  1. As you describe above, SearchProxy is created using only the modified properties exposed by changeMeta. When there are multiple sort properties, the order function used by binarySearch erroneously compares itemA to itemB when an unchanged sort property is used. This is because itemB['sortProperty'] in this case will always be undefined as you point out.
  2. When the value of a property changes on an object that is shared among multiple items (i.e. a common parent in a belongsTo relationship) and that property is a sort property, the order function compares the new sort property value of itemA to the old sort property value of itemB. It instead should compare using the values last used to sort the array, which are the old sort property value of itemA to the old sort property value of itemB.

I believe, in order to move past the second issue, it's going to be necessary to create a cache of sort property values so that a proper comparision between a itemA and itemB can take place such that the original value of a sort property value on itemA is used. A side effect of this approach is that we should be able to eliminate the need for a SearchProxy altogether which, in turn, should fix the first issue. I hope to have a pull request for this completed sometime soon.

@slindberg
Copy link
Contributor Author

Thanks @mmpestorich, glad to hear you're on top of it 😄

@stefanpenner
Copy link
Member

does array/reduce computed use Ember.Map or Ember.OrderedSet internally? cc @hjdivad

@mmpestorich
Copy link
Contributor

@stefanpenner Not that I can see. Internally, reduceComputed's DependentArrayObserver maintains a hash of changedItems in itemPropertyWillChange and then calls flushChanges in itemPropertyDidChange. Then flushChanges uses changedItems amongst others to create a changeMeta that is passed along to addItem and removeItem.

Originally, I thought that exposing changedItems via the changeMeta would be helpful but, rather then changing the internals of reduceComputed, I felt like using instanceMeta to maintain a cache of sort property values is a better approach - isn't that what instanceMeta is there for?

@ef4
Copy link
Contributor

ef4 commented Oct 18, 2014

This bug was really bugging me. Please see the above PR and let me know what you think.

@electricgraffitti
Copy link

Was this fix going to be in 1.8.0? I just updated to 1.8.1 and still get the "Index out of range" error on arrays that are using the computed.sort with any number of sort properties

@slindberg
Copy link
Contributor Author

@electricgraffitti it looks like the bugfix only made it in at v1.9.0-beta.1

@kelsin
Copy link

kelsin commented Jan 7, 2015

Sorry to comment on a closed issue, but I just updated to 1.9.1 hoping to catch this bug fix. Unfortunately my app is still getting the Index out of range error. Was it moved to another milestone? Is there a way to easily see what release #9356 will be available in?

@slindberg
Copy link
Contributor Author

@kelsin yes, there is a easy way to see: click on the PR (#9356), then find where it got merged and click on the merge commit (7b270f8). At the top , below the commit description, you can see that the commit is in v1.10.0-beta.3 … v1.9.0, meaning that the earliest release the fix is in is v1.9.0.

@kelsin
Copy link

kelsin commented Jan 7, 2015

Ok so my bug must be a variation of this one. Thanks so much!

@kimroen
Copy link
Contributor

kimroen commented Feb 2, 2015

I also have a bug that I think is very closely related to this one - here is the stack trace:

Error while processing route: product Index out of range Error: Index out of range
    at new Error (native)
    at Error.r (https://pam.no/assets/vendor-9f8fee9a7deaea9585c549a5c2cddb91.js:7:26)
    at Array.s.default.f.create.removeAt (https://pam.no/assets/vendor-9f8fee9a7deaea9585c549a5c2cddb91.js:10:31342)
    at M.removedItem (https://pam.no/assets/vendor-9f8fee9a7deaea9585c549a5c2cddb91.js:10:11031)
    at Object.m.flushChanges (https://pam.no/assets/vendor-9f8fee9a7deaea9585c549a5c2cddb91.js:10:4481)
    at Object.m.itemPropertyDidChange (https://pam.no/assets/vendor-9f8fee9a7deaea9585c549a5c2cddb91.js:10:4126)
    at Object.m.createPropertyObserver.e.observer (https://pam.no/assets/vendor-9f8fee9a7deaea9585c549a5c2cddb91.js:10:2031)
    at _ (https://pam.no/assets/vendor-9f8fee9a7deaea9585c549a5c2cddb91.js:8:7181)
    at f (https://pam.no/assets/vendor-9f8fee9a7deaea9585c549a5c2cddb91.js:7:2509)
    at v (https://pam.no/assets/vendor-9f8fee9a7deaea9585c549a5c2cddb91.js:7:27420)vendor-9f8fee9a7deaea9585c549a5c2cddb91.js:9 wvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:9 Q.errorvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:9 Cvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:14 lvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:14 i.triggervendor-9f8fee9a7deaea9585c549a5c2cddb91.js:14 (anonymous function)vendor-9f8fee9a7deaea9585c549a5c2cddb91.js:14 yvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:14 bvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:14 gvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:14 dvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:10 (anonymous function)vendor-9f8fee9a7deaea9585c549a5c2cddb91.js:4 r.invokeWithOnErrorvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:4 r.flushvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:4 n.flushvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:4 o.endvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:4 o.runvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:7 lvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:17 yt.extend.ajax.Ember.RSVP.Promise.s.successvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:2 J.Callbacks.cvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:2 J.Callbacks.h.fireWithvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:3 rvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:3 J.ajaxTransport.X.cors.e.crossDomain.send.tXMLHttpRequest.send (async)vendor-9f8fee9a7deaea9585c549a5c2cddb91.js:3 J.ajaxTransport.X.cors.e.crossDomain.sendvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:3 J.extend.ajaxvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:17 yt.extend.ajaxvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:14 _vendor-9f8fee9a7deaea9585c549a5c2cddb91.js:14 hvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:17 yt.extend.ajaxvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:17 yt.extend.findvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:16 $vendor-9f8fee9a7deaea9585c549a5c2cddb91.js:18 Ember.Object.extend.fetchRecordvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:18 rvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:18 Ember.Object.extend._flushPendingFetchForTypevendor-9f8fee9a7deaea9585c549a5c2cddb91.js:7 h.forEach.tvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:7 c.forEachvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:7 h.forEachvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:18 Ember.Object.extend.flushAllPendingFetchesvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:4 r.invokeWithOnErrorvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:4 r.flushvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:4 n.flushvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:4 o.endvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:4 o.runvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:7 lwebclient-7cb8831e571b29c1360aa0fb53e1aa73.js:4 u.extend.modelvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:8 _vendor-9f8fee9a7deaea9585c549a5c2cddb91.js:8 rvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:9 U.extend.deserializevendor-9f8fee9a7deaea9585c549a5c2cddb91.js:14 pvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:13 n.runSharedModelHookvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:14 a.getModelvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:14 rvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:14 yvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:14 bvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:14 gvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:10 (anonymous function)vendor-9f8fee9a7deaea9585c549a5c2cddb91.js:4 r.invokeWithOnErrorvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:4 r.flushvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:4 n.flushvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:4 o.endvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:4 o.runvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:4 o.joinvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:7 l.joinvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:12 d.default.y.extend._bubbleEventvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:12 (anonymous function)vendor-9f8fee9a7deaea9585c549a5c2cddb91.js:2 J.event.dispatchvendor-9f8fee9a7deaea9585c549a5c2cddb91.js:2 J.event.add.g.handlevendor-9f8fee9a7deaea9585c549a5c2cddb91.js:23 e.bugsnag

I'm not sure where to even start debugging this - should I start a new issue for it?

@ef4
Copy link
Contributor

ef4 commented Feb 2, 2015

@kimroen if you're using Ember 1.9 or newer, please file a new issue. Otherwise try upgrading first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants