Cursors and Pagination

[ghost]

Is there a way to offset a query, e.g: g.V().GetLimit(limit, offset)? Such a feature would be very handy for pagination purposes.

[barakmich]

What you really want (and I've been thinking about) is a cursor API. Eg, g.V().Cursor() (returns a cursor token) and then g.Cursor(token).Get(100) (returns 100 things from the prior query and a new cursor token).

This is effectively serialization of iterator trees for those playing at home, and is tangentially related to #64, in that you're building things that may need serialization, but is really just a feature unto itself.

[fisnik]

Any news on this feature - i think this is a must, without it, it would be hard to consider Cayley for production on anything dependent on pagination.

Also, how would the cursors be handled? Would the presence of cursors become an impediment on a live production deployment? If you have thousands of clients requesting paginated data, than you would have as many cursors as clients.

[jzelinskie]

Totally stealing @barakmich's words:

You can serialize the iterators and encode them and pass them as a token to the client. Thus, the client can just pass the token back and cayley deserializes it and continues iterating for results.

[iddan]

I think there should be a streamable content type for Gizmo to allow

[dennwc]

@iddan This would require some interactive API like WebSockets or gRPC, but it's easier to implement than serializing the iterator chain (although achievable).

[iddan]

This would require some interactive API like WebSockets or gRPC

I think writing multiple chunks to the HTTP message will be good enough. In Go's http I think you can use Response.Write() and Response.Flush(). We just need to use a different content type that supports partial messages like NDJSON or NQuads