Very slow serialization performance

[ajmurmann]

I've been working on a project where we have a very deeply nested structure that we need to serialize. We initially thought that a n+1 query was the cause for requests being slow. However, as we removed the n+1 query performance actually got worse. The request used to take ~5 seconds. 3 in AR, and 2 rendering. However, after we removed the query issue, the total time went to about 10 seconds. 2 seconds querying the DB and ~8s rendering the view. Ruby-prof revealed that the bulk of that time is being spent calling Class#new in the serializers. The resulting JSON output comes in at 85k with the data we currently have and probably has 378 objects embedded.

Is this a known issue? If so what are the recommended workarounds?

Unfortunately I cannot share any of our code, since I don't own it. If need be I might be able to find some time to reproduce this in a toy app.

We are using ActiveMode::Serializer 0.10.4, Rails 5.0.1 and ruby 2.3.3p222

Thanks for any help!

[bf4]

I'm guess you're not using any caching? Any insight into which Class.new is being called? Any idea of how many 'records' you're trying to render relative to the number of objects being created? Do you mean you're rendering 378 records in one response to 85k? That's big. Which adapter are you using?

I wouldn't say it's a known issue, but it's a known concern that I'd like to address. If you can provide a test case I can benchmark, I'd love to use that to dig into the performance issue.

[ajmurmann]

We are using the json_api adapter. Yes, it's 378 records. It's a large tree structure of many small objects that are usually just a String containing a sentence or two, a int ID and maybe a boolean. We tried caching in memory, but we got not reliably measurable performance improvement. Not sure if that's related to #1586

[ajmurmann]

It seems like we are just serializing a unreasonable amount of objects. I am still unsure why this got slower by a factor of 4 once our SQL query was improved, but resolved this issue by splitting out more sub-objects and separate endpoints and using Rails caching instead of active-model-serializer caching.

[mmahalwy]

@ajmurmann mind sharing what kind of rails caching you did?

[ajmurmann]

@mmahalwy We used Rails conditional GET: http://guides.rubyonrails.org/caching_with_rails.html#conditional-get-support

[sammyhenningsson]

I've also experienced poor performance with the json_api adapter. Serializing 378 records soons unhealthy and they should most likely be paginated. However if you assume that rendering I linear and divide the records by 20 you get less than 20 records then one would expect the time to render would be 8/20=0.4s. I think this is still way too slow.

[beauby]

@sammyhenningsson Have you tried an other JSON API lib (such as http://jsonapi-rb.org) to see whether perf issues were tied to the lib or to your business logic?

[sammyhenningsson]

I haven't tried any other library. Thanks for the advice! I will have look..

[arefaslani]

@beauby I'm using JSONAPI-RB for a few weeks. It's awesome, but unfortunately it has a week documentation and a small community around it...

[beauby]

@arefaslani Thanks for the feedback. It would help a lot if you could spare a few minutes opening an issue on jsonapi-rb listing the questions that you struggled with so that I could improve the docs.

[arefaslani]

@beauby Sure I will do this. Please continue the great job!