-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Excessive GC load related to use of Sqlexpr_sqlite_lwt #13
Comments
mfp
added a commit
to mfp/lwt
that referenced
this issue
Feb 28, 2016
aantron
pushed a commit
to ocsigen/lwt
that referenced
this issue
Jul 2, 2016
aantron
pushed a commit
to ocsigen/lwt
that referenced
this issue
Jul 2, 2016
Refer to http://caml.inria.fr/mantis/view.php?id=7158 #218. mfp/ocaml-sqlexpr#13 Fix #218. Maintainer: Resolves #219.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The Sqlexpr_sqlite_lwt worker loop uses the
Event
module to receive tasks across threads boundaries.As it turns out,
Event.sync
creates a newCondition.t
.This means in particular that a new condition variable (two in fact, also one for the sending side) is being created per returned row(!).
caml_condition_new
allocates the custom blocks withused=1
,max = Max_condition_number
. The latter was raised to 5000 in 2010, after being set to 1000 back in 1996.The end effect is that a full major GC run is performed when at most 5000 rows have been read from the SQLite3 DB.
This raises two questions:
used=0
,max = 1
Now, back to Sqlexpr: time to ditch the
Event
module and replace it with a single condition variable + manual handling.Note to self: send issue to ocaml project regarding the performance impact of
Event.sync
.Edit: also file Lwt issue, for it also uses
Event.sync
inLwt_preemptive
(so 1 full major GC run every 2500Lwt_preemptive.detach
at most)The text was updated successfully, but these errors were encountered: