Deal with VCS check plot memory and time issues #10
Comments
|
@doutriaux1, it's probably better to keep the issue (posted code, discussion) back in the vcs repo, no? If so which issue #? |
|
Let's start here because I suspect it's user error at this point. |
|
@doutriaux1 the two variants currently running (as it takes soooooooo long to plot these things, they're still running 12hrs later..) are: Both reference data on ocean |
|
@durack1 you had a template object created that you were not clearing, once this is fixed it seems stable. Will test again shortly when I get onsite because remotely it is painful. Might try wirth a mesa version as well just to see. |
|
@durack1 ok switching to |
|
@doutriaux1 what's the object name, and the command to delete it? I'll add it in and rerun my script/log.. |
|
@doutriaux1 I think that a trimmed down version of this script, along with a single timeslice of input data should be added to the test suite.. It seems it's a very sensitive example of timings and memory usage when turning |
|
@doutriaux1 would you like to revisit the "user error" #10 (comment) above? |
|
@doutriaux1 time to move discussions back into the vcs repo? I think it'll be cleaner to generate a dedicated issue and move comments there |
|
not fully revisiting the user erro, there was some left over templates which were creating additional object which you complained about it. So that was definitely a user error. There is only one textorientatation created per plot now. So that user-error part is nor solved. Need to figure out why contours create such a mem leak now though. |
|
What was the object that required deletion? You didn't answer my #10 (comment) above |
|
@doutriaux1 see CDAT/cdat#1424 (comment) it may in fact have already been identified by @chaosphere2112 The tool described in CDAT/cdat#1424 (comment) may also be really useful in determining the cause and finally sorting out this issue once and for all |
|
@durack1 rightfully pointe that using X we have no more leak! But is I switched to the mesa-based one I get: It is EXACTLY the same python underneath. @danlipsa I think it's either a |
|
@doutriaux1 @danlipsa and to fill in the holes, the reason why I can't use X11 is because with the VNC session (and |
|
@doutriaux1 it's also interesting to note that in your example above it's not just a single object all the time, so the transition: Some of these have a single additional python object created (the number behind the |
|
@durack1 some also have a great number of py obj going down. Probably gc related. To answer your question you had forgotten to remive the tmpl created in initVCS |
|
@durack1 when plotting s1s instead of s1 and using s1s for the diff instead of s1, which is what you actually meant we get a smaller memory print and the bug takes longer to appear: |
|
@doutriaux1 thanks.. This script is just a dummy/demo for me at the moment, so I don't really care about the png output.. Just was hoping to highlight the timing and memory issues in a repeatable way so that you and @danlipsa could dig in and figure out the persistent solution I think your comments on the |
|
@doutriaux1 the huge virtual memory footprint is also something that's concerning.. 12.3Gb when the script is generating a bunch of ~100Kb pngs is kinda concerning.. |
|
@durack1 I will send you the memory leak fixed script tomorrow. But yes with mesalib you get the memory leak nothing I can do about it right now. I will stop by tomorrow and try to get your png to be non black. |
The issues described in CDAT/cdat#1424 will be investigated within tweaks generated to solve the reported problems following VCS changes documented in CDAT/vcs#236
The text was updated successfully, but these errors were encountered: