Removing completed jobs automatically & atomically

[bradvogel]

It's our desire to remove jobs automatically upon completion. My understanding of the Bull API is that we should do this by listening to the completed event on the queue and call job.remove(), like this:

this.on('completed', job => job.remove());

However, we're finding that some completed jobs aren't getting removed. The likely reason for this is that the server is occasionally being killed right after the job is moved to "completed", but before job.remove() can be called. Since events are broadcast locally, there is no second server to pick up the message, the job stays around forever. Our current workaround is to run a cron job every hour that calls clean(10 * 1000, 'completed') on the queue.

I wanted to get your thoughts on the following possible approaches to clean up completed jobs:

1. Add a removeOnCompleted option to the job and have the moveToCompleted script atomically remove the job upon completion.
2. Use the new distributed events architecture to broadcast the 'completed' event to all workers, so one can remove the job.

[manast]

do you have any chance to test with master to see if it works better with the distributed events?
An atomic remove may be an interesting feature and not difficult to implement either.

[bradvogel]

Not yet. Will test though. I think we should internalize removal through a new removeOnComplete job option, pending the outcome of bug #356 which might affect this.

[bradvogel]

Fixed in https://github.com/OptimalBits/bull/pull/361/files