Schedule backup at next opportunity

[MatejKovacic]

I am using Vorta on Ubuntu and Linux Mint and set that it should run every time at 10:00.

Now, the problem is, that sometimes computer is turned off at 10:00. Su the user comes at 11:00, starts the computer, do some work and turn the computer off. And so the next day and the day after. Since computer is switched off several days at 10:00, scheduled backup does not run. So I am missing backups for a three days.

The solution would be, that backup will be run at 10:00 or at the next possible opportunity. So, Vorta should have an option to enable scheduled backup at a precise time, or at a precise time and if that was not successful, at the next possible opportunity.

So in latter case, user turns the computer on at 11:00, Vorta detects that backup at 10:00 was not run, so it runs it right away.

[m3nu]

Should be simple to do by adjusting APScheduler's misfire_grace_time. Currently it's just a minute or so to account for the process being at sleep. If you want to account for the computer being at sleep it would need to be a few hours.

[daniel-k]

I was researching the same thing, studied the code a bit and then found this issue. After taking a look into the docs of APScheduler, I think that just increasing misfire_grace_time wouldn't work in this case because (I assume) when the computer is turned off and started again, Vorta would start freshly too. Since it currently reconstructs the scheduled jobs at every program start from it's internal database and not by using a persisted job store, the job would not be "missed" because from APScheduler's point of view, it is still in the future.

I see two solutions to this:

1. use a persisted job store (but they bring some constraints)
2. keep track of the last execution and check if there's a missed "slot" since then

My personal motivation looking into schedulers is that I want an option to run the backup at a certain interval, e.g. "every 24h". There's no rule at which time of the day I'd use my laptop. So for me it makes no sense to have rule attached to a certain hour of the day. But still, I'd like to have daily backups.

@m3nu What's your take on this issue? I'd try to implement this and provide a PR, but I think it makes sense to agree on the implementation beforehand. Any guidance is well appreciated!

[m3nu]

@m3nu What's your take on this issue? I'd try to implement this and provide a PR, but I think it makes sense to agree on the implementation beforehand. Any guidance is well appreciated!

All improvements are welcome as long as they don't add a lot of code (to maintain in the future) for minor gains.

Before you implement anything, you should give an outline of the planned changes here, other maintainers a chance to give input.

[daniel-k]

So I toyed around with the scheduling during the last days and came up with a working prototype.

Goal:
Specify a "relative" backup interval, no matter which hour of the day. When Vorta wasn't running, honour the interval so don't start over again.

I achieved this by using a persistent jobstore for apscheduler and the interval trigger. This basically works nicely already, but I ran into an edge-case when at program start the borg version is checked, blocking an immediately scheduling backup job (all borg threads share a common mutex). To work around this and make it more robust, I implement a retry-mechanism for failed jobs.

The code is not 100% polished and I didn't touch the user interface yet. I'd propose to add a third scheduling method. However, I'm not sure about the naming. I don't like "relative interval" that much.

What are your thoughts? I'd be happy to submit a PR after cleaning up the code a bit more and integrating the new method into the user interface.

EDIT: Using a persistent jobstore btw. solves the initial issue brought up by @MatejKovacic
EDIT2: My changes are only touching code in the scheduler.py

[m3nu]

Cool. Neat idea to use the persistent jobstore and keep the library. Totally overlooked this option.

Looking forward to a PR, so others can weigh in and test.

[MatejKovacic]

It seems great. How can I test it?

[daniel-k]

@MatejKovacic see https://vorta.borgbase.com/contributing/ for how to setup a local dev setup. Just clone from my fork (git clone https://github.com/daniel-k/vorta.git) and checkout the feature branch (git checkout feature/trigger-everyday). I'd recommend to install everything in a virtual environment.

[sten0]

Ideally I'd like to see a config key for "Run on battery", because the "next opportunity" could be in a low-battery state; very few people want their last 20% of battery to be eaten by a CPU and IO intensive process. On the Linux Desktop udisk is standard, there is a Python lib/wrapper for it, and I would imagine OS X either provides its own interface, or that there is a Python lib/wrapper for this. Maybe there's an OS-agnostic one? :-)

[sten0]

P.S. UPower also has a DBus interface, so my wishlist condition could be solved using a similar method to #355. The less polling the better for battery life!

[renyhp]

My two cents: if the scheduled time is missed, the app could actually ask the user (with a popup or notification) if they want to backup. Maybe set that as an option. I don't know if that makes the problem simpler or harder to solve, but it would certainly give even more flexibility.

[MatejKovacic]

Yes and no. Sometimes you want backups to be totally seamless.

So... having an option to ask or to execute without the question? :)

[mauromol]

This would probably easier if Vorta used anacron, but #294 was rejected ☹

[samuel-w]

The issue with running the backups immediately is that multiple backups can be missed, and currently you are unable to run multiple backups at once. See #655.

[m3nu]

The interval schedule above would probably go a long way when combined with our own event log.

https://apscheduler.readthedocs.io/en/latest/modules/triggers/interval.html#module-apscheduler.triggers.interval

[sten0]

On Thu, 18 Feb 2021 at 11:46, Manu ***@***.***> wrote: The interval schedule above would probably go a long way when combined with our own event log. https://apscheduler.readthedocs.io/en/latest/modules/triggers/interval.html#module-apscheduler.triggers.interval

+1, apcscheduler looks like the right way forward to me, and I'm especially happy to read "The jitter option enables you to add a random component to the execution time. This might be useful if you have multiple servers and don’t want them to run a job at the exact same moment or if you want to prevent multiple jobs with similar options from always running concurrently". This feature is precisely what convinced me to consider it as a serious cron/anacron alternative for desktop-oriented software that doesn't have a backend daemon (eg: like bacula)...but I'm not sure how useful that will be, but https://apscheduler.readthedocs.io/en/stable/userguide.html#missed-job-executions-and-coalescing solves what I think it is the biggest problem for typical laptop users: laptop is closed during the time when the backup is scheduled, and the user is unlucky enough that the "jitter option" isn't enough to luckily coincide with a window of time when the laptop is open and active. I think between that and the event log it should also be possible to defend against the case where a user has been closing their laptop mid-backup, thus aborting backups, for a while, without realising this was occurring. It seems like apcscheduler will also provide what is needed to detect this, and perhaps warn the user that "no backup has succeeded in $period". Both of these are the two most significant issues I've identified in the Vorta users I personally support.