Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add weeks to schedule options #551

Closed
wants to merge 0 commits into from
Closed

Add weeks to schedule options #551

wants to merge 0 commits into from

Conversation

samuel-w
Copy link
Contributor

@samuel-w samuel-w commented Jul 18, 2020

This is another fairly large change, but it shouldn't break any existing setups.

This implements a persistent backup time storage, making #263 easier to implement.

The pretty_date localizes only the day of the week (Monday) but not the HH:MM am/pm
ui2

Fixes #487
Semifixes #263 by adding notification and extending gracetime so it runs missed backups if Vorta is not closed.

Edit: Fixes #343 (Last one hopefully)
This has become an enormous PR.

Discussions #804 #812

@m3nu
Copy link
Contributor

m3nu commented Aug 10, 2020

I'm all for having the every hour/day/week dropdown, but setting the starting time of Cron seems overkill.

@m3nu m3nu self-assigned this Aug 10, 2020
@samuel-w
Copy link
Contributor Author

samuel-w commented Aug 10, 2020

Its the only way I could think of for converting a QDateTime to a crontrigger.
Edit: Whoops, turns out there are multiple types of triggers. Will migrate to a date trigger https://apscheduler.readthedocs.io/en/stable/modules/triggers/date.html

Edit2: Peewee supports storing datetimefields, will change to that

@samuel-w samuel-w marked this pull request as draft August 10, 2020 23:23
@samuel-w samuel-w marked this pull request as ready for review August 10, 2020 23:51
@samuel-w
Copy link
Contributor Author

samuel-w commented Aug 11, 2020

Don't merge this yet, going to add #263.
Edit: Added it. Missed backups run immediately after startup.

@sudwhiwdh
Copy link

@samuel-w

Added it. Missed backups run immediately after startup.

Thank you, that's good news!

@m3nu

I'm all for having the every hour/day/week dropdown, but setting the starting time of Cron seems overkill.

Because?...

@m3nu
Copy link
Contributor

m3nu commented Aug 11, 2020

Because it clutters the UI/uses mental resources/adds more code to maintain while a majority of users doesn't care about it.

@sudwhiwdh
Copy link

sudwhiwdh commented Aug 11, 2020

@m3nu For me, the quality would be that I don't miss my backups if I as a user have the possibility to set a granular start time that fits well into my day. But this can also be solved elegantly by #263, right? Because that way I will never miss a backup in the end. Independent of the start time.

@samuel-w
Copy link
Contributor Author

samuel-w commented Aug 11, 2020

What type of trigger doesn't actually matter. In the end I'm using a run_date=datetime.datetime, eliminating the need for a trigger. This UI is less cluttered by removing the daily and hourly options.

Edit: I realize you're talking about the QDateTimeEdit

@samuel-w
Copy link
Contributor Author

samuel-w commented Aug 11, 2020

Initially, my UI concept was:

  • hourly: no time select
  • daily: hour select (QTimeEdit)
  • weekly: day select (QDateEdit)
  • monthly: day select (QDateEdit)

This would require switching between all the time periods, along with UI switching, requiring more code and maintenance. If for some reason we wanted to add or remove an option, it would be a lot more work this way than with the way I have in the PR.

Alternatively, it could be done with setDisplayFormat and enums (still requires switching code).

Also it would be confusing since the line length would shrink and grow as sections are removed/added, shifting text to the right of it around.

Edit: A search of "Backup scheduler" on google images reveals many UIs a lot more complex than this.

@samuel-w
Copy link
Contributor Author

samuel-w commented Aug 11, 2020

Basically what I'm saying is that this is the simplest it can get while supporting hours/days/weeks/months

Edit: As well as a persistent backup time store, needed for knowing if a backup was missed.

@samuel-w
Copy link
Contributor Author

Tests segfault on here, but on my own branch they work https://github.com/samuel-w/vorta-1/runs/1029008552?check_suite_focus=true. Weird.

@m3nu
Copy link
Contributor

m3nu commented Aug 30, 2020

I'm all for having the every hour/day/week dropdown, but setting the starting time of Cron seems overkill.

The dropdown option is great, but can we remove the starting at calendar field? Reasons explained above.

@samuel-w
Copy link
Contributor Author

samuel-w commented Aug 30, 2020

Initially, my UI concept was:

  • hourly: no time select
  • daily: hour select (QTimeEdit)
  • weekly: day select (QDateEdit)
  • monthly: day select (QDateEdit)

This would require switching between all the time periods, along with UI switching, requiring more code and maintenance. If for some reason we wanted to add or remove an option, it would be a lot more work this way than with the way I have in the PR.

Alternatively, it could be done with setDisplayFormat and enums (still requires switching code).

Also it would be confusing since the line length would shrink and grow as sections are removed/added, shifting text to the right of it around.

Edit: A search of "Backup scheduler" on google images reveals many UIs a lot more complex than this.

This was my reasoning behind the calendar field. TLDR: Calendar is the simplest way to do it in code and UI

Removing it and autosetting the time would not work either, since it removes functionality that was there before (run backup daily at specific time). Its a thing that is set infrequently anyways.

In case your talking about the calendar dropdown itself and not the QDateTimeEdit, its easier to visualize dates using a calendar than typing.

@m3nu
Copy link
Contributor

m3nu commented Aug 30, 2020

The drop-down and number field are great. Every X hours/days/weeks. I just question the usefulness of setting the start date. Won't it become meaningless after a while? User needs control of it?

@samuel-w
Copy link
Contributor Author

The start date is automatically bumped every time a scheduled backup is run, so it's always correct.

@m3nu
Copy link
Contributor

m3nu commented Aug 30, 2020

Don't we show the next backup run further down already? What's the purpose of the starting at field exactly? In your screenshot they show the same date and time.

@samuel-w
Copy link
Contributor Author

samuel-w commented Aug 30, 2020

The starting at field allows configuring the date and time, replicating the previous functionality of being able to set the hour interval for hourly backups and the time of day for daily backups. It adds the ability to set the day / hour for weekly backups and the day / hour for monthly backups. Removing it would create a feature regression.

@m3nu
Copy link
Contributor

m3nu commented Aug 31, 2020

I see. Now I get it. You removed the Daily option below.

Looking at the code, I think this PR is doing too many things at once with a very long and risky DB migration. The PRs you linked suggest two things:

  1. add weekly option
  2. deal with missed backups (can mean many things, like on startup, after hibernation)

I think the current form handles the hourly and daily options quite nicely. For weekly we have 2 options:

  1. Add a dropdown to the interval scheduler: Every [X] [hours, days, weeks] ... Not the best option, since hourly only sets the minute, while daily and weekly can set day of week and time of day.
  2. Add dropdown to fixed schedule: Backup [daily, weekly] at [10:15] on [Monday, Tuesday] ... Doesn't add another line and is easy to understand. But needs to hide the day of week part dynamically
  3. Add a new weekly option: Weekly on [Monday, Tuesday,..] at [10:15] ... similar to fixed daily option. Also easy to understand, but adds another line to UI.

Personally, I like option 2.

Next, the missed backup issue. I see you set a very large misfire-grace time for APScheduler. Have you verified that this will only do one missed backup and not 5, if 5 hourly backups were missed? If yes, this one setting would suffice. I wouldn't run missed backups on startup.

There is also the suggestion to prevent backups on battery power. If we can detect the battery state for both OS, this would be an interesting option too.

@samuel-w
Copy link
Contributor Author

samuel-w commented Aug 31, 2020

All the migration code is doing is converting the old time options into the new time options. It's long because it precisely converts the options, rather than resetting user scheduling. It should be safe as long as the user doesn't kill Vorta from the task manager while its upgrading the DB, which takes very fast unless someone has 100 profiles.

On the weekly thing, I don't think there is a need to add a day of the week selector, since if the weekly option is selected, it will run every n weeks on the same day of the week. Additionally, there aren't any requests to add running daily backups on certain days of the week.

The misfire_grace_time does need a coalese option added to it.

As for the battery power, this PR already does too much, I'll add the battery power in later https://github.com/samuel-w/vorta-1/tree/BatteryLimit

@samuel-w
Copy link
Contributor Author

samuel-w commented Sep 21, 2020

What other changes should I make, I already:

  • Added coalesce to the misfire time
  • Removed running missed backup on startup

For the earlier changes, see my comment above.

@mplorentz
Copy link

I would love to see this merged soon! This will fix a lot of issues I have been having with my laptop backups.

@m3nu
Copy link
Contributor

m3nu commented Nov 24, 2020

Needs more discussion. The current PR would loose backing up at a fixed time and the "starting at" isn't needed. Suggestions welcome (no code needed).

@samuel-w
Copy link
Contributor Author

samuel-w commented Nov 24, 2020

Backing up at a fixed time is still there. Just set it to something like 8pm and daily and it will run every day at 8pm. Same thing with weekly, set it to 8pm Monday and it will run every week at 8pm on Monday. Same thing with hourly, set it to every 3 hours starting at 8pm and it will run at 8pm,11pm,2am...

The super long conversion code is so that the daily and hourly settings are converted correctly.

As stated earlier, starting at is the only simple way of doing this, at least according to other backup. Scheduling is a set and forget thing, so I'd rather have marginally more complex UI and simpler code than the reverse.

@samuel-w
Copy link
Contributor Author

samuel-w commented Nov 24, 2020

We could always have four lines https://images.app.goo.gl/9ZNYWmZAyWyMdmhv9

That would remove the starting at that you don't like.

@m3nu
Copy link
Contributor

m3nu commented Nov 27, 2020

Looks very complex to be honest. Maybe we just keep it as-is, if there is no better solution. Just the migration code is massive.

@samuel-w
Copy link
Contributor Author

The migration code has to be massive, otherwise we lose data.

@samuel-w
Copy link
Contributor Author

Peek 2020-11-26 20-07

@m3nu
Copy link
Contributor

m3nu commented Nov 27, 2020

Wow. Looks very nice, but also complex.

We could remove old migration code after 6 months or so.

@samuel-w
Copy link
Contributor Author

Does the migration code really matter though? Its a run once thing.

Also, its not good to assume that everyone will update after 6 months. I don't want to break any existing installs.

@samuel-w
Copy link
Contributor Author

samuel-w commented Nov 29, 2020

So should we go with the current setup or the "starting at". I prefer the current setup just because I have sunk 3ish hours into it, and it avoids having an explicit datetime selector. It also keeps the similarity to the current UI so the change is less jarring.

@samuel-w
Copy link
Contributor Author

Hmm, should I keep the lambda function connections, or connect all boxes to the scheduler apply and read all the data there? First one is cleaner since it reads only the changed data, second one is easier to read and understand.

@samuel-w
Copy link
Contributor Author

samuel-w commented Dec 2, 2020

This removes the minutes past the hour setting for hourly backups. Going to fix that.

@samuel-w
Copy link
Contributor Author

samuel-w commented Dec 2, 2020

Done. The model updating code is less of a mess now.

@samuel-w
Copy link
Contributor Author

samuel-w commented Dec 5, 2020

So do we go with the current setup or the 'starting at' setup?

@stefanbuehler
Copy link

For me it is most important that a backup is done directly at startup when scheduled backups have been missed. I have some users that log in only occasionally. If I set the backup frequency to once per day with the present system, they may never get backed up. So I have to set it to 1h, just to catch them when they are there. To me that is the most important aspect of this thread.

@samuel-w
Copy link
Contributor Author

samuel-w commented Dec 8, 2020

OK, its back as a feature. Just waiting for it to be reviewed.
#551 (comment)

Edit: Had to remove it as multiple backups cannot run at a time.

@m3nu
Copy link
Contributor

m3nu commented Dec 9, 2020

Yeah, sorry. A bit slow since it's a bigger PR. But better take some time to do it right.

@samuel-w
Copy link
Contributor Author

samuel-w commented Dec 9, 2020

I'll try to make missed backups a notification, since its easier to do.

@m3nu m3nu removed the status:ready label Jan 18, 2021
@m3nu m3nu added this to the 0.7.3 milestone Jan 19, 2021
@m3nu m3nu removed this from the 0.7.3 (Next minor release) milestone Feb 15, 2021
@samuel-w
Copy link
Contributor Author

samuel-w commented Mar 1, 2021

Woohoo! 7 months old! 3 weeks until this PR's 8 month birthday.

@Hofer-Julian
Copy link
Collaborator

Yeah, waiting long for merges suck...

Don't forget though that @m3nu is maintaining Vorta pretty much on their own currently 😊

@samuel-w
Copy link
Contributor Author

samuel-w commented Mar 1, 2021

Yeah, I understand. Its just arguing that the datetime selector needs to be there to reduce code length, removing the datetime selector which increases code length, and then being shocked about the code length that I warned about is very discouraging.

@samuel-w samuel-w closed this Mar 2, 2021
@mplorentz
Copy link

😢

@m3nu
Copy link
Contributor

m3nu commented Mar 2, 2021

No need to close this, @samuel-w. This will be used, but more work is needed. I imagine the next scheduler iteration to:

@palto42
Copy link

palto42 commented Mar 7, 2021

A pity that this PR has been closed without implementation.
I really hope that this kind of schedule will be added anyway in not too far future.

@m3nu
Copy link
Contributor

m3nu commented Mar 7, 2021

See here for the current work on this feature, @palto42: #908

Still needs refinement, but implements all the items mentioned before.

You are welcome to test this PR and provide feedback before it’s merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add weekly option to scheduler tab Auto-apply scheduler settings and remove apply-button
8 participants