Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Database VANISHED!!! #247

Closed
onesnzerosimon opened this issue Apr 26, 2016 · 13 comments
Closed

Database VANISHED!!! #247

onesnzerosimon opened this issue Apr 26, 2016 · 13 comments

Comments

@onesnzerosimon
Copy link

Everything is GONE, and i'm feking DEVESTATED.
I opened Brewtarget just now, and EVERYTHING is gone.
My recipes, my hop information, my malts, my yeasts…
devestated

OSX 10.8.5
Lastest Brewtarget install.

@cgiacofei
Copy link

I just had the same thing happen this morning, database seemed to have been reset to defaults. Fortunately I had backups.

At the time it happened I mistakenly had Brewtarget open on 2 machines sharing a synced data directory. Not sure if that had anything to do with it.

Running 2.3.0 on Ubuntu.

@mikfire
Copy link
Contributor

mikfire commented Apr 26, 2016

Having two instances running against the same file is always
catastrophic. I think it has something to do with how sqlite locks the
files, how Qt responds to the lock and then how brewtarget interprets the
failure to open the database.

@onesnzersimon, I ... well crap. You may be able to recover if your
database was stored on something like a dropbox drive, maybe not. I really
wish I had something better to offer, but I am not sure how much I can do.

On Tue, Apr 26, 2016 at 2:26 PM, Chris Giacofei [email protected]
wrote:

I just had the same thing happen this morning, database seemed to have
been reset to defaults. Fortunately I had backups.

At the time it happened I mistakenly had Brewtarget open on 2 machines
sharing a synced data directory. Not sure if that had anything to do with
it.

Running 2.3.0 on Ubuntu.


You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#247 (comment)

In a world of ninja v. pirate, I pilot a Gundam

@tpiszcze
Copy link

tpiszcze commented Apr 27, 2016

My database disappeared just minutes ago. Last time I used BT was yesterday night. For sure I did not open two instances. All was good at that time. Today it started with only default entries as the fresh installation. No message was displayed.
Few times in the past it happened to me to start two BTs but there was always error displayed and I was careful to stop second instance first to not lose database.
BT is 2.1.0 running on Slackware 14.1

@recidive
Copy link

I've setup my equipment twice and added some recipes. No matter what I do, whenever I restart Brewtarget the database is always reset.

I just can't use the software and it's frustrating having hours of work lost.

It's completely broken on the Mac. And I'm not running two instances unless it's keeping it open in the background.

The software is great but not really usable.

@mikfire
Copy link
Contributor

mikfire commented May 18, 2016

I do not have a Mac, so I cannot easily test this to figure out what is
going on.

Are you getting any dialogs asking you to either save your changes, or to
restore/rollback?

Mik

On Wed, May 18, 2016 at 7:42 AM, Henrique Recidive <[email protected]

wrote:

I've setup my equipment twice and added some recipes. No matter what I do,
whenever I restart Brewtarget the database is always reset.

I just can't use the software and it's frustrating having hours of work
lost.

It's completely broken on the Mac. And I'm not running two instances
unless it's keeping it open in the background.

The software is great but not really usable.


You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#247 (comment)

In a world of ninja v. pirate, I pilot a Gundam

@onesnzerosimon
Copy link
Author

Since i lost a whole years work, not to mention the hours upon hours spent adding ingredients, I've bought Beersmith and I will never write another recipe in Brewtarget for as long as i live. I know, make more backups and all that, but damn.
I never had 2 instances going, but i recall i DID have 2 icons in launchpad for some reason.
Anyway, it's a real shame, but that was truly devastating. Once bitten...

@mikfire
Copy link
Contributor

mikfire commented May 20, 2016

You know. Just in case I haven't said it? I am honestly upset that this has
happened to you.

I think I will take the time and write some code so that we are
automatically backing things up for people. It isn't the best solution, as
the directory will grow with out bounds, but it would have helped in this
situation.

Again. I am honestly and deeply sorry this has happened to you.

Mik

On Thu, May 19, 2016 at 11:13 AM, onesnzerosimon [email protected]
wrote:

Since i lost a whole years work, not to mention the hours upon hours spent
adding ingredients, I've bought Beersmith and I will never write another
recipe in Brewtarget for as long as i live. I know, make more backups and
all that, but damn.
I never had 2 instances going, but i recall i DID have 2 icons in
launchpad for some reason.
Anyway, it's a real shame, but that was truly devastating. Once bitten...


You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#247 (comment)

In a world of ninja v. pirate, I pilot a Gundam

@xor-gate
Copy link

@mikfire you could implement backup rotation/compression so it will not go out of hand.

@mikfire
Copy link
Contributor

mikfire commented May 22, 2016

Yeah. I can't do the compression unless Qt offers something. I don't know enough about Windows and/or Mac to make sure I can find their built-in (if any) compression programs.

The design I have so far is something like this:

  1. On the database option screen, give an option for how many backups to retain and where to put them. This will give people some control over how much space we eat, and where. It will also allow people to use dropbox or google drive or a samba share or whatever to make the backups non-local. The values will default to 10 and the data directory.
  2. Every fourth clean start -- with "clean" defined as not seeing the rollback/restore dialog-- we backup the database before we open it. The backup name would be something like bt_database.isoTimeStamp, eg, bt_database.20160522.
  3. If the number of backups on disk exceeds the option, we remove the oldest backup.

I don't want to back up every clean start. I can see a case where a user has a corrupted database and opens brewtarget some number of times before realizing its gone pear shaped. If we backup every time, we stand the chance of overwriting the last good backup. It does mean the potential for lost work.

I am thinking I will allow -1, 0 and >0 as the number of backups to maintain. -1 means never remove them, 0 means never make them and >0 works as documented.

The hardest part is actually removing things. I am really considering just maintaining a list in the properties file of all the names. When the backup is made, the new name will be appended to the list. If the size of the list exceeds the number to maintain, we will start popping names off the list and removing them.

@xor-gate
Copy link

xor-gate commented May 22, 2016

I found out that QByteArray class has a zlib compression native builtin (cross-platform).
http://doc.qt.io/qt-5/qbytearray.html#qCompress with a example for zip/unzip http://www.antonioborondo.com/2014/10/22/zipping-and-unzipping-files-with-qt/

@mikfire
Copy link
Contributor

mikfire commented May 22, 2016

I don't have the plumbing in place, but I am thinking the option dialog screen gets rearranged. The directories tab goes away (which is something I wanted to do anyway), and you get this on the database tab if you have sqlite configured
SQLITE option

and this if you have postgres configured
PGSQL option

I am still working on the balance of the sqlite options, but that's the idea.

mikfire pushed a commit to mikfire/brewtarget that referenced this issue May 23, 2016
This doesn't fix the core problem. That comes next, because I think I can.
This implements automatic, under the cover backups.

The OptionDialog has been significantly re-arranged. The user can now select
how often (aka, frequency) to backup, and how many old copies to keep.
Frequency is defined by the number of times a user closes brewtarget and accepts
the changes. Frequency defaults to 4, the number defaults to 10. A frequency
of 1 means always backup on close. If the number to keep is -1, it means never
delete; 0 means never backup; >0 means that number of backups

When brewtarget is closed, we get a counter from the properties file. If that
counter % frequency is not 0, we simply store the updated counter and exit.

If the number to save is 0, we exit.

If we get this far, we pull the backup directory name and the list of existing
files from the properties file.

If counter % frequency is 0, we generate a filename like bt_database.yyyyMMdd.
We then check to see if that file exists in the backup directory. If it does,
we go into a loop appending 0001, 0002 ... until we get a filename that
doesn't exist. If we get to 10000, we give up and just use
bt_database.yyyyMMdd.

Once we have the name, we call backupToDir() which already existed. That just
copies the file for us. We then append the new name to the list.

The final step is to clean up the old copies if we need to. While the size of
the list is greater than the number of copies to keep, we pop the first (aka,
oldest) file name off the list, and delete it. Using a while loop allows the
user to adjust the number of preserved copies down and brewtarget will clean
up after itself.

Finally, we store the new list of files names back into the properties file
and exit.
@mikfire
Copy link
Contributor

mikfire commented May 23, 2016

Something like pull request #260. No compression, but I made things easy to configure so you can put the backups anywhere.

I still need to work on the dialog. It looks like crap, but it will take me some time to figure out what to do that doesn't look like crap.

mikfire pushed a commit to mikfire/brewtarget that referenced this issue May 24, 2016
This is the other half of the solution. I *think* I've found all the places
where we over write the databases when two run at the same time. I've found a
few edge cases I don't think we knew about -- like what happens when: there is
no temp backup, but the user discards changes. Hint: bye-bye database.

I've tried to protect every place I can find. No temp backup is created if we
cannot open the database. Database::load() fails if we cannot open the
database and run the pragmas. If the load fails, brewtarget bails.

I think I need to merge this with the other half, and then set about removing
the temporary backups. I like the idea, but there's just no good way to
implement it.
@rocketman768 rocketman768 added this to the v2.4.0 milestone May 29, 2016
@mikfire mikfire closed this as completed in ef633e3 Jun 3, 2016
@patrick-brian-mooney
Copy link

Me too. Entire database is just gone. Everything. Dozens of recipes, and all the yeast and other ingredient information I'd entered.

It just silently disappeared at some point between when I last used Brewtarget, maybe six weeks ago, and when I opened it again just now.

Everything's gone.

Linux Mint 20 on x64 with Brewtarget 2.3.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants