Cannot use kanji dictionary when permanent private browsing is enabled

[birtles]

Since releasing 0.1.2 and 0.1.3 which use IDB to store the kanji data, there have been a large number of OpenFailedErrors reported.

This is an error reported from Dexie but the inner error is InvalidStateError: A mutation operation was attempted on a database that did not allow mutations which is coming from Firefox.

Digging into the Firefox source for NS_ERROR_DOM_INDEXEDDB_NOT_ALLOWED_ERR there seem to be three main causes:

1. The user has disabled IDB using the dom.indexedDB.enabled pref
2. Permission for using IDB is denied
   • This happens in private windows but we are using IDB from the background script only, so that shouldn't be the case here. It's more likely the add-on has been denied storage access somehow.
3. Mis-use of IDB somehow (e.g. trying to use a cursor on a deleted database, trying to create an object store outside of a version change event).

I'm not sure which it is, but I notice that the dom.indexedDB.enabled pref was removed in Firefox 72 (bug 1488583), one reason being that it breaks extensions, and even the browser itself in some cases. I also note that I've yet to see a single case of this error on Firefox 72 (current Nightly).

So it's possible this will be fixed as 72 rolls out. If not, we will need to add more logging to see which of the above cases we're hitting.

[birtles]

I also note that I've yet to see a single case of this error on Firefox 72 (current Nightly).

For reference, at this point only 1.41% of users are on 72 (compared to 86.1% on 70, and 2.85% on 71). I have seen a few reports on 71, however which only has double the number of the users so 🤞.

[birtles]

I also notice that turning off dom.indexedDB.enabled breaks Youtube and Gmail (https://github.com/allo-/firefox-profilemaker/issues/130) and uBlock (https://gist.github.com/tioxy/b83342e32c018ff24dd3e0899b09f0dd#file-syspref-js-L143).

[birtles]

I'm not sure it's actually dom.indexedDB.enabled per se--if I disable that pref I get different errors and the whole browser basically breaks down. But I've still yet to see one single error of this type with Firefox 72 so I suspect whatever removed that pref also fixed this.

It's possible it's unrelated and is something else in version 72 fixed this, or there's some configuration that differs by virtue of 72 being Nightly. We'll see in the next few days as 72 goes to beta.

[birtles]

Now that Firefox 72 has gone to beta, I have started seeing these issues there as well. So there appears to be something different with Nightly and this problem won't magically fix itself 🙁.

There are still all sorts of unusual database errors being reported that I am having difficulty debugging since I can't reproduce them locally. At this point it might be simplest to remove the dependency on Dexie and use idb instead.

Doing so would simplify debugging and eliminate at least one known error where Dexie's initialization code calls a crypto method in order to get a native promise but that method sometimes throws when called from the extension process (seems like NSS has not been initialized, specifically this line).

[birtles]

I've released a version using idb instead of Dexie but still I see the same sort of errors and it has me absolutely stumped. I've dug through various Bugzilla issues and I can see that add-on background pages should typically have access to IndexedDB.

One comment suggested that they might still not have it in "permanent private browsing mode" (i.e. when you set Firefox to "Never remember history"). But when I try to install the add-on with that setting enabled, it doesn't even run. So I'm not sure if that's it.

If anyone notices the add-on failing in odd ways, I would love to hear about it, especially if you have any particular privacy settings enabled.

[birtles]

One comment suggested that they might still not have it in "permanent private browsing mode" (i.e. when you set Firefox to "Never remember history"). But when I try to install the add-on with that setting enabled, it doesn't even run. So I'm not sure if that's it.

I retried this today and the add-on installs fine in permanent private browsing mode and indeed it cannot use IndexedDB. This appears to be Mozilla bug 1427986.

At this stage the options for dealing with this appear to be either:

1. Re-introduce the flat-file dictionary file and code for looking it up and fall back to that when IndexedDB is not available, or
2. Wait for Mozilla bug 1427986 (or Mozilla bug 781982) to be fixed.

(1) is quite a bit of work and ongoing complexity, and would not allow, e.g. multi-lingual support, but I'm not sure how far off (2) is. I wouldn't want to start using IndexedDB for the word dictionary without either (1) or (2) happenning.

[datalowe]

Hi,

I use permanent private browsing and I believe I am experiencing a problem related to this issue.

System
Firefox 96.0.2 running on Mac OS Big Sur 11.5.2. (with 'always use private browsing' enabled)

Problem
Can't toggle from word to kanji or names dictionary.

Observed behavior
I hover over a name, say 松島. The word dictionary pops up and shows a definition for 松. If I try hitting the 'change dictionary' hotkey I've configured in the 10ten settings, nothing happens. The word dictionary pop-up remains the same, and I don't see any output in the console log. I've tried changing the 'change dictionary' hotkey (e.g. using 'n' rather than 'Enter'), but this doesn't appear to make any difference.

Configurations page error message
At the bottom of the 10ten configurations page I'm informed that 'Database storage utilization not possible'. If I press the button for 'try again' (presumably 'try accessing database storage again'), nothing appears to happen.

Observations when turning off 'always use private browsing'
If I toggle 'always use private browsing' and restart Firefox, I can toggle as expected between the words/kanji/names dictionaries. This includes in private browsing windows. If I visit 10ten's configurations page, I see a different error message, about failing to update one of the dictionaries as far as I understand:

Maybe this post is redundant as the root cause is a long-standing Firefox bug IIUC, but at least it might help make the issue more concrete for others who bump into this bug.

(sidenote: Thanks, to you and all other Rikai-devs, for all the great work! Rikai-x has helped me a lot through the years.)

[birtles]

Hi! Thank you very much for a very detailed and helpful bug description.

I'm afraid my reply will be brief since I'm about to head to sleep but I just wanted to confirm that this is indeed fundamentally a limitation in Firefox (bug 1427986 although there has been some traction on bug 1639542 over the last year which too would presumably also fix this bug).

Currently, 10ten ships with a snapshot of the database that includes only the word dictionary and English glosses. It downloads the updated data files including the name and kanji dictionaries in the background and stores them in IndexedDB. As a result, when IndexedDB is not available (as in private browsing mode) it cannot update the database nor download the kanji/names dictionaries.

That said, the other error you received about the version file not being accessible is a concern. Can you reproduce that consistently? I think some people have encountered that due to particular add-ons / firewall settings (specifically settings that block connections to cloudfront).

Also, getting back to the bug in question, I wonder if we can detect always-on private browsing mode and produce a more helpful error in that case.