Inconsistency with indexedDB in browser extensions

[scripthunter7]

Presently, for those looking to store binary data within a browser extension, the most dependable method available is through the use of indexedDB.

However, it's important to note that using indexedDB in extensions lacks complete standardization across different browsers.

From my experimentation, I observed that indexedDB functions effectively in Chromium-based browsers indexedDB is dropped in Chrome when the incognito session is closed (in split mode), and I also encountered issues when using it in Firefox. Specifically, when Firefox users activate the Privacy & Security -> History -> Never Remember History setting, indexedDB becomes entirely non-functional in the extension.

In light of these limitations, I propose the standardization of indexedDB for browser extensions.
Such a move would ensure its reliable usability across various browsers, paralleling the consistency we see with browser.storage.local.
This standardization would significantly enhance the functionality and developer experience across different browser platforms.

Edit (14/03/2024):

[patrickkettner]

it sounds like this is a bug in how firefox handles indexeddb - I am not sure it would warrant creating an entire new api that is extension specific? Have you spoken with the mozilla team about the issue?

[scripthunter7]

We are not seeking to request a new API; rather, our concern pertains to discrepancies observed in the existing implementations. The goal is to ensure that indexedDB functions in compliance with established standards within the context of browser extensions. While I have not reported this specific issue previously for Firefox, a closely related concern was raised six years ago:
https://bugzilla.mozilla.org/show_bug.cgi?id=1427986

Through this issue, I aim to highlight and address this ongoing problem.

[patrickkettner]

Ok, but your issue says "functions effectively in Chromium-based browsers, but I encountered issues when using it in Firefox". What work is there to do in this working group? It sounds like a bug in Firefox. Maybe @Rob--W or @zombie could ask the right folks to investigate, but otherwise I am not sure what action would be taken here.

[gorhill]

[indexedDB] functions effectively in Chromium-based browsers

It does not function properly when an extension's incognito key in manifest.json is set to split -- in which case indexedDB works fine, but is not persistent as it is wholly removed when exiting incognito mode.

[patrickkettner]

@gorhill interesting! Is that different behavior from Firefox/Safari? Are there any bugs filed about it?

[maximtop]

Based on what I found on the MDN site, it looks like Firefox and Safari aren't on board with the 'split' mode for extensions:
https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/manifest.json/incognito#browser_compatibility.

What's interesting, though, is that storage.local gets shared across modes as stated here https://developer.chrome.com/docs/extensions/reference/manifest/incognito#:~:text=The%20%22split%22%20mode%20means%20that,separate%20memory%2Donly%20cookie%20store, but indexedDB doesn't play by the same rules in Chrome. And in Firefox, indexedDB just doesn’t work at all when you're incognito. Definitely this is something that should be more consistent.

[Rob--W]

[indexedDB] functions effectively in Chromium-based browsers

It does not function properly when an extension's incognito key in manifest.json is set to split -- in which case indexedDB works fine, but is not persistent as it is wholly removed when exiting incognito mode.

For context: In Chrome, in incognito split mode, there is essentially a second instance of the extension, with isolated storage (not just indexedDB, but also things like chrome.storage.session, context menu registrations, declarativeContent, etc.).

On the Firefox aspect: When Never Remember History is enabled, Firefox operates in a permanent private browsing mode. IndexedDB used to be unavailable in private browsing, but that changed in https://bugzilla.mozilla.org/show_bug.cgi?id=1639542. I haven't tested recently, but I would expect indexedDB to work (but not persist).

[birtles]

I haven't tested recently, but I would expect indexedDB to work (but not persist).

I believe it works but only if the user has enabled the add-on to run in Private Browsing mode (see https://bugzilla.mozilla.org/show_bug.cgi?id=1427986#c11).

(I believe that is why I get some reports saying that since bug 1831058 IDB works when Never Remember History is enabled—but does not persist—and others reporting it does not work.)

[scripthunter7]

We made a small summary with @maximtop, which I added to the main issue comment.

[Rob--W]

Firefox has a bug tracking IndexedDB not working in extension pages in permanent private browsing mode, at https://bugzilla.mozilla.org/show_bug.cgi?id=1893821

[rdcronin]

From the Chrome perspective, this is working as intended. Chrome is the only browser that currently supports split mode extensions.

Extensions have the choice of whether they want to run in spanning mode or split mode in incognito. In spanning mode, a single extension process is used for both the regular and the incognito profile. In split mode, the regular profile and incognito profile have separate extension processes. Split mode is a utility to allow developers to more easily isolate the data they gather in incognito so that it is erased automatically when the incognito session ends.

Any data stored in web-based storage (IndexedDB, localStorage, etc) will be wiped when the incognito profile goes away. This allows developers to store data in these storage areas for the duration of the incognito session.

If an extension doesn't want this behavior at all (and wants all data persisted from the incognito session), that extension would typically then be a spanning mode extension -- with spanning mode, since there is only a single process, associated with the regular profile, all data is kept.

If an extension wants to store some limited data from an incognito session, we provide the chrome.storage API as a way to allow the incognito extension process to persist data from the session.

This is intended behavior for Chrome, with no plans to change it in the near future.

As far as standardizing this behavior across browsers, I think this makes sense for spanning mode extensions (where the behavior is consistent across browsers, except for the "Never remember history" setting in Firefox). Since spilt mode extensions are a Chrome-only feature, that behavior cannot be aligned between browsers.

[rdcronin]

(For want of a better label, I've added "implemented: chrome" here, since I don't think there's work left to be done on our side)