Support encodings other than UTF-8 (and binary)

[mlynch]

For filesystem API

[mlynch]

Lowering the priority on this now. I realized that even Cordova only supports UTF-8. Any other encoding is probably quite rare these days.

[geferon]

Is there any way we can get this to store binary files that are obtained by HTTP as blob objects?
I currently need this to download files on the client.

[sttz]

Just tried to write a binary file and the current documentation is misleading:
// The encoding to write the file in (defautls to utf8)

This should be the same as in FileReadOptions, where it mentions that setting no encoding expects the string to be base64 encoded.

Also, passing an invalid base64 string to writeFile on iOS fails silently. I assume the ? in Data(base64Encoded: data)? is hiding the error.

@geferon You can use the FileReader class of the standard JavaScript File API to read the contents of a Blob.

[geferon]

But I don't want to read, I want to WRITE binary files into the filesystem with obtained blobs from an HTTP request.

[diachedelic]

I have a similar requirement to @geferon except that I need to download fairly large videos to the filesystem. I feel uncomfortable about passing these around as strings. Despite the deprecation notice on cordova-plugin-file-transfer, it looks like it will do the job.

For storing arbitrary Blobs on the filesystem, looks like cordova-plugin-file might be the way to go.

[geferon]

@diachedelic That's exactly what I'm doing right now but it'd be nice if we had native Capacitor support instead of having to use a Cordova plugin

[sta55en]

So am I correct in saying that Capacitor cannot write binary files downloaded via HTTP to the filesystem?

[sttz]

Here's the code I'm using to write a binary blob to the file system:

readAsBinaryString(file: Blob): Promise<string> {
    return new Promise((resolve, reject) => {
        var reader = new FileReader();
        reader.onloadend = (event) => {
            resolve(reader.result as string);
        };
        reader.onerror = (event) => {
            reject(reader.error);
        };
        reader.readAsBinaryString(file);
    });
}

async save(blob: Blob, filename: string): void {
    try {
        let binaryString = await this.readAsBinaryString(blob);
        let base64String = btoa(binaryString);
        await Filesystem.writeFile({
            data: base64String,
            directory: FilesystemDirectory.Cache,
            path: filename
        });
    } catch (error) {
        console.error(error);
    }
}

[sta55en]

Thanks @sttz!

[jeremyaurelius]

It's good that there's workaround using base64, but I'm wondering what's the memory impact of converting to a data string? I remember that passing with base64 strings to Cordova often resulted in crashes.

[diachedelic]

@jairemix it has caused crashes for me with files several megabytes or larger, which is why we write the file incrementally (see https://gist.github.com/diachedelic/9aa3790e045e37327bffdcfaadd29253).

It is a shame that there does not seem to be any way that native code can directly read binary data from inside the webview - all data crossing the "bridge" must be strings.

[AshleyScirra]

For our game engine Construct (which has Cordova-based mobile publishing), it is important to have the capability to read binary files, so it would be good if the Capacitor APIs could be updated to cover that.

FWIW, cordova-plugin-file has some really terrible pitfalls. Significantly, it replaces the global File constructor with its own non-standard one with parameters in a different order, which is really a big headache for trying to write standardised code. Also its file APIs return these fake File objects that aren't real ones, so they don't work with other web APIs that accept File or Blob (e.g. createImageBitmap, Web Share files, etc.) - they only work with a small subset of magic APIs that they also polyfilled to handle their fake File object. Basically it's a big mess and needs replacing. Capacitor has a great opportunity to fix this. A method to read a file and get a real File object back (or a Blob would do) would be great.

[sta55en]

How is this a low priority?

[jcesarmobile]

Going to close since this is going off-topic and and most requests have nothing to do with the title. I think utf-8 and binary are enough and the cordova plugin only offers that.

As it has been mentioned, everything that goes through the bridge has to be a string, so it's not possible to pass a Blob or File.
There are some good comments here explaining how to do it with FileReader or other approaches to write a Blob.
What cordova file plugin does is to overwrite the File and FileReader objects as explained by AshleyScirra, which causes those mentioned issues, and also other different ones if using things like zone.js, so we won't be doing it ourselves.

So, if you need to download very big files, better use a native plugin that does the download and write by just passing the url string to it, that way there is no bridge limitation.

If it's not that big, use the FileReader or another of the proposed solutions.

For reading files, apart from Filesystem plugin, you can also use Capacitor.convertFileSrc("file/url/here"); to convert a file url to a url the WebView understands, this way you can use XHR/fetch on it and get the Blob, or set it as src/href on html elements.

[AshleyScirra]

Our use case is we need to read a binary file from the app package, e.g. a PNG image.

Currently the only option is to use cordova-plugin-file which does appalling things to compatibility, as I described, and we're desperate to get rid of it.

I don't see any other workarounds. We can't do something like convert to base64, because the image file is already in a binary format and needs to be read as binary.

There's a great opportunity for Capacitor to fix a huge pain point of Cordova here. Please reconsider! For example it could provide a method to read a binary file, internally do any base64 conversion to get it across a text bridge, and then return a File object to JS.

[jcesarmobile]

As I said, that's a separate issue from the one reported here, the issue went off-topic and you all reported different things.

And, as I said, using fetch/xhr with a converted url to the file path (if it's in the public folder you don't even need to convert, just use the relative url), you can get the image Blob with no plugins involved.

[jcesarmobile]

There is an issue about the blob/large files read/write #984

[AshleyScirra]

Fetch doesn't work on the file: protocol. Browser vendors are basically trying to phase it out.

[jcesarmobile]

That’s why you have to use Capacitor.convertFileSrc on the file url, and/or use an absolute path if the file is in public folder.

[seanharr11]

@jcesarmobile this running the Capacitor.convertFileSrc(image.path) yields this URI: capacitor://localhost/_capacitor_file_/private/var/mobile/Containers/Data/Application/0B36A6F4-4C63-4AD5-B3E1-765C7CDCA105/tmp/photo-1.jpg ... running fetch() on this URI yields an empty object. Any idea how this has changed since your comment above?

[diachedelic]

@seanharr11 possibly related to #2882? that usage works fine for me.

[ionitron-bot]

Thanks for the issue! This issue is being locked to prevent comments that are not relevant to the original issue. If this is still an issue with the latest version of Capacitor, please create a new issue and ensure the template is fully filled out.