electron-publish fails behind proxy server

[robertpatrick]

• Electron-Builder Version: 22.11.7

• Node Version: 14.17.3

• Electron Version: 14.0.1
• Electron Type (current, beta, nightly): current

• Electron Updater Version: 4.3.9

• Target: AppImage, NSIS, DMG/ZIP

I have forked the electron-updater-example and updated it to use current versions and include MacOS code-signing. You can find the repo at Electron Updater Example.

After publishing artifacts for MacOS and Linux (so far), the MacOS app seems to work fine from inside of our corporate network (which requires the use of a proxy server to reach the Internet).

The AppImage version of the app is failing while trying to check for updates inside our corporate network.

What am I missing? curl and firefox are able to find github.com without issue. How do I get electron-updater to use our proxy server on Linux?

[robertpatrick]

It seems clear that the mechanisms used by electron-updater clearly can work with a proxy server. The real issue is a lack of documentation on how to manually configure the autoUpdater to use a proxy.

While it appears that I can workaround this by doing `MyElectronApp.AppImage --proxy-server=my.proxy.host:80, I don't want our users to have to specify this on the command line. Is there another way to specify this information either with configuration or programmatically?

[robertpatrick]

By the way, the publish step also seems to require additional configuration to work with the proxy server on all platforms...

[mmaietta]

I'm not familiar with how the --proxy-server arg works but I did a quick dive into the code and am curious if this is what you're looking for?

electron-builder/packages/electron-updater/src/electronHttpExecutor.ts

Lines 51 to 57 in 8ebfc96

	// fix (node 7+) for making electron updater work when using AWS private buckets, check if headers contain Host property
	if (options.headers && options.headers.Host) {
	// set host value from headers.Host
	options.host = options.headers.Host
	// remove header property 'Host', if not removed causes net::ERR_INVALID_ARGUMENT exception
	delete options.headers.Host
	}

Is it possible to add a Host to your headers for electron-updater?

[robertpatrick]

Thanks, --proxy-server does allow me to get past the Checking for Updates issue on Linux.

The remaining problem is how to get electron-builder -p publishing to use the proxy server. Currently, it doesn't work on any platform from behind a proxy server. Any hints?

[mmaietta]

@robertpatrick would you be willing to provide any console logs with DEBUG=electron-builder set?
Trying to figure out where the publishing process fails. We currently handle all redirects manually, I think it's a requirement for all GitHub/s3 requests, not sure if that is being impacted by your proxy.

[robertpatrick]

@mmaietta I am willing to share this output with you directly but I am hesitant to put it on GitHub for all to see due to concerns about exposing sensitive information. If you want to send me an email to robert (dot) patrick at oracle (dot) com, I will send them to you. If you have some other way in which you would prefer me sharing this data, just let me know...

[robertpatrick]

@mmaietta OK, here is the output from my Windows machine (since it doesn't contain any sensitive info):
electron-builder-DEBUG.txt

[robertpatrick]

@mmaietta It seems clear to me that the publishing process itself is not honoring the proxy server and instead trying to push the artifacts directly to GitHub (140.82.112.5 -- lb-140-82-112-5-iad.github.com) rather than routing those requests through the proxy server. The rest of electron-builder seems to work with the proxy server so it would be great if we could figure out how to get uploading the artifacts during publishing to honor/use the proxy server.

[mmaietta]

I found this, would this help with routing? Allows you to specify a dedicated host. Perhaps you could route it internally to reach github? Might be a potential workaround, but just a shot in the dark.

electron-builder/packages/electron-publish/src/gitHubPublisher.ts

Lines 259 to 261 in 286d768

	private githubRequest<T>(path: string, token: string | null, data: { [name: string]: any } | null = null, method?: "GET" | "DELETE" | "PUT"): Promise<T> {
	// host can contains port, but node http doesn't support host as url does
	const baseUrl = parseUrl(`https://${this.info.host || "api.github.com"}`)

electron-builder/packages/builder-util-runtime/src/publishOptions.ts

Lines 96 to 100 in 286d768

	/**
	* The host (including the port if need).
	* @default github.com
	*/
	readonly host?: string | null

I'm not familiar without how to set up a debug environment for this. This is the first time I've read of a proxy not working tbh

[robertpatrick]

@mmaietta I spent some time looking at this code yesterday. It seems to me that the problem is that we need to be able to configure the http.Agent being used. For example, we use the https-proxy-agent package to make our outbound calls to the GitHub API work properly using it in conjunction with node-fetch. Unfortunately, the code where you would inject the Agent is buried under several layers and electron-builder doesn’t seem to expose any configuration to control the behavior.

I could get by with a mechanism to generate the latest-*.yml files locally and then devise my own mechanism to push the content to GitHub. Is there a reliable way to ensure the latest-*.yml file is generated by a build?

[robertpatrick]

For example, this code below is an example of one of the places electron-publish would need to change to make this work properly (assuming you wanted to get the proxy information from one of the standard environment variables). Unfortunately, there are other places that would also need to change since not all paths use githubRequest(); for example, doUploadFile().

    const HttpsProxyAgent = require('https-proxy-agent');
    ...
    githubRequest(path, token, data = null, method) {
        // host can contains port, but node http doesn't support host as url does
        const baseUrl = url_1.parse(`https://${this.info.host || "api.github.com"}`);
        const options = {
            hostname: baseUrl.hostname,
            port: baseUrl.port,
            path: this.info.host != null && this.info.host !== "github.com" ? `/api/v3${path.startsWith("/") ? path : `/${path}`}` : path,
            headers: { accept: "application/vnd.github.v3+json" },
        };
        
        const proxy = process.env.HTTPS_PROXY || process.env.HTTP_PROXY;
        if (proxy) {
            options.agent = new HttpsProxyAgent(proxy);
       }    
        return builder_util_runtime_1.parseJson(nodeHttpExecutor_1.httpExecutor.request(builder_util_runtime_1.configureRequestOptions(options, token, method), this.context.cancellationToken, data));
    }

[robertpatrick]

In looking at the electron-update package (that generally seems to work with the OS system proxy and --proxy-server argument), the main difference is that electron-update seems to use the electron.net API while electron-publish is using the Node.js https or http APIs directly (without accounting for any required proxy configuration).

electron-builder binary downloads also generally seem to work but I am still trying to find the code that does this to figure out why it works with the proxy server, if any.

[baparham]

For example, this code below is an example of one of the places electron-publish would need to change to make this work properly (assuming you wanted to get the proxy information from one of the standard environment variables). Unfortunately, there are other places that would also need to change since not all paths use githubRequest(); for example, doUploadFile().

    const HttpsProxyAgent = require('https-proxy-agent');
    ...
    githubRequest(path, token, data = null, method) {
        // host can contains port, but node http doesn't support host as url does
        const baseUrl = url_1.parse(`https://${this.info.host || "api.github.com"}`);
        const options = {
            hostname: baseUrl.hostname,
            port: baseUrl.port,
            path: this.info.host != null && this.info.host !== "github.com" ? `/api/v3${path.startsWith("/") ? path : `/${path}`}` : path,
            headers: { accept: "application/vnd.github.v3+json" },
        };
        
        const proxy = process.env.HTTPS_PROXY || process.env.HTTP_PROXY;
        if (proxy) {
            options.agent = new HttpsProxyAgent(proxy);
       }    
        return builder_util_runtime_1.parseJson(nodeHttpExecutor_1.httpExecutor.request(builder_util_runtime_1.configureRequestOptions(options, token, method), this.context.cancellationToken, data));
    }

From following through these code paths, it appears that all of these places end up using NodeHttpExecutor from builder-util at the low level to perform the request, so the fix in #6410 would cover these cases as well.

[robertpatrick]

@baparham That makes sense. I chimed in on your PR so hopefully it will be merged!

[mmaietta]

Released in 22.14.7