Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: fetch (POC) #618

Draft
wants to merge 44 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
58c7dc2
feat!: Upgrade to `node-fetch` v3
d-goog Apr 16, 2024
f8570d9
Merge branch 'main' into upgrade-node-fetch
danielbankhead Apr 16, 2024
7fd2a95
refactor: Use native `FormData` in testing
d-goog Apr 17, 2024
0cc7856
Merge branch 'upgrade-node-fetch' of github.com:googleapis/gaxios int…
d-goog Apr 17, 2024
23e7626
feat: `fetch` (POC)
d-goog Apr 19, 2024
a61cfa5
feat!: Improve spec compliance
d-goog Apr 19, 2024
35a73b1
test(temp): Run 18+
d-goog Apr 19, 2024
e7addeb
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Apr 19, 2024
9816229
feat: Improve types, streamline, and standardize
d-goog May 14, 2024
7315442
feat!: Do Not Treat Buffers as JSON By Default
d-goog May 15, 2024
93bf4a8
Merge branch 'main' of github.com:googleapis/gaxios into upgrade-node…
d-goog May 15, 2024
273e516
Merge branch 'upgrade-node-fetch' of github.com:googleapis/gaxios int…
d-goog May 15, 2024
cc2a68b
Merge branch 'upgrade-node-fetch' of github.com:googleapis/gaxios int…
d-goog May 15, 2024
9164b87
test: Adopt `Headers` type from merge
d-goog May 15, 2024
2e389c8
Merge branch 'upgrade-node-fetch' of github.com:googleapis/gaxios int…
d-goog May 15, 2024
d01ecde
Merge branch 'upgrade-node-fetch' of github.com:googleapis/gaxios int…
d-goog May 15, 2024
84faea8
feat: Introduce `GaxiosOptionsPrepared` for improved type guarantees
d-goog May 16, 2024
4e6de5f
feat: Ensure `GaxiosOptionsPrepared.url` is always a `URL`
d-goog May 16, 2024
e6c2371
refactor: Clean-up Types & Resolve lint warnings
d-goog May 16, 2024
2089d83
Merge branch 'upgrade-node-fetch' of github.com:googleapis/gaxios int…
d-goog May 16, 2024
a8b2a3d
Merge branch 'upgrade-node-fetch' of github.com:googleapis/gaxios int…
d-goog May 16, 2024
01ac8c7
chore: merge conflicts
d-goog May 16, 2024
10d557a
feat: Support `request(URL)`
d-goog May 16, 2024
7138ccb
chore: more merge resolutions
d-goog May 17, 2024
98f7009
refactor: streamline `.data` for `Response`
d-goog May 20, 2024
e66c4ba
Merge branch 'upgrade-node-fetch' of github.com:googleapis/gaxios int…
d-goog May 20, 2024
fe2133e
Merge branch 'upgrade-node-fetch' of github.com:googleapis/gaxios int…
d-goog May 20, 2024
4894e39
docs: Simplify example
d-goog May 22, 2024
3b5ad04
Merge branch 'upgrade-node-fetch' of github.com:googleapis/gaxios int…
d-goog May 22, 2024
9a070ef
chore: Remove `node-fetch` patch
d-goog May 22, 2024
b030c81
chore: Update summary
d-goog May 22, 2024
a7e2e83
refactor: streamline, no error case
d-goog May 22, 2024
0baab44
Merge branch 'main' of github.com:googleapis/gaxios into do-not-treat…
d-goog Jul 1, 2024
b2ec395
docs: clarification
d-goog Jul 1, 2024
8413f68
feat: Basic `GET` Support for `Request` objects
d-goog Sep 11, 2024
da7ba40
Merge branch 'upgrade-node-fetch' into native-fetch
d-goog Sep 11, 2024
e3220b9
chore: Improve `Request` Support
d-goog Sep 11, 2024
04e264d
test: remove `.only`
d-goog Sep 11, 2024
d163a90
refactor: simplify `httpMethodsToRetry`
d-goog Sep 11, 2024
705fad3
chore: update `nock`
d-goog Sep 11, 2024
9e52755
test: cleanup
d-goog Sep 11, 2024
e5baa94
Merge branch 'upgrade-node-fetch' into do-not-treat-buffer-as-json
d-goog Sep 11, 2024
ec5e4e0
Merge branch 'do-not-treat-buffer-as-json' into native-fetch
d-goog Sep 11, 2024
50321d6
docs: Document `Request` and `RequestInit`
d-goog Sep 11, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
123 changes: 39 additions & 84 deletions src/common.ts
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ export class GaxiosError<T = any> extends Error {

constructor(
message: string,
public config: GaxiosOptions,
public config: GaxiosOptionsPrepared,
public response?: GaxiosResponse<T>,
public error?: Error | NodeJS.ErrnoException
) {
Expand Down Expand Up @@ -111,45 +111,43 @@ export class GaxiosError<T = any> extends Error {
}

if (config.errorRedactor) {
config.errorRedactor<T>({
config.errorRedactor({
config: this.config,
response: this.response,
});
}
}
}

/**
* @deprecated use native {@link globalThis.Headers}.
*/
export interface Headers {
[index: string]: any;
}
export type GaxiosPromise<T = any> = Promise<GaxiosResponse<T>>;
type GaxiosResponseData =
| ReturnType<JSON['parse']>
| GaxiosOptionsPrepared['data'];

export type GaxiosPromise<T = GaxiosResponseData> = Promise<GaxiosResponse<T>>;

export interface GaxiosResponse<T = any> extends Response {
config: GaxiosOptions;
export interface GaxiosResponse<T = GaxiosResponseData> extends Response {
config: GaxiosOptionsPrepared;
data: T;
}

export interface GaxiosMultipartOptions {
headers: Headers;
headers: HeadersInit;
content: string | Readable;
}

/**
* Request options that are used to form the request.
*/
export interface GaxiosOptions extends Omit<RequestInit, 'headers'> {
export interface GaxiosOptions extends RequestInit {
/**
* Optional method to override making the actual HTTP request. Useful
* for writing tests.
*
* @deprecated Use {@link GaxiosOptions.fetchImplementation} instead.
*/
adapter?: <T = any>(
options: GaxiosOptions,
defaultAdapter: (options: GaxiosOptions) => GaxiosPromise<T>
adapter?: <T = GaxiosResponseData>(
options: GaxiosOptionsPrepared,
defaultAdapter: (options: GaxiosOptionsPrepared) => GaxiosPromise<T>
) => GaxiosPromise<T>;
url?: string | URL;
/**
Expand All @@ -167,16 +165,6 @@ export interface GaxiosOptions extends Omit<RequestInit, 'headers'> {
| 'OPTIONS'
| 'TRACE'
| 'PATCH';
/**
* Recommended: Provide a native {@link globalThis.Headers Headers} object.
*
* @privateRemarks
*
* This type does not have the native {@link globalThis.Headers Headers} in
* its signature as it would break customers looking to modify headers before
* providing to this library (new, unnecessary type checks/guards).
*/
headers?: Headers;
/**
* The data to send in the {@link RequestInit.body} of the request. Objects will be
* serialized as JSON, except for:
Expand Down Expand Up @@ -232,7 +220,7 @@ export interface GaxiosOptions extends Omit<RequestInit, 'headers'> {
* This is passed to {@link RequestInit.body}.
*/
multipart?: GaxiosMultipartOptions[];
params?: any;
params?: GaxiosResponseData;
/**
* @deprecated Use {@link URLSearchParams} instead and pass this directly to {@link GaxiosOptions.data `data`}.
*/
Expand All @@ -241,7 +229,7 @@ export interface GaxiosOptions extends Omit<RequestInit, 'headers'> {
/**
* @deprecated ignored
*/
onUploadProgress?: (progressEvent: any) => void;
onUploadProgress?: (progressEvent: GaxiosResponseData) => void;
/**
* If the `fetchImplementation` is native `fetch`, the
* stream is a `ReadableStream`, otherwise `readable.Stream`
Expand Down Expand Up @@ -324,24 +312,11 @@ export interface GaxiosOptions extends Omit<RequestInit, 'headers'> {
*/
errorRedactor?: typeof defaultErrorRedactor | false;
}
/**
* A partial object of `GaxiosOptions` with only redactable keys
*
* @experimental
*/
export type RedactableGaxiosOptions = Pick<
GaxiosOptions,
'body' | 'data' | 'headers' | 'url'
>;
/**
* A partial object of `GaxiosResponse` with only redactable keys
*
* @experimental
*/
export type RedactableGaxiosResponse<T = any> = Pick<
GaxiosResponse<T>,
'config' | 'data' | 'headers'
>;

export interface GaxiosOptionsPrepared extends GaxiosOptions {
headers: globalThis.Headers;
url: URL;
}

/**
* Configuration for the Gaxios `request` method.
Expand Down Expand Up @@ -397,7 +372,10 @@ export interface RetryConfig {
retryBackoff?: (err: GaxiosError, defaultBackoffMs: number) => Promise<void>;
}

function translateData(responseType: string | undefined, data: any) {
function translateData(
responseType: string | undefined,
data: GaxiosResponseData
) {
switch (responseType) {
case 'stream':
return data;
Expand All @@ -420,51 +398,30 @@ function translateData(responseType: string | undefined, data: any) {
*
* @experimental
*/
export function defaultErrorRedactor<T = any>(data: {
config?: RedactableGaxiosOptions;
response?: RedactableGaxiosResponse<T>;
}) {
export function defaultErrorRedactor<
O extends GaxiosOptionsPrepared,
R extends GaxiosResponse<GaxiosResponseData>,
>(data: {config?: O; response?: R}) {
const REDACT =
'<<REDACTED> - See `errorRedactor` option in `gaxios` for configuration>.';

function redactHeaders(headers?: Headers | globalThis.Headers) {
function redactHeaders(headers?: Headers) {
if (!headers) return;

function check(key: string) {
headers.forEach((_, key) => {
// any casing of `Authentication`
// any casing of `Authorization`
// anything containing secret, such as 'client secret'
return (
if (
/^authentication$/i.test(key) ||
/^authorization$/i.test(key) ||
/secret/i.test(key)
);
}

function redactHeadersObject(headers: Headers) {
for (const key of Object.keys(headers)) {
if (check(key)) headers[key] = REDACT;
}
}

function redactHeadersHeaders(headers: globalThis.Headers) {
headers.forEach((value, key) => {
if (check(key)) headers.set(key, REDACT);
});
}

// support `node-fetch` Headers and other third-parties
if (headers instanceof Headers || 'set' in headers) {
redactHeadersHeaders(headers as globalThis.Headers);
} else {
redactHeadersObject(headers);
}
)
headers.set(key, REDACT);
});
}

function redactString<T extends GaxiosOptions | RedactableGaxiosResponse>(
obj: T,
key: keyof T
) {
function redactString<T extends O | R>(obj: T, key: keyof T) {
if (
typeof obj === 'object' &&
obj !== null &&
Expand All @@ -482,9 +439,7 @@ export function defaultErrorRedactor<T = any>(data: {
}
}

function redactObject<T extends GaxiosOptions['data'] | GaxiosResponse>(
obj: T | null
) {
function redactObject<T extends O['data'] | R>(obj: T | null) {
if (!obj) {
return;
} else if (
Expand Down Expand Up @@ -523,7 +478,7 @@ export function defaultErrorRedactor<T = any>(data: {
redactObject(data.config.body);

try {
const url = new URL('', data.config.url);
const url = data.config.url;

if (url.searchParams.has('token')) {
url.searchParams.set('token', REDACT);
Expand All @@ -533,7 +488,7 @@ export function defaultErrorRedactor<T = any>(data: {
url.searchParams.set('client_secret', REDACT);
}

data.config.url = url.toString();
data.config.url = url;
} catch {
// ignore error - no need to parse an invalid URL
}
Expand Down
56 changes: 27 additions & 29 deletions src/gaxios.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ import {
GaxiosMultipartOptions,
GaxiosError,
GaxiosOptions,
GaxiosOptionsPrepared,
GaxiosPromise,
GaxiosResponse,
Headers,
defaultErrorRedactor,
} from './common';
import {getRetryConfig} from './retry';
Expand All @@ -50,7 +50,7 @@ export class Gaxios {
* Interceptors
*/
interceptors: {
request: GaxiosInterceptorManager<GaxiosOptions>;
request: GaxiosInterceptorManager<GaxiosOptionsPrepared>;
response: GaxiosInterceptorManager<GaxiosResponse>;
};

Expand All @@ -77,7 +77,7 @@ export class Gaxios {
}

private async _defaultAdapter<T>(
config: GaxiosOptions
config: GaxiosOptionsPrepared
): Promise<GaxiosResponse<T>> {
const fetchImpl =
config.fetchImplementation ||
Expand All @@ -89,7 +89,7 @@ export class Gaxios {
const preparedOpts = {...config};
delete preparedOpts.data;

const res = (await fetchImpl(config.url!, preparedOpts as {})) as Response;
const res = (await fetchImpl(config.url, preparedOpts as {})) as Response;
let data = await this.getResponseData(config, res);

// `node-fetch`'s data isn't writable. Native `fetch`'s is.
Expand Down Expand Up @@ -123,7 +123,7 @@ export class Gaxios {
* @param opts Set of HTTP options that will be used for this HTTP request.
*/
protected async _request<T = any>(
opts: GaxiosOptions = {}
opts: GaxiosOptionsPrepared
): GaxiosPromise<T> {
try {
let translatedResponse: GaxiosResponse<T>;
Expand Down Expand Up @@ -175,7 +175,7 @@ export class Gaxios {
}

private async getResponseData(
opts: GaxiosOptions,
opts: GaxiosOptionsPrepared,
res: Response
): Promise<any> {
if (
Expand Down Expand Up @@ -209,7 +209,7 @@ export class Gaxios {

#urlMayUseProxy(
url: string | URL,
noProxy: GaxiosOptions['noProxy'] = []
noProxy: GaxiosOptionsPrepared['noProxy'] = []
): boolean {
const candidate = new URL(url);
const noProxyList = [...noProxy];
Expand Down Expand Up @@ -257,21 +257,21 @@ export class Gaxios {
* Applies the request interceptors. The request interceptors are applied after the
* call to prepareRequest is completed.
*
* @param {GaxiosOptions} options The current set of options.
* @param {GaxiosOptionsPrepared} options The current set of options.
*
* @returns {Promise<GaxiosOptions>} Promise that resolves to the set of options or response after interceptors are applied.
* @returns {Promise<GaxiosOptionsPrepared>} Promise that resolves to the set of options or response after interceptors are applied.
*/
async #applyRequestInterceptors(
options: GaxiosOptions
): Promise<GaxiosOptions> {
options: GaxiosOptionsPrepared
): Promise<GaxiosOptionsPrepared> {
let promiseChain = Promise.resolve(options);

for (const interceptor of this.interceptors.request.values()) {
if (interceptor) {
promiseChain = promiseChain.then(
interceptor.resolved,
interceptor.rejected
) as Promise<GaxiosOptions>;
) as Promise<GaxiosOptionsPrepared>;
}
}

Expand All @@ -282,9 +282,9 @@ export class Gaxios {
* Applies the response interceptors. The response interceptors are applied after the
* call to request is made.
*
* @param {GaxiosOptions} options The current set of options.
* @param {GaxiosOptionsPrepared} options The current set of options.
*
* @returns {Promise<GaxiosOptions>} Promise that resolves to the set of options or response after interceptors are applied.
* @returns {Promise<GaxiosOptionsPrepared>} Promise that resolves to the set of options or response after interceptors are applied.
*/
async #applyResponseInterceptors(
response: GaxiosResponse | Promise<GaxiosResponse>
Expand All @@ -309,8 +309,10 @@ export class Gaxios {
* @param options The original options passed from the client.
* @returns Prepared options, ready to make a request
*/
async #prepareRequest(options: GaxiosOptions): Promise<GaxiosOptions> {
const opts: GaxiosOptions = extend(true, {}, this.defaults, options);
async #prepareRequest(
options: GaxiosOptions
): Promise<GaxiosOptionsPrepared> {
const opts = extend(true, {}, this.defaults, options);
if (!opts.url) {
throw new Error('URL is required.');
}
Expand Down Expand Up @@ -465,18 +467,10 @@ export class Gaxios {
(opts as {duplex: string}).duplex = 'half';
}

// preserve the original type for auditing later
if (opts.headers instanceof Headers) {
opts.headers = preparedHeaders;
} else {
const headers: Headers = {};
preparedHeaders.forEach((value, key) => {
headers[key] = value;
});
opts.headers = headers;
}

return opts;
return Object.assign(opts, {
headers: preparedHeaders,
url: opts.url instanceof URL ? opts.url : new URL(opts.url),
});
}

/**
Expand Down Expand Up @@ -531,8 +525,12 @@ export class Gaxios {
) {
const finale = `--${boundary}--`;
for (const currentPart of multipartOptions) {
const headers =
currentPart.headers instanceof Headers
? currentPart.headers
: new Headers(currentPart.headers);
const partContentType =
currentPart.headers['Content-Type'] || 'application/octet-stream';
headers.get('Content-Type') || 'application/octet-stream';
const preamble = `--${boundary}\r\nContent-Type: ${partContentType}\r\n\r\n`;
yield preamble;
if (typeof currentPart.content === 'string') {
Expand Down
2 changes: 1 addition & 1 deletion src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ export {
GaxiosError,
GaxiosPromise,
GaxiosResponse,
Headers,
GaxiosOptionsPrepared,
RetryConfig,
} from './common';
export {Gaxios, GaxiosOptions};
Expand Down
Loading
Loading