Skip to content

Commit

Permalink
usage reporting: add sendTraces, rename sendErrorsInTraces, etc
Browse files Browse the repository at this point in the history 
- We've decided to leave the usage reporting defaults in AS4 as "field
  level instrumentation on for all operations, send some operations as
  traces". But we'd like to at least provide a non-experimental
  mechanism for entirely disabling sending traces to Studio. So there's
  a new ApolloServerPluginUsageReporting: `sendTraces: false`.
- Rename the (new in AS4) usage reporting option `sendErrorsInTraces` to
  `sendErrors`, because it does also affect error statistics in stats
  reports if your `transform` function returns `null`.
- Remove Apollo-internal `internal_includeTracesContributingToStats`
  option. This enabled some internal consistency monitoring which we are
  no longer paying attention to.
- If you enable `debugPrintReports`, send the reports as `info` rather
  than `warn`, which primarily lets us delete some large comments (and
  makes sense because these debug messages are not warnings).

Fixes #6051. Fixes #6078.
  • Loading branch information
@glasser
glasser committed Aug 26, 2022
1 parent 24a9885 commit 6cd6f7c
Show file tree
Hide file tree
Showing 6 changed files with 105 additions and 89 deletions.
20 changes: 11 additions & 9 deletions docs/source/data/errors.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -438,7 +438,7 @@ You can use Apollo Studio to analyze your server's error rates. By default, the

If you _do_ want error information sent to Studio, you can send every error, or you can modify or redact specific errors before they're transmitted.

To send all errors to Studio you can pass `{ unmodified: true }` to `sendErrorsInTraces`, like so:
To send all errors to Studio you can pass `{ unmodified: true }` to `sendErrors`, like so:

```ts {7}
new ApolloServer({
Expand All @@ -447,23 +447,23 @@ new ApolloServer({
ApolloServerPluginUsageReporting({
// If you pass unmodified: true to the usage reporting
// plugin, Apollo Studio receives ALL error details
sendErrorsInTraces: { unmodified: true },
sendErrors: { unmodified: true },
}),
],
});
```

If you want to report specific errors or modify an error before reporting it, you can pass a function to the `sendErrorsInTraces.transform` option, like so:
If you want to report specific errors or modify an error before reporting it, you can pass a function to the `sendErrors.transform` option, like so:

```ts {4-6}
new ApolloServer({
// etc.
plugins: [
ApolloServerPluginUsageReporting({
sendErrorsInTraces: {
sendErrors: {
transform: (err) => {
if (err.extensions.code === 'MY_CUSTOM_CODE') {
// returning null will skip reporting this error
// returning null will skip reporting this error
return null;
}

Expand All @@ -483,6 +483,8 @@ The function you pass to `transform` is called for each error (`GraphQLError`) t
- Return a modified form of the error (e.g., by changing the `err.message` to remove potentially sensitive information)
- Return `null` to prevent the error from being reported entirely

Note that returning `null`` also affects Studio's aggregated statistics about how many operations contain errors and at what paths those errors appear.

[As mentioned above](#for-client-responses), you can use the `unwrapResolverError` (from `@apollo/server/errors`) to remove the `GraphQLError` wrapping an original error.

<!-- TODO(AS4) Replace link once plugin docs are updated -->
Expand All @@ -502,7 +504,7 @@ const server = new ApolloServer({
plugins: [
// highlight-start
ApolloServerPluginUsageReporting({
sendErrorsInTraces: {
sendErrors: {
transform: (err) => {
// Return `null` to avoid reporting `UNAUTHENTICATED` errors
if (err.extensions.code === 'UNAUTHENTICATED') {
Expand Down Expand Up @@ -536,7 +538,7 @@ const server = new ApolloServer({
resolvers,
plugins: [
ApolloServerPluginUsageReporting({
sendErrorsInTraces: {
sendErrors: {
transform: (err) => {
// Using a more stable, known error extension (e.g. `err.code`) would be
// more defensive, however checking the `message` might serve most needs!
Expand Down Expand Up @@ -565,7 +567,7 @@ If you _do_ want to send an error's details to Apollo Studio, but need to redact

For example, if there is personally identifiable information in the error `message`, like an API key:

```ts
```ts
import { GraphQLError } from 'graphql';

throw new GraphQLError(
Expand All @@ -584,7 +586,7 @@ const server = new ApolloServer({
resolvers,
plugins: [
ApolloServerPluginUsageReporting({
sendErrorsInTraces: {
sendErrors: {
transform: (err) => {
// Make sure that a specific pattern is removed from all error messages.
err.message = err.message.replace(/x-api-key:[A-Z0-9-]+/, 'REDACTED');
Expand Down
8 changes: 4 additions & 4 deletions docs/source/migration.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -929,7 +929,7 @@ ApolloServerPluginUsageReporting({

In Apollo Server 3, you can specify a function to rewrite errors before sending them to Apollo's server via the `rewriteError` option to `ApolloServerPluginUsageReporting` (for monoliths) and `ApolloServerPluginInlineTrace` (for subgraphs).

In Apollo Server 4, you specify the same function as the `transform` option on the `sendErrorsInTraces` option to `ApolloServerPluginUsageReporting` and the `includeErrors` option to `ApolloServerPluginInlineTrace`.
In Apollo Server 4, you specify the same function as the `transform` option on the `sendErrors` option to `ApolloServerPluginUsageReporting` and the `includeErrors` option to `ApolloServerPluginInlineTrace`.

(Additionally, the [default behavior has changed to mask errors](#usage-reporting-and-inline-trace-plugins-mask-errors-by-default).)

Expand All @@ -955,7 +955,7 @@ you can now write:
// monoliths
new ApolloServer({
plugins: [ApolloServerPluginUsageReporting({
sendErrorsInTraces: { transform: rewriteError },
sendErrors: { transform: rewriteError },
})],
// ...
})
Expand Down Expand Up @@ -1592,7 +1592,7 @@ To restore the Apollo Server 3 behavior, you can pass `{ unmodified: true }` to
// monoliths
new ApolloServer({
plugins: [ApolloServerPluginUsageReporting({
sendErrorsInTraces: { unmodified: true },
sendErrors: { unmodified: true },
})],
// ...
})
Expand All @@ -1606,7 +1606,7 @@ new ApolloServer({
})
```

(As [described above](#rewriteerror-plugin-option), the `rewriteError` option has been replaced by a `transform` option on `sendErrorsInTraces` or `includeErrors`.)
(As [described above](#rewriteerror-plugin-option), the `rewriteError` option has been replaced by a `transform` option on `sendErrors` or `includeErrors`.)

## Renamed packages

Expand Down
14 changes: 7 additions & 7 deletions packages/integration-testsuite/src/apolloServerTests.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1139,14 +1139,14 @@ export function defineIntegrationTestSuiteApolloServerTests(
});

describe('error munging', () => {
describe('sendErrorsInTraces', () => {
describe('sendErrors', () => {
it('new error', async () => {
throwError.mockImplementationOnce(() => {
throw new Error('transform nope');
});

await setupApolloServerAndFetchPair({
sendErrorsInTraces: {
sendErrors: {
transform: () =>
new GraphQLError('rewritten as a new error'),
},
Expand Down Expand Up @@ -1191,7 +1191,7 @@ export function defineIntegrationTestSuiteApolloServerTests(
});

await setupApolloServerAndFetchPair({
sendErrorsInTraces: {
sendErrors: {
transform: (err) => {
err.message = 'rewritten as a modified error';
return err;
Expand Down Expand Up @@ -1236,7 +1236,7 @@ export function defineIntegrationTestSuiteApolloServerTests(
});

await setupApolloServerAndFetchPair({
sendErrorsInTraces: { transform: () => null },
sendErrors: { transform: () => null },
});

const result = await apolloFetch({
Expand Down Expand Up @@ -1271,7 +1271,7 @@ export function defineIntegrationTestSuiteApolloServerTests(
});

await setupApolloServerAndFetchPair({
sendErrorsInTraces: {
sendErrors: {
// @ts-expect-error (not allowed to be undefined)
transform: () => undefined,
},
Expand Down Expand Up @@ -1319,7 +1319,7 @@ export function defineIntegrationTestSuiteApolloServerTests(
});

await setupApolloServerAndFetchPair({
sendErrorsInTraces: {
sendErrors: {
unmodified: true,
},
});
Expand Down Expand Up @@ -1370,7 +1370,7 @@ export function defineIntegrationTestSuiteApolloServerTests(
});

await setupApolloServerAndFetchPair({
sendErrorsInTraces: {
sendErrors: {
masked: true,
},
});
Expand Down
92 changes: 69 additions & 23 deletions packages/server/src/plugin/usageReporting/options.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,23 +14,63 @@ export interface ApolloServerPluginUsageReportingOptions<
> {
//#region Configure exactly which data should be sent to Apollo.
/**
* By default, Apollo Server does not send the values of any GraphQL variables to Apollo's servers, because variable
* values often contain the private data of your app's users. If you'd like variable values to be included in traces, set this option.
* This option can take several forms:
* Apollo Server's usage reports describe each individual request in one of
* two ways: as a "trace" (a detailed description of the specific request,
* including a query plan and resolver tree with timings and errors, as well
* as optional details like variable values and HTTP headers), or as part of
* aggregated "stats" (where invocations of the same operation from the same
* client program are aggregated together rather than described individually).
* Apollo Server uses an heuristic to decide which operations to describe as
* traces and which to aggregate as stats.
*
* By setting the `sendTraces` option to `false`, Apollo Server will describe
* *all* operations as stats; individual requests will never be broken out
* into separate traces. If you set `sendTraces: false`, the Traces view in
* Apollo Studio will not show any traces, but all other Studio functionality
* should be unaffected.
*
* Note that the values of `sendVariableValues`, `sendHeaders`, and
* `sendUnexecutableOperationDocuments` are irrelevant if you set
* `sendTraces: false`, because those options control data that is contained
* only in traces and not in stats.
*
* Setting `sendTraces: false` does *NOT* imply `fieldLevelInstrumentation:
* 0`. Apollo Server can still take advantage of field-level instrumentation
* (either directly for monolith servers, or via federated tracing for
* Gateways) in order to accurately report field execution usage in "stats".
* This option only controls whether data is sent to Apollo's servers as
* traces, not whether traces are internally used to learn about usage.
*/
sendTraces?: boolean;

/**
* By default, Apollo Server does not send the values of any GraphQL variables
* to Apollo's servers, because variable values often contain the private data
* of your app's users. If you'd like variable values to be included in
* traces, set this option. This option can take several forms:
* - { none: true }: don't send any variable values (DEFAULT)
* - { all: true}: send all variable values
* - { transform: ... }: a custom function for modifying variable values. Keys added by the custom function will
* be removed, and keys removed will be added back with an empty value. For security reasons, if an error occurs within this function, all variable values will be replaced with `[PREDICATE_FUNCTION_ERROR]`.
* - { exceptNames: ... }: a case-sensitive list of names of variables whose values should not be sent to Apollo servers
* - { onlyNames: ... }: A case-sensitive list of names of variables whose values will be sent to Apollo servers
* - { transform: ... }: a custom function for modifying variable values. Keys
* added by the custom function will be removed, and keys removed will be
* added back with an empty value. For security reasons, if an error occurs
* within this function, all variable values will be replaced with
* `[PREDICATE_FUNCTION_ERROR]`.
* - { exceptNames: ... }: a case-sensitive list of names of variables whose
* values should not be sent to Apollo servers
* - { onlyNames: ... }: A case-sensitive list of names of variables whose
* values will be sent to Apollo servers
*
* Defaults to not sending any variable values if both this parameter and the
* deprecated `privateVariables` are not set. The report will indicate each
* private variable key whose value was redacted by { none: true } or {
* exceptNames: [...] }.
*
* Defaults to not sending any variable values if both this parameter and
* the deprecated `privateVariables` are not set. The report will
* indicate each private variable key whose value was redacted by { none: true } or { exceptNames: [...] }.
* The value of this option is not relevant if you set `sendTraces: false`,
* because variable values only appear in traces.
*/
sendVariableValues?: VariableValueOptions;
/**
* By default, Apollo Server does not send the list of HTTP headers and values
* By default, Apollo Server does not send the HTTP request headers and values
* to Apollo's servers, as these headers may contain your users' private data.
* If you'd like this information included in traces, set this option. This
* option can take several forms:
Expand All @@ -44,6 +84,9 @@ export interface ApolloServerPluginUsageReportingOptions<
*
* Unlike with sendVariableValues, names of dropped headers are not reported.
* The headers 'authorization', 'cookie', and 'set-cookie' are never reported.
*
* The value of this option is not relevant if you set `sendTraces: false`,
* because request headers only appear in traces.
*/
sendHeaders?: SendValuesBaseOptions;
/**
Expand All @@ -65,10 +108,12 @@ export interface ApolloServerPluginUsageReportingOptions<
* (either a new error, or its potentially-modified argument) or `null`.
* This error is used in the report to Apollo servers; if `null`, the error
* is not included in traces or error statistics.
*
* If you set `sendTraces: false`, then the only relevant aspect of this
* option is whether you return `null` from a `transform` function or not
* (which affects aggregated error statistics).
*/
sendErrorsInTraces?: SendErrorsOptions;

// We should strongly consider changing the default to false in AS4.
sendErrors?: SendErrorsOptions;

/**
* This option allows you to choose if Apollo Server should calculate detailed
Expand Down Expand Up @@ -127,8 +172,9 @@ export interface ApolloServerPluginUsageReportingOptions<
* (Note that returning true here does *not* mean that the data derived from
* field-level instrumentation must be transmitted to Apollo Studio's servers
* in the form of a trace; it may still be aggregated locally to statistics.
* But either way this operation will contribute to the "field executions"
* statistic and timing hints.)
* Similarly, setting `sendTraces: false` does not affect
* `fieldLevelInstrumentation`. But either way this operation will contribute
* to the "field executions" statistic and timing hints.)
*
* The default `fieldLevelInstrumentation` is a function that always returns
* true.
Expand Down Expand Up @@ -211,6 +257,9 @@ export interface ApolloServerPluginUsageReportingOptions<
* and the operation name and signature will always be reported with a constant
* identifier. Whether the operation was a parse failure or a validation
* failure will be embedded within the stats report key itself.
*
* The value of this option is not relevant if you set `sendTraces: false`,
* because unexecutable operation documents only appear in traces.
*/
sendUnexecutableOperationDocuments?: boolean;

Expand All @@ -225,6 +274,9 @@ export interface ApolloServerPluginUsageReportingOptions<
* Apollo's servers perform their own sampling on received traces; not all
* traces sent to Apollo's servers can be later retrieved via the trace UI.)
*
* If you just want to send all operations as stats, set `sendTraces: false`
* instead of using this experimental hook.
*
* This option is highly experimental and may change or be removed in future
* versions.
*/
Expand Down Expand Up @@ -298,7 +350,7 @@ export interface ApolloServerPluginUsageReportingOptions<
/**
* If set, prints all reports as JSON when they are sent. (Note that for
* technical reasons, traces embedded in a report are printed separately when
* they are added to a report.)
* they are added to a report.) Reports are sent through `logger.info`.
*/
debugPrintReports?: boolean;
/**
Expand All @@ -307,12 +359,6 @@ export interface ApolloServerPluginUsageReportingOptions<
* about how the signature relates to the operation you executed.
*/
calculateSignature?: (ast: DocumentNode, operationName: string) => string;
/**
* This option includes extra data in reports that helps Apollo validate the
* stats generation code in this plugin. Do not set it; the only impact on
* your app will be a decrease in performance.
*/
internal_includeTracesContributingToStats?: boolean;
//#endregion
}

Expand Down
Loading

0 comments on commit 6cd6f7c

Please sign in to comment.