From cc64db1a73fd223200246e7d3f57c77b26c76f31 Mon Sep 17 00:00:00 2001 From: Timothy Sullivan Date: Thu, 12 Mar 2020 11:45:48 -0700 Subject: [PATCH 01/13] [Reporting] Screenshot Service RFC --- rfcs/text/0009_screenshot_mode_service.md | 262 ++++++++++++++++++++++ 1 file changed, 262 insertions(+) create mode 100644 rfcs/text/0009_screenshot_mode_service.md diff --git a/rfcs/text/0009_screenshot_mode_service.md b/rfcs/text/0009_screenshot_mode_service.md new file mode 100644 index 0000000000000..fd015b63b25e8 --- /dev/null +++ b/rfcs/text/0009_screenshot_mode_service.md @@ -0,0 +1,262 @@ +- Start Date: 2020-03-02 +- RFC PR: (leave this empty) +- Kibana Issue: (leave this empty) + +# Summary + +The Kibana Reporting team has been chasing ways of improving performance of +loading Kibana pages and capturing screenshots of the visualizations. We think +that performance is a barrier to adding more integrations in Kibana, however we +want to add more integrations and have Reporting be a factor of growth for +Kibana. However, it looks like there are too many constraints to streamlining +performance for Reporting if we only look for ways to change the Reporting code +to do it. In other words, it will take a cross-team effort to improve the main +performance issues. This RFC proposes one way to make cross-team efforts work +smoothly, which is to have a new plugin-provided service in Kibana that helps UIs and +visualizations work nicely when loaded for screenshot capture. + +**Screenshot mode service** + +Applications in Kibana should be aware of when their rendering is for a human +user using the page interactively, and when it's not. Sometimes the page is +loaded for display-purposes: there is no user in control of the +browser and no need for interactivity. Having Kibana support that purpose +allows higher-level features to be built on top of rapidly-available images of +Kibana pages. This turns out to be crucial for features like scheduled PDF +report generation, and automated alerts with embedded images. + +As a bridge between the Kibana applications screenshot-capture features is +needed as a service in Kibana. This RFC names that service the Screenshot +Mode Service (name TBD). Applications would read from this service while +loading UI components into the browser, and help the application make +case-by-case bases for loading components and the parameters that get passed +into the components. + +In most cases, the information coming from this service would help the +application determine which UI components should **not** be loaded, as applications will +use the service response for skipping components that aren't needed for +presenting the data in a screenshot, and thus work towards better performance +for the display-only purpose. + +More background on how Reporting currently works, including the lifecycle of +creating a PNG report, is here: https://github.com/elastic/kibana/issues/59396 + +# Basic example + +When Kibana loads initially, there is a Newsfeed component in the UI that +checks internally cached records to see if it must fetch the Elastic News +Service for newer items. When the Screenshot Mode Service is implemented, the +Newsfeed component has a source of information to check on whether or not it +should load in the Kibana UI. If it can avoid loading, it avoids an unnecessary +HTTP round trip, which weigh heavily on performance. + +# Motivation + +Kibana Reporting is a commercial feature in Elastic and is highly capable of +loading the application pages and converting them into a screenshot for PNG or +PDF export. However, the way it works has downsides with performance, +maintainability, and expanding it as a tool that can power higher-level +features. The solution to those downsides is to have Kibana pages themselves +become more capable at presenting themselves for screenshot capture reports. +With the Screenshot Mode Service available, Reporting could drop the +task that it currently has which hurt performance: wasted rendering that is +replaced with custom styles that make the page "reportable." + +The technical advantage of having such as service also leads to making Kibana +application pages "printable", in the sense that sending the page to a printer +for a hard copy results in something more meaningful and specialized for paper +than today's Kibana can guarantee. This isn't a big concern for Kibana since +there isn't the expectation to improve printing Kibana, but that technical +direction is appropriate for improving PDF report generation. + +# Detailed design + +The Screenshot Mode Service is a callable API available in the context of a +request on the server side, and as a pubic pluginsSetup Javascript API. + +The data provided by the Screenshot Mode are signals about: + - whether or not the context of the page load is for a screenshot capture + - layout and dimensions that the result image is expected to be + - time zone for which to format the page data + +To obtain the screenshot mode context data and provide it as a service to +plugins, this RFC recommends using a URL query string variable with raw data, +and provide an interface to it in the form of an observable. The Screenshot +Mode Service plugin will read the data in its `setup` method and provide the +observable to the data interface in its plugin contract. The interface is +described below as the `ScreenshotModeServiceApi`. + +The new query string parameter could appear in the URL of any Kibana +application. Adding new URL query string parameters is a non-breaking change, +and applications would not be impacted by having this additional data in the URL. + +Since the service data needs to be an object, the best way to encode it as data +in a URL is to use Rison encoding. An example of what that would look like in a +Kibana URL (with linebreaks added): + +``` +http://localhost:5601/app/kibana +?screenshots=(enabled:!t,layout:(height:500,width:400),timezone:PST) +#/discover?_g=()&_a=(columns:!(_source),index:'6a047550-4e87-11ea-ad93-21278bc67061',interval:auto,query:(language:kuery,query:''),sort:!()) + +``` + +That shows an example way of encoding this raw data into the URL as a global +variable called `screenshots`: +``` +{ + "enabled": true, + "layout": { + "height": 500, + "width": 400 + }, + "timezone": "PST" +} +``` + +The rest of the URL is needed by the application and is used for recreating a +certain state of data in the page. In order to be "reportable" these parts of +the URL must be totally sufficient for recreating everything the user expects +to see in a report. + +The `screenshot` object has to be crafted by a client that is generating a +request for a screenshot. That can be done by anything, but the implementation +concerns will be left out to keep the RFC about the details on the service. + +The Screenshot Mode Service would be a plugin designed to wrap the raw context data for +applications to consume, and it is up to an outside feature (i.e. Kibana +Reporting) to craft the raw context data. Applications are the parts that make +this design useful for Kibana, by reading the service data and using it to +inform its rendering. + +As a plugin, the screenshot mode service won't automatically work in the +background to inject the data into the various running stages of applications. +This is in line with the pattern on the client-side where all query parameters +are delegated to the individual applications, letting them do as they wish with +it. For this RFC, it's an important topic because plugins have to "wire-up" +this service in their router and UI app, but it will prevent "magical" behavior +happening. + +## Server side +On the server side, applications will have to interact with the screenshot +plugin to support custom endpoints include the screenshot data as part of the +route handling context. Example: +``` +const screenshotRouter = plugins.screenshots.wrapRouter(router); +screenshotRouter.get( + { path: '/my-endpoint' }, + async (context, req, res) => { + const isScreenshotMode = context.screenshots.isScreenshotMode(); + } +); +``` + +The `screenshotRouter` handlers could either replace the existing routes of the +application, or the application can feature custom endpoints for screenshot +export that use this plugin. + +## Front-end +On the front-end, the plugin service could be built as a simple utility that takes a +[History](https://developer.mozilla.org/en-US/docs/Web/API/History) +instance and returns an Observable that emits the screenshot data by listening +for location updates and computing the data based on the query parameters. + +``` +const screenshot$ = plugins.screenshots.readFrom(history); +``` + +## Types +The `request.screenshotMode` object is an instance of a class that has +functions to abstract it's internal raw data: + +``` +class ScreenshotModeServiceApi { + constructor(rawData) { + } + public isScreenshotMode (): boolean { + } + public getScreenshotDimensions (): ScreenshotLayoutDimensions { + } + public getTimezone (): Timezone { + } +} +``` + +Getting the screenshot dimensions would be needed only for very specialized use +cases such as the Dashboard application's "print layout" mode. + +# Drawbacks + +Why should we *not* do this? Please consider: + +- Plugins have to "wire-up" this service in their router and UI app +- Low-level of discoverability since this logic is hidden in a plugin rather than exposed on the CoreSetup API. +- Hard for application teams to create an environment to test against to check the screenshot mode rendering of their work. + +As a solution to the drawbacks of extra maintenance and tests needed, the +Reporting Services team could provide libraries to use in automated functional +tests that help verify the integration. There could also be a development tool +in the form of a test Kibana plugin or bookmarklet to help developers check their +work for screenshot mode. + +# Alternatives + +- Print media query CSS + Many things can be improved on capturing screenshots from the page using + `@print` CSS media query selecting. Also, CSS makes things hidden on the + page, but they are still loading in the DOM and take browser memory + resources. Print media query CSS are a good supplement to the Screenshot Mode + Service, but the best performance will come with streamlining the page + rendering in Javascript. See "further examples" in this RFC about how this + alternative solution can still be leveraged. + +# Adoption strategy + +Integrating applications with this service is only going to be necessary when +there are customizations needed to make the application more "reportable." +That will be done by examining the default results of how applications perform +for screenshot capture mode, and tuning the app code for performance as needed. + +Some of the first steps would be to have the Reporting services team hoist some +code out of the screenshot pipeline, and start adding the `screenshots` +variable to the URL as a query string parameter. Then, parts of the Reporting +code can be taken apart a bit. The areas that need customization for screenshot +capture will no longer rely on the Reporting plugin to inject those +customizations. The applications will check the Screenshot Mode Service and +inject the specialized customizations as needed. + +Having the Screenshot mode service would be a means towards fixing a few bugs +with Reporting rendering. The bug fixes will involve adopting the Screenshot +Mode Service throughout applications: especially in the Dashboard application. +Those fixes will be driven by the Reporting Services team initially. +Eventually, the Dashboard application could feature a few examples in the code +on how to integrate with the Screenshot Mode Service. + +Further adoption of this service would happen as we look for ways to improve +the performance and quality of Reporting services throughout Kibana. Those +fine-tuning adoptions can be driven by the Reporting services team, or +application teams, depending on bandwidth available. + +# How we teach this + +Not every Kibana developer will need to understand the screenshot mode service +to be effective at day-to-day work, but every Kibana developer should be at +least aware of its existence. That way, they can understand it when the time +comes to specialize their application for Reporting. As long as developers are +aware of its existence, they could understand it by reading the code and trying +the test Kibana plugin (if it exists). + +# Further examples + +- Applications can also use screenshot context to customize the way they load. + An example is Toast Notifications: by default they auto-dismiss themselves + after 30 seconds or so. That makes sense when there is a human there to + notice the message, read it and remember it. But if the page is loaded for + capturing a screenshot, the toast notifications should never disappear. The + message in the toast needs to be part of the screenshot for its message to + mean anything, so it should not force the screenshot capture tool to race + against the toast timeout window. +- Avoid collection and sending of telemetry from the browser when page is + loaded for screenshot capture. +- Turn off autocomplete features and auto-refresh features that weigh on + performance for screenshot capture. From abf1a168b2bb5181fe4cf314c6d85d132176fb26 Mon Sep 17 00:00:00 2001 From: Timothy Sullivan Date: Wed, 3 Mar 2021 12:24:00 -0700 Subject: [PATCH 02/13] rewrite summary --- rfcs/text/0009_screenshot_mode_service.md | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/rfcs/text/0009_screenshot_mode_service.md b/rfcs/text/0009_screenshot_mode_service.md index fd015b63b25e8..51093b3c55637 100644 --- a/rfcs/text/0009_screenshot_mode_service.md +++ b/rfcs/text/0009_screenshot_mode_service.md @@ -4,16 +4,11 @@ # Summary -The Kibana Reporting team has been chasing ways of improving performance of -loading Kibana pages and capturing screenshots of the visualizations. We think -that performance is a barrier to adding more integrations in Kibana, however we -want to add more integrations and have Reporting be a factor of growth for -Kibana. However, it looks like there are too many constraints to streamlining -performance for Reporting if we only look for ways to change the Reporting code -to do it. In other words, it will take a cross-team effort to improve the main -performance issues. This RFC proposes one way to make cross-team efforts work -smoothly, which is to have a new plugin-provided service in Kibana that helps UIs and -visualizations work nicely when loaded for screenshot capture. +Applications should be aware when their UI is rendered for purposes of +capturing a screenshot. This ability would improve the quality of the Kibana +Reporting feature for a few reasons: + - Fewer objects in the headless browser memory since interactive code doesn't run + - Fewer Reporting bugs in releases since App teams have more ownership and control over the reportability of their UI **Screenshot mode service** From f0b4cc7dc6064499dc30d04e699a64737e840042 Mon Sep 17 00:00:00 2001 From: Tim Sullivan Date: Wed, 3 Mar 2021 12:46:03 -0700 Subject: [PATCH 03/13] simplify design --- rfcs/text/0009_screenshot_mode_service.md | 194 +++++----------------- 1 file changed, 43 insertions(+), 151 deletions(-) diff --git a/rfcs/text/0009_screenshot_mode_service.md b/rfcs/text/0009_screenshot_mode_service.md index 51093b3c55637..4f15a03737d31 100644 --- a/rfcs/text/0009_screenshot_mode_service.md +++ b/rfcs/text/0009_screenshot_mode_service.md @@ -8,7 +8,8 @@ Applications should be aware when their UI is rendered for purposes of capturing a screenshot. This ability would improve the quality of the Kibana Reporting feature for a few reasons: - Fewer objects in the headless browser memory since interactive code doesn't run - - Fewer Reporting bugs in releases since App teams have more ownership and control over the reportability of their UI + - Fewer Reporting bugs in releases since App teams have more ownership and + control over the reportability of their UI **Screenshot mode service** @@ -66,180 +67,71 @@ direction is appropriate for improving PDF report generation. # Detailed design -The Screenshot Mode Service is a callable API available in the context of a -request on the server side, and as a pubic pluginsSetup Javascript API. +The Screenshot Mode Service is a callable API available as dependency for +Kibana applications. -The data provided by the Screenshot Mode are signals about: - - whether or not the context of the page load is for a screenshot capture - - layout and dimensions that the result image is expected to be - - time zone for which to format the page data +A method of the API tells the Application whether or not it should render +itself to optimize for non-interactivity. -To obtain the screenshot mode context data and provide it as a service to -plugins, this RFC recommends using a URL query string variable with raw data, -and provide an interface to it in the form of an observable. The Screenshot -Mode Service plugin will read the data in its `setup` method and provide the -observable to the data interface in its plugin contract. The interface is -described below as the `ScreenshotModeServiceApi`. +In a future phase, the API might also tell the application more about the +report, such as PDF page dimensions, or special layout types (print layout, +etc). -The new query string parameter could appear in the URL of any Kibana -application. Adding new URL query string parameters is a non-breaking change, -and applications would not be impacted by having this additional data in the URL. - -Since the service data needs to be an object, the best way to encode it as data -in a URL is to use Rison encoding. An example of what that would look like in a -Kibana URL (with linebreaks added): - -``` -http://localhost:5601/app/kibana -?screenshots=(enabled:!t,layout:(height:500,width:400),timezone:PST) -#/discover?_g=()&_a=(columns:!(_source),index:'6a047550-4e87-11ea-ad93-21278bc67061',interval:auto,query:(language:kuery,query:''),sort:!()) +## Interface +The `setupDeps.screenshotMode` object has a single purpose: tell the app if it +should render in an optimized way for screenshot capture: ``` - -That shows an example way of encoding this raw data into the URL as a global -variable called `screenshots`: -``` -{ - "enabled": true, - "layout": { - "height": 500, - "width": 400 - }, - "timezone": "PST" +interface ScreenshotModeServiceSetup { + isScreenshotMode: () => boolean; } ``` -The rest of the URL is needed by the application and is used for recreating a -certain state of data in the page. In order to be "reportable" these parts of -the URL must be totally sufficient for recreating everything the user expects -to see in a report. - -The `screenshot` object has to be crafted by a client that is generating a -request for a screenshot. That can be done by anything, but the implementation -concerns will be left out to keep the RFC about the details on the service. - -The Screenshot Mode Service would be a plugin designed to wrap the raw context data for -applications to consume, and it is up to an outside feature (i.e. Kibana -Reporting) to craft the raw context data. Applications are the parts that make -this design useful for Kibana, by reading the service data and using it to -inform its rendering. - -As a plugin, the screenshot mode service won't automatically work in the -background to inject the data into the various running stages of applications. -This is in line with the pattern on the client-side where all query parameters -are delegated to the individual applications, letting them do as they wish with -it. For this RFC, it's an important topic because plugins have to "wire-up" -this service in their router and UI app, but it will prevent "magical" behavior -happening. - -## Server side -On the server side, applications will have to interact with the screenshot -plugin to support custom endpoints include the screenshot data as part of the -route handling context. Example: -``` -const screenshotRouter = plugins.screenshots.wrapRouter(router); -screenshotRouter.get( - { path: '/my-endpoint' }, - async (context, req, res) => { - const isScreenshotMode = context.screenshots.isScreenshotMode(); - } -); -``` - -The `screenshotRouter` handlers could either replace the existing routes of the -application, or the application can feature custom endpoints for screenshot -export that use this plugin. +Internally, this object is constructed from a class that refers to information +sent via a custom proprietary header: -## Front-end -On the front-end, the plugin service could be built as a simple utility that takes a -[History](https://developer.mozilla.org/en-US/docs/Web/API/History) -instance and returns an Observable that emits the screenshot data by listening -for location updates and computing the data based on the query parameters. - -``` -const screenshot$ = plugins.screenshots.readFrom(history); ``` +interface HeaderData { + 'X-Screenshot-Mode': true +} -## Types -The `request.screenshotMode` object is an instance of a class that has -functions to abstract it's internal raw data: - -``` -class ScreenshotModeServiceApi { - constructor(rawData) { - } - public isScreenshotMode (): boolean { - } - public getScreenshotDimensions (): ScreenshotLayoutDimensions { - } - public getTimezone (): Timezone { - } +class ScreenshotModeServiceSetup { + constructor(rawData: HeaderData) {} + public isScreenshotMode (): boolean {} } ``` -Getting the screenshot dimensions would be needed only for very specialized use -cases such as the Dashboard application's "print layout" mode. - -# Drawbacks - -Why should we *not* do this? Please consider: - -- Plugins have to "wire-up" this service in their router and UI app -- Low-level of discoverability since this logic is hidden in a plugin rather than exposed on the CoreSetup API. -- Hard for application teams to create an environment to test against to check the screenshot mode rendering of their work. - -As a solution to the drawbacks of extra maintenance and tests needed, the -Reporting Services team could provide libraries to use in automated functional -tests that help verify the integration. There could also be a development tool -in the form of a test Kibana plugin or bookmarklet to help developers check their -work for screenshot mode. +This works because the headless browser that opens the page can inject custom +headers into the request. Teams can test how their app renders when loaded with +this header using a new configuration setting, or a web debugging proxy, or +some other tool that is TBD. # Alternatives - Print media query CSS - Many things can be improved on capturing screenshots from the page using - `@print` CSS media query selecting. Also, CSS makes things hidden on the - page, but they are still loading in the DOM and take browser memory - resources. Print media query CSS are a good supplement to the Screenshot Mode - Service, but the best performance will come with streamlining the page - rendering in Javascript. See "further examples" in this RFC about how this - alternative solution can still be leveraged. + If applications UIs supported printability using `@media print`, and Kibana + Reporting uses `page.print()` to capture the PDF, it would be easy for application + developers to test, and prevent bugs showing up in the report. + + However, this solution doesn't include performance benefits of reducing objects + in the headless browser memory: the headless browser still has to render the entire + page as a "normal" render before it is able to call `page.print()`. No one sees the + results of that initial render, so it is the same amount of wasted rendering cycles + during report generation that we have today. # Adoption strategy -Integrating applications with this service is only going to be necessary when -there are customizations needed to make the application more "reportable." -That will be done by examining the default results of how applications perform -for screenshot capture mode, and tuning the app code for performance as needed. - -Some of the first steps would be to have the Reporting services team hoist some -code out of the screenshot pipeline, and start adding the `screenshots` -variable to the URL as a query string parameter. Then, parts of the Reporting -code can be taken apart a bit. The areas that need customization for screenshot -capture will no longer rely on the Reporting plugin to inject those -customizations. The applications will check the Screenshot Mode Service and -inject the specialized customizations as needed. - -Having the Screenshot mode service would be a means towards fixing a few bugs -with Reporting rendering. The bug fixes will involve adopting the Screenshot -Mode Service throughout applications: especially in the Dashboard application. -Those fixes will be driven by the Reporting Services team initially. -Eventually, the Dashboard application could feature a few examples in the code -on how to integrate with the Screenshot Mode Service. - -Further adoption of this service would happen as we look for ways to improve -the performance and quality of Reporting services throughout Kibana. Those -fine-tuning adoptions can be driven by the Reporting services team, or -application teams, depending on bandwidth available. +The Reporting Services team should create an few example in a developer example plugin +on how to integrate an App with the Screenshot Mode Service. From there, the team should +work with App teams in a consulting role to establish usage of this service. # How we teach this -Not every Kibana developer will need to understand the screenshot mode service -to be effective at day-to-day work, but every Kibana developer should be at -least aware of its existence. That way, they can understand it when the time -comes to specialize their application for Reporting. As long as developers are -aware of its existence, they could understand it by reading the code and trying -the test Kibana plugin (if it exists). +The Reporting Services team can offer statistics in a weekly update about how many +Reporting-enabled applications are using this service. This will help people remember +that it is available. If the team can also provide statistics about how many bugs this is +fixing, how much time it saves in generating a report, etc, then it will also help +people understand why it is important. # Further examples From 61d526fc31b58ccb43bf39c9bbfa1900c0606329 Mon Sep 17 00:00:00 2001 From: Tim Sullivan Date: Wed, 3 Mar 2021 12:58:34 -0700 Subject: [PATCH 04/13] Update 0009_screenshot_mode_service.md --- rfcs/text/0009_screenshot_mode_service.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rfcs/text/0009_screenshot_mode_service.md b/rfcs/text/0009_screenshot_mode_service.md index 4f15a03737d31..fa599fe01679f 100644 --- a/rfcs/text/0009_screenshot_mode_service.md +++ b/rfcs/text/0009_screenshot_mode_service.md @@ -82,7 +82,7 @@ The `setupDeps.screenshotMode` object has a single purpose: tell the app if it should render in an optimized way for screenshot capture: ``` -interface ScreenshotModeServiceSetup { +interface IScreenshotModeServiceSetup { isScreenshotMode: () => boolean; } ``` @@ -95,7 +95,7 @@ interface HeaderData { 'X-Screenshot-Mode': true } -class ScreenshotModeServiceSetup { +class ScreenshotModeServiceSetup implements IScreenshotModeServiceSetup { constructor(rawData: HeaderData) {} public isScreenshotMode (): boolean {} } From abdf4fdc4faee1022ed83039cc56be1fa5238ece Mon Sep 17 00:00:00 2001 From: Timothy Sullivan Date: Tue, 23 Mar 2021 14:24:52 -0700 Subject: [PATCH 05/13] mention the 3 screenshot report apps --- rfcs/text/0009_screenshot_mode_service.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/rfcs/text/0009_screenshot_mode_service.md b/rfcs/text/0009_screenshot_mode_service.md index 4f15a03737d31..a80dd97f34a92 100644 --- a/rfcs/text/0009_screenshot_mode_service.md +++ b/rfcs/text/0009_screenshot_mode_service.md @@ -11,6 +11,11 @@ Reporting feature for a few reasons: - Fewer Reporting bugs in releases since App teams have more ownership and control over the reportability of their UI +Currently, the applications that support screenshot reports are: + - Dashboard + - Visualize Editor + - Canvas + **Screenshot mode service** Applications in Kibana should be aware of when their rendering is for a human From 4dde01a9e171de15a3123d968f2cbf6f0ebd5ed9 Mon Sep 17 00:00:00 2001 From: Timothy Sullivan Date: Tue, 23 Mar 2021 14:37:48 -0700 Subject: [PATCH 06/13] try not to say this is a high-level service --- rfcs/text/0009_screenshot_mode_service.md | 33 ++++++++++------------- 1 file changed, 14 insertions(+), 19 deletions(-) diff --git a/rfcs/text/0009_screenshot_mode_service.md b/rfcs/text/0009_screenshot_mode_service.md index a80dd97f34a92..ebe448d453f91 100644 --- a/rfcs/text/0009_screenshot_mode_service.md +++ b/rfcs/text/0009_screenshot_mode_service.md @@ -19,25 +19,20 @@ Currently, the applications that support screenshot reports are: **Screenshot mode service** Applications in Kibana should be aware of when their rendering is for a human -user using the page interactively, and when it's not. Sometimes the page is -loaded for display-purposes: there is no user in control of the -browser and no need for interactivity. Having Kibana support that purpose -allows higher-level features to be built on top of rapidly-available images of -Kibana pages. This turns out to be crucial for features like scheduled PDF -report generation, and automated alerts with embedded images. - -As a bridge between the Kibana applications screenshot-capture features is -needed as a service in Kibana. This RFC names that service the Screenshot -Mode Service (name TBD). Applications would read from this service while -loading UI components into the browser, and help the application make -case-by-case bases for loading components and the parameters that get passed -into the components. - -In most cases, the information coming from this service would help the -application determine which UI components should **not** be loaded, as applications will -use the service response for skipping components that aren't needed for -presenting the data in a screenshot, and thus work towards better performance -for the display-only purpose. +user using the page interactively, and when it's not. At a high level, the application +can support that awareness through a customized URL for screenshot reports. Canvas +demonstrates a great implementation of this today. However, Canvas has dependencies +on other UI plugins, and can not control how they render or what type of network requests +they send. Those plugins, and lower-level plugins that run on every page, do not have +a way to control themselves when the page is rendering for taking a screenshot. + +This RFC proposes a Screenshot Mode Service as a low-level plugin that allows +other plugins (UI code) to make choices when the page is rendering for a screenshot. + +In most cases, the information coming from this service would help the UI code +decide what should **not** be loaded: for example, the NewsFeed component and +Telemetry code should not load itself when the page is rendering for +screenshots. More background on how Reporting currently works, including the lifecycle of creating a PNG report, is here: https://github.com/elastic/kibana/issues/59396 From dff5d27d93a189adc163045b489c46ad3a40bfcc Mon Sep 17 00:00:00 2001 From: Timothy Sullivan Date: Tue, 23 Mar 2021 14:42:45 -0700 Subject: [PATCH 07/13] clarify that print media css is just ok --- rfcs/text/0009_screenshot_mode_service.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/rfcs/text/0009_screenshot_mode_service.md b/rfcs/text/0009_screenshot_mode_service.md index ebe448d453f91..b9001c68d0e58 100644 --- a/rfcs/text/0009_screenshot_mode_service.md +++ b/rfcs/text/0009_screenshot_mode_service.md @@ -113,9 +113,10 @@ some other tool that is TBD. Reporting uses `page.print()` to capture the PDF, it would be easy for application developers to test, and prevent bugs showing up in the report. - However, this solution doesn't include performance benefits of reducing objects - in the headless browser memory: the headless browser still has to render the entire - page as a "normal" render before it is able to call `page.print()`. No one sees the + However, this proposal only provides high-level customization over visual rendering, which the + application already has if it uses a customized URL for rendering the layout for screenshots. It + has a performance downside, as well: the headless browser still has to render the entire + page as a "normal" render before we can call `page.print()`. No one sees the results of that initial render, so it is the same amount of wasted rendering cycles during report generation that we have today. From 6a8181f455213af3f520537021efe232c5f93991 Mon Sep 17 00:00:00 2001 From: Timothy Sullivan Date: Tue, 23 Mar 2021 14:51:21 -0700 Subject: [PATCH 08/13] clarify the intent --- rfcs/text/0009_screenshot_mode_service.md | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/rfcs/text/0009_screenshot_mode_service.md b/rfcs/text/0009_screenshot_mode_service.md index b9001c68d0e58..35d36d49cbbf7 100644 --- a/rfcs/text/0009_screenshot_mode_service.md +++ b/rfcs/text/0009_screenshot_mode_service.md @@ -122,17 +122,19 @@ some other tool that is TBD. # Adoption strategy -The Reporting Services team should create an few example in a developer example plugin -on how to integrate an App with the Screenshot Mode Service. From there, the team should -work with App teams in a consulting role to establish usage of this service. +Using this service doesn't mean that anything needs to be replaced or thrown away. It's an add on +that any plugin or even application can use to add conditionals that previously weren't possible. +The Reporting Services team should create an example in a developer example plugin on how to build +a UI that is aware of Screenshot Mode Service. From there, the team would work on updating +whichever code that would benefit from this the most, which we know from analyzing debugging logs +of a report job. The team would work across teams to get it accepted by the owners. # How we teach this -The Reporting Services team can offer statistics in a weekly update about how many -Reporting-enabled applications are using this service. This will help people remember -that it is available. If the team can also provide statistics about how many bugs this is -fixing, how much time it saves in generating a report, etc, then it will also help -people understand why it is important. +The Reporting Services team will continue to analyze debug logs of reporting jobs to find if there +is UI code running during a report job that could be optimized by this service. The team would +reach out to the code owners and determine if it makes sense to use this service to improve +screenshot performance of their code. # Further examples From a491a7217d59584ae2a0ad5e6e89cb748e1b7d4d Mon Sep 17 00:00:00 2001 From: Timothy Sullivan Date: Wed, 24 Mar 2021 14:20:13 -0700 Subject: [PATCH 09/13] drop the `app` --- rfcs/text/0009_screenshot_mode_service.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/rfcs/text/0009_screenshot_mode_service.md b/rfcs/text/0009_screenshot_mode_service.md index 054feabeef610..ad047d75aae60 100644 --- a/rfcs/text/0009_screenshot_mode_service.md +++ b/rfcs/text/0009_screenshot_mode_service.md @@ -78,8 +78,9 @@ report, such as PDF page dimensions, or special layout types (print layout, etc). ## Interface -The `setupDeps.screenshotMode` object has a single purpose: tell the app if it -should render in an optimized way for screenshot capture: +The `setupDeps.screenshotMode` object has a single purpose: tell a low-level +plugin if the page is rendering in screenshot capture mode. The plugin decides +what to do with that information. ``` interface IScreenshotModeServiceSetup { From 7eb12f64b91bea46c44cfdf5ae3234606fd46b89 Mon Sep 17 00:00:00 2001 From: Timothy Sullivan Date: Wed, 24 Mar 2021 14:21:53 -0700 Subject: [PATCH 10/13] add the possibility to test screenshot mode through a URL parameter --- rfcs/text/0009_screenshot_mode_service.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/rfcs/text/0009_screenshot_mode_service.md b/rfcs/text/0009_screenshot_mode_service.md index ad047d75aae60..72cd6c4b3bd74 100644 --- a/rfcs/text/0009_screenshot_mode_service.md +++ b/rfcs/text/0009_screenshot_mode_service.md @@ -102,6 +102,8 @@ class ScreenshotModeServiceSetup implements IScreenshotModeServiceSetup { } ``` +There can be a URL parameter which triggers screenshot mode to be enabled, for easier testing. + This works because the headless browser that opens the page can inject custom headers into the request. Teams can test how their app renders when loaded with this header using a new configuration setting, or a web debugging proxy, or From 20609cf2d5e5c70af54a69668c997c15ed6f7127 Mon Sep 17 00:00:00 2001 From: Timothy Sullivan Date: Wed, 24 Mar 2021 14:46:37 -0700 Subject: [PATCH 11/13] keep it more low-level --- rfcs/text/0009_screenshot_mode_service.md | 58 ++++++++--------------- 1 file changed, 20 insertions(+), 38 deletions(-) diff --git a/rfcs/text/0009_screenshot_mode_service.md b/rfcs/text/0009_screenshot_mode_service.md index 72cd6c4b3bd74..38fc02f7cf6fd 100644 --- a/rfcs/text/0009_screenshot_mode_service.md +++ b/rfcs/text/0009_screenshot_mode_service.md @@ -4,22 +4,20 @@ # Summary -Applications should be aware when their UI is rendered for purposes of -capturing a screenshot. This ability would improve the quality of the Kibana -Reporting feature for a few reasons: - - Fewer objects in the headless browser memory since interactive code doesn't run - - Fewer Reporting bugs in releases since App teams have more ownership and - control over the reportability of their UI - Currently, the applications that support screenshot reports are: - Dashboard - Visualize Editor - Canvas +Kibana UI code should be aware when the page is rendering for the purpose of +capturing a screenshot. This ability would improve the quality of the Kibana +Reporting feature for a few reasons: + - Fewer objects in the headless browser memory since interactive code doesn't run + - Fewer API requests made by the headless browser for features that don't apply in a non-interactive context + **Screenshot mode service** -Applications in Kibana should be aware of when their rendering is for a human -user using the page interactively, and when it's not. At a high level, the application +At a high level, the application can support that awareness through a customized URL for screenshot reports. Canvas demonstrates a great implementation of this today. However, Canvas has dependencies on other UI plugins, and can not control how they render or what type of network requests @@ -29,11 +27,6 @@ a way to control themselves when the page is rendering for taking a screenshot. This RFC proposes a Screenshot Mode Service as a low-level plugin that allows other plugins (UI code) to make choices when the page is rendering for a screenshot. -In most cases, the information coming from this service would help the UI code -decide what should **not** be loaded: for example, the NewsFeed component and -Telemetry code should not load itself when the page is rendering for -screenshots. - More background on how Reporting currently works, including the lifecycle of creating a PNG report, is here: https://github.com/elastic/kibana/issues/59396 @@ -48,34 +41,23 @@ HTTP round trip, which weigh heavily on performance. # Motivation -Kibana Reporting is a commercial feature in Elastic and is highly capable of -loading the application pages and converting them into a screenshot for PNG or -PDF export. However, the way it works has downsides with performance, -maintainability, and expanding it as a tool that can power higher-level -features. The solution to those downsides is to have Kibana pages themselves -become more capable at presenting themselves for screenshot capture reports. -With the Screenshot Mode Service available, Reporting could drop the -task that it currently has which hurt performance: wasted rendering that is -replaced with custom styles that make the page "reportable." - -The technical advantage of having such as service also leads to making Kibana -application pages "printable", in the sense that sending the page to a printer -for a hard copy results in something more meaningful and specialized for paper -than today's Kibana can guarantee. This isn't a big concern for Kibana since -there isn't the expectation to improve printing Kibana, but that technical -direction is appropriate for improving PDF report generation. +The Reporting-enabled applications should use the recommended practice of +having a customized URL for Reporting, where UI features like navigation and +auto-complete disabled. The customized reporting URLs can solve any rendering +issue in a PDF or PNG, without needing extra CSS to be injected into the page. -# Detailed design +However, many low-level plugins have been added to the UI over time. These run +on every page and an application can not turn them off. Reporting performance +is negatively affected by this type of code. -The Screenshot Mode Service is a callable API available as dependency for -Kibana applications. +# Detailed design -A method of the API tells the Application whether or not it should render -itself to optimize for non-interactivity. +The plugin is low-level as it has no dependencies of its own, so other +low-level plugins can depend on it. -In a future phase, the API might also tell the application more about the -report, such as PDF page dimensions, or special layout types (print layout, -etc). +The Screenshot Mode Service is an entirely new plugin that has an API method +that returns a Boolean. The return value tells the plugin whether or not it +should render itself to optimize for non-interactivity. ## Interface The `setupDeps.screenshotMode` object has a single purpose: tell a low-level From b577f0985b7bb35e57d54ee85cb204611759b43c Mon Sep 17 00:00:00 2001 From: Timothy Sullivan Date: Wed, 24 Mar 2021 14:59:30 -0700 Subject: [PATCH 12/13] keep the discussion high level --- rfcs/text/0009_screenshot_mode_service.md | 72 ++++++++++++++--------- 1 file changed, 43 insertions(+), 29 deletions(-) diff --git a/rfcs/text/0009_screenshot_mode_service.md b/rfcs/text/0009_screenshot_mode_service.md index 38fc02f7cf6fd..4103e3f9396c1 100644 --- a/rfcs/text/0009_screenshot_mode_service.md +++ b/rfcs/text/0009_screenshot_mode_service.md @@ -10,19 +10,23 @@ Currently, the applications that support screenshot reports are: - Canvas Kibana UI code should be aware when the page is rendering for the purpose of -capturing a screenshot. This ability would improve the quality of the Kibana -Reporting feature for a few reasons: +capturing a screenshot. There should be a service to interact with low-level +code for providing that awareness. Reporting would interact with this service +to improve the quality of the Kibana Reporting feature for a few reasons: + - Fewer objects in the headless browser memory since interactive code doesn't run - Fewer API requests made by the headless browser for features that don't apply in a non-interactive context **Screenshot mode service** -At a high level, the application -can support that awareness through a customized URL for screenshot reports. Canvas -demonstrates a great implementation of this today. However, Canvas has dependencies -on other UI plugins, and can not control how they render or what type of network requests -they send. Those plugins, and lower-level plugins that run on every page, do not have -a way to control themselves when the page is rendering for taking a screenshot. +The Reporting-enabled applications should use the recommended practice of +having a customized URL for Reporting. The customized URL renders without UI +features like navigation, auto-complete, and anything else that wouldn't make +sense for non-interactive pages. + +However, applications are one piece of the UI code in a browser, and they have +dependencies on other UI plugins. Apps can't control plugins and other things +that Kibana loads in the browser. This RFC proposes a Screenshot Mode Service as a low-level plugin that allows other plugins (UI code) to make choices when the page is rendering for a screenshot. @@ -32,7 +36,7 @@ creating a PNG report, is here: https://github.com/elastic/kibana/issues/59396 # Basic example -When Kibana loads initially, there is a Newsfeed component in the UI that +When Kibana loads initially, there is a Newsfeed plugin in the UI that checks internally cached records to see if it must fetch the Elastic News Service for newer items. When the Screenshot Mode Service is implemented, the Newsfeed component has a source of information to check on whether or not it @@ -41,28 +45,38 @@ HTTP round trip, which weigh heavily on performance. # Motivation -The Reporting-enabled applications should use the recommended practice of -having a customized URL for Reporting, where UI features like navigation and -auto-complete disabled. The customized reporting URLs can solve any rendering -issue in a PDF or PNG, without needing extra CSS to be injected into the page. +The Reporting team wants all applications to support a customized URLs, such as +Canvas does with its `#/export/workpad/pdf/{workpadId}` UI route. The +customized URL is where an app can solve any rendering issue in a PDF or PNG, +without needing extra CSS to be injected into the page. However, many low-level plugins have been added to the UI over time. These run on every page and an application can not turn them off. Reporting performance -is negatively affected by this type of code. +is negatively affected by this type of code. When the Reporting team analyzes +customer logs to figure out why a job timed out, we sometimes see requests for +the newsfeed API and telemetry API: services that aren't needed during a +reporting job. -# Detailed design +In 7.12.0, using the customized `/export/workpad/pdf` in Canvas, the Sample +Data Flights workpad loads 163 requests. Most of thees requests don't come from +the app itself but from the application container code that Canvas can't turn +off. -The plugin is low-level as it has no dependencies of its own, so other -low-level plugins can depend on it. +# Detailed design The Screenshot Mode Service is an entirely new plugin that has an API method -that returns a Boolean. The return value tells the plugin whether or not it +that returns a Boolean. The return value tells the plugin whether or not it should render itself to optimize for non-interactivity. +The plugin is low-level as it has no dependencies of its own, so other +low-level plugins can depend on it. + ## Interface -The `setupDeps.screenshotMode` object has a single purpose: tell a low-level -plugin if the page is rendering in screenshot capture mode. The plugin decides -what to do with that information. +A plugin would depend on `screenshotMode` in kibana.json. That provides +`screenshotMode` as a plugin object. The plugin's purpose is to know when the +page is rendered for screenshot capture, and to interact with plugins through +an API. It allows plugins to decides what to do with the screenshot mode +information. ``` interface IScreenshotModeServiceSetup { @@ -70,8 +84,9 @@ interface IScreenshotModeServiceSetup { } ``` -Internally, this object is constructed from a class that refers to information -sent via a custom proprietary header: +The plugin knows the screenshot mode from request headers: this interface is +constructed from a class that refers to information sent via a custom +proprietary header: ``` interface HeaderData { @@ -84,12 +99,11 @@ class ScreenshotModeServiceSetup implements IScreenshotModeServiceSetup { } ``` -There can be a URL parameter which triggers screenshot mode to be enabled, for easier testing. - -This works because the headless browser that opens the page can inject custom -headers into the request. Teams can test how their app renders when loaded with -this header using a new configuration setting, or a web debugging proxy, or -some other tool that is TBD. +The Reporting headless browser that opens the page can inject custom headers +into the request. Teams should be able to test how their app renders when +loaded with this header. They could use a web debugging proxy, or perhaps the +new service should support a URL parameter which triggers screenshot mode to be +enabled, for easier testing. # Alternatives From f68c0d400011d2bff821f438738a1bc7d6179072 Mon Sep 17 00:00:00 2001 From: Timothy Sullivan Date: Wed, 24 Mar 2021 15:35:35 -0700 Subject: [PATCH 13/13] move a sectioin of text --- rfcs/text/0009_screenshot_mode_service.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/rfcs/text/0009_screenshot_mode_service.md b/rfcs/text/0009_screenshot_mode_service.md index 4103e3f9396c1..11ceae29b5f14 100644 --- a/rfcs/text/0009_screenshot_mode_service.md +++ b/rfcs/text/0009_screenshot_mode_service.md @@ -34,15 +34,6 @@ other plugins (UI code) to make choices when the page is rendering for a screens More background on how Reporting currently works, including the lifecycle of creating a PNG report, is here: https://github.com/elastic/kibana/issues/59396 -# Basic example - -When Kibana loads initially, there is a Newsfeed plugin in the UI that -checks internally cached records to see if it must fetch the Elastic News -Service for newer items. When the Screenshot Mode Service is implemented, the -Newsfeed component has a source of information to check on whether or not it -should load in the Kibana UI. If it can avoid loading, it avoids an unnecessary -HTTP round trip, which weigh heavily on performance. - # Motivation The Reporting team wants all applications to support a customized URLs, such as @@ -105,6 +96,15 @@ loaded with this header. They could use a web debugging proxy, or perhaps the new service should support a URL parameter which triggers screenshot mode to be enabled, for easier testing. +# Basic example + +When Kibana loads initially, there is a Newsfeed plugin in the UI that +checks internally cached records to see if it must fetch the Elastic News +Service for newer items. When the Screenshot Mode Service is implemented, the +Newsfeed component has a source of information to check on whether or not it +should load in the Kibana UI. If it can avoid loading, it avoids an unnecessary +HTTP round trip, which weigh heavily on performance. + # Alternatives - Print media query CSS