Add a configuration option to only run a specific plugin and its dependencies-Focus mode

[joshdover]

Most developers only work within a single plugin when working on Kibana. However, it is not easy to configure Kibana to only run a single plugin, so most developers run all the plugins at once. This results in a slower development environment.

We should add a config option that allows for this usecase. For example, a --run-only=apm CLI flag would only start the APM plugin + any plugin it depends on. This setting should impact both the runtime as well as the @kbn/optimizer to avoid building bundles that are not going to be used.

In some cases it may also be helpful to run the plugins that depend on your plugin. For instance, maybe I want to work on the embeddables plugin along with any plugins that implement an embeddable. We could add an additional config option like --include-dependents which also run any plugins that list embeddable as a required or optional dependency.

It may make sense to expand this as a feature to users at some point as well, but to start we should probably only allow this in dev mode so that we can ensure all of the dependencies are correctly defined.

[elasticmachine]

Pinging @elastic/kibana-platform (Team:Platform)

[elasticmachine]

Pinging @elastic/kibana-operations (Team:Operations)

[spalger]

Would be happy to support this "focus mode" in @kbn/optimizer, we'll need to add dependency awareness for #64843 anyways.

[lizozom]

Just realized I needed this yesterday. :)

[mshustov]

However, it is not easy to configure Kibana to only run a single plugin, so most developers run all the plugins at once. This results in a slower development environment.

Will this problem be so significant after we ban asynchronous cycles? Or the plugin build time is the main concern?

From #66621 (comment)

For "focus mode" specifically, I would want to ensure that security and spaces are both enabled here, even if not explicitly depended on. Disabling these plugins at dev time will make it too easy for engineers to ignore the complexities of the security model, and/or multi-space setups.

I'm a bit worried about the necessity to maintain such a list manually. It's easy to forget or miss such an implicit dependency and a result to get different behavior in prod & dev modes. I'd say it's a valid counter-argument not to add this functionality.

[joshdover]

Will this problem be so significant after we ban asynchronous cycles? Or the plugin build time is the main concern?

Build time is the main one, but also load/parse/compile time of the client-side JS bundles. Build time could be solved in other ways. In my experience, it seems that the bundle cache is often invalidated due to the package.json changes. If we could improve the caching to be intelligent enough not to bust the cache if no changes were made to a plugin's dependencies, we may be able to avoid the frequent cache busts.

I'm a bit worried about the necessity to maintain such a list manually. It's easy to forget or miss such an implicit dependency and a result to get different behavior in prod & dev modes. I'd say it's a valid counter-argument not to add this functionality.

I agree this is a risk, but I also think we should think about the tradeoff in value here. It may require experimenting first with a subset of developers, but I suspect that the speed / dev XP gains may outweigh the risks here and if anything may free up more time for thorough testing.

That said, we should continue to eliminate the possibilities for implicit dependencies. Enforcing that cross-plugin imports respect the kibana.json dependencies would be a great first step. I don't think we can completely eliminate the possibility but if we can eliminate 80%, that may be good enough.

[mshustov]

but also load/parse/compile time of the client-side JS bundles

that's almost negligible comparing to the build time.

Build time could be solved in other ways... If we could improve the caching to be intelligent

Yeah, I didn't mean any particular solution until we identify a problem to solve. Probably, we need to focus on improving the build time? @elastic/kibana-operations

That said, we should continue to eliminate the possibilities for implicit dependencies

To note: most of the plugins do not depend on Security nor Spaces directly. However, I agree that the number of plugins affecting the whole Kibana runtime is known in advance.

[smith]

I have a branch where I've been working on a hack script that does some of this by outputting yaml that you can paste into your config: master...smith:nls/kill-all-plugins

[dgieselaar]

I'm not sure if this is feasible, but if this could be done via kibana.yml, that would be incredibly useful in terms of ease-of-use (esp if we at some point start scaling Kibana for specific workloads, e.g. 3 Kibana Alerting instances)

[sorenlouv]

that's almost negligible comparing to the build time.

I've been faithfully disabling plugins in the past to squeeze more perf out of Kibana, and I had a feeling that it made it faster but wanted to make sure and did a benchmark just now:

Only APM related plugins: 89.4mb / 6.1 s
All plugins: 169mb / 9.9s

Savings: 79.6mb / 3.8s

Thus, running the trimmed down version gives a speed improvement of ~40% - that's not bad for a frontend dev who refreshes many times a day. Multiply that with the number of people working on Kibana daily and you have a pretty big time saver :)

[mshustov]

that's almost negligible comparing to the build time.

Build time ~5min after bootstrap. 3-5 sec. is nothing comparing to this value.

Savings: 79.6mb / 3.8s

Kibana should be fast by default. Our user might be not a technical person and might not know how to squeeze another second from the loading time. To make it possible Kibana developers should work toward optimizing loading & initialization time: to load code lazily, to minimize dependencies, don't block execution in lifecycles, etc. If every developer disables all the plugins, they don't understand how Kibana works as a whole, what experience the end-users have.

Just to note: it's not a big deal to add this feature, but I want to make sure we understand all the potential problems. It's okay a temporary solution, but in long term it will hurt more.

[sorenlouv]

Build time ~5min after bootstrap. 3-5 sec. is nothing comparing to this value.

Agree, build time is also a problem. But in my experience many ui developers refresh a lot more than they restart the stack. I wouldn't be surprised if the ratio was more than 1:100.

So while I'd LOVE to see the build time reduced, I think that reducing a repetitive operation (a hot path you might call it 🙂) by a small amount is equally important.

Kibana developers should work toward optimizing loading & initialization time: to load code lazily, to minimize dependencies, don't block execution in lifecycles, etc

Agree, we should! And I can see the dilemma of chasing a local optimum instead of a global. However, I think getting to the perfect solution is going to be a longer journey, and if we can get this improvement today I'd take it.

[smith]

I did some benchmarking for the use case of working on an API endpoint and
waiting for server restarts.

These numbers are from putting console.time in the constructor of
src/core/server/http/http_server.ts and timeEnd at the end of its start method,
then saving that file with Kibana running.

It includes the startup time but not the shutdown time (which is usually around
5-8s on this machine.)

SHA used: 951c0f6. Running on a 2019 MBP 2.4 GHz 8-Core Intel Core i9 w/ 32G RAM

Restarting - all plugins enabled

13659.246
15697.114
15295.798
16061.203
15881.497
14787.836
14172.419
14664.298
16390.889
15627.312

Average 15223.76

Restarting - with everything but APM and its transitive dependencies and base required plugins enabled

12132.613
14111.772
11919.642
14193.937
11616.219
13317.972
11599.760
12382.730
12031.357
14777.742

Average 12808.37

Restarting with all (possible) plugins disabled

13656.353
12069.611
12908.988
17548.867
14255.815
11882.885
12719.409
13057.826
14227.450
15873.836

Average 13820.10

Conclusion

The speedup from disabling plugins is apparent, but 2-3 seconds is not enough to
really matter in the development workflow.

We should be working on finding the bottlenecks in the whole system and attacking
those rather than focusing on this IMO.

I'd be OK with this feature being added, but I'm not working on the branch I
started any more since the improvements seen won't really solve my problem.

[kobelb]

While I agree that this approach could be utilized short-term to improve developer productivity, it comes with a rather large amount of risk. In the worst-case scenario, this approach could lead to developers writing new features without taking into consideration the ramifications that this feature has on security, spaces, or any other platform service that isn't required as a direct dependency.

While we could have a static-list of plugins that shouldn't be disabled in focus mode, this is an incredibly fragile solution that will require constant maintenance. Plugins interact with each other in very interesting ways, and the responsibilities that a single plugin has is constantly evolving.

While some of this risk can be mitigated by writing automated tests that run against a production version of Kibana, our current test coverage does not mitigate this risk in actuality.

[joshdover]

Thanks all for the thoughts on this idea. At this point, I am going to close this issue. Many of the issues this was designed to solve are now either solved by the bundle size reductions we've done or will be solved by the new Bazel build system.