Created EmptyQuickstart command #8024
Conversation
| Thread.sleep(5000); | ||
| } | ||
|
|
||
| public static void printStatus(Color color, String message) { |
There was a problem hiding this comment.
this should go to QuickStartBase.
| return null; | ||
| } | ||
|
|
||
| protected void waitForBootstrapToComplete(QuickstartRunner runner) |
There was a problem hiding this comment.
Fixed in all referencing quickstarts
Codecov Report
@@ Coverage Diff @@
## master #8024 +/- ##
============================================
- Coverage 71.39% 68.12% -3.28%
+ Complexity 4222 4142 -80
============================================
Files 1597 1202 -395
Lines 82903 60120 -22783
Branches 12369 9267 -3102
============================================
- Hits 59189 40956 -18233
+ Misses 19707 16271 -3436
+ Partials 4007 2893 -1114
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
| @@ -75,6 +75,8 @@ | |||
| private final boolean _enableTenantIsolation; | |||
| private final String _authToken; | |||
| private final Map<String, Object> _configOverrides; | |||
| private final String _zkAddress; | |||
There was a problem hiding this comment.
call it _externalZkAddress? And add comment that if this is set, Quickstart won't start an embedded zk.
| @@ -93,9 +95,31 @@ public QuickstartRunner(List<QuickstartTableRequest> tableRequests, int numContr | |||
| _enableTenantIsolation = enableIsolation; | |||
| _authToken = authToken; | |||
| _configOverrides = configOverrides; | |||
| _zkAddress = null; | |||
| _deleteExistingData = true; | |||
There was a problem hiding this comment.
this should be added as a command-line parameter for QuickStartCommand?
There was a problem hiding this comment.
I've refactored the constructor to make this simpler. For deleting existing data, if the flag is set to false for any of the other quickstart commands, there will be an exception on startup when attempting to create a data directory that already exists. I've added _deleteExistingData as a constructor argument for future quickstart commands.
|
This is going to be super useful. I am wondering if we can have the option to set certain parameters as well to make debugging using Quickstart easier: CommonConstants.java:
Also, if possible:
|
Yes, let me look into this. We're discussing adding configuration overrides to QuickStarts as another PR. We're still sorting out the best design philosophy for this, as configuration keys are applied to separate components. I think ideally we would be able to provide an easier zero-downtime configuration manager, such as using a config server with auto-refresh scope. This would allow a user to use a git repository as a way to version control and manage configurations for each of the different components. Any thoughts on this @amrishlal? |
We should add support for quickstart to take extra configs or a config file to override existing configs. E.g. enabling grpc or auth for quick test. |
* Added empty quickstart command. * Quickstart fixes per reviewer feedback. * Fixed typo in QuickStartCommand * Fixing bug with minion quickstart
This is more generalized (and probably more useful) approach than what I had in mind, but I was mainly thinking about something very simple like specifying a command line parameter ("-debug" or "-dev") that would increase the value of |
Added in #8059 |
Description
Created a new Pinot quickstart command called
EmptyQuickstart.The entire point of this command is that we previously had no quickstart that would save your data after a restart. This will be used to update the Homebrew formula so that users can run Pinot as a single component local service that will save your data/tables/schemas/segments after a restart. It can also be used for Docker when specifying a host volume data directory.
See the Documentation section below for usage notes.
Upgrade Notes
Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion)
backward-incompat, and complete the section below on Release Notes)Does this PR fix a zero-downtime upgrade introduced earlier?
backward-incompat, and complete the section below on Release Notes)Does this PR otherwise need attention when creating release notes? Things to consider:
release-notesand complete the section on Release Notes)Documentation
To launch a single component installation of Pinot that saves data after a restart, you can use the
EmptyQuickstartcommand.-zkAddress).-- If no argument is provided, an embedded Zookeeper instance will be launched.
-- When an argument is provided, no embedded Zookeeper instance will be launched and controller/broker/server will use the provided ZK URL.
-dataDir).-- If no argument is provided, a temporary data directory will be generated.
-- When an argument is provided, embedded Zookeeper files are stored and reused on restart.
-- When an argument is provided, Pinot controller/broker/server data files are stored and reused on restart.
To start the quickstart command, run the following in your terminal.