Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

migrate vtorc to use cobra commands #13917

Merged
merged 7 commits into from
Sep 6, 2023
Merged

migrate vtorc to use cobra commands #13917

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
569d3e1
migrate vtorc to use cobra commands
Sep 5, 2023
7b55653
update testdata
Sep 5, 2023
920d9e7
remove unused shim code
Sep 5, 2023
2805a22
refactor to support generated docs
Sep 5, 2023
983297c
Update go/cmd/vtorc/cli/cli.go
frouioui Sep 5, 2023
df67dd1
Update go/cmd/vtorc/docgen/main.go
frouioui Sep 5, 2023
687ccfd
move plugin imports so they are present when we generate website docs
Sep 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 104 additions & 0 deletions go/cmd/vtorc/cli/cli.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
/*
Copyright 2023 The Vitess Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package cli
frouioui marked this conversation as resolved.
Show resolved Hide resolved

import (
"fmt"

"github.com/spf13/cobra"

"vitess.io/vitess/go/acl"
_flag "vitess.io/vitess/go/internal/flag"
"vitess.io/vitess/go/viperutil"
viperdebug "vitess.io/vitess/go/viperutil/debug"
"vitess.io/vitess/go/vt/log"
"vitess.io/vitess/go/vt/servenv"
"vitess.io/vitess/go/vt/vtorc/config"
"vitess.io/vitess/go/vt/vtorc/inst"
"vitess.io/vitess/go/vt/vtorc/logic"
"vitess.io/vitess/go/vt/vtorc/server"
)

var (
configFile string
Main = &cobra.Command{
Use: "vtorc",
Short: "", // TODO
Args: cobra.NoArgs,
Version: servenv.AppVersion.String(),
PreRunE: func(cmd *cobra.Command, args []string) error {
_flag.TrickGlog()

watchCancel, err := viperutil.LoadConfig()
if err != nil {
return fmt.Errorf("%s: failed to read in config: %s", cmd.Name(), err)
}

servenv.OnTerm(watchCancel)
servenv.HTTPHandleFunc("/debug/config", viperdebug.HandlerFunc)
return nil
},
Run: run,
}
)

func run(cmd *cobra.Command, args []string) {
servenv.Init()
config.UpdateConfigValuesFromFlags()
inst.RegisterStats()

log.Info("starting vtorc")
if len(configFile) > 0 {
config.ForceRead(configFile)
} else {
config.Read("/etc/vtorc.conf.json", "conf/vtorc.conf.json", "vtorc.conf.json")
}
if config.Config.AuditToSyslog {
inst.EnableAuditSyslog()
}
config.MarkConfigurationLoaded()

// Log final config values to debug if something goes wrong.
config.LogConfigValues()
server.StartVTOrcDiscovery()

server.RegisterVTOrcAPIEndpoints()
servenv.OnRun(func() {
addStatusParts()
})

// For backward compatability, we require that VTOrc functions even when the --port flag is not provided.
// In this case, it should function like before but without the servenv pages.
// Therefore, currently we don't check for the --port flag to be necessary, but release 16+ that check
// can be added to always have the serenv page running in VTOrc.
servenv.RunDefault()
}

// addStatusParts adds UI parts to the /debug/status page of VTOrc
func addStatusParts() {
servenv.AddStatusPart("Recent Recoveries", logic.TopologyRecoveriesTemplate, func() any {
recoveries, _ := logic.ReadRecentRecoveries(false, 0)
return recoveries
})
}

func init() {
logic.RegisterFlags(Main.Flags())
server.RegisterFlags(Main.Flags())
config.RegisterFlags(Main.Flags())
acl.RegisterFlags(Main.Flags())
Main.Flags().StringVar(&configFile, "config", "", "config file name")
}
36 changes: 36 additions & 0 deletions go/cmd/vtorc/docgen/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
/*
Copyright 2023 The Vitess Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
frouioui marked this conversation as resolved.
Show resolved Hide resolved

import (
"github.com/spf13/cobra"

"vitess.io/vitess/go/cmd/internal/docgen"
"vitess.io/vitess/go/cmd/vtorc/cli"
)

func main() {
var dir string
cmd := cobra.Command{
Use: "docgen [-d <dir>]",
Comment on lines +26 to +29
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the current plan for documentation generation? This program seems very specific to only vtorc, should we have a top level generation tool that generates the docs for all binaries?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

each binary currently needs a docgen directory (primarly because you need to import that command's root cobra.Command which is hard to parameterize). We can look into codegen with a template if it becomes too laborious, but right now I think it's going to be a set-and-forget situation (write this directory once, then further changes to the actual binary don't require any changes here).

You can see how this is being used (also for vtctldclient) here: vitessio/website#1576

RunE: func(cmd *cobra.Command, args []string) error {
return docgen.GenerateMarkdownTree(cli.Main, dir)
},
}

cmd.Flags().StringVarP(&dir, "dir", "d", "doc", "output directory to write documentation")
_ = cmd.Execute()
}
114 changes: 14 additions & 100 deletions go/cmd/vtorc/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,120 +17,34 @@
package main

import (
"strings"
"flag"

_ "github.com/go-sql-driver/mysql"
"github.com/spf13/pflag"
_ "modernc.org/sqlite"

"vitess.io/vitess/go/acl"
"vitess.io/vitess/go/cmd/vtorc/cli"
_flag "vitess.io/vitess/go/internal/flag"
"vitess.io/vitess/go/vt/log"
"vitess.io/vitess/go/vt/servenv"
"vitess.io/vitess/go/vt/vtorc/config"
"vitess.io/vitess/go/vt/vtorc/inst"
"vitess.io/vitess/go/vt/vtorc/logic"
"vitess.io/vitess/go/vt/vtorc/server"
)

// transformArgsForPflag turns a slice of raw args passed on the command line,
// possibly incompatible with pflag (because the user is expecting stdlib flag
// parsing behavior) and transforms them into the arguments that should have
// been passed to conform to pflag parsing behavior.
//
// the primary function is to catch any cases where the user specified a longopt
// with only a single hyphen (e.g. `-myflag`) and correct it to be
// double-hyphenated.
//
// note that this transformation does _not_ actually validate the arguments; for
// example if the user specifies `--myflag`, but the FlagSet has no such flag
// defined, that will still appear in the returned result and will (correctly)
// cause a parse error later on in `main`, at which point the CLI usage will
// be printed.
//
// note also that this transformation is incomplete. pflag allows interspersing
// of flag and positional arguments, whereas stdlib flag does not. however, for
// vtorc specifically, with the exception of `vtorc help <topic>`, the CLI only
// consumes flag arguments (in other words, there are no supported subcommands),
// so this is a non-issue, and is not implemented here in order to make this
// function a bit simpler.
func transformArgsForPflag(fs *pflag.FlagSet, args []string) (result []string) {
for i, arg := range args {
switch {
case arg == "--":
// pflag stops parsing at `--`, so we're done transforming the CLI
// arguments. Just append everything remaining and be done.
result = append(result, args[i:]...)
return result
case strings.HasPrefix(arg, "--"):
// Long-hand flag. Append it and continue.
result = append(result, arg)
case strings.HasPrefix(arg, "-"):
// Most complex case. This is either:
// 1. A legacy long-hand flag that needs a double-dash (e.g. `-myflag` => `--myflag`).
// 2. One _or more_ pflag shortopts all shoved together (think `rm -rf` as `rm -r -f`).
//
// In the latter case, we don't need to do any transformations, but
// in the former, we do.
name := strings.SplitN(arg[1:], "=", 2)[0] // discard any potential value (`-myflag` and `-myflag=10` both have the name of `myflag`)
if fs.Lookup(name) != nil || name == "help" {
// Case 1: We have a long opt with this name, so we need to
// prepend an additional hyphen.
result = append(result, "-"+arg)
} else {
// Case 2: No transformation needed.
result = append(result, arg)
}
default:
// Just a flag argument. Nothing to transform.
result = append(result, arg)
}
}

return result
}

// main is the application's entry point. It will spawn an HTTP interface.
func main() {
servenv.RegisterDefaultFlags()
servenv.RegisterFlags()

var configFile string
servenv.OnParseFor("vtorc", func(fs *pflag.FlagSet) {
logic.RegisterFlags(fs)
server.RegisterFlags(fs)
config.RegisterFlags(fs)
acl.RegisterFlags(fs)
cli.Main.Flags().AddFlagSet(servenv.GetFlagSetFor("vtorc"))

fs.StringVar(&configFile, "config", "", "config file name")
})
servenv.ParseFlags("vtorc")
servenv.Init()
config.UpdateConfigValuesFromFlags()
inst.RegisterStats()
// glog flags, no better way to do this
_flag.PreventGlogVFlagFromClobberingVersionFlagShorthand(cli.Main.Flags())
cli.Main.Flags().AddGoFlag(flag.Lookup("logtostderr"))
cli.Main.Flags().AddGoFlag(flag.Lookup("alsologtostderr"))
cli.Main.Flags().AddGoFlag(flag.Lookup("stderrthreshold"))
cli.Main.Flags().AddGoFlag(flag.Lookup("log_dir"))

log.Info("starting vtorc")
if len(configFile) > 0 {
config.ForceRead(configFile)
} else {
config.Read("/etc/vtorc.conf.json", "conf/vtorc.conf.json", "vtorc.conf.json")
}
if config.Config.AuditToSyslog {
inst.EnableAuditSyslog()
}
config.MarkConfigurationLoaded()

// Log final config values to debug if something goes wrong.
config.LogConfigValues()
server.StartVTOrcDiscovery()
// TODO: viperutil.BindFlags()

server.RegisterVTOrcAPIEndpoints()
servenv.OnRun(func() {
addStatusParts()
})

// For backward compatability, we require that VTOrc functions even when the --port flag is not provided.
// In this case, it should function like before but without the servenv pages.
// Therefore, currently we don't check for the --port flag to be necessary, but release 16+ that check
// can be added to always have the serenv page running in VTOrc.
servenv.RunDefault()
if err := cli.Main.Execute(); err != nil {
log.Exit(err)
}
}
48 changes: 0 additions & 48 deletions go/cmd/vtorc/main_test.go
View file
Open in desktop

This file was deleted.

30 changes: 0 additions & 30 deletions go/cmd/vtorc/status.go
View file
Open in desktop

This file was deleted.

9 changes: 5 additions & 4 deletions go/flags/endtoend/vtorc.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
Usage of vtorc:
Usage:
vtorc [flags]

Flags:
--allow-emergency-reparent Whether VTOrc should be allowed to run emergency reparent operation when it detects a dead primary (default true)
--alsologtostderr log to standard error as well as files
--audit-file-location string File location where the audit logs are to be stored
Expand All @@ -25,13 +28,12 @@ Usage of vtorc:
--grpc_keepalive_timeout duration After having pinged for keepalive check, the client waits for a duration of Timeout and if no activity is seen even after that the connection is closed. (default 10s)
--grpc_max_message_size int Maximum allowed RPC message size. Larger messages will be rejected by gRPC with the error 'exceeding the max size'. (default 16777216)
--grpc_prometheus Enable gRPC monitoring with Prometheus.
-h, --help display usage and exit
-h, --help help for vtorc
--instance-poll-time duration Timer duration on which VTOrc refreshes MySQL information (default 5s)
--keep_logs duration keep logs for this long (using ctime) (zero to keep forever)
--keep_logs_by_mtime duration keep logs for this long (using mtime) (zero to keep forever)
--lameduck-period duration keep running at least this long after SIGTERM before stopping (default 50ms)
--lock-timeout duration Maximum time for which a shard/keyspace lock can be acquired for (default 45s)
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
--log_dir string If non-empty, write log files in this directory
--log_err_stacks log stack traces for errors
--log_rotate_max_size uint size in bytes at which logs are rotated (glog.MaxSize) (default 1887436800)
Expand Down Expand Up @@ -87,5 +89,4 @@ Usage of vtorc:
--topo_zk_tls_key string the key to use to connect to the zk topo server, enables TLS
--v Level log level for V logs
-v, --version print binary version
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
--wait-replicas-timeout duration Duration for which to wait for replica's to respond when issuing RPCs (default 30s)
Loading