Release scripts for modules

[ptabor]

Release: Scripts to change versions in all go.mod files and push tags upstream.

Exemplar invocations:

Editing of go.mod files such that all etcd modules are pointing on given version (e.g. "v3.5.13"):

 DRY_RUN=false TARGET_VERSION="v3.5.13" ./scripts/release_mod.sh update_versions

Tagging latest commit with current version numbers for all the modules and pushing it upstreamt o origin:

% DRY_RUN=true REMOTE_REPO="origin" ./scripts/release_mod.sh push_mod_tags

The arbitrary choice I'm proposing in the PR is how we tag versions for module:

go.etcd.io/etcd/client/v2

This is v2 module stored in the repository and kept in-sync with v3 version hierarchy.
The v2 modules need to have version of v2.x.y . So I'm proposing to version them as v2.30x.y (e.g. v2.305.12) when the rest of code is versioned as v3.5.12. This creates very intuitive mapping and is future proof till etcd 3.99.y.

[ptabor]

% DRY_RUN=false TARGET_VERSION="v3.5.0-alpha.1" ./scripts/release_mod.sh update_versions
Making sure everything is committed.
DRY_RUN       : false
TARGET_VERSION: v3.5.0-alpha.1

v3version: v3.5.0-alpha.1
v2version: v2.305.0-alpha.1
% (cd api && 'go' 'list' '-f' '{{if not .Main}}{{if not .Indirect}}{{.Path}}{{end}}{{end}}' '-m' 'all')
% (cd pkg && 'go' 'list' '-f' '{{if not .Main}}{{if not .Indirect}}{{.Path}}{{end}}{{end}}' '-m' 'all')
% (cd raft && 'go' 'list' '-f' '{{if not .Main}}{{if not .Indirect}}{{.Path}}{{end}}{{end}}' '-m' 'all')
% (cd raft && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/pkg/[email protected]')
% (cd client/v2 && 'go' 'list' '-f' '{{if not .Main}}{{if not .Indirect}}{{.Path}}{{end}}{{end}}' '-m' 'all')
% (cd client/v2 && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/api/[email protected]')
% (cd client/v2 && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/pkg/[email protected]')
% (cd client/v3 && 'go' 'list' '-f' '{{if not .Main}}{{if not .Indirect}}{{.Path}}{{end}}{{end}}' '-m' 'all')
% (cd client/v3 && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/api/[email protected]')
% (cd client/v3 && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/pkg/[email protected]')
% (cd server && 'go' 'list' '-f' '{{if not .Main}}{{if not .Indirect}}{{.Path}}{{end}}{{end}}' '-m' 'all')
% (cd server && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/api/[email protected]')
% (cd server && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/client/[email protected]')
% (cd server && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/pkg/[email protected]')
% (cd server && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/raft/[email protected]')
% (cd server && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/client/[email protected]')
% (cd etcdctl && 'go' 'list' '-f' '{{if not .Main}}{{if not .Indirect}}{{.Path}}{{end}}{{end}}' '-m' 'all')
% (cd etcdctl && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/api/[email protected]')
% (cd etcdctl && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/client/[email protected]')
% (cd etcdctl && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/pkg/[email protected]')
% (cd etcdctl && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/raft/[email protected]')
% (cd etcdctl && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/server/[email protected]')
% (cd etcdctl && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/client/[email protected]')
% (cd tests && 'go' 'list' '-f' '{{if not .Main}}{{if not .Indirect}}{{.Path}}{{end}}{{end}}' '-m' 'all')
% (cd tests && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/api/[email protected]')
% (cd tests && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/client/[email protected]')
% (cd tests && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/etcdctl/[email protected]')
% (cd tests && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/pkg/[email protected]')
% (cd tests && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/raft/[email protected]')
% (cd tests && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/server/[email protected]')
% (cd tests && 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/client/[email protected]')
% 'go' 'list' '-f' '{{if not .Main}}{{if not .Indirect}}{{.Path}}{{end}}{{end}}' '-m' 'all'
% 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/api/[email protected]'
% 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/client/[email protected]'
% 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/pkg/[email protected]'
% 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/raft/[email protected]'
% 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/server/[email protected]'
% 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/tests/[email protected]'
% 'go' 'mod' 'edit' '-require' 'github.com/ptabor/etcd/client/[email protected]'


% git commit -a                                                                         
[release-3.5 59f7116bf] DRY_RUN=false TARGET_VERSION="v3.5.0-alpha.1" ./scripts/release_mod.sh update_versions
 7 files changed, 32 insertions(+), 32 deletions(-)



%  DRY_RUN=false REMOTE_REPO="ptabs" ./scripts/release_mod.sh push_mod_tags
Making sure everything is committed.
REMOTE_REPO:  ptabs
Tags for: github.com/ptabor/etcd/api/v3 version:v3.5.0-alpha.1 tag:api/v3.5.0-alpha.1
% 'git' 'tag' '-f' 'api/v3.5.0-alpha.1'
Tags for: github.com/ptabor/etcd/pkg/v3 version:v3.5.0-alpha.1 tag:pkg/v3.5.0-alpha.1
% 'git' 'tag' '-f' 'pkg/v3.5.0-alpha.1'
Tags for: github.com/ptabor/etcd/raft/v3 version:v3.5.0-alpha.1 tag:raft/v3.5.0-alpha.1
% 'git' 'tag' '-f' 'raft/v3.5.0-alpha.1'
Tags for: github.com/ptabor/etcd/client/v2 version:v2.305.0-alpha.1 tag:client/v2.305.0-alpha.1
% 'git' 'tag' '-f' 'client/v2.305.0-alpha.1'
Tags for: github.com/ptabor/etcd/client/v3 version:v3.5.0-alpha.1 tag:client/v3.5.0-alpha.1
% 'git' 'tag' '-f' 'client/v3.5.0-alpha.1'
Tags for: github.com/ptabor/etcd/server/v3 version:v3.5.0-alpha.1 tag:server/v3.5.0-alpha.1
% 'git' 'tag' '-f' 'server/v3.5.0-alpha.1'
Tags for: github.com/ptabor/etcd/etcdctl/v3 version:v3.5.0-alpha.1 tag:etcdctl/v3.5.0-alpha.1
% 'git' 'tag' '-f' 'etcdctl/v3.5.0-alpha.1'
Tags for: github.com/ptabor/etcd/tests/v3 version:v3.5.0-alpha.1 tag:tests/v3.5.0-alpha.1
% 'git' 'tag' '-f' 'tests/v3.5.0-alpha.1'
Tags for: github.com/ptabor/etcd/v3 version:v3.5.0-alpha.1 tag:v3.5.0-alpha.1
% 'git' 'tag' '-f' 'v3.5.0-alpha.1'
% 'git' 'push' '-f' 'ptabs' 'api/v3.5.0-alpha.1' 'pkg/v3.5.0-alpha.1' 'raft/v3.5.0-alpha.1' 'client/v2.305.0-alpha.1' 'client/v3.5.0-alpha.1' 'server/v3.5.0-alpha.1' 'etcdctl/v3.5.0-alpha.1' 'tests/v3.5.0-alpha.1' 'v3.5.0-alpha.1'

stderr: To github.com:ptabor/etcd.git
stderr: * [new tag]             api/v3.5.0-alpha.1 -> api/v3.5.0-alpha.1
stderr: * [new tag]             pkg/v3.5.0-alpha.1 -> pkg/v3.5.0-alpha.1
stderr: * [new tag]             raft/v3.5.0-alpha.1 -> raft/v3.5.0-alpha.1
stderr: * [new tag]             client/v2.305.0-alpha.1 -> client/v2.305.0-alpha.1
stderr: * [new tag]             client/v3.5.0-alpha.1 -> client/v3.5.0-alpha.1
stderr: * [new tag]             server/v3.5.0-alpha.1 -> server/v3.5.0-alpha.1
stderr: * [new tag]             etcdctl/v3.5.0-alpha.1 -> etcdctl/v3.5.0-alpha.1
stderr: * [new tag]             tests/v3.5.0-alpha.1 -> tests/v3.5.0-alpha.1
stderr: * [new tag]             v3.5.0-alpha.1 -> v3.5.0-alpha.1                                                                                                                                                                              

% mkdir /tmp/exmod
% cd /tmp/exmod
% go mod init exmod

%go list -m --versions github.com/ptabor/etcd/client/v3
github.com/ptabor/etcd/client/v3 v3.5.0-alpha.0 v3.5.0-alpha.1

% go get github.com/ptabor/etcd/client/v3               
go: downloading github.com/ptabor/etcd/client/v3 v3.5.0-alpha.1
go: github.com/ptabor/etcd/client/v3 upgrade => v3.5.0-alpha.1

go: downloading github.com/ptabor/etcd/pkg/v3 v3.5.0-alpha.1
go: downloading github.com/ptabor/etcd/api/v3 v3.5.0-alpha.1

% go get github.com/ptabor/etcd/client/v2
go: downloading github.com/ptabor/etcd/client/v2 v2.305.0-alpha.1
go: github.com/ptabor/etcd/client/v2 upgrade => v2.305.0-alpha.1

% cat go.mod 
module exmod

go 1.15

require (
	github.com/ptabor/etcd/client/v2 v2.305.0-alpha.1 // indirect
	github.com/ptabor/etcd/client/v3 v3.5.0-alpha.1 // indirect
)


% GOPROXY="direct" go get github.com/ptabor/etcd/client/[email protected]
go: downloading github.com/ptabor/etcd/client/v3 v3.5.0-alpha.1.0.20201104164215-9faeda3c1690

% cat go.mod 
module exmod

go 1.15

require github.com/ptabor/etcd/client/v3 v3.5.0-alpha.1.0.20201104164215-9faeda3c1690 // indirect

[ptabor]

Could you please review / consider merging. Thank you.

[jingyih]

Actually I am not super familiar with the release process. The release was mostly done by @gyuho and sometimes @jpbetz. I believe the script used for previous releases is https://github.com/etcd-io/etcd/blob/master/scripts/release. As I am reading through the existing script, it looks like there are some overlaps between the existing one and the one you are adding. Is it possible to merge the new script into the existing one? (Am I making sense?)

[ptabor]

I'm definitely planning to integrate them. Although I think that tar-ball release we will need in O(1 quarter) when we are ready to ship 3.5.0-rc version, so there is some time for doing the integration properly. For now I'm proposing following step-by-step plan:

1. submit the script
2. tag the repository as e.g. 3.5.0-alpha.0 to resolve issues like: go get is failing #12412
   (potentially after branching release-3.5 as @gyuho suggested on the last community meeting,
   I don't have opinion whether its worth to branch now and keep merging from master till 3.5 release,
   or for now just tag the master branch)
3. publish documentation with examples using version 3.5.0-alpha.0
4. perform the full release script integration (knowing my-self it might by substantial PR on its own)

[gyuho]

@ptabor Thanks for the great initiatives. Should we create a separate issue to track the release process?

[sagikazarmark]

Should we create a separate issue to track the release process?

I think that'd be great. People are eagerly waiting for the release and right now there is not a lot of information about it.

[ptabor]

The issue tracking release: #12498