could athens support mirror?

[daqingshu]

Is your feature request related to a problem? Please describe.
a simple way to replace in athens, not in go.mod
batch config in athens

Describe the solution you'd like
Mirrors = [
"golang.org/x/(.*) => github.com/golang/$1",
"google.golang.org/appengine => github.com/golang/appengine",
"google.golang.org/genproto => github.com/google/go-genproto",
"google.golang.org/grpc => github.com/grpc/grpc-go",
"google.golang.org/api => github.com/googleapis/google-api-go-client",
"cloud.google.com/go => github.com/googleapis/google-cloud-go",
]

Describe alternatives you've considered
if impl..ed many project just simple write go.mod file

Additional context
NA

[arschles]

@daqingshu do you have a use case for this?

[daqingshu]

@daqingshu do you have a use case for this?
thank you a lot.
in case of inside the enterprise, there is strict network control, only some people can directly access github, we want create some mirror repos, sync them every week, then others can use the athens with your great idea "alias" feature get the repos from mirrors

[blueworrybear]

I am also interesting in this idea. In my department, we host private repository with Gitea. I had to initial my go module as domain/user/repo.git instead of domain/user/repo to make go mod recognize it. It would be great if we could alias domain/user/repo as domain/user/repo.git with Athens!

Is there anyone working on this kind of feature? I would like to involve in. 😄

[ferryvg]

Yep, it's needed! Dep is support it by name option in constraint

[arschles]

Is there anyone working on this kind of feature? I would like to involve in. 😄

@blueworrybear not that I know of. if you have ideas, feel free to write them in here!

@ferryvg @daqingshu behind the scenes, Athens does a go mod download when it is asked for a module that it doesn't currently have in storage. If Athens is configured with "google.golang.org/appengine => github.com/golang/appengine", for example, I have two questions:

• Would it need to do a go mod download google.golang.org/appengine behind the scenes?
• Are you aiming to be able to do a go get github.com/golang/appengine?

[daqingshu]

@arschles in my opinion, Athens is a transparent proxy, wont change 'go mod download' s result, user can get same result as no proxy also include error.
so

1. if user exec 'go mod download google.golang.org/appengine' can download it to $GOPATH/pkg/mod/google.golang.org/appengine/v1.x.y
2. user exec 'go get github.com/golang/appengine' will got a error.

[arschles]

@daqingshu I'm still not sure if we can make this happen technically. If the user executes go mod download google.golang.org/appengine, then we tell Athens to translate that to github.com/golang/appengine. Athens will then run go mod download github.com/golang/appengine, and that currently fails on my machine. Here is the output of the go mod download and also the go get output:

Aarons-Mac-mini :: ~/Desktop/modtest » go mod download github.com/golang/appengine
module github.com/golang/appengine: not a known dependency
Aarons-Mac-mini :: ~/Desktop/modtest » go get github.com/golang/appengine                                                                    1 ↵
go: finding github.com/golang/appengine v1.6.5
go: downloading github.com/golang/appengine v1.6.5
go: extracting github.com/golang/appengine v1.6.5
go get: github.com/golang/[email protected]: parsing go.mod:
	module declares its path as: google.golang.org/appengine
	        but was required as: github.com/golang/appengine

The problem lies in the go.mod file that declares module google.golang.org/appengine. That tells the Go tool to only allow the module to be fetched with that name.

Do you know of a way around that restriction? If you do, I think we can make this work, and I would love to do it 😄

[daqingshu]

go get github.com/golang/appengine

go mod download should with version, like:

go mod download github.com/golang/appengine@latest
go mod download github.com/golang/[email protected]

go get github.com/golang/appengine got error I think this is correct, because go get is legacy mechanism, and wont through GOPROXY

[arschles]

@daqingshu I see! So, Athens can do the download on the backend, and then when it serves a request for the github.com/golang/appengine module to a client, would the client be able to use that module, even though the go.mod says the module name is not github.com/golang/appengine?

[daqingshu]

@daqingshu I see! So, Athens can do the download on the backend, and then when it serves a request for the github.com/golang/appengine module to a client, would the client be able to use that module, even though the go.mod says the module name is not github.com/golang/appengine?

my opnion, the config effect athens, user side should not perceptive other
with GOPROXY, user now can download package
without GOPROXY, user got network error

example:
if user's go.mod use google.golang.org/appengine, it's OK
if use github.com/golang/appengine, not OK, he must add 'replace'

[arschles]

I don't think that is going to be possible because of the way that the Go client works. Whatever GOPROXY is set to, if you run a go get my/module, that module's go.mod file needs to have module my/module in it.

If Athens supported server-side replace directives, the user could run go get github.com/golang/appengine would return the go.mod file with google.golang.org/appengine in it. So they would need the replace directive anyway and we would be back to the beginning. What do you think?

[daqingshu]

I think Athens won't supported server-side replace directives, just transparent proxy.
like:
user's go.mod config "github.com/golang/appengine v1.6.5"
server side config
alias github.com git.corp.com

then user compile, actually the cmd is go mod download
go mod download github.com/golang/[email protected]

this cmd to athens server-side
athens just download from git.corp.com/golang/[email protected]
then transfer the content to user side

let me see, did you said the impossible point is 'athens just download from git.corp.com/golang/[email protected]' ?

[arschles]

@daqingshu sorry for the delay.

yes, the difficult point is having Athens download from git.corp.com/golang/[email protected] - that is because the go.mod file doesn't match up.

Athens uses go get to fetch modules at the moment. We could download code on our own, and avoid this problem. I would like to keep that code outside of Athens, but we can certainly build it, and have Athens interact with it via an API.

What do you think about that?

[daqingshu]

@daqingshu sorry for the delay.

yes, the difficult point is having Athens download from git.corp.com/golang/[email protected] - that is because the go.mod file doesn't match up.

Athens uses go get to fetch modules at the moment. We could download code on our own, and avoid this problem. I would like to keep that code outside of Athens, but we can certainly build it, and have Athens interact with it via an API.

What do you think about that?

Thank you. I agree your opinion.
and could you please give the API declaration, so that volunteers(include me) could contribute implement code.

[marwan-at-work]

From what I understand, you have a git server at mycompany.com/mymodule.git and you'd like to declare its import path as something like mycompany.com/mymodule or really any other name such as mymodule.mycompany.com -- is that assumption correct?

If so, wouldn't a vanity import server solve your use case?