Git Fetch does not work with Azure DevOps repos

[jessesuen]

We use go-git in our K8s GitOps Deployment tool and users are reporting issues with Azure DevOps git repos. It appears that go-git does not work with Azure DevOps git repos. Fetch fails with the following error:
unexpected client error: unexpected requesting "https://[email protected]/jsuen0437/jsuen/_git/jsuen/git-upload-pack" status code: 400

The go-git call we are making are as follows:

	err = repo.Fetch(&git.FetchOptions{
		RemoteName: git.DefaultRemoteName,
		Tags:       git.AllTags,
		Force:      true,
	})

The equivalent git CLI command will succeed git fetch origin --tags --force.

I traced the error down to the upload_pack.go::UploadPack() when it is making the HTTP request:

func (s *upSession) UploadPack(
	ctx context.Context, req *packp.UploadPackRequest,
) (*packp.UploadPackResponse, error) {

	if req.IsEmpty() {
		return nil, transport.ErrEmptyUploadPackRequest
	}

	if err := req.Validate(); err != nil {
		return nil, err
	}

	url := fmt.Sprintf(
		"%s/%s",
		s.endpoint.String(), transport.UploadPackServiceName,
	)

	content, err := uploadPackRequestToReader(req)
	if err != nil {
		return nil, err
	}

	res, err := s.doRequest(ctx, http.MethodPost, url, content)  // <<<<< fails here
	if err != nil {
		return nil, err
	}
...
}

The values to the doRequest() call are:

url: https://[email protected]/jsuen0437/jsuen/_git/jsuen/git-upload-pack
content: 004cwant 58594acf362b1452349e2a15163d3357b9160663 side-band-64k no-progress
00000009done

I tested with the following go-git versions:

• v4.7.0
• v4.9.1

EDIT: updated to note that this fails even with public repos. Please feel free to use:
https://[email protected]/jsuen0437/jsuen/_git/jsuen for testing this.

[jessesuen]

Another data point is that another user reports that SSH also fails with azure: remote: Command git-upload-pack '/YYY/PROJECT/REPONAME' is not in expected format.

[mcuadros]

Its related with #335, looks like MS didn't implement, the basic protocol on his services. And only implements multi-ack protocol.

In short, we are not going to implement multi-ack (at least by @src-d), instead, we are already working on the implementation of the new git wire protocol, the v2. Can you test against azure, a git client with v2? (https://opensource.googleblog.com/2018/05/introducing-git-protocol-version-2.html)

[jessesuen]

instead, we are already working on the implementation of the new git wire protocol, the v2. Can you test against azure, a git client with v2? (https://opensource.googleblog.com/2018/05/introducing-git-protocol-version-2.html)

@mcuadros Sure I'd be happy to! However, I'm not sure what exactly you mean by testing "a git client v2". Is there a pointer to some example code which is doing this?

[mcuadros]

I mean using, git with v2 protocol against Azure. You have an example in the link I pasted

git clone https://chromium.googlesource.com/chromium/src.git
cd src
export GIT_TRACE_PACKET=1

# Using protocol version 2
git -c protocol.version=2 fetch --no-tags origin master

[jessesuen]

Yes this seems to work:

$ git clone https://[email protected]/jsuen0437/jsuen/_git/jsuen
Cloning into 'jsuen'...
remote: Azure Repos
remote: Found 237 objects to send. (22 ms)
Receiving objects: 100% (237/237), 46.04 KiB | 7.67 MiB/s, done.
Resolving deltas: 100% (120/120), done.

$ cd jsuen/

$ export GIT_TRACE_PACKET=1

$ git -c protocol.version=2 fetch --no-tags origin master
23:21:13.422461 pkt-line.c:80           packet:          git< # service=git-upload-pack
23:21:13.422819 pkt-line.c:80           packet:          git< 0000
23:21:13.422825 pkt-line.c:80           packet:          git< 58594acf362b1452349e2a15163d3357b9160663 HEAD\0 multi_ack thin-pack side-band side-band-64k no-progress multi_ack_detailed no-done shallow allow-tip-sha1-in-want symref=HEAD:refs/heads/master
23:21:13.422838 pkt-line.c:80           packet:          git< 58594acf362b1452349e2a15163d3357b9160663 refs/heads/master
23:21:13.422842 pkt-line.c:80           packet:          git< 0000
From https://dev.azure.com/jsuen0437/jsuen/_git/jsuen
 * branch            master     -> FETCH_HEAD

[jessesuen]

What's the timeframe for v2? We'd just like to understand if we should change the implementation to fork/exec git CLI to workaround this.

[delaman]

I am also experiencing this issue with the following setup:

go 1.12

require (
	github.com/gliderlabs/ssh v0.2.2 // indirect
	github.com/google/go-cmp v0.3.0 // indirect
	github.com/kr/pty v1.1.5 // indirect
	github.com/stretchr/objx v0.2.0 // indirect
	golang.org/x/crypto v0.0.0-20190618222545-ea8f1a30c443 // indirect
	golang.org/x/net v0.0.0-20190619014844-b5b0513f8c1b // indirect
	golang.org/x/sys v0.0.0-20190620070143-6f217b454f45 // indirect
	golang.org/x/text v0.3.2 // indirect
	golang.org/x/tools v0.0.0-20190620154339-431033348dd0 // indirect
	gopkg.in/src-d/go-git.v4 v4.12.0
)

Any word on when this issue will be resolved?

[memark]

What's the timeframe for v2? We'd just like to understand if we should change the implementation to fork/exec git CLI to workaround this.

He tried your command as requested (v2 against Azure) and it worked. Do you have a response? :-)

Is there any timeframe for v2, @mcuadros?