x/tools/gopls: using a lot of memory

[ajeecai]

What version of Go are you using (go version)?

$ go version
go version go1.13.4 linux/amd64

Does this issue reproduce with the latest release?

Yes, I have update gopls with GO111MODULE=on go get golang.org/x/tools/gopls@master golang.org/x/tools@master

$ gopls version
golang.org/x/tools/gopls master
golang.org/x/tools/[email protected] h1:cDdkE3V1eIXwN6tdzqVXVYz3RkYVbE8y0n+sn6klUn4=

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE="auto"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ajee/.cache/go-build"
GOENV="/home/ajee/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/ajee/go"
GOPRIVATE=""
GOPROXY="https://goproxy.io"
GOROOT="/usr/lib/go-1.13"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.13/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build734902289=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Use vs code 1.42.0, download https://github.com/asticode/go-astilectron-demo and browse its code, after a bit change then save to make gopls activated.

What did you expect to see?

From htop, this memory used by gopls should not be excessive.

What did you see instead?

The memory usage is soar, up to 9G, sometimes 12G by gopls, and it never goes down.

The similar issue is discussed in #30309, @stamblerre encourages to open a new issue.

The pprofile svg file is attached as required (in zip file as github required). Any information needed, pls tell me.

Thanks

pprof001.zip

[gopherbot]

Thank you for filing a gopls issue! Please take a look at the Troubleshooting guide, and make sure that you have provided all of the relevant information here.

[stamblerre]

It seems like analysis is causing the high memory usage. Can you try disabling it by adding the following to your VS Code settings?

"gopls": {
	"experimentalDisabledAnalyses": [
		"asmdecl", "assign", "atomic", "atomicalign", 
		"bools", "buildtag", "cgocall", "composite", "copylock",      
		"errorsas", "httpresponse", "loopclosure", "lostcancel",
		"nilfunc", "printf", "shift", "stdmethods", 
		"structtag", "tests", "unmarshal", "unreachable", "unsafeptr",
		"unusedresult", "deepequalerrors", 
		"sortslice", "testinggoroutine"
	],
},

We also just released gopls/v0.3.2 - can you try downloading that to see if affects anything? (GO111MODULE=on go get golang.org/x/tools/[email protected]).

[heschi]

Can you check your /tmp directory for files named gopls.*? It should have written out some profiles. They're named after the size of the heap; please upload the ones from the biggest heaps.

[ajeecai]

Hi, I just tried, adding the options in vs code setting doesn't help. And I also update to 0.3.2, looks the same, memory up to 9G. Please see the attached zip file which includes svg for memory profile. Also I didn't find any files name gopls.* under /tmp, instead in my home directory there is a folder named pprof which contains some files with gopls, then I include one is generated at the same timestamp in the zip. Anything else please tell me.

Thanks
pp.zip

[gopherbot]

Change https://golang.org/cl/219897 mentions this issue: internal/lsp/cache: debugging CL for golang/go#37223

[stamblerre]

Thanks for the profiles. Looks like the inspect analysis is still using the majority of the memory. I've disabled in analyses in CL 219897 - can you try rebuilding gopls with that CL patched in and see if that resolves the problem?

This can be done with the following steps:

$ cd $(mktemp -d)
$ git clone https://go.googlesource.com/tools
$ cd tools
$ git fetch "https://go.googlesource.com/tools" refs/changes/97/219897/2 && git cherry-pick FETCH_HEAD
$ cd gopls
$ go install

This supports @heschik's theory that analyses aren't being canceled in time, and that we are running too many analyses concurrently.

[gopherbot]

Change https://golang.org/cl/219958 mentions this issue: internal/lsp: limit diagnostics concurrency

[ajeecai]

Hi, good news with CL 219897, now the memory stays low even I remove experimentalDisabledAnalyses from the vs code config.

[heschi]

Cool, that confirms that it was a problem with analyses. Can you see if https://golang.org/cl/219958 (which is on master now) also helps?

[ajeecai]

Hi @heschik , I just git clone https://go.googlesource.com/tools and go install from head, it seems working, no memory soar.

[stamblerre]

Thanks for confirming! I'll mark this as closed, and https://golang.org/cl/219958 will be part of the gopls/v0.4.0 release.

[nikkolasg]

Just want to report my case here since related: Using vim-go, that uses gopls underneath, I see my memory usage just growing, need to restart vim after a couple of hours each time. With latest updated binaries using vim-go :GoUpdateBinaries.

[stamblerre]

@nikkolasg: Just to confirm, the output of gopls version is v0.3.4?

[nikkolasg]

golang.org/x/tools/gopls v0.3.4
    golang.org/x/tools/[email protected] h1:4GC7q/pXQ/tsxHBGVdsMdlB4gCxVC06m/7rIXg1Px4E=

Yes! Happy to provide more info if I can help.

[stamblerre]

Thanks! Yes, hopefully we can debug this. There should be some files named gopls.PID- in your temporary directory, named after heap sizes. Please upload the pair for the largest heap -- I would like to see a goroutine dump that matches this heap dump.

[nikkolasg]

Any idea where I can find this file ? Only thing relevant I find in /tmp is gopls_stderrsq5q_qoo.log but it's empty.

[stamblerre]

Maybe try looking in whatever the value of TMPDIR is on your machine? (This might be helpful.)

[nikkolasg]

my tmpdir is empty , so I'm using the same tmpdir as what go returns I guess ?
find . -name gopls.PID didn't return anything in /tmp ... Maybe vim-go changes that but I don't see it.

[stamblerre]

Ah, the PID means it will be a process ID, so you may want to run something like find . -name "gopls.*". If nothing turns up still, then maybe @heschik can advise.

[nikkolasg]

nothing turns out there as well :/
/tmp:% find . -name "gopls.*"

[EnriqueL8]

I'm hitting the same problem as well. Same version and nothing found with the find . -name "gopls.*".

[heschi]

It would be very, very surprising if recent versions of gopls hadn't written any of those files to /tmp while using a problematic amount of memory. Can you triple-check that there isn't an old version anywhere, and that it's actually gopls using the memory?

[nikkolasg]

I ran the search on / ;) The only file gopls creates is of the form /tmp/gopls_stderr9hteqnwy.log and it's empty.
And yes, I see a bunch of gopls threads/processes always on top with htop when sorting by memory. And I gain a bunch of free GB by calling pkill gopls.

The only thing I am not sure of is which version vim-go uses, even when doing :GoUpdateBinaries but the one I have when type gopls version is v0.3.4. I never go get it by hand so I think vim-go did it for me.

[heschi]

Sorry, I don't know how vim-go works at all, but by far the most likely explanation for what you're describing is still that you're using an old version of gopls. If you're on Linux, you can see where the binary comes from using ls -l /proc/$PID/exe.

[nikkolasg]

Nice ! In fact it was YouCompleteMe that used an old version, no vim-go, I found it by running your command. Updated it manually and so far it works good !
Thanks a lot !

[ericwq]

i might find the correct setup method for gopls. see the post