Building TiDB on Windows

[rapoth]

Hi - First of all, thank you for your wonderful work on this project! I am trying to run TiDB/TiKV/PD on Windows to test the system out.

To begin with, I was trying to get TiDB to run. I setup Go and the associated GOPATH etc., after which I ran go build but saw some compiler errors. Any help in getting this to work on Windows would be greatly appreciated!

Thank you!

Summary:

1. What did you do?
If possible, provide a recipe for reproducing the error.

1. On Windows, install Go and setup the necessary paths (e.g., GOPATH)
2. Clone TiDB into GOPATH\github.com\pingcap\
3. cd into the directory and run go build

2. What did you expect to see?

Project getting built successfully.

3. What did you see instead?

PS C:\Go\work\src\github.com\pingcap\tidb> go build
# github.com/pingcap/tidb/parser
C:\Go\src\github.com\pingcap\tidb\parser\lexer.go:26: cannot convert Scanner literal (type *Scanner) to type yyLexer:
        *Scanner does not implement yyLexer (missing Error method)
C:\Go\src\github.com\pingcap\tidb\parser\lexer.go:151: cannot use s (type *Scanner) as type yyLexer in argument to toInt:
        *Scanner does not implement yyLexer (missing Error method)
C:\Go\src\github.com\pingcap\tidb\parser\lexer.go:153: cannot use s (type *Scanner) as type yyLexer in argument to toFloat:
        *Scanner does not implement yyLexer (missing Error method)
C:\Go\src\github.com\pingcap\tidb\parser\lexer.go:155: cannot use s (type *Scanner) as type yyLexer in argument to toDecimal:
        *Scanner does not implement yyLexer (missing Error method)
C:\Go\src\github.com\pingcap\tidb\parser\lexer.go:157: cannot use s (type *Scanner) as type yyLexer in argument to toHex:
        *Scanner does not implement yyLexer (missing Error method)
C:\Go\src\github.com\pingcap\tidb\parser\lexer.go:159: cannot use s (type *Scanner) as type yyLexer in argument to toBit:
        *Scanner does not implement yyLexer (missing Error method)
C:\Go\src\github.com\pingcap\tidb\parser\parser.go:11228: undefined: parser in parser.startOffset
C:\Go\src\github.com\pingcap\tidb\parser\parser.go:11229: undefined: parser in parser.endOffset
C:\Go\src\github.com\pingcap\tidb\parser\parser.go:11231: undefined: parser in parser.src
C:\Go\src\github.com\pingcap\tidb\parser\parser.go:11505: yylex.Errorf undefined (type yyLexer has no field or method Errorf)
C:\Go\src\github.com\pingcap\tidb\parser\parser.go:11505: too many errors
# google.golang.org/grpc/transport
_vendor\src\google.golang.org\grpc\transport\http_util.go:481: f.fr.SetReuseFrames undefined (type *http2.Framer has no field or method SetReuseFrames)

4. What version of TiDB are you using (tidb-server -V)?

I checked out the master branch.

[choleraehyq]

You shouldn't run go build directly, because we need to compile goyacc and build parser before compiling tidb itself.

[rapoth]

@choleraehyq Thank you for your prompt response!

As per your suggestion, I tried building the parser:

PS C:\Go\work\src\github.com\pingcap\tidb> goyacc.exe -o temp.xxx -xegen temp .\parser\parser.y
Parse table entries: 1172549 of 2447560, x 16 bits == 2345098 bytes
PS C:\Go\work\src\github.com\pingcap\tidb> goyacc.exe -o .\parser\parser.go -xe temp .\parser\parser.y
Parse table entries: 1172549 of 2447560, x 16 bits == 2345098 bytes

I verified that the parser.y and parser.go exist. After that, I ran a go build for tidb-server and got the following:

PS C:\Go\work\src\github.com\pingcap\tidb> go build -o tidb-server .\tidb-server\main.go
# github.com/pingcap/tidb/parser
C:\Go\src\github.com\pingcap\tidb\parser\lexer.go:26: cannot convert Scanner literal (type *Scanner) to type yyLexer:
        *Scanner does not implement yyLexer (missing Error method)
C:\Go\src\github.com\pingcap\tidb\parser\lexer.go:151: cannot use s (type *Scanner) as type yyLexer in argument to toInt:
        *Scanner does not implement yyLexer (missing Error method)
C:\Go\src\github.com\pingcap\tidb\parser\lexer.go:153: cannot use s (type *Scanner) as type yyLexer in argument to toFloat:
        *Scanner does not implement yyLexer (missing Error method)
C:\Go\src\github.com\pingcap\tidb\parser\lexer.go:155: cannot use s (type *Scanner) as type yyLexer in argument to toDecimal:
        *Scanner does not implement yyLexer (missing Error method)
C:\Go\src\github.com\pingcap\tidb\parser\lexer.go:157: cannot use s (type *Scanner) as type yyLexer in argument to toHex:
        *Scanner does not implement yyLexer (missing Error method)
C:\Go\src\github.com\pingcap\tidb\parser\lexer.go:159: cannot use s (type *Scanner) as type yyLexer in argument to toBit:
        *Scanner does not implement yyLexer (missing Error method)
C:\Go\src\github.com\pingcap\tidb\parser\parser.go:11229: undefined: parser in parser.startOffset
C:\Go\src\github.com\pingcap\tidb\parser\parser.go:11230: undefined: parser in parser.endOffset
C:\Go\src\github.com\pingcap\tidb\parser\parser.go:11232: undefined: parser in parser.src
C:\Go\src\github.com\pingcap\tidb\parser\parser.go:11506: yylex.Errorf undefined (type yyLexer has no field or method Errorf)
C:\Go\src\github.com\pingcap\tidb\parser\parser.go:11506: too many errors
# google.golang.org/grpc/transport
_vendor\src\google.golang.org\grpc\transport\http_util.go:481: f.fr.SetReuseFrames undefined (type *http2.Framer has no field or method SetReuseFrames)

I'll continue looking into this. If you have any other suggestions, please let me know.

[rapoth]

I thought perhaps I am unable to build master so I also tried building this release: 6e0ff81

In the Makefile, I did not see any instructions for generating scanner.l as was noted in a previous attempt to run tidb on Windows (#130). I am not certain if this step is still needed.

When I try to use golex, I get the following error:

PS C:\Go\work\src\github.com\pingcap\tidb> C:\go\work\bin\golex.exe -o .\parser\scanner.o .\parser\parser.y
Error: .\parser\parser.y:41:2:"syntax error"

2017/08/23 10:29:00 main.go:111: .\parser\parser.y:41:2:"syntax error"

I am new to Go and its ecosystem and am eager to try out TiDB - would greatly appreciate any help you could provide. Thank you for your time!

[shenli]

@rapoth Firstly, You need to build the goyacc binary. Please refer to our Makefile:
https://github.com/pingcap/tidb/blob/master/Makefile#L49
Then you need to build the parser: https://github.com/pingcap/tidb/blob/master/Makefile#L52
At last, you can build the tidb-server binary.

I am not sure if the whole project could be built on Windows platform. But it is a nice try.

[rapoth]

@shenli: Thank you! I did manage to do everything you suggested. Here are the precise commands I have tried until now:

go build -o goyacc.exe .\parser\goyacc\main.go
.\goyacc.exe -o temp.remove -xegen temp.xxx .\parser\parser.y
.\goyacc.exe -o .\parser\parser.go -xe temp.xxx .\parser\parser.y
go build -o tidb-server .\tidb-server\main.go // does not work and throws the error above

The Makefile does not explicily run golex so I am not sure if I missed any other steps before running go build tidb-server.

[rapoth]

@choleraehyq @shenli : I'd highly appreciate any suggestions from your end on how to proceed. I am not totally familiar with the Go ecosystem so I'm out of ideas from my end :) Thank you in advance for your time!

[shenli]

@rapoth I will try it myself in my spare time.

[rapoth]

@shenli Thank you so much! I really appreciate your time. Please do let me know if you manage to build it.

[tohghua]

@rapoth I can sucessfully build tidb in windows with the following commands:
set GOPATH=D:\test\src\github.com\pingcap\tidb\_vendor;d:\test
cd /d D:\test\src\github.com\pingcap\tidb
go build -o bin/goyacc.exe parser/goyacc/main.go
bin\goyacc -o parser\parser.go parser\parser.y
ren bench_test.go bench_test.go1
ren y.go y.go1
go build -o bin\tidb-server.exe tidb-server\main.go

The result exe file size is 37MB.
I find it can run without anything else (e.g. tikv , pd)

[shenli]

@tohghua You can run tidb-server with a local storage engine without tikv. :)

[tohghua]

@shenli I ignored this line in Makefile, does it matter? It seems no effect at all.
sed -i -e 's|//line.*||' -e 's/yyEofCode/yyEOFCode/' parser/parser.go

[shenli]

@tohghua You need to keep this or replace it with equivalent scripts.

[jacobsy]

@tohghua
set GOPATH=D:\test\src\github.com\pingcap\tidb_vendor;d:\test or
set GOPATH=D:\test\src\github.com\pingcap\tidb\_vendor;d:\test ?

[tohghua]

You are right: tidb\_vendor

[wwar]

This is now possible using WSL. Should this be closed?

[ghost]

I am going to close this issue as stale. It is not something that is explicitly supported, but it should be possible to build TiDB on Windows with WSL. Thanks!