errors with osx moduleset builds

[totaam]

Issue migrated from trac ticket # 971

component: platforms | priority: blocker | resolution: fixed | keywords: osx moduleset

2015-08-31 09:54:00: antoine created the issue

---

Related to Xpra-org/xpra#533.

I've hit some weird gtk module import problems which forced me to pull the download link for 0.15.5 on osx.

It looked like some modules / libraries needed to be rebuilt.
So I tried a few guesses (gtk), and when that didn't work I took the big hammer:

jhbuild clean

(and skip the ones that fail to clean)
Then:

jhbuild list | xargs

And rebuild all the modules from this list (since I can't find an easy to force a rebuild of all the modules).

jhbuild buildone -f THELIST

Which failed on:

• harfbuzz:

...
  CXX      libharfbuzz_icu_la-hb-icu.lo
  CXX      main-main.o
  CXX      test-test.o
  CXX      test_buffer_serialize-test-buffer-serialize.o
hb-coretext.cc: In function 'hb_bool_t _hb_coretext_shape(hb_shape_plan_t*, hb_font_t*, hb_buffer_t*, const hb_feature_t*, unsigned int)':
hb-coretext.cc:753: error: 'kCTTypesetterOptionForcedEmbeddingLevel' was not declared in this scope  CXX      test_size_params-test-size-params.o

make[4]: *** [libharfbuzz_la-hb-coretext.lo] Error 1
make[4]: *** Waiting for unfinished jobs....
make[3]: *** [all-recursive] Error 1
make[2]: *** [all] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

• gstreamer:

Making all in parse
  GEN    lex.priv_gst_parse_yy.c
  CC     libgstparse_la-lex.priv_gst_parse_yy.lo
lex.priv_gst_parse_yy.c:1008: warning: redundant redeclaration of 'priv_gst_parse_yyget_column'
lex.priv_gst_parse_yy.c:4: warning: previous declaration of 'priv_gst_parse_yyget_column' was here
lex.priv_gst_parse_yy.c:1010: warning: redundant redeclaration of 'priv_gst_parse_yyset_column'
lex.priv_gst_parse_yy.c:5: warning: previous declaration of 'priv_gst_parse_yyset_column' was here
  GEN    grammar.tab.c
./grammar.y: warning: 37 shift/reduce conflicts [-Wconflicts-sr]
./grammar.y: warning: 5 reduce/reduce conflicts [-Wconflicts-rr]
  CC     libgstparse_la-grammar.tab.lo
grammar.tab.c: In function 'priv_gst_parse_yyparse':
grammar.tab.c:1819: error: too few arguments to function 'priv_gst_parse_yylex'
make[4]: *** [libgstparse_la-grammar.tab.lo] Error 1
make[3]: *** [all-recursive] Error 1
make[2]: *** [all] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

• libtheora stalls on checking for SDL in the configure script...
• gmplib:

checking size of unsigned long... 4
checking size of mp_limb_t... 4
configure: error: Oops, mp_limb_t is 32 bits, but the assembler code
in this configuration expects 64 bits.
You appear to have set $CFLAGS, perhaps you also need to tell GMP the
intended ABI, see "ABI and ISA" in the manual.

---

Using the modulesets was meant to make it easier to rebuild things reliably... now I'm not so sure!

This is not the first time either, a recent update to ffmpeg (or was it x264) caused the gstreamer plugins to fail until they got rebuilt against the update lib.

@smo: did those packages get updated? when? if so, why didn't we automatically rebuild them? why does it break? and more importantly, how do I fix it..
How do we prevent these kinds of problems in the future? should at least have a way of easily locking a particular version so we can rebuild it reliably?

PS: it is possible that some of these errors are specific to my OSX 10.5 gcc 4.2 compiler, but I am not about to switch my build vm, as this one has worked reliably for years (something that is not so easy with OSX virtualized - no other version has been this reliable for me). But I would consider building a new gcc as part of the moduleset build if that helps (who knows what osx patches are required for that though..)

[totaam]

2015-09-02 02:54:18: smo commented

---

I will have to look into when these have changed recently as I haven't had that much time to work on them.

Most of our issues here is because I have not implemented good dependency checking. Jhbuild does provide this but I did not add these. This is something I could spend some time on so we can avoid it in the future.

Usually I find it best to just

mv ~/gtk ~/gtk.old
jhbuild bootstrap
jhbuild build 

Just start over from scratch I know this isn't ideal but this is what I usually do. It takes my machine about 3-4hours to get everything compiled again.

I don't think these are specific to your machine but something probably I updated then you built it and we failed to build stuff dependent on it.

[totaam]

2015-09-02 09:34:31: antoine commented

---

OK, started again from scratch, which got rid of all the issues above, but I hit a new one instead!
Maybe you're building on a newer system with 64-bit support?

configure: error: Oops, mp_limb_t is 32 bits, but the assembler code
in this configuration expects 64 bits.
You appear to have set $CFLAGS, perhaps you also need to tell GMP the
intended ABI, see "ABI and ISA" in the manual.
*** Error during phase configure of gmplib: ########## Error running ./configure --prefix /Users/osx/gtk/inst --libdir '/Users/osx/gtk/inst/lib'    *** [71/97]

Which I have worked around by running configure with:

ABI=32 ./configure ...

@smo: please confirm, apply and/or close.

[totaam]

2015-09-17 18:58:22: smo commented

---

My build system is still 32bit so I will have to conform this. If I add that environment variable we will have to make note of it or it will break on 64 bit builds which we are working towards.

[totaam]

2015-09-18 05:54:46: antoine commented

---

@smo: maybe we can stick this in jhbuildrc-custom somewhere?
or run configure like this:

ABI=`python -c 'import struct;print( 8 * struct.calcsize("P"))'` ./configure ...

This will give 32 or 64 depending on the architecture python is built for.

[totaam]

2015-09-24 22:34:20: smo commented

---

Yes this can be added.

I'm going to do some updates from the upstream osx jhbuild stable moduleset before I continue with this.

Next is to get an environment for myself to build this 64bit stuff so I can work on these errors and bring 64bit up to par.

[totaam]

2015-09-24 23:00:03: smo commented

---

Updates in r10674

I kept back gmp-6.0.0 which we had upgraded some time back but I noticed again that the stable-moduleset upstream uses gmp 4.3.2 and a few patches.

I would think having the newer version would be better and has worked for us?

Maybe there is something I don't know.

[totaam]

2015-09-25 10:37:35: antoine commented

---

Yes, we do want the latest gmp, 4.3.2 is way too old for such a critical low level library.

[totaam]

2015-10-08 23:07:37: smo commented

---

I can't reproduce the issue with gmp that you were seeing I see this from configure

checking ABI=32
checking whether /usr/bin/gcc-4.2 is gcc... yes
checking compiler /usr/bin/gcc-4.2 -arch i386 -I/Developer/SDKs/MacOSX10.5.sdk/usr/include -arch i386 -I/Developer/SDKs/MacOSX10.5.sdk/usr/include
 -isysroot /Developer/SDKs/MacOSX10.5.sdk -mmacosx-version-min=10.5 -isysroot /Developer/SDKs/MacOSX10.5.sdk -mmacosx-version-min=10.5 -I/Users/sp
ikesinc/gtk/inst/include -arch i386 -I/Developer/SDKs/MacOSX10.5.sdk/usr/include -I/Users/osx/gtk/inst/include -arch i386 -I/Developer/SDKs/
MacOSX10.5.sdk/usr/include -isysroot /Developer/SDKs/MacOSX10.5.sdk -isysroot /Developer/SDKs/MacOSX10.5.sdk... yes
checking compiler /usr/bin/gcc-4.2 -arch i386 -I/Developer/SDKs/MacOSX10.5.sdk/usr/include -arch i386 -I/Developer/SDKs/MacOSX10.5.sdk/usr/include
 -isysroot /Developer/SDKs/MacOSX10.5.sdk -mmacosx-version-min=10.5 -isysroot /Developer/SDKs/MacOSX10.5.sdk -mmacosx-version-min=10.5 has sizeof(
long)==4... yes
checking for gcc... /usr/bin/gcc-4.2
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether /usr/bin/gcc-4.2 accepts -g... yes
checking for /usr/bin/gcc-4.2 option to accept ISO C89... none needed
checking for /usr/bin/gcc-4.2 option to accept ISO C99... -std=gnu99
checking for /usr/bin/gcc-4.2 -std=gnu99 option to accept ISO Standard C... (cached) -std=gnu99
checking how to run the C preprocessor... /usr/bin/gcc-4.2 -std=gnu99 -E
checking build system compiler /usr/bin/gcc-4.2 -std=gnu99... yes
checking for build system preprocessor... /usr/bin/gcc-4.2 -std=gnu99 -E
checking for build system executable suffix... 
checking whether build system compiler is ANSI... yes
checking for build system compiler math library... -lm
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking if the assembler knows about MMX instructions... yes
checking if the assembler knows about SSE2 instructions... yes
using ABI="32"
      CC="/usr/bin/gcc-4.2 -std=gnu99"
      CFLAGS="-arch i386 -I/Developer/SDKs/MacOSX10.5.sdk/usr/include -arch i386 -I/Developer/SDKs/MacOSX10.5.sdk/usr/include -isysroot /Developer
/SDKs/MacOSX10.5.sdk -mmacosx-version-min=10.5 -isysroot /Developer/SDKs/MacOSX10.5.sdk -mmacosx-version-min=10.5"

Is it possible because I am doing this on hardware and you are on a VM?

If it doesn't hurt anything I will add comment:5 to the configure line in the moduleset

[totaam]

2015-10-27 21:28:47: smo commented

---

I tried adding this to gmplib in our moduleset but it has issues parsing it. We are able to add os.environ["ABI"] = "32" in .jhbuildrc-custom will that be enough to fix it?

[totaam]

2015-10-28 02:56:05: antoine commented

---

Works:

rm -fr ~/gtk/source/gmp*
echo 'os.environ["ABI"] = "32"' >> .jhbuildrc-custom
export PATH=$PATH:~/.local/bin/
jhbuild buildone -f gmplib

Now rebuilding everything with jhbuild -f just to make sure this doesn't break anything else.

[totaam]

2015-10-29 18:27:56: smo commented

---

I also tested something similar with no issues. Closing this for now.