Build in Alpine #293

gitHusband · 2020-07-30T03:32:31Z

Hi!

I got some problem when I tried to build gearmand in Docker-Alpine(gcc v9.3.0).

As you know, when build gearmand with version 1.1.91.1 and below, there has a bug which was fixed in #289.
error: 'environ' was declared 'extern' and later 'static' [-fpermissive]

So I try to build master.

There are two bugs for now:

git clone https://github.com/gearman/gearmand && cd gearmand && ./bootstrap.sh -a,
got an error: ./bootstrap.sh:317: An attempt was made to set an invalid VENDOR=alpine
You may need to fix this bug.
But I can do these things using CentOS and then clone the codes into Docker-Alpine, so I can make it in Alpine.
./configure -- done!
make - Bug occurred:

      CXX      libhashkit/libhashkit_libhashkitinc_la-function.lo
**libhashkit/fnv_64.cc:41:5: error: "__WORDSIZE" is not defined, evaluates to 0 [-Werror=undef]**
   41 | #if __WORDSIZE == 64 && defined(HAVE_FNV64_HASH)
      |     ^~~~~~~~~~
  CXX      libhashkit/libhashkit_libhashkitinc_la-has.lo
cc1plus: all warnings being treated as errors
make[2]: *** [Makefile:6803: libhashkit/libhashkit_libhashkitinc_la-fnv_64.lo] Error 1
make[2]: *** Waiting for unfinished jobs....
libhashkit/has.cc:47:5: error: "__WORDSIZE" is not defined, evaluates to 0 [-Werror=undef]
   47 | #if __WORDSIZE == 64 && defined(HAVE_FNV64_HASH)
      |     ^~~~~~~~~~
cc1plus: all warnings being treated as errors
make[2]: *** [Makefile:6817: libhashkit/libhashkit_libhashkitinc_la-has.lo] Error 1
make[2]: Leaving directory '/tmp/gearmand-master-configure'
make[1]: *** [Makefile:8307: all-recursive] Error 1
make[1]: Leaving directory '/tmp/gearmand-master-configure'
make: *** [Makefile:3069: all] Error 2

Can you please help to fix these bugs? Thanks!

The text was updated successfully, but these errors were encountered:

gitHusband · 2020-07-30T06:27:55Z

I fix libhashkit/common.h in my computer and then I can make successfully.

  #ifndef __WORDSIZE                
  # ifdef __MINGW32__               
  #  define __WORDSIZE 32           
+ # else                            
+ #  define __WORDSIZE 64    
  # endif                           
  #endif

p-alik · 2020-07-30T09:43:44Z

Thank you, @gitHusband!
Patched master could be faultless build on Ubuntu 18.04 with gcc 7.5.0.

gitHusband · 2020-07-30T13:11:00Z

Thank you, @gitHusband!
Patched master could be faultless build on Ubuntu 18.04 with gcc 7.5.0.

It's my pleasure. Would you fix the two bugs when you have time? @p-alik

p-alik · 2020-07-30T13:43:47Z

Yes, I'll do that.

gitHusband · 2020-07-30T13:45:13Z

Cool! Thanks!

esabol · 2020-07-30T23:19:09Z

Using a modified bootstrap.sh and libhashkit/common.h, I was able to bootstrap, build, and test. Here's the Dockerfile I used:

FROM alpine
MAINTAINER gearmand

# Install packages
RUN apk add musl-dev gcc g++ autoconf automake m4 git libtool make bash py3-sphinx \
            file util-linux-dev libuuid libevent-dev gperf boost-dev openssl-dev

# Switch to non-root user
RUN adduser --disabled-password --shell /bin/bash gearman
USER gearman

ARG GEARMAN_REPO=https://github.com/gearman/gearmand
RUN cd /tmp && git clone --depth 1 --branch master ${GEARMAN_REPO}.git
WORKDIR /tmp/gearmand
COPY bootstrap.sh /tmp/gearmand/
COPY common.h /tmp/gearmand/libhashkit/
RUN ./bootstrap.sh -a
RUN ./configure --enable-ssl 2>&1 | tee ./configure.log
RUN make 2>&1 | tee ./build.log
RUN make test 2>&1 | tee ./test.log

Here are the diffs on my bootstrap.sh just for comparison:

--- /path/to/github/gearmand/bootstrap.sh    2019-11-27 17:02:43.416800000 -0500
+++ ./bootstrap.sh      2020-07-30 18:51:38.209550000 -0400
@@ -194,6 +194,9 @@
     opensuse*)
       VENDOR_DISTRIBUTION='opensuse'
       ;;
+    alpine)
+      VENDOR_DISTRIBUTION='alpine'
+      ;;
     *)
       die "attempt to set an invalid VENDOR_DISTRIBUTION=$dist"
       ;;
@@ -263,6 +266,9 @@
     opensuse)
       VENDOR_RELEASE="$release"
       ;;
+    alpine)
+      VENDOR_RELEASE="$release"
+      ;;
     unknown)
       die "attempt to set VENDOR_RELEASE without setting VENDOR_DISTRIBUTION"
       ;;
@@ -273,13 +279,16 @@
 }
 
 
-#  Valid values are: apple, redhat, centos, canonical, oracle, suse
+#  Valid values are: apple, redhat, centos, canonical, oracle, suse, alpine
 set_VENDOR ()
 {
   local vendor
   vendor="$(echo "$1" | tr '[:upper:]' '[:lower:]')"
 
   case $vendor in
+    alpine)
+      VENDOR='alpine'
+      ;;
     apple)
       VENDOR='apple'
       ;;
@@ -372,6 +381,10 @@
     # shellcheck disable=SC1091
     source '/etc/lsb-release'
     set_VENDOR 'canonical' "$DISTRIB_ID" "$DISTRIB_CODENAME"
+  elif [[ -f '/etc/alpine-release' ]]; then
+    local alpine_version
+    alpine_version="$(cat /etc/alpine-release)"
+    set_VENDOR 'alpine' 'alpine' "$alpine_version"
   fi
 
   rebuild_host_os

Hope this helps!

p-alik · 2020-07-31T06:57:00Z

@esabol, could you put the snippets in a PR, please?
Regarding libhashkit, it's a part of geamand and libmemcached.
geamand version of the lib is a bit elder. For instance update_continuum seems to be useless

08:34 $ git grep update_continuum
libhashkit/common.h:/* int update_continuum(hashkit_st *hashkit); */
libhashkit/ketama.cc:int update_continuum(hashkit_st *hashkit)

Removing update_continuum from libhashkit/common.h appears to be harmless. The remaining md5_signature seems to be alike libhashkit_md5_signature

git grep "md5_signature(const"
libhashkit-1.0/algorithm.h:void libhashkit_md5_signature(const unsigned char *key, size_t length, unsigned char *result);
libhashkit/algorithm.cc:void libhashkit_md5_signature(const unsigned char *key, size_t length, unsigned char *result)
libhashkit/common.h:void md5_signature(const unsigned char *key, unsigned int length, unsigned char *result);
libhashkit/md5.cc:void md5_signature(const unsigned char *key, unsigned int length, unsigned char *result)

Wouldn't replacement of md5_signature by libhashkit_md5_signature make libhashkit/common.h obsolete?

esabol · 2020-08-07T04:55:44Z

Wouldn't replacement of md5_signature by libhashkit_md5_signature make libhashkit/common.h obsolete?

I think libhashkit has been copied here from another project (libmemcached?). If so, I think it makes sense to just keep it as it is in order to facilitate comparisons with the upstream codebase.

p-alik · 2020-08-07T08:44:12Z

I think libhashkit has been copied here from another project (libmemcached?)

@dormando, @BrianAker, could you share your knowledge regarding libhashkit. Would you recommend to keep it in gearmand code base?

If so, I think it makes sense to just keep it as it is in order to facilitate comparisons with the upstream codebase.

I'm of the opposite opinion because:

there is no code improvement in gearmand version of libhash since migration from https://code.launchpad.net/gearmand
by using appropriate libhash we could get rid of bunch of redundant code and issues with regard to libhash

$ wc -l `find libhashkit -name "*.cc"`|tail -n1
  4564 total

esabol · 2020-08-10T19:22:07Z

@p-alik: Looking through the code, it appears to me that libhashkit_md5_signature is the C++ interface and md5_signature is the C interface. Am I understanding that correctly?

Are you also saying libhashkit_md5_signature is the only routine used by gearmand? And the memcached plugin doesn't use it either? If so, you make a convincing case.

p-alik · 2020-08-10T19:57:15Z

@p-alik: Looking through the code, it appears to me that libhashkit_md5_signature is the C++ interface and md5_signature is the C interface. Am I understanding that correctly?

libhashkit_md5_signature and md5_signature are mostly equal.

Yeah, I saw that, but one is in an extern "C" block and the other one isn't. I suspect one is the public (external) interface and the other is the private (internal) implementation of the public interface. In other words, I do not think anyone is supposed to use md5_signature and only libhashkit_md5_signature is supposed to be used. (I don't actually know this. I'm just inferring.)

Are you also saying libhashkit_md5_signature is the only routine used by gearmand? And the memcached plugin doesn't use it either?

It seems there is very limited demand for libhashkit at all. But I'll check it accurately.

I see it's used by libgearman. If you eliminate it, that will change the symbols in libgearman.so, I think? And that will require bumping version numbers on libgearman.so, right?

Issue #293: Bootstrap on Alpine

SpamapS · 2020-08-27T02:57:51Z

Can you please confirm whether or not master builds for you on Alpine now?

p-alik · 2020-08-28T08:44:30Z

Can you please confirm whether or not master builds for you on Alpine now?

#295 provided successful build on Alpine

esabol · 2020-08-31T00:22:37Z

My Docker image of the master branch builds and tests fine:

FROM alpine

MAINTAINER gearmand

# Install packages
RUN apk add --no-cache \
            musl-dev gcc g++ autoconf automake m4 git libtool make bash file \
            py3-sphinx util-linux-dev libuuid libevent-dev gperf boost-dev \
            openssl-dev

# Switch to non-root user
RUN adduser --disabled-password --shell /bin/bash gearman
USER gearman

ARG GEARMAN_REPO=https://github.com/gearman/gearmand

RUN cd /tmp && git clone --depth 1 --branch master ${GEARMAN_REPO}.git
WORKDIR /tmp/gearmand
RUN ./bootstrap.sh -a
RUN ./configure --enable-ssl 2>&1 | tee ./configure.log
RUN make 2>&1 | tee ./build.log
RUN make test 2>&1 | tee ./test.log

============================================================================
Testsuite summary for gearmand 67407a4
============================================================================
# TOTAL: 35
# PASS:  26
# SKIP:  7
# XFAIL: 2
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================

gitHusband · 2020-09-01T07:38:43Z

Cool, I can build Docker Alpine image of the master branch successfully now. Thanks! @SpamapS @p-alik @esabol

Remove libhashkit subdirectory and use instead officially by libmemcached provided hashkit http://docs.libmemcached.org/hashkit_functions.html See discussion in issue gearman#293 (comment)

thibaultduponchelle mentioned this issue Aug 1, 2020

Add github ci for alpine and static #295

Closed

esabol added a commit to esabol/gearmand that referenced this issue Aug 1, 2020

Issue gearman#293: Bootstrap on Alpine

bc5ff83

esabol added a commit to esabol/gearmand that referenced this issue Aug 1, 2020

Issue gearman#293: Bootstrap on Alpine

6fe7421

esabol mentioned this issue Aug 1, 2020

Issue #293: Bootstrap on Alpine #296

Merged

esabol self-assigned this Aug 7, 2020

SpamapS added a commit that referenced this issue Aug 27, 2020

Merge pull request #296 from esabol/issue-293-bootstrap-on-alpine-redux

67407a4

Issue #293: Bootstrap on Alpine

SpamapS added enhancement incomplete labels Aug 27, 2020

p-alik closed this as completed Sep 1, 2020

p-alik mentioned this issue Nov 22, 2020

use hashkit library provided by libmemcached #313

Closed

esabol added portability and removed incomplete labels Nov 20, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build in Alpine #293

Build in Alpine #293

gitHusband commented Jul 30, 2020 •

edited

Loading

gitHusband commented Jul 30, 2020 •

edited

Loading

p-alik commented Jul 30, 2020

gitHusband commented Jul 30, 2020 •

edited

Loading

p-alik commented Jul 30, 2020

gitHusband commented Jul 30, 2020

esabol commented Jul 30, 2020 •

edited

Loading

p-alik commented Jul 31, 2020

esabol commented Aug 7, 2020

p-alik commented Aug 7, 2020

esabol commented Aug 10, 2020

p-alik commented Aug 10, 2020 •

edited by esabol

Loading

SpamapS commented Aug 27, 2020

p-alik commented Aug 28, 2020

esabol commented Aug 31, 2020

gitHusband commented Sep 1, 2020

Build in Alpine #293

Build in Alpine #293

Comments

gitHusband commented Jul 30, 2020 • edited Loading

gitHusband commented Jul 30, 2020 • edited Loading

p-alik commented Jul 30, 2020

gitHusband commented Jul 30, 2020 • edited Loading

p-alik commented Jul 30, 2020

gitHusband commented Jul 30, 2020

esabol commented Jul 30, 2020 • edited Loading

p-alik commented Jul 31, 2020

esabol commented Aug 7, 2020

p-alik commented Aug 7, 2020

esabol commented Aug 10, 2020

p-alik commented Aug 10, 2020 • edited by esabol Loading

SpamapS commented Aug 27, 2020

p-alik commented Aug 28, 2020

esabol commented Aug 31, 2020

gitHusband commented Sep 1, 2020

gitHusband commented Jul 30, 2020 •

edited

Loading

gitHusband commented Jul 30, 2020 •

edited

Loading

gitHusband commented Jul 30, 2020 •

edited

Loading

esabol commented Jul 30, 2020 •

edited

Loading

p-alik commented Aug 10, 2020 •

edited by esabol

Loading