build from source #63
Comments
|
This builds like any other rabbitmq plugin. Have you tried it? |
|
Yes, tried |
|
I need to build a custom version to test dcorbacho@db4762d but can't remember how to build this 🤔 |
|
With some help from @acogoluegnes, we've concluded that it's just a matter of zipping @deadtrickster do you want me to wrap this into a |
ugh never knew about these vars, I usually check-out tags.
Yeah, would be nice. When I want to update deps (like prometheus.erl, accept, etc) I use https://github.com/deadtrickster/rebar3_archive_plugin. Maybe this could help. |
|
I am yet to test the resulting You will notice in the diff that I removed the These changes enabled me to create diff --git a/.gitignore b/.gitignore
index 9953f1e..db249c9 100644
--- a/.gitignore
+++ b/.gitignore
@@ -15,3 +15,4 @@
/rabbitmq_management_metrics.d
erl_crash.dump
+*.ez
diff --git a/Makefile b/Makefile
index a2d7610..71802e4 100644
--- a/Makefile
+++ b/Makefile
@@ -1,13 +1,22 @@
PROJECT = prometheus_rabbitmq_exporter
+PROJECT_DESCRIPTION = RabbitMQ Prometheus.io metrics exporter
+RABBITMQ_VERSION = 3.7.$(shell date +'%Y%m%d')
+# PROJECT_VERSION gets set in rabbitmq-components.mk to RABBITMQ_VERSION
+EZ = $(PROJECT)-$(PROJECT_VERSION)
+PROJECT_APP_EXTRA_KEYS = {maintainers, ["Ilya Khaprov"]}, \
+ {licenses, ["MIT"]}, \
+ {links, [{"Github", "https://github.com/deadtrickster/prometheus_rabbitmq_exporter"}]}
-dep_prometheus = hex 3.5.1
-dep_prometheus_process_collector = hex 1.3.1
-dep_prometheus_httpd = hex 2.1.8
dep_accept = hex 0.3.3
+dep_prometheus = hex 3.5.1
dep_prometheus_cowboy = hex 0.1.4
+dep_prometheus_httpd = hex 2.1.8
+
+DEPS = rabbit rabbitmq_management \
+ prometheus prometheus_cowboy prometheus_httpd
-DEPS = rabbit_common rabbit rabbitmq_management prometheus prometheus_httpd accept \
- prometheus_process_collector prometheus_cowboy
+# We do not want these deps defined as applications in app
+BUILD_DEPS = rabbit_common rabbitmq_management_agent accept
DEP_EARLY_PLUGINS = rabbit_common/mk/rabbitmq-early-plugin.mk
DEP_PLUGINS = rabbit_common/mk/rabbitmq-plugin.mk
@@ -18,10 +27,13 @@ DEP_PLUGINS = rabbit_common/mk/rabbitmq-plugin.mk
ERLANG_MK_REPO = https://github.com/rabbitmq/erlang.mk.git
ERLANG_MK_COMMIT = rabbitmq-tmp
+RABBITMQ_BRANCH ?= v3.7.x
+RABBITMQ_CURRENT_FETCH_URL ?= https://github.com/rabbitmq/
+
include rabbitmq-components.mk
include erlang.mk
-.PHONY: docker_build docker_push docker_latest docker_pure docker_alpine
+.PHONY: up docker_build docker_push docker_latest docker_pure docker_alpine
docker_build:
docker build -t deadtrickster/rabbitmq_prometheus\:3.7.8 .
@@ -47,3 +59,28 @@ docker_pure:
docker_alpine:
-docker run -p15672\:15672 deadtrickster/rabbitmq_prometheus\:latest-alpine
+
+up: $(abspath .)+up $(DEPS:%=$(DEPS_DIR)/%+up) $(BUILD_DEPS:%=$(DEPS_DIR)/%+up)
+ @:
+
+%+up: fetch-deps
+ $(exec_verbose) cd $*; \
+ git fetch -p && \
+ if [ '$(RABBITMQ_BRANCH)' ]; then \
+ git checkout $(RABBITMQ_BRANCH) || : ; \
+ fi && \
+ if git symbolic-ref -q HEAD >/dev/null; then \
+ branch=$$(git symbolic-ref --short HEAD); \
+ remote=$$(git config branch.$$branch.remote); \
+ merge=$$(git config branch.$$branch.merge | sed 's,refs/heads/,,'); \
+ if [ "$$remote" -a "$$merge" ]; then \
+ git merge --ff-only "$$remote/$$merge"; \
+ fi; \
+ fi && \
+ echo
+
+ez: up all
+ @rm -fr $(EZ) && mkdir $(EZ) && \
+ cp -r ebin include priv $(EZ) && \
+ rm -f $(EZ).ez && \
+ zip --move --recurse-paths --test $(EZ).ez $(EZ)
diff --git a/rabbitmq-components.mk b/rabbitmq-components.mk
index 6d4d9c9..0789b1c 100644
--- a/rabbitmq-components.mk
+++ b/rabbitmq-components.mk
@@ -40,7 +40,6 @@ endif
dep_amqp_client = git_rmq rabbitmq-erlang-client $(current_rmq_ref) $(base_rmq_ref) master
dep_amqp10_client = git_rmq rabbitmq-amqp1.0-client $(current_rmq_ref) $(base_rmq_ref) master
dep_amqp10_common = git_rmq rabbitmq-amqp1.0-common $(current_rmq_ref) $(base_rmq_ref) master
-dep_ra = git_rmq ra $(current_rmq_ref) $(base_rmq_ref) master
dep_rabbit = git_rmq rabbitmq-server $(current_rmq_ref) $(base_rmq_ref) master
dep_rabbit_common = git_rmq rabbitmq-common $(current_rmq_ref) $(base_rmq_ref) master
dep_rabbitmq_amqp1_0 = git_rmq rabbitmq-amqp1.0 $(current_rmq_ref) $(base_rmq_ref) master
@@ -51,7 +50,6 @@ dep_rabbitmq_auth_backend_ldap = git_rmq rabbitmq-auth-backend-ldap $(cur
dep_rabbitmq_auth_mechanism_ssl = git_rmq rabbitmq-auth-mechanism-ssl $(current_rmq_ref) $(base_rmq_ref) master
dep_rabbitmq_aws = git_rmq rabbitmq-aws $(current_rmq_ref) $(base_rmq_ref) master
dep_rabbitmq_boot_steps_visualiser = git_rmq rabbitmq-boot-steps-visualiser $(current_rmq_ref) $(base_rmq_ref) master
-dep_rabbitmq_clusterer = git_rmq rabbitmq-clusterer $(current_rmq_ref) $(base_rmq_ref) master
dep_rabbitmq_cli = git_rmq rabbitmq-cli $(current_rmq_ref) $(base_rmq_ref) master
dep_rabbitmq_codegen = git_rmq rabbitmq-codegen $(current_rmq_ref) $(base_rmq_ref) master
dep_rabbitmq_consistent_hash_exchange = git_rmq rabbitmq-consistent-hash-exchange $(current_rmq_ref) $(base_rmq_ref) master
@@ -71,7 +69,6 @@ dep_rabbitmq_management = git_rmq rabbitmq-management $(current_rm
dep_rabbitmq_management_agent = git_rmq rabbitmq-management-agent $(current_rmq_ref) $(base_rmq_ref) master
dep_rabbitmq_management_exchange = git_rmq rabbitmq-management-exchange $(current_rmq_ref) $(base_rmq_ref) master
dep_rabbitmq_management_themes = git_rmq rabbitmq-management-themes $(current_rmq_ref) $(base_rmq_ref) master
-dep_rabbitmq_management_visualiser = git_rmq rabbitmq-management-visualiser $(current_rmq_ref) $(base_rmq_ref) master
dep_rabbitmq_message_timestamp = git_rmq rabbitmq-message-timestamp $(current_rmq_ref) $(base_rmq_ref) master
dep_rabbitmq_metronome = git_rmq rabbitmq-metronome $(current_rmq_ref) $(base_rmq_ref) master
dep_rabbitmq_mqtt = git_rmq rabbitmq-mqtt $(current_rmq_ref) $(base_rmq_ref) master
@@ -111,20 +108,18 @@ dep_rabbitmq_public_umbrella = git_rmq rabbitmq-public-umbrella $(curre
# all projects use the same versions. It avoids conflicts and makes it
# possible to work with rabbitmq-public-umbrella.
-dep_cowboy = hex 2.2.2
-dep_cowlib = hex 2.1.0
-dep_jsx = hex 2.8.2
-dep_lager = hex 3.6.3
-dep_ranch = hex 1.5.0
-dep_ranch_proxy_protocol = hex 1.5.0
-dep_recon = hex 2.3.2
-
-dep_sockjs = git https://github.com/rabbitmq/sockjs-erlang.git 405990ea62353d98d36dbf5e1e64942d9b0a1daf
+dep_cowboy = hex 2.6.1
+dep_cowlib = hex 2.7.0
+dep_jsx = hex 2.9.0
+dep_lager = hex 3.6.5
+dep_ra = git https://github.com/rabbitmq/ra.git master
+dep_ranch = hex 1.7.1
+dep_recon = hex 2.3.6
+dep_sysmon_handler = hex 1.1.0
RABBITMQ_COMPONENTS = amqp_client \
amqp10_common \
amqp10_client \
- ra \
rabbit \
rabbit_common \
rabbitmq_amqp1_0 \
@@ -135,7 +130,6 @@ RABBITMQ_COMPONENTS = amqp_client \
rabbitmq_auth_mechanism_ssl \
rabbitmq_aws \
rabbitmq_boot_steps_visualiser \
- rabbitmq_clusterer \
rabbitmq_cli \
rabbitmq_codegen \
rabbitmq_consistent_hash_exchange \
@@ -155,7 +149,6 @@ RABBITMQ_COMPONENTS = amqp_client \
rabbitmq_management_agent \
rabbitmq_management_exchange \
rabbitmq_management_themes \
- rabbitmq_management_visualiser \
rabbitmq_message_timestamp \
rabbitmq_metronome \
rabbitmq_mqtt \
@@ -233,7 +226,7 @@ export base_rmq_ref
# If cloning from this computed location fails, we fallback to RabbitMQ
# upstream which is GitHub.
-# Maccro to transform eg. "rabbit_common" to "rabbitmq-common".
+# Macro to transform eg. "rabbit_common" to "rabbitmq-common".
rmq_cmp_repo_name = $(word 2,$(dep_$(1)))
# Upstream URL for the current project.
diff --git a/src/prometheus_rabbitmq_exporter.app.src b/src/prometheus_rabbitmq_exporter.app.src
deleted file mode 100644
index 3a79b6c..0000000
--- a/src/prometheus_rabbitmq_exporter.app.src
+++ /dev/null
@@ -1,19 +0,0 @@
-{application, prometheus_rabbitmq_exporter,
- [{description, "RabbitMQ Prometheus.io metrics exporter"},
- %% 3.7.2 or newer 3.7.x
- %% {broker_version_requirements, ["3.7.2"]},
- {vsn, "3.7.2.4"},
- {modules, []},
- {registered, []},
- {env, []},
- {applications, [kernel,
- stdlib,
- rabbit,
- rabbitmq_management,
- prometheus,
- prometheus_cowboy,
- prometheus_httpd]},
-
- {maintainers, ["Ilya Khaprov"]},
- {licenses, ["MIT"]},
- {links, [{"Github", "https://github.com/deadtrickster/prometheus_rabbitmq_exporter"}]}]}. |
|
I'm ok removing process_collector. Classic, but I don't have any problems with it :-(. Still will be including it in releasing anyway. |
|
The resulting I will be done when all ez files can be built from within this plugin with a single command: |
|
btw any plans to have a PR for that Core metrics exporter? |
|
I've managed to build |
That was the initial plan, but we've discovered that core metrics don't need |
|
I've started pushing changes required in different deps to make this come together. I have noticed that prometheus_rabbitmq_exporter is using prometheus.erl v3.5.1, yet the latest is v4.2.2. Is there anything keeping prometheus_rabbitmq_exporter pinned to prometheus.erl v3.x? |
|
I can see that https://github.com/deadtrickster/prometheus_process_collector/tree/v1.4.0 already has the rebar3_archive_plugin deps. Is there any reason why prometheus_rabbitmq_exporter can't use this version? It's currently referencing prometheus_process_collector v1.3.1. |
|
This is something new that I've just started hitting. @deadtrickster have you seen this before? Same rebar archiving accept: |
OTP 19 support
I'm not sure, I just tried 1.4.x works OK.
no :-) can you run it with DEBUG=1? |
|
It's time for this and related plugins to maintain two release branches, much like RabbitMQ does. RabbitMQ master requires OTP 21.2, I don't see any reasons for new releases to cling on OTP 19 but if @deadtrickster would like to support it, it should be on a separate branch (and release series). |
|
nah, I'll happily abandon otp19 and move prometheus to 4.x here |
|
OK, bumping to prometheus.erl v4.2.2 prometheus_cowboy v0.1.6 fails to archive: This is what I get with |
|
I just published prometheus_cowboy 0.1.7. Turns out rebar3 doesn't play well with this hex.pm version metadata |
|
published archive plugin with the callback |
|
So I guess we have the answer for this now. Closing. Thanks @gerhard! Still, would be nice to have dcorbacho@db4762d |
Will submit as is, even though we've discovered 2 important aspects that we would like to address soon-ish:
Would it be hard to disable the dependency on |
|
Management used for getting metrics and exposing. Of course given prometheus_httpd we can expose even without it, using just inets. I would be happy to switch to node-only metrics. I agree current mix is confusing. The only way to workaround it is to configure cluster-wide collectors on one node and disable them on others, leaving just beam/process. |
Yes please!
Why would there be any cluster-wide collectors? The sum across all nodes of certain metrics (e.g. queues, connections, channels) are the total for the entire cluster. Other than users & exchanges, which are just data, everything else is per-node. Even vhosts, while they are the same across all nodes, the Erlang processes which represent them are per-node and we might want to capture extra metrics for them one day. I can't think of any reason why we would want cluster-wide collectors. Am I missing something? Do you want to weigh in @dcorbacho @michaelklishin @kjnilsson ? |
Unless I'm completely off, I feel like some metrics are duplicated on all nodes, exactly because of management plugin or the way they aggregated. i think someone even complained about that. So I proposed workaround, which is of course just a hack and not really robust - collect replicated metrics only from one node. |
|
But this is not a collection problem, it's an aggregation problem. |
|
If you can push the local metrics (core or "collected") and aggregate them outside of RabbitMQ that would be the ideal solution. |
Please provide some guide to build project from sources
The text was updated successfully, but these errors were encountered: