Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bitnami/mongodb] Inaccurate Checks in Bitnami Readiness and Startup Probes #21957

Closed
eli-hasson opened this issue Jan 10, 2024 · 4 comments
Closed
Labels
mongodb solved stale 15 days without activity tech-issues The user has a technical issue about an application

Comments

@eli-hasson
Copy link

Name and Version

bitnami/mongodb

What architecture are you using?

None

What steps will reproduce the bug?

I found that the current readiness and startup probes , which utilizes the bitnami scripts /bitnami/scripts/readiness-probe.sh and /bitnami/scripts/startup-probe.sh (stored within the docker image docker.io/bitnami/mongodb:5.0.23-debian-11-r3 ) , returns an unexpected exit code (0) even when encountering an error. Here’s a snippet of the issue:

I have no name!@common-mongodb-2:/$ mongosh $TLS_OPTIONS --port $MONGODB_PORT_NUMBER --eval 'db.hello().isWritablePrimary || db.hello().secondary' | grep 'true'
Connecting to: mongodb://127.0.0.1:27017/?directConnection=true&serverSelectionTimeoutMS=2000&appName=mongosh+2.1.1
MongoNetworkError: connect ECONNREFUSED 127.0.0.1:27017

I have no name!@common-mongodb-2:/$ echo $?
0

The issue arises from the fact that the pattern directConnection=true matches the grep true filter, resulting in an incorrect exit code (0). This misleading exit code makes it seem like the health check is passing.
When we used our custom probes leveraging the mongo shell instead of mongosh, the output was different and did not contain the word ‘true’:

I have no name!@common-mongodb-2:/$ mongo --eval 'db.hello().isWritablePrimary || db.hello().secondary'                 
MongoDB shell version v5.0.23
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("8b83775b-9e40-47b2-9293-23387ee36fd1") }
MongoDB server version: 5.0.23
true

A swift solution is to add the --quiet argument to silence the output during the connection process, ensuring only the accurate MongoDB status is returned:

mongosh $TLS_OPTIONS --port $MONGODB_PORT_NUMBER --quiet --eval 'db.hello().isWritablePrimary || db.hello().secondary' | grep -q 'true'

By incorporating --quiet, we eliminate unwanted output and guarantee a reliable status from MongoDB.

Are you using any custom parameters or values?

No response

What is the expected behavior?

No response

What do you see instead?

readiness and startup checks succeeded even when mongodb pod has not reached SECONDAY or PRIMARY state.

Additional information

No response

@eli-hasson eli-hasson added the tech-issues The user has a technical issue about an application label Jan 10, 2024
@github-actions github-actions bot added the triage Triage is needed label Jan 10, 2024
@javsalgar javsalgar changed the title Inaccurate Checks in Bitnami Readiness and Startup Probes [bitnami/mongodb] Inaccurate Checks in Bitnami Readiness and Startup Probes Jan 11, 2024
@github-actions github-actions bot removed the triage Triage is needed label Jan 11, 2024
@github-actions github-actions bot assigned CeliaGMqrz and unassigned javsalgar Jan 11, 2024
hackerwins added a commit to yorkie-team/yorkie that referenced this issue Jan 25, 2024
This PR adds a Helm chart for MongoDB, supporting both standalone mode
and sharded cluster mode.
Shard rules, including shard keys, shard methods, and unique
constraints, are defined and managed in the values.yaml.

This chart depends on Bitnami's mongodb-sharded chart
(https://github.com/bitnami/charts/tree/main/bitnami/mongodb-sharded).

There are a few known issues with it:

A problem with livenessProbe and readinessProbe
  - Issues: bitnami/charts#21957
  - Solution: use custom livenessProbe and readinessProbe instead

Lack of ARM64 support in the Bitnami mongodb-sharded container
- Issues:
  - bitnami/charts#7305 (comment)
  - bitnami/containers#40947
- Solution: use the official Mongo 6.0 image instead and set up the cluster via a Job.

---------

Co-authored-by: Youngteac Hong <[email protected]>
hackerwins added a commit to yorkie-team/yorkie that referenced this issue Jan 25, 2024
This PR adds a Helm chart for MongoDB, supporting both standalone mode
and sharded cluster mode.
Shard rules, including shard keys, shard methods, and unique
constraints, are defined and managed in the values.yaml.

This chart depends on Bitnami's mongodb-sharded chart
(https://github.com/bitnami/charts/tree/main/bitnami/mongodb-sharded).

There are a few known issues with it:

A problem with livenessProbe and readinessProbe
  - Issues: bitnami/charts#21957
  - Solution: use custom livenessProbe and readinessProbe instead

Lack of ARM64 support in the Bitnami mongodb-sharded container
- Issues:
  - bitnami/charts#7305 (comment)
  - bitnami/containers#40947
- Solution: use the official Mongo 6.0 image instead and set up the cluster via a Job.

---------

Co-authored-by: Youngteac Hong <[email protected]>
Copy link

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

@github-actions github-actions bot added the stale 15 days without activity label Jan 27, 2024
Copy link

github-actions bot commented Feb 1, 2024

Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.

@github-actions github-actions bot added the solved label Feb 1, 2024
@bitnami-bot bitnami-bot closed this as not planned Won't fix, can't repro, duplicate, stale Feb 1, 2024
@gal-rosencovich
Copy link

gal-rosencovich commented Sep 24, 2024

I wonder why this was closed..?🤔
Seems like simple, yet critical matter to resolve

@carrodher
Copy link
Member

Thank you for bringing this issue to our attention. We appreciate your involvement! If you're interested in contributing a solution, we welcome you to create a pull request. The Bitnami team is excited to review your submission and offer feedback. You can find the contributing guidelines here.

Your contribution will greatly benefit the community. Feel free to reach out if you have any questions or need assistance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
mongodb solved stale 15 days without activity tech-issues The user has a technical issue about an application
Projects
None yet
Development

No branches or pull requests

6 participants