mongodb-3_4, mongodb-3_6: drop

[wegank]

Description of changes

#203814 seems to be stalled, so let's try again.

The original PR drops boost160 while keeping boost159. I think it would be better to drop them both after #207373.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 23.05 Release Notes (or backporting 22.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
  • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
• Fits CONTRIBUTING.md.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/1620

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-already-reviewed/2617/762

[gador]

Just for anyone dropping by:

This harsh update from version 3.4 to 6.0 breaks unifi which uses pkgs.mongodb by default.
For the next time, can we please update more gradually? I know, mongodb 3.4 is EOL for quite some time now, but other services and packages depend on it and should have more gradually move to the new version to allow the updates to not break the database.

For others who noticed the database getting corrupted:

nix-shell -p mongodb-3_4 mongodb-tools
mongod --dbpath /var/lib/unifi/data/db  --logpath /var/log/unifi/repair.log --repair
mongod --dbpath /var/lib/unifi/data/db  --logpath /var/log/unifi/repair.log --journal --fork
mongodump --out=/root/mongodump
pkill mongod
exit
nix-shell -p mongodb-4_2 mongodb-tools
# remove old data dir and recreate it
mv /var/lib/unifi/data/db /var/lib/unifi/data/db_bak
mkdir /var/lib/unifi/data/db
mongod --dbpath /var/lib/unifi/data/db  --logpath /var/log/unifi/repair.log --journal --fork
mongorestore /root/mongodump
pkill mongod
nix-shell -p mongodb-5_0 mongodb-tools
(rinse and repeat)
nix-shell -p mongodb-6_0 mongodb-tools
(rinse and repeat)

I'm not sure whether thes gradual procedure is necessary, but at least it works.

EDIT:
It seems to be an isolated problem with unifi which defaults to pkgs.mongodb. This should obviously be fixed in the unifi service to be a fixed version

[Artturin]

This needs a release note and upgrade instructions if needed

[gador]

What would be the best way to tackle this?
I just tested this in a VM with an older commit (677ed08) with the older mongodb and a more recent commit with the newer mongodb-4_2.
Unfortunately even the update from 3_4 to 4_2 breaks the database. Should we add a check for stateVersion and a warning to first backup the database?

I just feel a release note isn't enough, since every user who updates to 23.05 will break their unifi installation.

[wegank]

Perhaps by specifying in the release note that MongoDB upgrades must be done (and progressively, following the official instruction 3.4 -> 3.6 -> 4.0 -> 4.2 (-> 4.4)) in 22.11 before upgrading to 23.05?

[gador]

@wegank and @mweinelt
I prepared a gist: https://gist.github.com/gador/a740144a6949b1272991cda5e35b8457 for an update of the mongo database.
This is tested on 22.11 with migration to the newer mongodb version before update as well as on current master with migration after the update. Maybe you could have a look.

Should we link it in the release notes? Include it in NixOS? I'm not sure what the best approach would be.

I also though about an automatic migration in the NixOS service, but I'm afraid this wouldn't be robust.

[RaitoBezarius]

@wegank and @mweinelt I prepared a gist: https://gist.github.com/gador/a740144a6949b1272991cda5e35b8457 for an update of the mongo database. This is tested on 22.11 with migration to the newer mongodb version before update as well as on current master with migration after the update. Maybe you could have a look.

Should we link it in the release notes? Include it in NixOS? I'm not sure what the best approach would be.

I also though about an automatic migration in the NixOS service, but I'm afraid this wouldn't be robust.

Awesome, thank you. I do not advise for automatic migrations as you remarked it would not be robust.

@riotbib, the release editor for 23.05 can help you find a way to format those instructions properly in the release notes.

[wegank]

@riotbib @winterqt Could you please take a look at including the instructions in the release notes?

[riotbib]

@riotbib @winterqt Could you please take a look at including the instructions in the release notes?

Sure, @wegank. Added to my TODOs. Thanks for the reminder!

[Ralith]

This bit me today, after reading the release notes and seeing no mention of breaking changes :(