Upgrade to rails 7.0

[Taeir]

Turns out your application was not using defaults for rails 5 either, so this essentially updates the configuration to the defaults for rails 5.0 -> 5.1 -> 5.2 -> 6.0 -> 6.1 -> 7

Testing

Changes were tested in a production setting on rails 6.1 / rails 7 for several weeks (Ruby 2.7, using passenger rather than puma).

Upgrade process

The upgrade can be done in one go, but the Redis cache must be cleared after shutdown and before starting the new version as the data in the cache is not compatible with the old version of rails. (Rails.cache.clear or in redis-cli use FLUSHDB).
NOTE: in the case of a rollback, all cookies are invalidated and users will have to log in again (and client-side stored settings are cleared).

If you want to avoid the potential of some users losing their cookies upon a rollback, then add an initializer (config/initializers/upgrading_rails_6.rb) with the following content:

Rails.application.config.action_dispatch.use_cookies_with_metadata = false
Rails.application.config.action_dispatch.cookies_same_site_protection = nil
Rails.application.config.action_controller.urlsafe_csrf_tokens = false

Then you can upgrade in 2 steps. First upgrade (clear cache!), check whether application is stable, and if it is remove the initializer and restart. Otherwise, you can shutdown, clear cache and rollback.

Considerations

All setups

• (PUMA) I upgraded to puma 5. It depends on how fancy the production setup is whether any settings for puma need to be changed, but if the configuration is not too fancy, it should work out of the box.
• (7.0) Make sure to install the libvips library, as image processing moves from imagemagick to that one.
• (6.0/6.1) Cookies set by rails 6 cannot be read by rails 5. This sounds scary, but in practice this means that Rails 5 will reject any session cookie set by Rails 6 and just throw it away. So in the case a rollback is done, users are just logged out and need to log back in. See above for alternative.
• (5.2) Cache versioning is now enabled. This means that in redis, things are now stored under a base key that does not include a version number (i.e. timestamp). This should decrease stale data in cache. See e.g. https://www.bigbinary.com/blog/rails-adds-support-for-recyclable-cache-keys
• (5.1) URLs to elements which are not in the asset pipeline do not fallback to the public/assets folder. Since there are multiple assets only available in the public folder and not in the app/assets folder, this may be an issue for your setup. On the other hand, I believe you serve the public folder from NGINX, in which case it will work just fine.

Other

If you are self-hosting a QPixel instance with a custom setup or have made code changes:

• Rails 6 default settings drop support for Internet Explorer 8 (Windows XP) and below. If this is an issue, the setting Rails.application.config.action_view.default_enforce_utf8 = true must be set.
• If you have mail which uses the deliver_later option and use sidekiq, you may need to add a new queue to sidekiq. The normal codebase does not use jobs and does not have delayed mail delivery, so it is unlikely you have these set up.

Dependency Notes

• QPixel now requires at least ruby 2.7+. I added a note in the installation guide.
• All packages were updated to their latest major version (which supports ruby 2.6), unless indicated below.
• Omniauth should be updated from v1 to v2, but this requires testing. Omniauth is used for grabbing user data from stack exchange, which is hard for me to test. Therefore I left it as is.
• While sass-rails is upgraded to sassc-rails (that is what v6 does), neither gem is still maintained. The real solution is to switch asset processing to cssbundler (requires more work).
• Stripe did 2 major releases this year, but I cannot easily test this. Therefore I left it at the last version of the currently used v5.

"Fun" things

Interesting things I came across, but which are not so relevant to this PR

The following section is not fully relevant, just me rambling about the changes being annoying to apply.

Single Quotes vs Double Quotes

Rails apparently decided to use double quotes everywhere, and it put them in everywhere when updating the configs. I really don't care, but rubocop does, so I made everything single quotes.

Why so many changes

It's mainly because of rubocop. It seems some new rules were added which require layout changes across the project, causing a large number of very small changes.

Why are settings moved around?

I move the settings to minimize the diffs with the rails upgrade config file suggestion (i.e. match the locations as described by rails as accurately as possible). Apparently they don't have consensus on where things should be, so stuff moves around.

Something something webpacker

Don't even get me started. Apparently we need to change how things are being done with every single rails release. Rails 6 introduces webpacker as the "new default you should switch to" and rails 7 says "webpacker is retired" (as in, will no longer be getting any new features). Instead, for Rails 7 we should now be switching to jsbundling-rails (see below). I guess we are lucky that we waited with upgrading to rails 6 until rails 7 had already deprecated the new things rails 6 introduced, such that we don't have to bother rewriting things again to fix it.

Something something asset pipeline

The internet is moving forward. NPM/nodejs is cool, sprockets is not cool. Rails 6.1 still has sprockets as required dependency, but rails 7 makes it optional. Looks like the asset pipeline is no more in rails 7.

Instead, we have the great and amazing definitely-not(-unnecessarily?)-more-complex jsbundling and cssbundling which essentially throw yarn into the mix to compile and prepare assets and throw the resulting files in the assets directory. Or you can use webpacker (though that was already deprecated again). The gist of it is that people got used to the NodeJS/NPM ways and thus rails moved to it too.

Why didn't you update to jsbundling/cssbundling?
Because the production setup required is a bit different. Don't get me wrong, it should probably be done, but it requires some effort that I'm not yet ready for. Perhaps it's not so hard, or perhaps it breaks all the css/js loading and requires a complete change to assets and stuff.

SASS(y)

The SASS team had not enough dev power to maintain their ruby-sass library (deprecated in 2017), nor their libsass C++ compiler (deprecated in 2019) (on which the new ruby-sassc was based). Instead, they decided to go all-in on dart-sass. Now, by this time, the asset pipeline was already no longer cool, so noone (that I know of) made a ruby library wrapper around dart-sass. Instead, since the world was already moving to NodeJS/NPM/Yarn for handling js and css, they can also handle sass/scss, which is the new recommended way (not in this MR). If the switch is made to jsbundling/cssbundling, then sass can/should/must also be switched to do it in cssbundling/yarn.

[ArtOfCode-]

Looks good so far!

[ArtOfCode-]

This is definitely unmaintained but AFAICT still works. However, we should review whether we're actually using this - I think our pagination may have been separated from this some time ago now that we use our own design framework but the dependency hasn't been removed? Not critical, can be left for now.

[Taeir]

I checked by removing it, and it is definitely still being used for the pagination of posts.

[ArtOfCode-]

Does this no longer work as one clause with the upgrade?

[Taeir]

It triggers a deprecation warning because rails 6.1 will change the behaviour of having multiple items in one not clause. It changes from !a && !b to !(a && b) aka !a || !b.

[Taeir]

@ArtOfCode- This is ready for review.
We ran with rails 6.1 for a while and then upgraded further to rails 7. I can add those changes (not so many) into this PR or add the upgrade to rails 7 in a separate one. What do you prefer? If anything breaks it is a little easier to pin the problems down if less stuff changes at once, but assuming no major issues it will take more time and effort to upgrade twice of course.