Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a pretty important functional change. The work is complete, however it is marked as WIP as I have so far only tested the changes on a sqlite3 database.
I would very much like some help with testing this change, particularly on postgres, but also anyone out there who has a modified DB or a large dataset - your testing would be awesome.
What to test
FORCE_MIGRATION=true
FORCE_MIGRATION=true
I've already tested these things on sqlite3, and it all seems to be working ok.
Why this is changing
A little while ago, I did a PR to add the ability to drop columns in our database migrations/upgrades: #6467. This didn't work as expected, as on migrations where a table was deleted, we'd build commands to delete the table, followed by deleting all the (already deleted) columns. This prevented upgrades from 002 -> 003 from working, but they are sufficiently old (0.5.0 was released 2014-08-11) that it was safe to just remove these old migrations.
However, this got me thinking about how, although migrations have always been destructive in that they would delete any table not defined in schema.js, we'd never previously had destructive migrations for columns. This would adversely affect any users who had, for example, added an extra column to their posts tables.
Further still, because all the migration operations were automatically calculated by figuring out the differences between the existing DB structure, and what was in the schema file, there were absolutely no guarantees what might happen when a user ran a migration.
This makes it very hard to reason about upgrades, and makes them almost impossible to test - the test coverage for migrations has always been absolute zero.
All of these reasons added up together resulted in me deciding to remove the old automatic migrations in favour of explicit tasks which describe what needs to happen. These new, explicit tasks have 100% test coverage. The downside is that they're a bit harder to write, and we're now responsible for ensuring that the populate & upgrade flow result in the same DB structure, however I believe this is still an improvement particularly as we are able to test much more thoroughly now.
refs #6301