State of repository vs Community Outpost's Patch goals

[xezon]

Community Outpost wants to roll out Patches with conservative changes in small steps. The first release(s) are supposed to be match compatible with 1.04. This is an increased complexity for both code and data. It is good that we know this early, so we can plan for it in the Code repository and it should be no problem, but this Patch repository was never setup to go this route, so there are challenges.

This repository contains dirt.

Branches

This is probably not a big concern, but this repository owns too many branches (~200). Basically we have created branches directly on here, instead of the contributor's forks. We will change this and no longer allow feature branches in this repository and slowly phase them out after they are merged or closed.

A relevant question here is though, is the git memory cleared after branches are deleted? I hope we have not grown this repository unnecessarily large by doing that.

LFS

At some point we experimented with LFS for data. We noticed it is useless because it causes high costs. We abandoned the LFS. The LFS data that we pushed is still asociated with this repository. It is about 500 MB large. I was unable to figure out how to delete it. We still have the branch that contains the links to the LFS, and perhaps it can be cleaned up through that. It does not seem to affect behaviour of the repository, but it is dirt that would be nice to clean.

https://github.com/TheSuperHackers/GeneralsGamePatch/tree/main-lfs

Early commit mess

In the early days of this repository we have not been diligent with our commits. This concerns the first 100 or so commits. We added and edited files as we went ahead without setting up a clean foundation of all original 1.04 plain text files first. In hindsight this was a mistake, because now we cannot simply create a new branch from an older base and cherry pick changes to slowly build a new version.

The bad:

• No clean foundation. Game files were added while other changes were already applied. Here is a big offender: Fix: USA Microwave Tank structure bug #3 It was still very early, but as you can see INI files were added together with edits while other edits were already applied in previous commits.
• WND, STR file(s) were added in other commits much later in time.

Not all is bad though. The good:

• Plain text game files (INI, WND, STR, etc) were never moved once we put them in a folder
• Plain text game files were never refactored/formatted at any point
• Binary assets have little to no revisions
• The vast majority of changes are very clean, doing just one thing, with no unnecessary refactorings around it
• WND and STR have clean revisions with clear steps

It is unclear to me if this can be repaired. Changing commits this low will rewrite the whole history and break hash links with github records and could have bad side effects in regards to pending pull requests.

Considerations

With the current goals for the Community Patch in mind, it looks like we would perhaps not have just one main branch, but have

main -> contains all changes
main-noncontroversial -> contains no controversial changes
main-non104mismatch -> contains no changes that mismatch with 1.04 game state

How we would be able to achieve this is not clear to me. I hope someone smarter than me has some ideas :-)

Also, one consideration here, if we have 3 main lines, will cherry picks of data to different commit lines incur new copies or will git reuse data properly? If data would be duplicated then this would be bad for the repository size and we would then have to consider moving binary assets (textures, audio) into seperate repositories, at which point it would make sense to start all repositories from scratch, because this repository already contains big assets in its commit history.

If we would not go with 3 main lines, then we would have to go with the Bundle packs approach, where we use markers in INI files to compile out or compile in portions for different packs. We do have the tech for that already in place, but it will be a lot of work to put these markers and it may make the INI's very complicated to maintain for scripters.

The INI markers would look like so:

;patch104p-core-begin
PrimaryDamage = 60.0
;patch104p-core-end
;patch104p-core-controversial-begin
PrimaryDamage = 70.0
;patch104p-core-controversial-end