Tool: Build main game data from Game.agf

[ivan-mogilko]

A stand-alone tool which may be run from command-line (no GUI), which parses Game.agf xml and writes main game data in binary format.
Presumably C++ is better choice here, as it's more portable than C# , and also there's already a game-loading code in the engine that may be reused.

NOTE: this task is exclusively in writing this tool, adjusting Editor is a separate task, so no need to concern yourself with that if you are doing this. In fact it's best to assume that Editor will not be present to ensure tool's work result has no reliance on it.

Input:

• an xml file in AGS 3.* project format (path to file);
• path to output file;

Output:

• file containing main game data in binary format (options, global objects, and so on).

---

Details

TODO: info on how this data is stored in Game.agf

Writing binary format

The C++ code which reads game data may be found in a dedicated h/cpp pair:
https://github.com/adventuregamestudio/ags/blob/master/Common/game/main_game_file.h
https://github.com/adventuregamestudio/ags/blob/master/Common/game/main_game_file.cpp
Relevant functions are OpenMainGameFile() and ReadGameData().

The C++ code that writes same data was once present in the AGSNative dll sources, but it was very "dirty", and it was removed after C# variant was implemented (it was too bad to keep, and one can recreate it using reading code anyway).
In C# writing is performed by DataFileWriter class: https://github.com/adventuregamestudio/ags/blob/master/Editor/AGS.Editor/DataFileWriter.cs
But if the tool would be written in C++ I'd recommend using C++ reading functions as the main reference.

In fact, it will be fine to put data writing in the same main_game_file.cpp, for consistency, and link it to the tool.

To be amended.

---

See Also

#1146 - task on another tool which also parses Game.agf but for different purpose.
General overview of the game building process: https://github.com/adventuregamestudio/ags/wiki/AGS-Game-Build-process-(3.5.*)

[ericoporto]

I am reading the C# implementation

ags/Editor/AGS.Editor/DataFileWriter.cs

Line 1384 in 4934cd0

public static bool SaveThisGameToFile(string fileName, Game game, CompileMessages errors)

It appears it would also need a dir containing the binary of the built scripts as input of the tool, since the script binaries are later stored inside the game28.dta structure.

---

Some notes here... The GameSetupStructBase code is already replicated in both C# and C++ (no idea if we ever test the c++ code path, is this a case of matching data and savegame?)

ags/Editor/AGS.Editor/DataFileWriter.cs

Line 498 in 4934cd0

private static void WriteGameSetupStructBase(BinaryWriter writer, Game game, out long ext_off_pos)

ags/Common/ac/gamesetupstructbase.cpp

Line 192 in 4934cd0

void GameSetupStructBase::WriteToFile(Stream *out, const SerializeInfo &info) const

I guess the reading functions in gamesetupstruct that don't have a write pair needs to have then written, like custom properties, and interactions_scripts (I don't know if these are relevant?) and some other things there (perhaps things that don't make into savegames?)

And then once this is done the main_game_file.cpp can have the write functions added, I think in this particular file the writing function can be a single one, it looks like the multiple reading functions are trying to figure out retrocompatibility stuff.

[ivan-mogilko]

It appears it would also need a dir containing the binary of the built scripts as input of the tool, since the script binaries are later stored inside the game28.dta structure.

I recommend not saving scripts inside game28.dta. Engine supports reading separate script assets since 3.6.0 (iirc). It would be very beneficial to split scripts out of regular game data. Then "game data" writing tool will not have to bother about them. The scripts are written by compiler alone, and agspak packages everything together.

[ericoporto]

The name game28.dta is a reserved name that has to be this, right?

I think the tool can then take two parameters, the Game.agf file and a directory to output the file - which is game28.dta now but may be something else in the far future, but will always be a fixed named afaict. It doesn't look like it needs any additional options then.

Needing a name... I guess agf2maindata could work.

[ivan-mogilko]

I like shorter names more, could be agf2dta, for example.

[ericoporto]

Just to check I am not doing something wrong

• in Tools/data/game_utils.h, ALL properties from Game.agf xml files must be there (currently only a small few are there)
• we will read from these from the xml and write to LoadedGameEntities (in agf2dta code)
• in Common/game/main_data_file, have a function that does the reverse of ReadGameData to perform the write to a file (from a LoadedGameEntities).

[ivan-mogilko]

Just to check I am not doing something wrong

• in Tools/data/game_utils.h, ALL properties from Game.agf xml files must be there (currently only a small few are there)
• we will read from these from the xml and write to LoadedGameEntities (in agf2dta code)
• in Common/game/main_data_file, have a function that does the reverse of ReadGameData to perform the write to a file (from a LoadedGameEntities).

That sounds right.

I am not particularly certain about LoadedGameEntities, because it was filled without much plan, and for example has few members of reference type instead of actual objects. The latter may be fixed (I just forgot about them). If it turns out inconvenient for the purpose of writing things back, then it may be replaced by something else, but in the end it might also be a struct with a "collection" of all the game data.

[ericoporto]

There is a struct called GameRef in Tools/data/game_utils.h, and in there there is GUIRef, and so on, and some things are using some EntityRef, can I make these structs complete? Meaning, make they hold all properties of the Xml file.

My approach is make each property be either an int, a bool, or a String, so I can make the reading the XML and filling this struct a thing, and then something else has to solve the translation of that to the actual LoadedGameEntities - mostly a few Strings need to become an enum.

I would like to separate the XML reading from the "translation" to LoadedGameEntities, because it's a lot of things and going this route I think it will be easier to debug/test things.

[ivan-mogilko]

The "xRef" structs were meant as minimal necessary object description, only with its id and subobject refs. There are functions that read only "Ref" struct, because some processing don't need anything else (such as ReadEntityRef and ReadAllEntityRefs).

If these "ref" structs would be replaced with full structs, then the functions that read these "refs" will have to be adjusted too. I can't tell right now what would be a most convenient approach here. E.g. these full structs may inherit EntityRef, and passed relayed into a vector as EntityRef* pointers. But I suppose all this is a matter of finding a good code organization.

[ericoporto]

Hey, should the full structs have other name, even if they inheirit from the Ref structs (say they use xData, like GUIData, GUILabelData and so on) and have their own set of functions to fill them?

[ivan-mogilko]

Hey, should the full structs have other name, even if they inheirit from the Ref structs (say they use xData, like GUIData, GUILabelData and so on) and have their own set of functions to fill them?

Yes, I suppose they should not be called "xRef", since i used that name to indicate that they are merely "reference" of something rather than full data.