New Error Handler with backtrace and symbol support

[vladikcomper]

This PR aims to install a new error handler from the MD Debugger project to SGDK. The old error handler is still preserved for compatibility and is available by setting LEGACY_ERROR_HANDLER to 1 in config.h.

It also enhances symbol.txt files by adding source-line information (file path and line number for every symbol, where applicable).

Overview

• Displays debug symbols from the source code with "debug" build profile, displays raw offsets only with "release" profile.
• Symbols are attached to the end of ROM in custom compressed format by the new bin\convsym.exe utility.
• More readable exception screens with formatting and a few built-in debuggers:
  • Press B to display backtrace;
  • Press A to display address register info (decoding address register values as symbols, if applicable);
  • Press C or Start to return to the main exception screen.

New error handler with "debug" profile

Backtrace in action

SGDK API changes

1. Freed about 88 bytes in RAM by removing busErrorCB, addressErrorCB and other error callbacks as well as registerState and other global variables used by the old error handler;
2. Removed all legacy error handler functions from sys.c (unless LEGACY_ERROR_HANDLER flag is set);
3. Modified boot/sega.s to include the new error handler as a pre-compiled blob;
4. Added bin/convsym.exe utility version 2.12 (compiled from https://github.com/vladikcomper/md-modules/tree/master/utils/convsym) to convert symbols for the MD Debugger;
5. Modified makefile.gen to add new injectSymbolsInROM target for "debug" build profile, so debug symbols are only injected in debug ROMs.
6. Symbol injection is performed by convsym.exe after building the ROM: it appends symbols to the end of the ROM and injects symbol data pointer at the offset of MDDBG_SymbolDataPtr variable defined in sega.s (barebones linking).
7. symbol.txt enhancement: modify makefile.gen to compile object files with -g flag to include source-line information, invoke nm with -l flag to dump this information.

Error handler implementation details

1. Error handler was designed to be stand-alone and portable. It aims to support all major 68K assemblers (now GNU As as well), so it comes as a pre-compiled blob with minimal bindings;
2. It doesn't require any memory, everything is placed on the stack, memory consumption is small (survives with 128 byte stack);
3. Supports symbol tables in my custom DEB2 format (generated by convsym.exe); format is optimized for fast searching (faster than O(log n)), symbol data is compressed;
4. Flexible console engine with buffered IO and automatic line breaks (long lines won't be cut off).

How to test

You can apply the following patch to sample/game/sonic (my favourite game):

diff --git a/sample/game/sonic/src/main.c b/sample/game/sonic/src/main.c
index 0ea29d42..3d5128b1 100644
--- a/sample/game/sonic/src/main.c
+++ b/sample/game/sonic/src/main.c
@@ -134,6 +135,8 @@ static void joyEvent(u16 joy, u16 changed, u16 state)
     {
         paused = !paused;
         HUD_setVisibility(paused);
+
+        asm("illegal");
     }

You should now see it crash when pressing Start. You should see crash offset as joEvent+XXX with "debug" build profile.

Compatibility

1. Existing SGDK projects must update their local boot/sega.s.
2. You must rebuild SGDK library if LEGACY_ERROR_HANDLER is disabled.

[vladikcomper]

Added some comments explaining key changes.

[Stephane-D]

So far it's an important but perfectly well prepared and detailed PR 👍 Thanks a ton for it ! it's a terribly useful and great addition to SGDK which ease a lot debugging :)
The only change I would request if possible, is to include the sources of your neat convsym tool in the SGDK tools subfolder so people using others OS (unix mainly) can rebuild the tool from sources without searching for hours for it ;)

[vladikcomper]

@Stephane-D Thanks a lot for reviewing it quickly and liking my debugger in the first place :)

I made a few updates following your feedback:

• ConvSym's base source code is included (I didn't include test suite though, it'd be an overkill: I extensively test symbol conversion against listings of huge projects and disassemblies, that's millions lines of codes, so my tests are over 70 MiB 🙂)
• I split everything error vector related from boot/sega.s to error_a.s in the library itself. And it's much cleaner and easier to maintain now, this was definitely the right move.

One tiny thing: Codacy freaked out about about ConvSym's source code (mostly unusual Markdown syntax, haha), so I think it's better to add ./tools/convsym to ignored files in Codacy settings.

[vladikcomper]

@Stephane-D I also recommend rebuilding libmd on your machine just so builds are consistent. I'm not sure why libmd.a is almost 1 MiB bigger now, but libmd_debug.a is 18.7 KiB smaller :)

[Stephane-D]

@Stephane-D Thanks a lot for reviewing it quickly and liking my debugger in the first place :)

I made a few updates following your feedback:

* ConvSym's base source code is included (I didn't include test suite though, it'd be an overkill: I extensively test symbol conversion against listings of huge projects and disassemblies, that's millions lines of codes, so my tests are over 70 MiB 🙂)

* I split everything error vector related from `boot/sega.s` to `error_a.s` in the library itself. And it's much cleaner and easier to maintain now, this was definitely the right move.

One tiny thing: Codacy freaked out about about ConvSym's source code (mostly unusual Markdown syntax, haha), so I think it's better to add ./tools/convsym to ignored files in Codacy settings.

Thanks, that sounds perfect 👍
About Codacy reporting, well anything in tools isn't so much a problem, it was more to monitor SGDK sources code itself ;)