Add some way to trace program read/writes

[JomerDev]

Hello,

first of all I really love this project. I use it to simulate/trace the program code of an old dmx light controller (MA LightCommander 12/2) which uses the MC68008, reading the assembly is one thing but it is much nicer to let it run and actually see how it works.

To trace memory access better I tried to find a way to somehow get the current program counter in the read and write methods of Addressable devices, to be able to print out not only what address was read or written but also from where in the program it took place.

Sadly, after multiple tries I did not succeed in finding a good way, so I am now asking you if you have a suggestion how to make it work.
I tried adding a write with program counter command to run_command of the M68k, however to use it I would need mutable access to system in my device, which I couldn't make work.
Then I tried storing the log message in the device and call a print out method after every step with the current program counter, however that needs access to the cpu outside of the device, which I was unable to get without unsafe casting from the device.
Additionally I thought about adding a new error type which would add the program counter to the inner string, but that also stops the execution of the cpu, which I wanted to avoid.

My final idea (which I haven't tried yet) would be to instead of clock add a context variable to the read and write methods. It would be passed by reference through all addressable methods that currently have clock. The context variable would be a struct with at least the program counter and the clock in them. It would be created in the cpu on read/write and passed through every addressable in the chain (M68kBusPort -> Bus -> Any addressable device)
That way and device that supports addressable can also log some access information if it needs to. It could also help to create better error messages when a read/write address is outside of the valid segments

If this idea sounds workable to you I'm happy to write a PR as well

Best, Jomer

[JomerDev]

I have for now modified get_address_sized and set_address_sized to print out the program counter and the address and value that is read or written. I'm also planning to play with the tracing crate a little bit and see how well that could work

[transistorfet]

Hi!

Thanks! I'm glad you're finding it useful. I love to hear how people are using this project so I have a better idea of how to improve it. That sounds like a really cool project your working on! Did you just extract the ROM from the controller and are running the code in the emulator to see what I/O it's performing?

As you mentioned, the get_address_sized functions, or the functions in m68k/src/memory.rs, which all m68k memory accesses go through, is a good place to put some print statements, and for something more permanent, adding some kind of tracing that can be conditionally compiled in would be best. Perhaps storing tracing baggage in the System object somehow. Or the tracing info could be internal to the m68k and use the Inspect trait impl's to recall and print the data, along with the built-in debugger's trace command to step the simulation without breaking printing the Inspect info.

It would be nice if there was a way to downcast objects, but I think I ran into trouble when using trait objects at the same time as the Any trait. I'm in the process of rewriting the core here: https://github.com/transistorfet/moa/compare/transistor/use-emulator-hal-in-z80...transistor/rewrite-core-to-use-emulator-hal-interfaces?expand=1 but I've stalled due to work eating all my time. I was planning to improve the emulator-hal traits as well. Initially I had wanted the Instant type variable to be usable as anything, including as a context object like you mentioned, but I had to make it implement a time trait in order to make it useful for it's intended purpose. It should be possible to implement that trait on a custom context object though. It's a bit of a kludge but it could work.

It also might be possible using the emulator-hal traits to make a top level custom System-style object that then uses an enum or direct references to the devices instead of making every device a trait object stored in a HashMap or Vec. That would then allow device-specific methods to be called in between steps instead of making everything go through traits. That is kind of a long term goal, of making it possible to have a custom system that's faster or a trait-object based system that's more flexible, while using all the same implementations, but that might be too wishful on my part.

I'd love to see any custom code you have, if you're willing to share it. I've only seen my own code, but I find it useful to see the approaches other people take.

[JomerDev]

Pretty much, yes. I currently have two of the controllers to play with, bought second hand (and partially broken). I got very lucky and have two different software version binaries. Afaik the last software update the controllers ever got by the company that built them is from 2007.
I have looked at the disassembled binaries a lot and know which areas are doing what, but it is still helpful to see the order the code gets run in and to debug it as well.
My larger goal is to create an extension for the controller which slots in the eeprom slot (since that is the only ic that is socketed) and provides both the eeprom and wifi functionality. For that it'll also be really helpful to emulate the additional code for the wifi extension and see if it works and to debug it. The controller itself has a watchdog for the processor build in along with other pcb connections that make debugging directly in the controller very hard (though not impossible)

In this branch of my moa fork I've got my current progress. I added logging via tracing to some parts of the m68k cpu implementation, the result of which you can see in the log file in the new lightcommander frontend. (I also fixed a tiny bug here where the display implementation wrote out the wrong instruction).
I'm currently somewhat stuck, the program crashes because it tries to clear parts of the eeprom (which is of course read only), but after trying to figure it out for some hours I doubt the emulator is to blame. The decoded instructions are correct, this may actually just be a bug in the controller which never got caught since on the actual pcb the eeprom doesn't care that it got written to. (It was also very annoying to find, I forgot to set the eeprom to read only at first which then meant that some instructions were cleared, which then turned into the program suddenly jumping to non-existent addresses). The alternative explanation might be that in the real controller there was always an interrupt before this point.

Which brings me to the first of my two current issues:
I need interrupts. I can see that you've worked on them a bit, there are some code bits commented out, but as far as I can tell they are currently not ready to be used. I have an idea for a potential solution where the interrupts are passed alongside either the address bus or a separate interrupt bus . One issue where I don't have a complete idea yet is that in my case there needs to be a way for the interrupts to be muxed. On the pcb of the controller some interrupt lines are combined in an AND gate.
I've been hesitant so far to put my idea into code since I could see that some work on the interrupts already existed and that the idea behind them was different.

The second issue is pretty much what you described in your second to last paragraph, for some devices it would be very helpful to be able to access them between steps. The biggest reason why I added the tracing printouts was because I couldn't find a way to access the cpu registers between steps. I also have ideas for a tauri build frontend for my tests (I already had to hardcode a peripheral value to return a specific value once, since I currently haven't added a way to actually press the two buttons it expects me to press)
I'm not really sure on a good way to achieve the access to the devices, but I'll think on it.

All in all, I'm happy to contribute my fixes and if you've got interest my alternative take on the interrupts as well.

P.s. if you want to run my code yourself, I haven't pushed the rom binary since the company still exists and still provide support for the controllers, but if you're interested in it I can send it to you via email or something. And if you don't want to share your email in an github issue you can send it to me on bluesky (I'm Jomer there as well)

Edit: Oh, I have one more question: How do the bcc/bra/bsr implementations work when the sign of offset is ignored? Is it just based around the overflow? Why not use u32::wrapping_add_signed?

[transistorfet]

That sounds like a cool project. Thanks for finding that bsr/bra bug!

Related to the issue of crashing, yeah... I made the emulator a bit overly strict at first, which helped me catch some bugs in the emulator early on, but then I later found that a lot of roms do weird things that would cause a crash or overwrite rom, so for systems like the Sega Genesis, I ended up using read-only mode for the roms and put a big chunk of memory that fills up most of the address space so that it doesn't crash, because I still want the opportunity to know if there's a bug when I'm testing my own 68k code.

For u32::wrapping_add_signed, that function actually wasn't available until 1.66, and I wrote the CPU implementation before then, but I also wanted to just pass around u32s everywhere and pass in the size as a second parameter, instead of doing type conversions everywhere, so it can control the output and preserve the parts that don't change, which is required to accurately emulate the CPU registers where the upper part of the register will be unchanged.

For interrupts, they do actually work and are used for Computie for the timer interrupt, and for the Sega Genesis for horizontal and vertical sync, and for the controllers. Sega Genesis games won't actually run and update the screen without them working because the vertical sync is usually used to advance the game's time. I made an object called Signal which can be passed around to send the interrupt signal between different devices, which is passed in when creating devices, and the step functions can then use the interrupt controller object in System to raise an interrupt in the CPU. I've rethought it and tried to refactor it many times without settling on a better method. I can't remember what I was going to do for the System rewrite, but there's something in progress here: main...transistor/rewrite-core-to-use-emulator-hal-interfaces#diff-94b711a0f71fa602139de897e5bd96ca9f0fff031adfa4c8c00d291191176689R325-R374.

To use what's already there for interrupts, I'd take a look at peripherals/motorola/src/mc68681.rs for a simple example, and systems/genesis/src/peripherals/controllers.rs and ym7101.rs in the same dir for a more complicated example.

For accessing the devices, the manual way of directly stepping the CPU instead of wrapping it in a Device is probably the easiest way. If I get time, I'll try to fix up the work in progress and revisit the idea of somehow accessing the object within Device. If you have an idea of how to do that, I'd be happy to talk about it. I'd want to avoid breaking what already works though, so it might be good to chat a bit before you sink a lot of time into a PR. We can chat in Github Issues (you could open a new issue for each topic), or my email address is on my website linked to my profile =P

I wouldn't mind getting the ROM too so I can give it a try, but it might be a couple weeks before I have a decent amount of time. I'm in a bit of a crunch at work at the moment, but I'm hoping things will quiet down in the summer. I really want to get the refactor complete and improve the performance of the web assembly version

[JomerDev]

Thanks, and you're welcome

Fair enough, I might have to do the same thing. Or implement a memory device that wraps the Memory Block but ignores/logs all writes.

Also fair, I did try replacing wrapping_add(offset as u32) with wrapping_add_signed(offset) for bcc/bra/bsr and ran my program with the logging again. There not a single difference, so should you want to remove the casts in those methods, you can.

Ohh, that is good to know, in that case I probably read the code that was in the middle of the refactor. I'll try and play with the interrupts a bit more as I have time over the next few days.

For the interrupts in general I realized today that I'll need a system that is a bit generic, in my case not only are two interrupts ANDed together, one of them originates actually from one of the I/O pins of the MC68681. So I thought of a system where I/O pins could be connected via the system to allow for these kind of interactions to happend. While I have a pretty good idea on how the system could look like in terms of functionality, I did not have enough time to envision how it could look like in code. But I'm definitely happy to discuss.

Regarding the system/accessing objects within a Device I realized that the trait system you've currently set up reminds me a lot of an ECS and wondered how an actual ECS handles the same challenges. I found evenio (among multiple other ECS crates) which combines an ECS with events. I haven't played with it yet but from what I saw it seems like a good project to either actually use or at least find some inspiration on how to store the different devices and get access to them. For this too I am happy to discuss.

Warning

Edit: After actually trying to figure out how an ECS could apply, I've found that the current system seems to almost be the > opposite -> trait driven components instead of data driven components. There are less similarities than I initially thought

Alright, I'll send you the ROM to play with via email :D
For chatting, issues seem a bit heavy handed to me but email is fine. I also realized that since you mentioned the embedded-hal crates in your repo for emulator-hal, an alternative might also be Matrix. I've been hanging out in the rust embedded and embassy rooms under pretty much the same name

[transistorfet]

I responded over email. Thanks for the PR #9. I'll take a closer look when I have time. I also hope to get the refactoring that's in progress finished up!