Editor Testing Frameworks

[persn]

I think we should add projects that allow writing tests for the editor.

• Decide on a test framework
• Decide on a mock framework
• Add test projects for the editor projects that use the previously mentioned frameworks
• Add some few tests to begin with (We can use these tests to discuss guidelines for writing tests if we want)
• Add test runs to the CI

[persn]

I propose NUnit and NSubstitute, simply because these are the tools I use, and I know they are very good. If no one makes objections in a reasonable amount of time (a few days?) I'll make a PR for it

[rofl0r]

I think we should add projects that allow writing tests for the editor.

what kind of tests ?

[persn]

Unit Tests to begin with

[ghost]

I think the question is rather, what functionality do you want to test.
This is important, because there's an interest in changing the editor workflow, such as the way it works with the source files, and decoupling game building functions into standalone tools.

[persn]

The idea for starting out is just to make the infrastructure for testing available, I don't have plans to write 90% plus test coverage or even 5%. This is important because I've fixed bugs in the past where having tests could have made my workflow smoother instead of manually testing every case for every change for every feedback. So no specific functionality is what I'm trying to say I guess

My motivation for having test frameworks available is because I'm researching if I'm capable of solving the issue of "All the room management should be reimplemented in .NET." as stated in #469, and I think it might help to have tests that checks that room handling works as it should before I start messing with it.

[rofl0r]

what kind of tests ?

I think the question is rather, what functionality do you want to test.

right. i'm asking because "writing tests for the editor" actually sounds as if it would require some click automation software like autoit in order to assert that clicking certain stuff in GUI produces the desired result.

[morganwillcock]

I've twice started something similar, using XUnit. I think it would be OK to add it tests for specific parts which are being modified (and write new code to be testable), but I'm not sure how much of the existing C# code will exist long term (so aiming for total test coverage could end up being a waste) as there isn't a long term plan at the moment.

It may be beneficial to continually track the latest Visual Studio project version and increase the .NET Framework version to the highest compatible level, if builds are then depending on tests that run in .NET. I think that the XUnit console runner needed 4.5.2, otherwise it forces the environment to have a mass of extra Nuget packages.

[persn]

right. i'm asking because "writing tests for the editor" actually sounds as if it would require some click automation software like autoit in order to assert that clicking certain stuff in GUI produces the desired result.

Well like I said unit tests to begin with, which excludes click automation, UI tests are on the completely opposite end of the test pyramid spectrum. On unit test level it's still useful to simulate click events, which mocking frameworks like NSubstitute is able to do

[persn]

Fixed in #1040