Document why we ignore all tests but the first

[petertseng]

We should document this somewhere in this track. We may incorporate by reference any other documentation elsewhere in Exercism such as exercism/exercism#856, but it is convenient to have it on hand somewhere in this track, so we can explain in what way the Exercism-wide policy affects this track.

This documentation should go in the README (or some suitable place) so it is visible to contributors who may wish to contribute a new exercise.

It would probably also be good to show this in a place visible to students, so they understand why their tests have #[ignore] everywhere. Consider placing it in docs/TESTS.md I suppose? https://github.com/exercism/xrust/blob/master/docs/TESTS.md - it shows up on http://exercism.io/languages/rust/tests.

Note that before #266, GETTING_STARTED.md used to say that tests were ignored but still did not explain why. Now there is only one test (which is not ignored) in hello-world, so it it is no longer the right place to put this explanation, just providing a bit of history.

I am unsure of whether to nominate this for good first patch. On the one hand, I figure this is a decision with historical context behind it, and the explanation might be best written by someone with that context. On the other hand, the explanation might instead be best written by a newcomer who benefitted from one-test-at-a-time and so can explain it firsthand.

So, feel free to chime in if you have experienced the benefits and want to explain it.

[petertseng]

Hello. There is a paragraph in https://github.com/exercism/docs/blob/master/contributing-to-language-tracks/exercise-test-suites.md added by exercism/docs#34 that explains that some tracks do this, and gives some explanation.

Your contribution might simply be to link to that document in this track's README (or other suitable place).

If you believe that that document sufficiently explains why this track does this, that would be all you do in your contribution. If you believe that extra explanation is required, you would of course add it.

[IanWhitney]

Why did you remove good first patch, @petertseng? This seems like a perfect candidate for it.

[petertseng]

Why did you remove good first patch

explained

I am unsure of whether to nominate this for good first patch. On the one hand, I figure this is a decision with historical context behind it, and the explanation might be best written by someone with that context. On the other hand, the explanation might instead be best written by a newcomer who benefitted from one-test-at-a-time and so can explain it firsthand.

so it sounds like you agree with the latter.

[sacherjj]

As someone who started learning Rust two weeks ago, two things to comment on:

1 - Rust is actually at a disadvantage compared to some other languages for a new user, due to the intelligence of the compiler. While the tests are supposed to be incremental, #[ignore]'d or not, all tests must compile. If this is implementing multiple functions and other things, it would be easier to comment them out. (I have done this early on.) I'm not sure of an elegant solution for this.

I reverted to doing cargo build, rather than cargo test, then moving back once cargo build works. Maybe we can make this more explicit.

2 - I don't know if I've read anywhere about cargo test -- --ignored, to run all the ignored tests without having to delete each #[ignore]. I think that would be helpful for the user. They may get it in one shot. If not, they can always setup by step it.

[ijanos]

Regarding to your first point, one way to avoid this is to provide lib.rs files, we discussed this a few times, start reading #117 if interested.

[petertseng]

Actually, I'm going to call this resolved by #334 if that's OK, because the README links to https://github.com/exercism/docs/blob/master/language-tracks/exercises/anatomy/test-suites.md and the example test suite in the README shows the second test being ignored. If it needs to be more explicitly called out that second-and-subsequent tests should be ignored, just reopen.

[DanielSank]

I just started using exercism (I have a bit of experience with Rust) and I am puzzled by the tests being ignored.

[coriolinus]

At the top of the Rust track README there is this:

At the most basic level, Exercism is all about the tests. You can read more about how we think about test suites in the Exercism documentation.

Following that link, we get this:

In order to provide a better user experience, some language tracks only leave the first test in an exercise test suite active. All subsequent tests are marked as "skipped" or "pending" using whatever method or directive that is provided by the testing framework. This means that the person solving the exercise must manually edit the test suite to activate the tests one by one as they write their solution.

In short, the Rust track encourages incremental TDD by ignoring all but the first test, and implying that students should un-ignore one test at a time as they write a solution which passes all of them.

This is arguably a wart in the Rust track; I certainly prefer to run all the tests every time. However, it's a long-standing policy which we haven't had occasion to revisit.

Some potential ways forward, if you feel strongly about this:

• Open a PR adding some documentation about this to the README
• Open a PR which removes all #[ignore] annotations through the track and adds a documentation note that the Rust track no longer uses this style of TDD. If accepted, this PR would change track policy.