Feature request: TODO tests

[ghost]

In Perl's TAP testing framework, the developer can mark certain tests as "TODO", which are useful when you know what the correct result should be, but the code doesn't produce it yet:

http://testanything.org/tap-specification.html

Could this be added to the various expect_* and is_* functions, perhaps as an extra todo argument, or maybe as a style=c('test','todo','skip') argument that allows different options (to prevent future arg proliferation)?

[HarlanH]

The skip function is sorta the right idea, but I was hoping I'd be able to do something like:

skip(expect_equal(not_implemented_fn(), "cat"))

And have the result be a warning that displays the argument...

[krlmlr]

I'd be fine with the summary reporter displaying the location of skipped tests (if no failures or errors occur). Or would a dedicated reporter be better?

[richierocks]

Rather than having extra code in the test to say that the thing to be tested isn't ready yet, it seems to make more sense (to me at least) that the function uses.NotYetImplemented from the base package. Then reporters can optionally choose to display something different for cases of a not-ye-implemented error.

From a test-driven-development point of view this seems semantically more correct. The test should fail because the code isn't correct, but you still have the information there to report a sensible reason as to why it failed.

[kenahoo]

I've looked in #343, but I don't see how this supports marking certain tests as TODO. Is there a writeup of it somewhere else?

@richierocks It's not always feasible or even desirable to handle it in the package side. For example, you may have a bug and written a test for it that currently fails, but have no idea where the bug sits in the package. Specifying the TODO semantics in the tests is a good way to define the desired behavior even before you know how/where you're going to implement it.

[krlmlr]

If you call skip("TODO: ...") in a test, it will remind you that the test (or the code that it tests) doesn't work yet. We have a bunch of these in dplyr.

[kenahoo]

Oh okay, thanks @krlmlr .