Add an initial setup for RSpec examples. #381
Conversation
Adds: * Makefile commands for: * Seeding each exercise with a common RSpec setup * Running RSpec for each exercise * Running RSpec for all exercises * An example implementation of what the RSpec solution would look like for the "wordy" exercise. Also: ignores only the root `Gemfile.lock`. We need `exercises/**/Gemfile.lock` to exist so that the version of RSpec that exists in each one is pinned. I've chosen here to do a somewhat literal translation of the minitest exercises into RSpec. That is that there is no usage of subject, let, before, nested contexts etc. The reason for that is that it feels to me like exercism exercises are mostly done by beginners learning to do TDD and so slamming them with extra RSpec concepts is unwise. Also, at least for this problem, it doesn't feel like we're proving out boolean cases so much as we are adding one test at a time to force complexity. One RSpec feature I've taken advantage of that isn't used in minitest is providing a docstring to skip, basically making it extremely explicit in which order the examples should be satisfied. I feel like exercism users are used to going from the top down anyway, but I figured the extra degree of inline docs is useful. What do people think? In terms of spec naming style I've gone with the authoritative "it can/it can't" do things. One other style that could be used here is the "it adds two numbers together", "it multiplies and then adds numbers". I personally think that gets harder to read as we add more cases, but I'd love to hear feedback on that. Regarding documentation: I'm aware that at the moment the documentation reflects the fact that minitest is built in to ruby, and so most people can just immediately get started. One open question is what are all the places we need to document how to work with RSpec? If we decide we like this approach, I'm happy to go ahead. I'd love to get feedback on copy and style choices before I plough ahead with adding specs to all the examples though. Thanks ^_^
|
I like the idea! The order of the tests really aren't ultimately important. The skips are there in keeping the "while you are solving the problem" noise down. Seeing Ultimately, the order of the tests shouldn't not matter in the order of running. Where to document testing with RSpec is probably in the The working comparison of "can" or "must", "must" is stronger, a requirement, where "can" is an allowance. Looking at this closer, though. Thanks for the PR! |
|
If the Gemfile and Gemfile.lock are stored in the exercise folder, do we then download those files using the exercism client, so that the solutions, if using rspec, are influenced by the rspec version? |
| end | ||
|
|
||
| it "can add two to fifty three" do | ||
| skip "fix this test second - delete the `skip` lines to enable individual tests" |
There was a problem hiding this comment.
I thought pending was the rspec way to "skip"?
There was a problem hiding this comment.
skip is preferable here. See our (RSpec's) own documentation:
https://relishapp.com/rspec/rspec-core/v/3-4/docs/pending-and-skipped-examples/skip-examples
https://relishapp.com/rspec/rspec-core/v/3-0/docs/pending-and-skipped-examples/pending-examples
There was a problem hiding this comment.
Fair enough, it has been a long time since I have used RSpec.
When you say "Here" do you mean "Here as in at RSpec" or "Here as in on exercism"?
There was a problem hiding this comment.
here as in "for the exorcism examples"
There was a problem hiding this comment.
Glad I asked.
If it is idiomatic for RSpec to use pending, then we should use pending. Exercism should not influence this, the tests, if we provide them, should be idiomatic and instructive. The documentation states to use skip only due to the use of it in regards to Minitest.
There was a problem hiding this comment.
pending is pretty confusing in Rspec, I think we should stick with skip. pending actually runs the example and reports errors/failures. That noise is confusing because despite big ugly stack traces, pending examples won't fail the run. Quite the opposite, actually. pending examples actually fail the run if a pending scenario passes.
There was a problem hiding this comment.
I think skipped is what we're looking for. It should always be skipped, until they delete it. It's not temporarily skipped because we think we have a bug.
Is there a way to not have the verbose output about it being skipped? I'm trying to avoid noisy output. In this context (exercism) it's not intended as team-wide communication.
|
I am strongly against including RSpec tests. Newbie Infrastructure concerns: with Minitest all you need is Ruby. RSpec requires a lot more infrastructure to run, and diagnosing problems getting it all running is no fun.
If I'm running Minitest tests I don't need this file at all. Duplication: of test maintenance, and Exercism testing infrastructure Concepts: Anyone who wants to use RSpec should be familiar with the basic testing concepts used by Minitest even if they do not prefer the syntax. Caveat: I'd be more fine with it if it required an extra argument to fetch eg: Translation: Is automated translation from the Minitest tests possible/desirable? Framework Explosion: I'd also be more fine with it there was only one test framework supported, Is anyone making the argument that we should migrate all the tests from Minitest to RSpec? |
|
No-one is stopping the student writing their own RSpec tests to test-drive their solution. Are the students unable to read the Minitest specs? |
| @@ -1,4 +1,4 @@ | |||
| .bundle | |||
| bin/configlet | |||
| bin/configlet.exe | |||
| Gemfile.lock | |||
| /Gemfile.lock | |||
There was a problem hiding this comment.
I don't know about this but I wonder if this should be:
Gemfile.lock
# Do track the exercises Gemfile.lock files
!/exercises/*/Gemfile.lock
This way, any other ones are still ignored, or must be explicitly added to the project.
There was a problem hiding this comment.
This tells git to ignore the Gemfile.lock at the repo root, so it will work (in fact is preferred).
There was a problem hiding this comment.
Which "This"? The one that I propose does the same, but specifically also says to NOT ignore the Gemfile.lock in each of the exercise subfolders (but only the subfolder.), while also ignoring Gemfile.lock anywhere else.
|
@Insti brings up some really good points. Supporting both minitest and rspec is double the work and I'm not sure I see much benefit to exercists, but I do see a lot of possible confusion. |
|
Responding to #381 (comment) by @samphippen
Agreed.
Yeah, that sounds right to me.
It feels noisy to me. The skips should be removed in order. The tests are run in pseudorandom order in minitest, and I'm assuming that's also the case in RSpec. The tests aren't order-dependent per se, but since we're emulating TDD, the order that they get activated does matter.
Does authoritative mean that the RSpec bosses said it's the right way to do it? I have to admit that never really understood that style of naming. It seems like unnecessary hedging: Please make it so, kind of, if it's OK with you, I think. Perhaps. It's not that the code is able to do it (of course it's able), it's that the code does it.
This would still be the case. The RSpec should be a second choice. Something like: If you don't have a strong preference, or if you don't know which to choose, then use minitest. The point of adding RSpec would be to make the people who really, really care happy. Everyone else can keep doing what they're doing.
Responding to @Insti's comments in #381 (comment)
I agree that only people who have a strong desire to do so should use RSpec. If they don't have a preference, or they don't have a reason to pick one over the other, then they should pick Minitest.
Fair enough. We've been talking about implementing track-level files: https://github.com/exercism/x-api/issues/29 If we do that first, then we could just have one Gemfile.lock at the top level.
If we have people who care about RSpec helping out with maintenance of the Ruby track on Exercism, then I don't mind this. Also, we have generators for some exercises. Maybe this would be the impetus to write generators across the board, since that would be some work up front, but then easy to support later.
I agree and disagree. Yes, they should be familiar with the basic testing concepts. And yes, they should be able to practice using the framework that their courses are having them use. There's an aspect of fluency here: when you're very, very new to things, it doesn't take much to add a whole lot of friction.
I feel much more strongly about avoiding track-specific stuff in the CLI than about avoiding duplication in the tracks themselves. If we can come up with something in the config that would let us do this in a generic way, then I'd be fine with that.
Yeah, I do think that we should prioritize using generators to avoid the thing where we are always changing things in two places. Doing that work up front feels like it would be worth it. This wouldn't necessarily be a direct translation from minitest, but a way to create both test suites based on canonical test data.
No, I don't think anyone is making that argument. And if they did I'd be very, very strongly against it (basically for the reasons that you brought up). Responding to @Insti on #381 (comment)
This is an interesting question. Why do you think the students want exercism to come with RSpec and aren't writing the specs themselves? |
|
As someone who was rather recently a newbie, I wanted to add my support for the idea here. I came to exercism after I already had a fair amount of experience with Ruby, and I remember working on a patch to the API that serves exercises. The tests were all in Minitest, which at the time I had no experience with, and that was greatly intimidating. I'd imagine that others who are in that same boat (<2 years experience) might feel similarly, and it would possibly turn them off from exercism all together. Really, to me, this seems like a great way to try and reduce the initial friction of getting users "in the door", per se. If you have the option of using all the tools that you already know and love, you'll be more likely to make the leap. Hopefully in the process of going through the exercises you'll also learn Minitest, but it shouldn't feel like a prerequisite (even though it really isn't). It certainly adds additional burden to the maintainers, but in my eyes the potential return on that investment (helping users feel more welcome/less intimidated by seeing things they know and are comfortable with) is significant. Anytime there's less of a feeling of "oh my gosh, I don't know how to use this tool - maybe this isn't for me?", that's a good thing! |
I think that's reasonable, reading back on this style, I'm not totally in love with it for exercism examples. I think e.g.
👍
I've added a
This would be run
👍 In summary: This is all really good feedback. I'll start making some updates. In particular, I'm not totally sure about the |
Uhm... I keep meaning to document that, but I keep not getting to it :/ Sorry about that. The short version is that there are scripts in bin/generate-* This is a very short shell script that loads some generic generator stuff (lib/generator.rb) then loads some exercise-specific stuff for reading the canonical test inputs/outputs that live in x-common (json files) (lib/ Please ask questions :) |
|
Limited documentation to get you there is in the CONTRIBUTING.md, this link is directed more toward updating. It is what I used to get started though. I am happy to help you get started with this, though. |
Remove reference to 'black' and 'white' in description.
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
I'm going to suggest that we close this with the following conclusion: Exercism is meant to teach people the basics of Ruby and the Ruby standard library. 3rd party libraries are out of scope. I do think that people want to learn RSpec, but I don't think this is the right place to do that. I would encourage anyone who wants to make a practice corpus for RSpec to consider creating a set of stand-alone RSpec koans (or similar). They can, of course, be inspired by or based off of Exercism's exercises, since they are licensed under MIT. |
Originated from this conversation on twitter: https://twitter.com/kytrinyx/status/744411580654190593. /cc @kytrinyx ❤️
Adds:
for the "wordy" exercise.
Also: ignores only the root
Gemfile.lock. We needexercises/**/Gemfile.lockto exist so that the version of RSpec thatexists in each one is pinned.
I've chosen here to do a somewhat literal translation of the minitest
exercises into RSpec. That is that there is no usage of subject, let,
before, nested contexts etc. The reason for that is that it feels to me
like exercism exercises are mostly done by beginners learning to do TDD
and so slamming them with extra RSpec concepts is unwise. Also, at least
for this problem, it doesn't feel like we're proving out boolean cases
so much as we are adding one test at a time to force complexity.
One RSpec feature I've taken advantage of that isn't used in minitest is
providing a docstring to skip, basically making it extremely explicit in
which order the examples should be satisfied. I feel like exercism users
are used to going from the top down anyway, but I figured the extra
degree of inline docs is useful. What do people think?
In terms of spec naming style I've gone with the authoritative "it
can/it can't" do things. One other style that could be used here is the "it
adds two numbers together", "it multiplies and then adds numbers". I
personally think that gets harder to read as we add more cases, but I'd
love to hear feedback on that.
Regarding documentation: I'm aware that at the moment the documentation
reflects the fact that minitest is built in to ruby, and so most people
can just immediately get started. One open question is what are all the
places we need to document how to work with RSpec?
If we decide we like this approach, I'm happy to go ahead. I'd love
to get feedback on copy and style choices before I plough ahead with
adding specs to all the examples though. Thanks ^_^