Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High-Scores: Add immutability test #1486

Merged
merged 6 commits into from
Feb 15, 2022
Merged

High-Scores: Add immutability test #1486

merged 6 commits into from
Feb 15, 2022

Conversation

jeffdparker
Copy link
Contributor

I don't know the syntax, but would like to add a test that asks for top three (causing a sort) followed by a latest to see if the program sorted in place.

I don't know the syntax, but would like to add a test that asks for top three (causing a sort) followed by a latest to see if the program sorted in place.
@rpottsoh rpottsoh changed the title Add test High-Scores: Add test Mar 24, 2019
@rpottsoh
Copy link
Member

This proposed addition seems similar to what was removed in #1459.

@jeffdparker
Copy link
Contributor Author

Sorry to have overlooked this history. I am back to request a test that calls latest() twice, due to a solution that used pop() to implement it.

@rpottsoh
Copy link
Member

No problem. Can you post the code that has driven you to want a test added? It may be that the test need only be added for that track. But lets see what you have. I don't quite follow your reasoning for a new case at this point.

@jeffdparker
Copy link
Contributor Author

I can't find it in my lists, but briefly the solution stores the results in a list, and then calls pop(), which removes and returns the last item on the list.

Like the issue with sorting, this method destroys the integrity of the data.

@sshine
Copy link
Contributor

sshine commented Mar 25, 2019

This does sound like you want a track-specific test.

The current canonical data describes the expected output of running functions in isolation. So if you'd like to test that a function is immutable, our current canonical data format assumes them to be and doesn't really support this explicitly.

Also, I don't think it should. If mentors/maintainers of other tracks disagree, feel free to object to this.

But I do think you should add a test for this in your track if this is something you expect to occur again.

@emcoding
Copy link
Contributor

It looks like a perfect Talking Point to add to the Mentor Notes, rather than adding a test.

@jeffdparker
Copy link
Contributor Author

jeffdparker commented Mar 25, 2019

There is already note about this in the python mentoring guide.

The problem of making sure the representation provides an accurate version of the state seems like the kind of thing that would be useful to test, especially as we try to automate what we can.

Is it difficult to make two calls inside a test? Is the reluctance to test this driven by technology or by pedagogy?

@rpottsoh
Copy link
Member

rpottsoh commented Mar 26, 2019

Perhaps this would be acceptable:

       {
          "description": "Recent after a call to sort",
          "property": "latest",
          "input": {
            "personalTopThree": {
              "scores": [10, 20, 30, 40, 25]
            }
          },
          "expected": [25]
        }

The test would need to be last and probably shouldn't fall within the grouping of "Top 3 scores".

@rpottsoh
Copy link
Member

rpottsoh commented Mar 26, 2019

My implementation of this test would look like:

procedure THighScoresTest.Recent_after_a_call_to_sort;
var
  Scores: IScores;
begin
  Scores := NewScores([10, 20, 30, 40, 25]);
  Scores := NewScores(Scores.personalTopThree.ToArray);
  Assert.AreEqual(25, Scores.latest);
end;

Only latest is tested.

@petertseng
Copy link
Member

petertseng commented Mar 26, 2019

as you have seen, and as I will also now point you to #1225 to see, the schema optimises for f(input) = output; support for sending two (or more) messages to the same receiver was not given first-priority consideration in the design. If you would like to see some examples of how it was done, take a look at https://github.com/exercism/problem-specifications/blob/master/exercises/circular-buffer/canonical-data.json . Possibly react and zipper as well.

this comment should not be taken as a recommendation (either positive or negative) on whether this case should be added

@ErikSchierboom
Copy link
Member

Also, I don't think it should. If mentors/maintainers of other tracks disagree, feel free to object to this.

I agree with @sshine. I understand the thoughts behind the test case, but I'm not in favor of adding it to the canonical data.

@sshine
Copy link
Contributor

sshine commented Mar 29, 2019

Is it difficult to make two calls inside a test?

No, but do you mean inside a test file for an exercise on a language track, or in canonical-data.json?

Is the reluctance to test this driven by technology or by pedagogy?

Technology, entirely!

My main point of objection would be that the schema of canonical-data.json is challenged up to a point where we wrap a bunch of human-readable sentences in JSON and rule out automated unit test generation.

So while some tracks may benefit from the added tests, other tracks for which automated test generation based on canonical-data.json can no longer run their test generator. This exercise was implemented by the following tracks: scala, ecmascript, delphi, nim, javascript, fsharp, csharp, python, ruby. According to exercism/discussions#155, scala, csharp and ruby have test generators.

I think your (property) test should be added to canonical-data.json.

But I'd prefer if canonical-data.json supported properties, then.

I suppose we could

  1. Add a property test to high-scores' canonical-data.json that may break test generators, work on making canonical-data.json compatible with properties (one way or another, as continued in Schema optimises for testing f(inputs) = output. What of other types of tests? #1225), and then refactor this property test.
    Pros: All tracks benefit from the extra test case. Test is stored in central location.
    Cons: May break test generators. Format of canonical-data.json is abused.
  2. Add track-specific tests to high-scores, work on making canonical-data.json compatible with properties (one way or another, as continued in Schema optimises for testing f(inputs) = output. What of other types of tests? #1225), add the property test to high-scores' canonical-data.json and sync the test version of the track(s) that added the local test case.
    Pros: Doesn't break test generators. Keep format of canonical-data.json pure.
    Cons: Track-specific tests don't propagate to other tracks. We may never resolve the discussion in Schema optimises for testing f(inputs) = output. What of other types of tests? #1225.
  3. Something else entirely...

As a last note, I have to admit that I already abused canonical-data.json to store property-based tests in dnd-character; I reasoned that because this was a new exercise, it wouldn't break existing test generators, and because it operates on randomness, testing becomes very track-specific (mocking? injecting seed? property-based?), making test generation difficult if not impossible. But this still violates at least one argument I'm using against adding a property-based test to high-scores.

I'm not sure how we proceed from here. I withdraw my objection and am open to proceed in any way.

@cmccandless cmccandless changed the title High-Scores: Add test High-Scores: Add immutability test Apr 1, 2019
Base automatically changed from master to main January 27, 2021 15:31
@ErikSchierboom
Copy link
Member

@jeffdparker Are you still interested in working on this?

@jeffdparker
Copy link
Contributor Author

jeffdparker commented Jan 5, 2022 via email

@ErikSchierboom
Copy link
Member

Great. Let's discuss what needs to happen when you're back.

@kotp
Copy link
Member

kotp commented Jan 24, 2022

Yes, I am interested. I’m out of town now, but will be back at my desk
Monday

@jeffdparker Which monday? ;)

@jeffdparker
Copy link
Contributor Author

jeffdparker commented Jan 24, 2022 via email

@glennj
Copy link
Contributor

glennj commented Jan 24, 2022

This is what groovy does:

    def "Personal top three does not mutate"() {
        given:
        def hs = new HighScores(scores)
        def top3 = hs.personalTopThree()

        expect:
        hs.latest() == expected

        where:
        scores           || expected
        [40, 20, 10, 30] || 30
    }

But I don't know how to express that with the canonical-data-schema

@jeffdparker
Copy link
Contributor Author

jeffdparker commented Jan 24, 2022 via email

@ErikSchierboom
Copy link
Member

@jeffdparker A separate PR added two test cases to verify immutability: https://github.com/exercism/problem-specifications/blob/main/exercises/high-scores/canonical-data.json#L87-L110 Could you check to see if those work for your suggestion too?

@jeffdparker
Copy link
Contributor Author

jeffdparker commented Jan 25, 2022 via email

@jeffdparker
Copy link
Contributor Author

jeffdparker commented Feb 3, 2022 via email

@jeffdparker
Copy link
Contributor Author

jeffdparker commented Feb 7, 2022 via email

@ErikSchierboom
Copy link
Member

  1. I believe that I made the changes, but in a parallel repository, csci1,
    with the same problem statement.

This is likely the problem.

  1. If Erik thinks there is a problem, what does he think the solution is?
    Why would it fail after rebasing?

That's git protecting you from overwriting remote changes with local changes. In essence: git rebase will have changed the history which means that git has no way of merging the remote changes with the local changes because they are two different trees.

  1. If I didn't make the changes to the exercism branch, do you have any
    advice on the next steps?

The branch for this PR is in this repo: https://github.com/jeffdparker/problem-specifications/tree/patch-2 The changes should be applied to the branch of this repo and then pushed.

@jeffdparker
Copy link
Contributor Author

jeffdparker commented Feb 8, 2022 via email

Add tests for immutability after calling personal best

Some solutions sort the array to find the largest.  This breaks any subsequent call for recent scores, and isn't as fast as calling max.  These two tests call recent and scores to verify that the scores have not changed.
@jeffdparker
Copy link
Contributor Author

jeffdparker commented Feb 9, 2022 via email

Add missing commas
@jeffdparker
Copy link
Contributor Author

jeffdparker commented Feb 9, 2022 via email

@ynfle
Copy link
Contributor

ynfle commented Feb 9, 2022

Hi @jeffdparker,

It seems that your push didn't fix it.

Check out https://github.com/exercism/problem-specifications#automated-tests for running at least some of the tests locally.
You can also checkout the GitHub Actions CI at https://github.com/exercism/problem-specifications/blob/main/.github/workflows/ci.yml if you're willing to dive in to that

Good luck

@ynfle
Copy link
Contributor

ynfle commented Feb 9, 2022

It seems that property should be an array with [] around the 2 strings

Copy link
Member

@ErikSchierboom ErikSchierboom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one comma missing I believe.

@@ -112,6 +112,30 @@
},
"expected": [25]
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a comma missing here.

},
{
"description": "Recent after a call to sort",
"property": "personalTopThree", "latest",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not valid JSON. Is this test case still needed? I sort of assumed that the two newer ones would handle this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The property value must be a string. This looks like an array, but without the [ and ] characters.

@jeffdparker
Copy link
Contributor Author

jeffdparker commented Feb 10, 2022 via email

Copy link
Member

@ErikSchierboom ErikSchierboom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tests come in pairs: one of these tests checks the most recent (last)
element of the list. The other test in each pair asks for the full list,
and checks that it is unchanged. The tests that use the latest score have
the advantage that they don't need access beyond the API.

Correct, but the PR is now adding three tests. I think the first one should be removed, right?

},
{
"description": "Recent after a call to sort",
"property": "personalTopThree", "latest",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The property value must be a string. This looks like an array, but without the [ and ] characters.

@jeffdparker
Copy link
Contributor Author

jeffdparker commented Feb 12, 2022 via email

Removed redundant test that two property strings, rather than one

          "property": "personalTopThree", "latest",
@jeffdparker
Copy link
Contributor Author

jeffdparker commented Feb 12, 2022 via email

@jeffdparker
Copy link
Contributor Author

jeffdparker commented Feb 12, 2022 via email

@ErikSchierboom
Copy link
Member

/format

@loziniak
Copy link
Member

seems already addressed by #1767 .

@ErikSchierboom
Copy link
Member

seems already addressed by #1767 .

This is a new addition.

@jeffdparker
Copy link
Contributor Author

jeffdparker commented Feb 15, 2022 via email

@kotp kotp merged commit 88ce356 into exercism:main Feb 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants