Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check if all testdata repositories are well-formed just once, and not in every task run #263

Closed
8 tasks done
ruiAzevedo19 opened this issue Jul 12, 2024 · 2 comments
Closed
8 tasks done
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@ruiAzevedo19
Copy link
Contributor

ruiAzevedo19 commented Jul 12, 2024

Goal: check if the testdata repositories are well-formed just once and not in every run.

Currently, we check if a task repository is well-formed when we run the task itself. Since we can have multiple runs, we are checking the same thing over and over again. There needs to be an extra step before running all the tasks where we validate the testdata repositories.

TODO

  • add a new method to the repository interface Validate() error
  • each task file contains a validation function that knows how to validate the repository
    • write-tests task
    • code-repair task
    • transpilation task
  • the repository validation needs to happen in 2 places in evaluate/evaluate.go:Evaluate
    • when we run the evaluation for the plain repository (after calculating the temporaryRepositories and before the evaluation loop)
    • when we run the evaluation for all repositories (after calculating the temporaryRepositories and before the evaluation loop)
@bauersimon
Copy link
Member

Would basically just extend the

type Repository interface { 
	Validate() error
...

and that way you can just call that function in the two locations you mentioned above right after creating the repository

and internally you will probably do a switch based on the task identifiers that the repository has... and do one special validation function per task... would probably even leave the validation logic itself where the tasks are (i.e. task-write-test.go) that way the validation for a task is still where the task happens

@bauersimon
Copy link
Member

so basically i would not do a function that knows how to validate all types of repository... I think it is still good that this logic is separated... but the way that the repository validation is invoked should just be via the Repository interface and only once when the repository is created

ruiAzevedo19 added a commit that referenced this issue Jul 12, 2024
ruiAzevedo19 added a commit that referenced this issue Jul 12, 2024
… for the "write-tests" task is well-formed

Part of #263
ruiAzevedo19 added a commit that referenced this issue Jul 12, 2024
… for the "write-tests" task is well-formed

Part of #263
ruiAzevedo19 added a commit that referenced this issue Jul 12, 2024
ruiAzevedo19 added a commit that referenced this issue Jul 12, 2024
… for the "write-tests" task is well-formed

Part of #263
ruiAzevedo19 added a commit that referenced this issue Jul 12, 2024
… for the "write-tests" task is well-formed

Part of #263
ruiAzevedo19 added a commit that referenced this issue Jul 15, 2024
ruiAzevedo19 added a commit that referenced this issue Jul 15, 2024
… for the "write-tests" task is well-formed

Part of #263
ruiAzevedo19 added a commit that referenced this issue Jul 15, 2024
ruiAzevedo19 added a commit that referenced this issue Jul 15, 2024
… for the "write-tests" task is well-formed

Part of #263
ruiAzevedo19 added a commit that referenced this issue Jul 16, 2024
ruiAzevedo19 added a commit that referenced this issue Jul 16, 2024
… for the "write-tests" task is well-formed

Part of #263
ruiAzevedo19 added a commit that referenced this issue Jul 16, 2024
ruiAzevedo19 added a commit that referenced this issue Jul 16, 2024
… for the "write-tests" task is well-formed

Part of #263
ruiAzevedo19 added a commit that referenced this issue Aug 1, 2024
ruiAzevedo19 added a commit that referenced this issue Aug 1, 2024
…rom a package into a helper, for reusability

Part of #263
ruiAzevedo19 added a commit that referenced this issue Aug 1, 2024
…g the evaluation, so it is checked just once

Closes #263
ruiAzevedo19 added a commit that referenced this issue Aug 1, 2024
…lidation tests, so cross-OS error check can be done

Part of #263
ruiAzevedo19 added a commit that referenced this issue Aug 1, 2024
…g the evaluation, so it is checked just once

Closes #263
ruiAzevedo19 added a commit that referenced this issue Aug 1, 2024
ruiAzevedo19 added a commit that referenced this issue Aug 1, 2024
…rom a package into a helper, for reusability

Part of #263
ruiAzevedo19 added a commit that referenced this issue Aug 1, 2024
…lidation tests, so cross-OS error check can be done

Part of #263
Munsio pushed a commit that referenced this issue Aug 28, 2024
Munsio pushed a commit that referenced this issue Aug 28, 2024
…rom a package into a helper, for reusability

Part of #263
Munsio pushed a commit that referenced this issue Aug 28, 2024
…lidation tests, so cross-OS error check can be done

Part of #263
Munsio pushed a commit that referenced this issue Aug 28, 2024
…g the evaluation, so it is checked just once

Closes #263
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants