Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test fixtures non functional #2382

Closed
phillxnet opened this issue Jun 21, 2022 · 1 comment
Closed

Test fixtures non functional #2382

phillxnet opened this issue Jun 21, 2022 · 1 comment

Comments

@phillxnet
Copy link
Member

When our unit tests were last re-vamped (read updated to work again), post some major API and error reporting changes in the interim at that time, we have had a completely non functional fixture arrangement. See roughly 4 years ago:

"Standardise error text format and move to string.format part-1. Fixes #1913" #1914

Fixtures are the fake db content used to present tests with the desired database context in which the test is to be set. This short coming (know bug) was circumvented at the time, and since, by what may be considered excessive object mocking. This is less than desirable (read more 'artificial') than presenting tests with actual database dumps, or parts there of: a common origin of test fixtures.

Since our recent Django update via:
"Update Django within current Python 2 constrains #2254" #2365
our current 'broken fixtures' arrangement has now run it's course due to the increased sanity checks imposed on database / object interconnects; in turn increasing the complexity of mocking objects for our tests. See: (t) Failing unit tests #2371 for which this issue is a spin-off to address individually the failing fixtures issue now holding back our required test maintenance.

Thanks to @FroggyFlox for doing the vast majority of the 'solving' on this one.
Pull request to follow with fixture instantiation fix and small sample of adapted unit tests as proof of fix.
@phillxnet phillxnet mentioned this issue Jun 21, 2022
Closed
14 tasks
phillxnet added a commit to phillxnet/rockstor-core that referenced this issue Jun 21, 2022
@phillxnet
(t) Test fixtures non functional rockstor#2382
f923808 
Remark out model mocks, that are currently incompatible for our recently
updated Django, in test_pool_scrub.py so it can be used as a proof of
fix for the outstanding fixtures issue.

Includes:
- prior model mocks on some unit tests.
- minor test fix re inadvertent follow= parameter in test_appliances.py
- notes on creating appropriate test_pool_scrub_balance and test_pools
fixtures/db-dumps.
- minor formatting fixes.
- Logging comments in pool_scrub.py _validate_pool() to ascertain
current pool objects.
phillxnet pushed a commit to phillxnet/rockstor-core that referenced this issue Jun 21, 2022
@FroggyFlox @phillxnet
(t) Ensure fixtures are loaded in all API tests rockstor#2382
7e58740 
- Modify APITestMixin to
  - not skip `setUpClass()`
  - fetch User from fixture
- Do not inherit from APITestCase in `PoolScrubTests()` as APITestMixin
already inherits from it.
- Create new fixture that includes newly-added oauth2_provider
migrations.
phillxnet pushed a commit to phillxnet/rockstor-core that referenced this issue Jun 21, 2022
@FroggyFlox @phillxnet
(t) Keep models for API auth in their own fixture rockstor#2382
6dad5af 
Keep fixture for test_pool_scrub.py to minimal models needed.
phillxnet added a commit to phillxnet/rockstor-core that referenced this issue Jun 21, 2022
@phillxnet
(t) Remove 2 redundant fixtures & minimise init test fixture for rock…
77057ae 
…stor#2382

Remove extraneous oauth2_provider from fixture instance & comment. We
are now attempting to minimalise all fixtures. The oauth2_provider
entries in test_api.json look to be redundant.

Includes
- Fix typo in comment on fixture creation.
- Remove redundant test_pool_scrub_balance related fixtures.
- Remark out debug logging associated with issue.
Not removing entirely as we are very likely to need these in related
test issues to follow shortly.
@phillxnet phillxnet mentioned this issue Jun 21, 2022
Merged
phillxnet added a commit that referenced this issue Jun 22, 2022
@phillxnet
Merge pull request #2383 from phillxnet/2382_Test_fixtures_non_functi…
afa3e41 
…onal

(t) Test fixtures non functional #2382
@phillxnet
Copy link
Member Author

Closing as:
Fixed by #2383

@phillxnet phillxnet mentioned this issue Jun 22, 2022
Closed
phillxnet added a commit to phillxnet/rockstor-core that referenced this issue Jun 23, 2022
@phillxnet
(t) improve test_api.py re setUpClass/tearDownClass supers rockstor#2382
52eafdc 
 rockstor#2384

Thanks to FroggyFlox on Github for follow-up work in this area.
phillxnet added a commit to phillxnet/rockstor-core that referenced this issue Jun 28, 2022
@phillxnet
(t) Refresh/remove all test fixtures and update tests accordingly roc…
7ccae50 
…kstor#2384

As a result of major Django changes enacted in 'Update Django within current Python 2
constrains rockstor#2254' rockstor#2365, and following on from the fixture re-enablement spin-off of
the former: '(t) Test fixtures non functional rockstor#2382' rockstor#2383 we have a test wide
requirement to update or remove our now mostly outdated and newly re-enabled fixtures.

Our prior work-around for the previously dysfunctional fixtures was extensive in-test
db object mocking. This has become unworkable and incompatible with our newer Django.

## Includes
- remove now outdated trailing '/' in task post calls rockstor#2384
Thanks to FroggyFlox for pointers: in GitHub issue rockstor#2370 we recently
removed the trailing slash for post calls on tasks. Adjust task unit
tests accordingly.
- minor existing fixture update to be compatible with newer auth arrangement via
independent fixture.
- del redundant/legacy fixture/inheritance - multiple tests.
- Addition on now independent test_api.json auth feature.
- Black formatting.
- del redundant/legacy fixture/inheritance - test_pool_balance.
- Remove redundant fixtures and their references.
- Addition on now independent test_api.json auth feature.
- Add test run command.
- Addition instructions on fixture content and creation.
- Removing numerous now incompatible db object mocking as we now have working fixtures.
- Improve test_api.py re setUpClass/tearDownClass supers rockstor#2382
Thanks to FroggyFlox on Github for follow-up work in this area.
- Restore previously remarked out, existing snapshot delete test.
- move smart run_test mock down to system - test_disk_smart.py.
Thanks to FroggyFlox on GitHub. Our initial mock was failing, covered up by as yet
only local testing. Moving the mock to the source of its definition resolves a failed
mock.
- Reviving a number of previously unworkable tests.
- Improving test group names for clarity.
- Test re-ordering for clarity.
- Breaking out more tests into their own sets rather than repeat for all raid levels.
- Improved/simplified mocking.
- add todo, debug logging, and comments to pool.py concerning issue rockstor#2384
phillxnet added a commit to phillxnet/rockstor-core that referenced this issue Jun 29, 2022
@phillxnet
(t) Refresh/remove all test fixtures and update tests accordingly roc…
976871e 
…kstor#2384

As a result of major Django changes enacted in 'Update Django within current Python 2
constrains rockstor#2254' rockstor#2365, and following on from the fixture re-enablement spin-off of
the former: '(t) Test fixtures non functional rockstor#2382' rockstor#2383 we have a test wide
requirement to update or remove our now mostly outdated and newly re-enabled fixtures.

Our prior work-around for the previously dysfunctional fixtures was extensive in-test
db object mocking. This has become unworkable and incompatible with our newer Django.

## Includes
- remove now outdated trailing '/' in task post calls rockstor#2384
Thanks to FroggyFlox for pointers: in GitHub issue rockstor#2370 we recently
removed the trailing slash for post calls on tasks. Adjust task unit
tests accordingly.
- minor existing fixture update to be compatible with newer auth arrangement via
independent fixture.
- del redundant/legacy fixture/inheritance - multiple tests.
- Addition on now independent test_api.json auth feature.
- Black formatting.
- del redundant/legacy fixture/inheritance - test_pool_balance.
- Remove redundant fixtures and their references.
- Addition on now independent test_api.json auth feature.
- Add test run command.
- Addition instructions on fixture content and creation.
- Removing numerous now incompatible db object mocking as we now have working fixtures.
- Improve test_api.py re setUpClass/tearDownClass supers rockstor#2382
Thanks to FroggyFlox on Github for follow-up work in this area.
- Restore previously remarked out, existing snapshot delete test.
- move smart run_test mock down to system - test_disk_smart.py.
Thanks to FroggyFlox on GitHub. Our initial mock was failing, covered up by as yet
only local testing. Moving the mock to the source of its definition resolves a failed
mock.
- Reviving a number of previously unworkable tests.
- Improving test group names for clarity.
- Test re-ordering for clarity.
- Breaking out more tests into their own sets rather than repeat for all raid levels.
- Improved/simplified mocking.
- add todo, debug logging, and comments to pool.py concerning issue rockstor#2384
- Simplify mocking of pool_usage in line with recent tested code
changes.
phillxnet added a commit to phillxnet/rockstor-core that referenced this issue Jul 4, 2022
@phillxnet
(t) Refresh/remove all test fixtures and update tests accordingly roc…
f0c091a 
…kstor#2384

As a result of major Django changes enacted in 'Update Django within current Python 2
constrains rockstor#2254' rockstor#2365, and following on from the fixture re-enablement spin-off of
the former: '(t) Test fixtures non functional rockstor#2382' rockstor#2383 we have a test wide
requirement to update or remove our now mostly outdated and newly re-enabled fixtures.

Our prior work-around for the previously dysfunctional fixtures was extensive in-test
db object mocking. This has become unworkable and incompatible with our newer Django.

## Includes
- remove now outdated trailing '/' in task post calls rockstor#2384
Thanks to FroggyFlox for pointers: in GitHub issue rockstor#2370 we recently
removed the trailing slash for post calls on tasks. Adjust task unit
tests accordingly.
- minor existing fixture update to be compatible with newer auth arrangement via
independent fixture.
- del redundant/legacy fixture/inheritance - multiple tests.
- Addition on now independent test_api.json auth feature.
- Black formatting.
- del redundant/legacy fixture/inheritance - test_pool_balance.
- Remove redundant fixtures and their references.
- Addition on now independent test_api.json auth feature.
- Add test run command.
- Addition instructions on fixture content and creation.
- Removing numerous now incompatible db object mocking as we now have working fixtures.
- Improve test_api.py re setUpClass/tearDownClass supers rockstor#2382
Thanks to FroggyFlox on Github for follow-up work in this area.
- Restore previously remarked out, existing snapshot delete test.
- move smart run_test mock down to system - test_disk_smart.py.
Thanks to FroggyFlox on GitHub. Our initial mock was failing, covered up by as yet
only local testing. Moving the mock to the source of its definition resolves a failed
mock.
- Reviving a number of previously unworkable tests.
- Improving test group names for clarity.
- Test re-ordering for clarity.
- Breaking out more tests into their own sets rather than repeat for all raid levels.
- Improved/simplified mocking.
- add todo, debug logging, and comments to pool.py concerning issue rockstor#2384
- Simplify mocking of pool_usage in line with recent tested code
changes.
- re-awaken more raid migration tests.
- resolving prior PoolBalance anomaly - savepoint for issue rockstor#2384
We have strongly suspected code malfunction regarding pool balance
status reporting. Adding initial comments to define new tests to
establish expected behaviour to be used in another spin-off issue/pr
set.
- improve mocking accuracy test_pools.py.
Depends on issue-pr (rockstor#2390/rockstor#2391) to abstract our balance command
generation to improve future mocking and testing capabilities.
phillxnet added a commit to phillxnet/rockstor-core that referenced this issue Oct 7, 2022
@phillxnet
(t) Refresh/remove all test fixtures and update tests accordingly roc…
84ba78b 
…kstor#2384

As a result of major Django changes enacted in 'Update Django within current Python 2
constrains rockstor#2254' rockstor#2365, and following on from the fixture re-enablement spin-off of
the former: '(t) Test fixtures non functional rockstor#2382' rockstor#2383 we have a test wide
requirement to update or remove our now mostly outdated and newly re-enabled fixtures.

Our prior work-around for the previously dysfunctional fixtures was extensive in-test
db object mocking. This has become unworkable and incompatible with our newer Django.

## Includes
- remove now outdated trailing '/' in task post calls.
Thanks to FroggyFlox for pointers: in GitHub issue rockstor#2370 we recently
removed the trailing slash for post calls on tasks. Adjust task unit
tests accordingly.
- minor existing fixture update to be compatible with newer auth arrangement via
independent fixture.
- del redundant/legacy fixture/inheritance - multiple tests.
- Addition on now independent test_api.json auth feature.
- Black formatting.
- del redundant/legacy fixture/inheritance - test_pool_balance.
- Remove redundant fixtures and their references.
- Addition on now independent test_api.json auth feature.
- Add test run command.
- Addition instructions on fixture content and creation.
- Removing numerous now incompatible db object mocking as we now have working fixtures.
- Improve test_api.py re setUpClass/tearDownClass supers rockstor#2382
Thanks to FroggyFlox on Github for follow-up work in this area.
- Restore previously remarked out, existing snapshot delete test.
- move smart run_test mock down to system - test_disk_smart.py.
Thanks to FroggyFlox on GitHub. Our initial mock was failing, covered up by as yet
only local testing. Moving the mock to the source of its definition resolves a failed
mock.
- Reviving a number of previously unworkable tests.
- Improving test group names for clarity.
- Test re-ordering for clarity.
- Breaking out more tests into their own sets rather than repeat for all raid levels.
- Improved/simplified mocking.
- add todo, debug logging, and comments to pool.py concerning issue
- Simplify mocking of pool_usage in line with recent tested code
changes.
- re-awaken more raid migration tests.
- resolving prior PoolBalance anomaly - savepoint for issue
We have strongly suspected code malfunction regarding pool balance
status reporting. Adding initial comments to define new tests to
establish expected behaviour to be used in another spin-off issue/pr
set.
- improve mocking accuracy test_pools.py.
Depends on issue-pr (rockstor#2390/rockstor#2391) to abstract our balance command
generation to improve future mocking and testing capabilities.
- Includes reordering and splitting out more tests
- By modifying our tid entries for the fixture PoolBalance
we can aid test failure diagnosis via existing tid
logging.
- separate pool scrub and pool balance fixtures.
- Add balance status reporting for cli initiated balance operations rockstor#2393
-- Rework PoolBalance to be more robust and cli balance aware.
-- Improve ordering robustness by using order_by("start_time") as our
prior by "id" can more fragile over such things as db imports.
- API http post on balance status fails serialisation rockstor#2396
-- When we have no balance record avoid serialization failure
by passing through our empty response from _balance_status().
- extend post cli balance status to multiple pools
We initially tested on our data pool (in fixture) with two
existing balance records. Extend to use also the ROOT pool
(no balance records in fixture) to test our behaviour when
no balance history exists: i.e. first record creation etc.
- HTTP_204_NO_CONTENT for POST balance status empty rockstor#2396
When we have no last record we indicate as such via 204.
Our existing behaviour on POST balance status command is
to return latest record. When there isn't one we inform
our caller of this via NO_CONTENT.
- (t) ensure we message on all new cli balance records rockstor#2393
- (t) account for HTTP_204 in test_pool_balance.py
Propose code now returns 204 when no prior balance exists
and we do a POST balance status command. Inform our tests
of this recent proposed code change in variable success
returns.
- t) improve post_valid_status test_pool_balance.py rockstor#2384
Now includes asserting latest return and recent NO_CONTENT
return behaviour when there is no prior balance record.
- POC for Huey aware balance reporting.
@phillxnet phillxnet mentioned this issue Oct 7, 2022
Merged
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@phillxnet