Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rule request: [Test class having setup method, but no teardown] #3452

Closed
2 tasks done
kenji21 opened this issue Dec 2, 2020 · 6 comments
Closed
2 tasks done

Rule request: [Test class having setup method, but no teardown] #3452

kenji21 opened this issue Dec 2, 2020 · 6 comments

Comments

@kenji21
Copy link

kenji21 commented Dec 2, 2020

New Issue Checklist

New rule request

Please describe the rule idea, format
this issue's title as Rule Request: [Rule Name] and describe:

  1. Why should this rule be added?

It's a good pratique to "rollback" in teardown what we do in setup on unit test
And XCtest create an instance of the test class for each method, if we don't set objects to nil in the teardown method, we use more memory than necessary, and attribute not setted back to nil can make side effects

  1. Provide several examples of what would and wouldn't trigger violations.
    setUp (or setUpWithError) without tearDown (or tearDownWithError()) method in a file with Test/Tests in the end of its name (convention)

  2. Should the rule be configurable, if so what parameters should be configurable?
    No i don't see any possible configuration

  3. Should the rule be opt-in or enabled by default? Why?
    To lower memory consumption in tests, and prevent side effects

@agarmash
Copy link
Contributor

agarmash commented Dec 6, 2020

It's a common case for not having a tearDown method, consider this example:

class StorageTests: XCTestCase {

    var storage: StorageProtocol!

    func setUp() {
        storage = InMemoryStorage()
    }

    func testSomething() {
        ...
    }
}

In this case, there wouldn't be any side effects since there's a brand new external dependency generated before each test. And the old one will be successfully deallocated by ARC, so memory consumption is also not an issue.

Such rule will make more harm forcing developers to have redundant empty tearDown implementation just to silence the warning.

@kenji21
Copy link
Author

kenji21 commented Dec 6, 2020

I found the original article pointing to me that teardown are necessary: https://qualitycoding.org/xctestcase-teardown/

protocol StorageProtocol {
    var i: Int { get }
}
class InMemoryStorage: NSObject, StorageProtocol {
    var i = 1
    
    deinit {
        print("\(self) gets deinit")
    }
}

class StorageWithTearDownTests: XCTestCase {
    var storage: StorageProtocol!
    
    deinit {
        print("\(self) gets deinit")
    }

    override func setUp() {
        storage = InMemoryStorage()
    }

    override func tearDown() {
        storage = nil
    }

    func testSomething() {
        XCTAssertTrue(storage.i == 1)
    }
}

class StorageTests: XCTestCase {
    var storage: StorageProtocol!

    deinit {
        print("\(self) gets deinit")
    }

    override func setUp() {
        storage = InMemoryStorage()
    }

    func testSomething() {
        XCTAssertTrue(storage.i == 1)
    }
}

/// Z prefix to be the last test runned
class ZdebugMemoryGraph: XCTestCase {
    func testZLastTestToRun_debugMemoryGraph_allowingToFindMemoryLeaks() {
        usleep(1000)
        XCTAssertTrue(true)
    }
}

Only one deinit gets called:

Test Suite 'All tests' started at 2020-12-06 19:20:17.867
Test Suite 'TestMemoryCycleTests.xctest' started at 2020-12-06 19:20:17.867
Test Suite 'StorageTests' started at 2020-12-06 19:20:17.867
Test Case '-[TestMemoryCycleTests.StorageTests testSomething]' started.
Test Case '-[TestMemoryCycleTests.StorageTests testSomething]' passed (0.001 seconds).
Test Suite 'StorageTests' passed at 2020-12-06 19:20:17.868.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.001 (0.001) seconds
Test Suite 'StorageWithTearDownTests' started at 2020-12-06 19:20:17.868
Test Case '-[TestMemoryCycleTests.StorageWithTearDownTests testSomething]' started.
<TestMemoryCycleTests.InMemoryStorage: 0x6000015e8e40> gets deinit
Test Case '-[TestMemoryCycleTests.StorageWithTearDownTests testSomething]' passed (0.000 seconds).
Test Suite 'StorageWithTearDownTests' passed at 2020-12-06 19:20:17.868.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.000 (0.000) seconds
Test Suite 'TestMemoryCycleTests' started at 2020-12-06 19:20:17.868
Test Case '-[TestMemoryCycleTests.TestMemoryCycleTests testExample]' started.
Test Case '-[TestMemoryCycleTests.TestMemoryCycleTests testExample]' passed (0.000 seconds).
Test Suite 'TestMemoryCycleTests' passed at 2020-12-06 19:20:17.871.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.000 (0.003) seconds
Test Suite 'ZdebugMemoryGraph' started at 2020-12-06 19:20:17.871
Test Case '-[TestMemoryCycleTests.ZdebugMemoryGraph testZLastTestToRun_debugMemoryGraph_allowingToFindMemoryLeaks]' started.
Test Case '-[TestMemoryCycleTests.ZdebugMemoryGraph testZLastTestToRun_debugMemoryGraph_allowingToFindMemoryLeaks]' passed (0.929 seconds).
Test Suite 'ZdebugMemoryGraph' passed at 2020-12-06 19:20:18.801.
	 Executed 1 test, with 0 failures (0 unexpected) in 0.929 (0.930) seconds
Test Suite 'TestMemoryCycleTests.xctest' passed at 2020-12-06 19:20:18.801.
	 Executed 4 tests, with 0 failures (0 unexpected) in 0.930 (0.934) seconds
Test Suite 'All tests' passed at 2020-12-06 19:20:18.802.
	 Executed 4 tests, with 0 failures (0 unexpected) in 0.930 (0.935) seconds

@agarmash
Copy link
Contributor

agarmash commented Dec 6, 2020

Whoa, I didn't know that. Additionally checked the sources in swift-corelibs-xctest to figure out that it's absolutely true

@kenji21
Copy link
Author

kenji21 commented Dec 8, 2020

That's one reason why I this such a rule should be added and opted-in

@otaviocc
Copy link
Contributor

Hi @kenji21, let me know if the PR makes sense to you :-)

@kenji21
Copy link
Author

kenji21 commented Jan 20, 2021

Looks nice, you have covered all cases ( #3495 (comment) )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants