Merge pull request #2 from mytestopia/rules

rules docs

README.md

@@ -1,8 +1,6 @@
 # flake8-vedro
 Flake8 based linter for [Vedro](https://vedro.io/) framework
 
-All validation rules description is in progress, but you can find them [here](https://github.com/mytestopia/flake8-vedro/blob/version-1.0.0/flake8_vedro/errors/errors.py).
-
 ## Installation
 
 ```bash
@@ -26,3 +24,32 @@ Some rules in linter should be configurated:
 scenario_params_max_count = 8  # VDR109
 allowed_to_redefine_list = page,page2  # VDR311
 ```
+
+## Rules
+
+### Scenario Rules
+1. [VDR001. Decorator @vedro.only should not be presented](./flake8_vedro/rules/VDR101.md)
+2. [VDR002. Scenario should be inherited from class vedro.Scenario](./flake8_vedro/rules/VDR102.md)
+3. [VDR103. Scenario should be located in the folder "scenarios/”](./flake8_vedro/rules/VDR103.md)
+4. [VDR104. Scenario should have a subject](./flake8_vedro/rules/VDR104.md)
+5. [VDR105. Scenario subject should not be empty](./flake8_vedro/rules/VDR105.md)
+6. [VDR106. Scenario should have only one subject](./flake8_vedro/rules/VDR106.md)
+7. [VDR107. Subject is not parameterized*](./flake8_vedro/rules/VDR107.md)
+8. [VDR108. Calling functions in parametrization](./flake8_vedro/rules/VDR108.md)
+9. [VDR109. Limit the amount of parameters in a parametrized scenario](./flake8_vedro/rules/VDR109.md)
+
+
+###  Scenario Steps Rules
+1. [VDR300. Step name should start with..](./flake8_vedro/rules/VDR300.md)
+2. [VDR301. Steps name should be in right order](./flake8_vedro/rules/VDR301.md)
+3. [VDR302. Interface should not be used in given or asserted steps](./flake8_vedro/rules/VDR302.md)
+4. [VDR303. Scenario should have a "when" step](./flake8_vedro/rules/VDR303.md)
+5. [VDR304. Scenario should have only one "when" step](./flake8_vedro/rules/VDR304.md)
+6. [VDR305. Scenario should have a "then" step](./flake8_vedro/rules/VDR305.md)
+7. [VDR306. Scenario should have only one "then" step](./flake8_vedro/rules/VDR306.md)
+8. [VDR307. Step should have an assertion](./flake8_vedro/rules/VDR307.md)
+9. [VDR308. Step should have specific assertions](./flake8_vedro/rules/VDR308.md)
+10. [VDR309. Step should not have comparison without assert](./flake8_vedro/rules/VDR309.md)
+11. [VDR310. Some steps should not have an assertion](./flake8_vedro/rules/VDR310.md)
+12. [VDR311. Scope variables should not be redefined](./flake8_vedro/rules/VDR311.md)
+13. [VDR312. Scope variables should not be partially redefined](./flake8_vedro/rules/VDR312.md)

flake8_vedro/rules/VDR101.md

@@ -0,0 +1,31 @@
+# VDR101. Decorator @vedro.only should not be presented
+Decorator `@vedro.only` is used to tell the Vedro framework to run only the decorated scenario and ignore the others.
+In order not to ignore other scenarios in CI, this decorator is allowed to be used only in dev environment
+
+```python
+# scenarios/example/register_user.py
+import vedro
+
+@vedro.only
+class Scenario(vedro.Scenario):
+    subject = "register new user"
+
+
+# scenarios/example/login_user.py
+import vedro
+
+class Scenario(vedro.Scenario):
+    subject = "login registered user"
+```
+
+Only one scenarios with decorator will be executed:
+
+```shell
+$ vedro run
+Scenarios
+* example
+ ✔ register new user (1.02s)
+```
+
+### Additional links
+- https://vedro.io/docs/basics/selecting-and-ignoring#selecting-specific-scenarios

flake8_vedro/rules/VDR102.md

@@ -0,0 +1,19 @@
+# VDR102. Scenario should be inherited from class vedro.Scenario
+Scenarios, not inherited from `vedro.Scenario`, are not included in execution. 
+
+
+### ❌ Anti-pattern
+****
+```python
+class Scenario:
+    subject = "register new user"
+```
+
+### ✅ Best practice
+
+```python
+import vedro
+
+class Scenario(vedro.Scenario):
+    subject = "register new user"
+```

flake8_vedro/rules/VDR103.md

@@ -0,0 +1,19 @@
+# VDR103. Scenario should be located in the folder "scenarios/”
+
+Test located in other folders (even if they are inherited from vedro.Scenario class) are not included in executions
+
+Correct structure files for vedro framework:
+
+    e2e/
+       contexts/
+       helpers/
+       interfaces/
+       scenarios/
+         register/
+           register_new_user.py
+       schemas/
+
+
+### Additional links
+- https://vedro.io/docs/tutorial
+

flake8_vedro/rules/VDR104.md

@@ -0,0 +1,52 @@
+#  VDR104. Scenario should have a subject
+A scenario subject is not required for a successful test execution. But when it is empty, then the framework uses the name of the scenario file. And this practice may confuse, especially in parametrized or complicated scenarios. Also, subject availability can help reviewers to understand the test faster.
+
+
+### ❌ Anti-pattern
+```python
+# scenarios/register_user.py
+class Scenario(vedro.Scenario):
+    def when(self)
+      self.response = ...
+
+    def then(self):
+      assert self.response.status_code == 200
+```
+
+```shell
+$ vedro run
+Scenarios
+* register
+ ✔ register user
+
+# 1 scenario, 1 passed, 0 failed, 0 skipped (0.12s)
+```
+
+
+### ✅ Best practice
+```python
+# scenarios/register_user.py
+class Scenario(vedro.Scenario):
+    subject = "register new user"
+
+    def when(self)
+      self.response = ...
+
+    def then(self):
+      assert self.response.status_code == 200
+```
+
+```shell
+$ vedro run
+Scenarios
+* register
+ ✔ register new user
+
+# 1 scenario, 1 passed, 0 failed, 0 skipped (0.12s)
+```
+
+
+### Additional links
+
+- https://vedro.io/docs/tutorial/api/chapter1-first-steps#subject
+

flake8_vedro/rules/VDR105.md

@@ -0,0 +1,24 @@
+# VDR105. Scenario subject should not be empty
+The subject is used to represent user intentions, and intentions can not be empty.
+
+### ❌ Anti-pattern
+```python
+# scenarios/register_user.py
+class Scenario(vedro.Scenario):
+    subject = ""
+    def when(self):
+      self.response = ...
+```
+
+### ✅ Best practice
+```python
+# scenarios/register_user.py
+class Scenario(vedro.Scenario):
+    subject = "register new user"
+    def when(self):
+      self.response = ...
+```
+### Additional links
+
+- https://vedro.io/docs/tutorial/api/chapter1-first-steps#subject
+

flake8_vedro/rules/VDR106.md

@@ -0,0 +1,25 @@
+# VDR106. Scenario should have only one subject
+Several subjects in one scenario could confuse both scenario author and scenario reviewers. 
+
+### ❌ Anti-pattern
+```python
+# scenarios/register_user.py
+class Scenario(vedro.Scenario):
+    subject = "register"
+    subject = "register new user"
+
+    def when(self)
+      self.response = ...
+```
+
+
+### ✅ Best practice
+```python
+# scenarios/register_user.py
+class Scenario(vedro.Scenario):
+    subject = "register new user"
+
+    def when(self)
+      self.response = ...
+```
+

flake8_vedro/rules/VDR107.md

@@ -0,0 +1,72 @@
+# VDR107. Subject is not parameterized*
+*if the test assumes that
+
+The subject serves to represent user intentions and is displayed as a scenario name in test report. In the event of an unparameterized subject within a parameterized scenario, the log will display multiple duplicate scenarios. 
+
+### ❌Anti-pattern
+```python
+import vedro
+from vedro import params
+
+class Scenario(vedro.Scenario):
+    subject = "get objects when auth returned error"
+
+    @params(500)
+    @params(404)
+    def __init__(self, status):
+        self.status = status
+```
+
+```shell
+$ vedro run
+Scenarios
+* get
+ ✔ get objects when auth returned error (0.12s)
+ ✔ get objects when auth returned error (12.11s)
+```
+
+
+### ✅ Best practice
+```python
+import vedro
+from vedro import params
+
+class Scenario(vedro.Scenario):
+    subject = "get objects when auth returned error {status}"
+
+    @params(500)
+    @params(404)
+    def __init__(self, status):
+        self.status = status
+```
+
+or
+
+```python
+import vedro
+from vedro import params
+
+class Scenario(vedro.Scenario):
+    subject = "{subject}"
+
+    @params("get objects when auth returned error 500", 500)
+    @params("get objects when auth returned error 404", 404)
+    def __init__(self, subject, status):
+        self.subject = subject
+        self.status = status
+```
+```shell
+$ vedro run
+Scenarios
+* get
+ ✔ get status 200 (0.12s)
+ ✔ get status 404 (12.11s)
+```
+
+
+
+### Additional links
+
+- https://vedro.io/docs/features/parameterized-scenarios
+
+

flake8_vedro/rules/VDR108.md

@@ -0,0 +1,90 @@
+#  VDR108. Calling functions in parametrization
+Functions called in scenario parameters are executed before all tests, resulting in:
+
+- a delay in the execution of other scenarios,
+- introducing potentially unexpected behavior in tests.
+
+Only contexts’ function call is considered to be a bad practice in order to avoid warnings to object constructors and other third-party libraries.
+
+### ❌ Anti-pattern
+```python
+    # contexts/example.py
+    def greetings(string):
+        print(f"executing greetings({string})")
+        return f"Hello, {string}!"
+
+    def goodbye(string):
+        print(f"executing goodbye({string})")
+        return f"Buy, {string}!"
+
+    # scenarios/example/lambda.py
+    from contexts import greetings, goodbye
+    class Scenario(vedro.Scenario):
+        subject = "example subject"
+
+        @params(greetings("Mike"))
+        @params(goodbye("Jane"))
+        def __init__(self, tmp):
+            self.tmp = tmp
+
+    # scenarios/example/another_scenario.py
+    class Scenario(vedro.Scenario):
+        subject = "another scenario subject"
+
+        def given_user(self):
+           pass
+
+        def when_user_registers(self):
+          pass
+```
+
+```shell
+$ vedro run --subject="another scenario subject"
+executing greetings(Mike)
+executing goodbye(Jane)
+Scenarios
+* example
+ ✔ another scenario subject (0.0s) 
+```
+
+
+### ✅ Best practice
+
+It’s better to use composition or lambda in parametrization.
+
+```python
+# scenarios/example/lambda.py
+from contexts import greetings, goodbye
+
+class Scenario(vedro.Scenario):
+    subject = "example subject"
+
+    @params(greetings, "Mike")
+    @params(goodbye, "Jane")
+    def __init__(self, func, tmp):
+        self.tmp = func(tmp)
+
+    # or
+    @params(lambda: greetings("Mike"))
+    @params(lambda: greetings("Mike"))
+    def __init__(self, f):
+       self.tmp = f()
+
+
+# scenarios/example/another_scenario.py
+class Scenario(vedro.Scenario):
+    subject = "another scenario subject"
+
+    def given_user(self):
+       pass
+
+    def when_user_registers(self):
+      pass
+```
+
+```shell
+$ vedro run --subject="another scenario subject"
+Scenarios
+* example
+ ✔ another scenario subject (0.0s) 
+```