Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Radon support #19

Merged
merged 40 commits into from
Apr 12, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
41eddcd
Added param and params to whitelist
GirZ0n Mar 14, 2021
d176f77
Fixed flake8-spellcheck issue
GirZ0n Mar 14, 2021
1f77c30
Added new test
GirZ0n Mar 16, 2021
79f229b
Added wps-light support
GirZ0n Mar 21, 2021
3e00358
Fixed tests
GirZ0n Mar 21, 2021
fc32b5f
Merge branch 'develop' into wps-light-support
GirZ0n Mar 21, 2021
c295334
Update build.yml
GirZ0n Mar 21, 2021
5c9c329
Small test fix
GirZ0n Mar 21, 2021
542068e
Small code refactoring:
GirZ0n Mar 22, 2021
aa3a6e7
Added support for flake8-broken-line
GirZ0n Mar 23, 2021
f8c2a87
Code refactoring
GirZ0n Mar 23, 2021
48ef2af
Added support for flake8-string-format
GirZ0n Mar 26, 2021
d142938
Added tests for flake8-string-format
GirZ0n Mar 26, 2021
cff0776
Added support for flake8-commas
GirZ0n Mar 27, 2021
a5a6c82
Added tests for flake8-commas
GirZ0n Mar 27, 2021
0c88f53
Added multiline
GirZ0n Mar 27, 2021
323f748
Added C812 to ignore
GirZ0n Mar 27, 2021
4287cf0
Added support for cohesion
GirZ0n Mar 29, 2021
afaf0dd
Fixed tests
GirZ0n Mar 29, 2021
4b2930c
Merge remote-tracking branch 'origin/new-flake8-plugins' into new-fla…
GirZ0n Mar 29, 2021
7b79c6b
Added H601 (cohesion) to ignore
GirZ0n Mar 29, 2021
0d2a9cb
Added test for cohesion
GirZ0n Mar 30, 2021
923c18a
Added sqrt
GirZ0n Mar 30, 2021
43d5193
Added Radon inspector and maintainability index metric
GirZ0n Apr 3, 2021
47cb02e
Added tests for Radon inspector and maintainability metric
GirZ0n Apr 3, 2021
f2d3377
Remove Halstead complexity placeholders
GirZ0n Apr 6, 2021
5a7378f
Add comments
GirZ0n Apr 6, 2021
72075ce
Merge branch 'develop' into radon-support
GirZ0n Apr 6, 2021
ffd212e
Fixed F401
GirZ0n Apr 6, 2021
528599a
Small import refactoring
GirZ0n Apr 6, 2021
9556546
Fixed W293
GirZ0n Apr 6, 2021
ffbcf4c
Trailing commas fix (#20)
GirZ0n Apr 8, 2021
81356dc
Added convert_percentage_of_value_to_lack_of_value
GirZ0n Apr 10, 2021
06553f3
Replaced __get_cohesion_lack with convert_percentage_of_value_to_lack…
GirZ0n Apr 10, 2021
ac4349b
typo fix
GirZ0n Apr 10, 2021
a442f84
Small code refactoring:
GirZ0n Apr 10, 2021
6367493
Removed unused import
GirZ0n Apr 10, 2021
805a492
Small code refactoring:
GirZ0n Apr 10, 2021
62600e3
Added maintainability index tip
GirZ0n Apr 10, 2021
bacb25f
Fixed test
GirZ0n Apr 10, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Added Radon inspector and maintainability index metric
  • Loading branch information
GirZ0n committed Apr 3, 2021
commit 43d519315b94a04d13c8fbfde7e39b9d0d8cf28e
7 changes: 5 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,10 @@ Python language:
- [x] Pylint [GNU LGPL v2]
* [Site and docs](https://www.pylint.org/)
* [Repository](https://github.com/PyCQA/pylint)


- [x] Radon [MIT]
* [Site and docs](https://radon.readthedocs.io/en/latest/)
* [Repository](https://github.com/rubik/radon)

Java language:

Expand Down Expand Up @@ -92,7 +95,7 @@ Argument | Description
--- | ---
**‑h**, **‑‑help** | show the help message and exit.
**‑v**, **‑‑verbosity** | choose logging level according [this](https://docs.python.org/3/library/logging.html#levels) list: `1` - **ERROR**; `2` - **INFO**; `3` - **DEBUG**; `0` - disable logging (**CRITICAL** value); default value is `0` (**CRITICAL**).
**‑d**, **‑‑disable** | disable inspectors. Available values: for **Python** language: `pylint` for [Pylint](https://github.com/PyCQA/pylint), `flake8` for [flake8](https://flake8.pycqa.org/en/latest/), `python_ast` to check different measures providing by AST; for **Java** language: `checkstyle` for the [Checkstyle](https://checkstyle.sourceforge.io/), `pmd` for [PMD](https://pmd.github.io/); for `Kotlin` language: detekt for [Detekt](https://detekt.github.io/detekt/); for **JavaScript** language: `eslint` for [ESlint](https://eslint.org/). Example: `-d pylint,flake8`.
**‑d**, **‑‑disable** | disable inspectors. Available values: for **Python** language: `pylint` for [Pylint](https://github.com/PyCQA/pylint), `flake8` for [flake8](https://flake8.pycqa.org/en/latest/), `radon` for [Radon](https://radon.readthedocs.io/en/latest/), `python_ast` to check different measures providing by AST; for **Java** language: `checkstyle` for the [Checkstyle](https://checkstyle.sourceforge.io/), `pmd` for [PMD](https://pmd.github.io/); for `Kotlin` language: detekt for [Detekt](https://detekt.github.io/detekt/); for **JavaScript** language: `eslint` for [ESlint](https://eslint.org/). Example: `-d pylint,flake8`.
**‑‑allow-duplicates** | allow duplicate issues found by different linters. By default, duplicates are skipped.
**‑‑language-version**, **‑‑language_version** | specify the language version for JAVA inspectors. Available values: `java7`, `java8`, `java9`, `java11`. **Note**: **‑‑language_version** is deprecated. Will be deleted in the future.
**‑‑n-cpu**, **‑‑n_cpu** | specify number of _cpu_ that can be used to run inspectors. **Note**: **‑‑n_cpu** is deprecated. Will be deleted in the future.
Expand Down
1 change: 1 addition & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ flake8-broken-line==0.3.0
flake8-string-format==0.3.0
flake8-commas==2.0.0
cohesion==1.0.0
radon==4.5.0

# extra libraries and frameworks
django==3.0.8
Expand Down
2 changes: 2 additions & 0 deletions src/python/review/inspectors/inspector_type.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ class InspectorType(Enum):
PYLINT = 'PYLINT'
PYTHON_AST = 'PYTHON_AST'
FLAKE8 = 'FLAKE8'
RADON = 'RADON'

# Java language
PMD = 'PMD'
Expand All @@ -29,6 +30,7 @@ def available_values(cls) -> List[str]:
InspectorType.PYLINT.value,
InspectorType.FLAKE8.value,
InspectorType.PYTHON_AST.value,
InspectorType.RADON.value,

# Java language
InspectorType.PMD.value,
Expand Down
11 changes: 11 additions & 0 deletions src/python/review/inspectors/issue.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ class IssueType(Enum):
COHESION = 'COHESION'
CLASS_RESPONSE = 'CLASS_RESPONSE'
METHOD_NUMBER = 'METHOD_NUMBER'
MAINTAINABILITY = 'MAINTAINABILITY'


# Keys in results dictionary
Expand All @@ -44,6 +45,7 @@ class IssueData(Enum):
BOOL_EXPR_LEN = 'bool_expr_len'
CYCLOMATIC_COMPLEXITY = 'cc_value'
COHESION_LACK = 'cohesion_lack'
MAINTAINABILITY_LACK = 'maintainability_lack'

@classmethod
def get_base_issue_data_dict(cls, file_path: Union[str, Path], inspector_type: InspectorType, line_number: int = 1,
Expand Down Expand Up @@ -178,3 +180,12 @@ class MethodNumberIssue(BaseIssue, Measurable):

def measure(self) -> int:
return self.method_number


@dataclass(frozen=True)
class MaintainabilityLackIssue(BaseIssue, Measurable):
maintainability_lack: int
type = IssueType.MAINTAINABILITY

def measure(self) -> int:
return self.maintainability_lack
75 changes: 75 additions & 0 deletions src/python/review/inspectors/radon/radon.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
import re
from math import floor
from pathlib import Path
from typing import List

from src.python.review.common.subprocess_runner import run_in_subprocess
from src.python.review.inspectors.base_inspector import BaseInspector
from src.python.review.inspectors.inspector_type import InspectorType
from src.python.review.inspectors.issue import BaseIssue, IssueData, IssueType, MaintainabilityLackIssue


class RadonInspector(BaseInspector):
inspector_type = InspectorType.RADON

@classmethod
def inspect(cls, path: Path, config: dict) -> List[BaseIssue]:
mi_command = [
"radon", "mi",
"--max", "F",
"--show",
path
]

hal_command = [
"radon", "hal",
path
]

mi_output = run_in_subprocess(mi_command)
mi_issues = cls.mi_parse(mi_output)

hal_output = run_in_subprocess(hal_command)
hal_issues = cls.hal_parse(hal_output)

return mi_issues + hal_issues

@classmethod
def mi_parse(cls, mi_output: str) -> List[BaseIssue]:
"""
Parses the results of the "mi" command.

:param mi_output: "mi" command output.
:return: list of issues.
"""
row_re = re.compile(r"^(.*) - \w \((.*)\)$", re.M)

issues: List[BaseIssue] = []
for groups in row_re.findall(mi_output):
file_path = Path(groups[0])
maintainability_lack = cls.__get_maintainability_lack(float(groups[1]))

issue_data = IssueData.get_base_issue_data_dict(file_path, cls.inspector_type)
issue_data[IssueData.DESCRIPTION.value] = "" # TODO: add tip
issue_data[IssueData.MAINTAINABILITY_LACK.value] = maintainability_lack
issue_data[IssueData.ISSUE_TYPE.value] = IssueType.MAINTAINABILITY

issues.append(MaintainabilityLackIssue(**issue_data))

return issues

@staticmethod
def __get_maintainability_lack(maintainability_index: float) -> int:
"""
Converts maintainability index to lack of maintainability.
Calculated by the formula: floor(100 - maintainability_index).

:param maintainability_index: value in the range from 0 to 100.
:return: lack of maintainability.
"""
return floor(100 - maintainability_index)

# TODO: add Halstead complexity
@classmethod
def hal_parse(cls, hal_output: str) -> List[BaseIssue]:
return []
5 changes: 5 additions & 0 deletions src/python/review/quality/evaluate_quality.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,10 @@
LANGUAGE_TO_COHESION_RULE_CONFIG,
CohesionRule,
)
from src.python.review.quality.rules.maintainability_scoring import (
LANGUAGE_TO_MAINTAINABILITY_RULE_CONFIG,
MaintainabilityRule,
)


def __get_available_rules(language: Language) -> List[Rule]:
Expand All @@ -56,6 +60,7 @@ def __get_available_rules(language: Language) -> List[Rule]:
ResponseRule(LANGUAGE_TO_RESPONSE_RULE_CONFIG[language]),
WeightedMethodsRule(LANGUAGE_TO_WEIGHTED_METHODS_RULE_CONFIG[language]),
CohesionRule(LANGUAGE_TO_COHESION_RULE_CONFIG[language]),
MaintainabilityRule(LANGUAGE_TO_MAINTAINABILITY_RULE_CONFIG[language]),
]


Expand Down
68 changes: 68 additions & 0 deletions src/python/review/quality/rules/maintainability_scoring.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
from dataclasses import dataclass
from typing import Optional

from src.python.review.common.language import Language
from src.python.review.inspectors.issue import IssueType
from src.python.review.quality.model import Rule, QualityType


@dataclass
class MaintainabilityRuleConfig:
maintainability_lack_good: int
maintainability_lack_moderate: int
maintainability_lack_bad: int


common_maintainability_rule_config = MaintainabilityRuleConfig(
maintainability_lack_good=50,
maintainability_lack_moderate=80,
maintainability_lack_bad=90,
)

LANGUAGE_TO_MAINTAINABILITY_RULE_CONFIG = {
Language.JAVA: common_maintainability_rule_config,
Language.PYTHON: common_maintainability_rule_config,
Language.KOTLIN: common_maintainability_rule_config,
Language.JS: common_maintainability_rule_config,
}


class MaintainabilityRule(Rule):
def __init__(self, config: MaintainabilityRuleConfig):
self.config = config
self.rule_type = IssueType.MAINTAINABILITY
self.maintainability_lack: Optional[int] = None

def apply(self, maintainability_lack):
self.maintainability_lack = maintainability_lack
if maintainability_lack > self.config.maintainability_lack_bad:
self.quality_type = QualityType.BAD
self.next_level_delta = maintainability_lack - self.config.maintainability_lack_bad
elif maintainability_lack > self.config.maintainability_lack_moderate:
self.quality_type = QualityType.MODERATE
self.next_level_delta = maintainability_lack - self.config.maintainability_lack_moderate
elif maintainability_lack > self.config.maintainability_lack_good:
self.quality_type = QualityType.GOOD
self.next_level_delta = maintainability_lack - self.config.maintainability_lack_good
else:
self.quality_type = QualityType.EXCELLENT
self.next_level_delta = 0
self.next_level_type = self.__get_next_quality_type()

def __get_next_quality_type(self) -> QualityType:
if self.quality_type == QualityType.BAD:
return QualityType.MODERATE
elif self.quality_type == QualityType.MODERATE:
return QualityType.GOOD
return QualityType.EXCELLENT

def merge(self, other: 'MaintainabilityRule') -> 'MaintainabilityRule':
config = MaintainabilityRuleConfig(
min(self.config.maintainability_lack_bad, other.config.maintainability_lack_bad),
min(self.config.maintainability_lack_moderate, other.config.maintainability_lack_moderate),
min(self.config.maintainability_lack_good, other.config.maintainability_lack_good)
)
result_rule = MaintainabilityRule(config)
result_rule.apply(max(self.maintainability_lack, other.maintainability_lack))

return result_rule
2 changes: 2 additions & 0 deletions src/python/review/reviewers/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
from src.python.review.inspectors.detekt.detekt import DetektInspector
from src.python.review.inspectors.eslint.eslint import ESLintInspector
from src.python.review.inspectors.flake8.flake8 import Flake8Inspector
from src.python.review.inspectors.radon.radon import RadonInspector
from src.python.review.inspectors.issue import BaseIssue
from src.python.review.inspectors.pmd.pmd import PMDInspector
from src.python.review.inspectors.pyast.python_ast import PythonAstInspector
Expand All @@ -24,6 +25,7 @@
PylintInspector(),
Flake8Inspector(),
PythonAstInspector(),
RadonInspector(),
],
Language.JAVA: [
CheckstyleInspector(),
Expand Down
4 changes: 4 additions & 0 deletions src/python/review/reviewers/utils/code_statistics.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ class CodeStatistics:

max_cyclomatic_complexity: int
max_cohesion_lack: int
max_maintainability_lack: int
max_func_len: int
max_bool_expr_len: int

Expand All @@ -40,6 +41,7 @@ def issue_type_to_statistics_dict(self) -> Dict[IssueType, int]:

IssueType.CYCLOMATIC_COMPLEXITY: self.max_cyclomatic_complexity,
IssueType.COHESION: self.max_cohesion_lack,
IssueType.MAINTAINABILITY: self.max_maintainability_lack,
IssueType.FUNC_LEN: self.max_func_len,
IssueType.BOOL_EXPR_LEN: self.max_bool_expr_len,

Expand Down Expand Up @@ -85,6 +87,7 @@ def gather_code_statistics(issues: List[BaseIssue], path: Path) -> CodeStatistic
func_lens = __get_max_measure_by_issue_type(IssueType.FUNC_LEN, issues)
cyclomatic_complexities = __get_max_measure_by_issue_type(IssueType.CYCLOMATIC_COMPLEXITY, issues)
cohesion_lacks = __get_max_measure_by_issue_type(IssueType.COHESION, issues)
maintainabilities = __get_max_measure_by_issue_type(IssueType.MAINTAINABILITY, issues)

# Actually, we expect only one issue with each of the following metrics.
inheritance_depths = __get_max_measure_by_issue_type(IssueType.INHERITANCE_DEPTH, issues)
Expand All @@ -101,6 +104,7 @@ def gather_code_statistics(issues: List[BaseIssue], path: Path) -> CodeStatistic
max_func_len=func_lens,
n_line_len=issue_type_counter[IssueType.LINE_LEN],
max_cohesion_lack=cohesion_lacks,
max_maintainability_lack=maintainabilities,
max_cyclomatic_complexity=cyclomatic_complexities,
inheritance_depth=inheritance_depths,
class_response=class_responses,
Expand Down
4 changes: 4 additions & 0 deletions src/python/review/reviewers/utils/issues_filter.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
from src.python.review.quality.rules.method_number_scoring import LANGUAGE_TO_METHOD_NUMBER_RULE_CONFIG
from src.python.review.quality.rules.weighted_methods_scoring import LANGUAGE_TO_WEIGHTED_METHODS_RULE_CONFIG
from src.python.review.quality.rules.cohesion_scoring import LANGUAGE_TO_COHESION_RULE_CONFIG
from src.python.review.quality.rules.maintainability_scoring import LANGUAGE_TO_MAINTAINABILITY_RULE_CONFIG


def __get_issue_type_to_low_measure_dict(language: Language) -> Dict[IssueType, int]:
Expand All @@ -25,6 +26,9 @@ def __get_issue_type_to_low_measure_dict(language: Language) -> Dict[IssueType,
IssueType.CLASS_RESPONSE: LANGUAGE_TO_RESPONSE_RULE_CONFIG[language].response_good,
IssueType.WEIGHTED_METHOD: LANGUAGE_TO_WEIGHTED_METHODS_RULE_CONFIG[language].weighted_methods_good,
IssueType.COHESION: LANGUAGE_TO_COHESION_RULE_CONFIG[language].cohesion_lack_bad,
IssueType.MAINTAINABILITY: LANGUAGE_TO_MAINTAINABILITY_RULE_CONFIG[
language
].maintainability_lack_good
}


Expand Down
2 changes: 2 additions & 0 deletions test/python/inspectors/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ class IssuesTestInfo:
n_bool_expr_len: int = 0
n_other_complexity: int = 0
n_cohesion: int = 0
n_maintainability: int = 0


def gather_issues_test_info(issues: List[BaseIssue]) -> IssuesTestInfo:
Expand All @@ -88,6 +89,7 @@ def gather_issues_test_info(issues: List[BaseIssue]) -> IssuesTestInfo:
n_bool_expr_len=counter[IssueType.BOOL_EXPR_LEN],
n_other_complexity=counter[IssueType.COMPLEXITY],
n_cohesion=counter[IssueType.COHESION],
n_maintainability=counter[IssueType.MAINTAINABILITY]
)


Expand Down