Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(linters): Introduce ruff and fix issues #831

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

MaxymVlasov
Copy link
Collaborator

@MaxymVlasov MaxymVlasov commented Feb 27, 2025

TBD:
After @webknjaz will approve config, split to 4 PRs:

  1. Config
  2. Automatic ruff fixes
  3. Manual fixes
  4. Unrelated + disable git blame to s.2

Copy link

coderabbitai bot commented Feb 27, 2025

📝 Walkthrough

Summary by CodeRabbit

  • Chores

    • Updated automated code quality tooling with refined configurations for container, shell, and Python linting.
  • Refactor / Style

    • Improved internal code formatting and documentation for better readability.
    • Enhanced CLI warning messages by including clearer migration guidance for deprecated functionality.
  • Tests

    • Adjusted test structures and assertions to boost consistency and reliability.

Walkthrough

The changes update linting configurations and perform code cleanup. The pre-commit configuration now includes new repositories for Hadolint and Bash linters with explicit ignore rules, while older configurations were removed. A new ruff.toml file sets Python linting and formatting rules. In the CLI modules and tests, import statements, docstrings, and error handling have been reformatted for consistency and clarity without affecting functionality.

Changes

File(s) Change Summary
.pre-commit-config.yaml, ruff.toml Lint configuration updates: Removed old entries and added new Hadolint rules and Bash lint hooks, plus a new ruff.toml for Python linting and formatting.
src/.../__main__.py, src/.../_cli.py, src/.../_cli_parsing.py, src/.../_cli_subcommands.py, src/.../_errors.py, src/.../_types.py, src/.../terraform_docs_replace.py CLI and utility modules: Revised import formatting, enhanced docstrings with return type annotations, refined error handling (including FIXME comments), and performed minor code cleanup without altering functionality.
tests/pytest/_cli_test.py, tests/pytest/terraform_docs_replace_test.py Test files: Reorganized imports, added @staticmethod decorators to helper methods, and updated assertion and parameterization formats for improved clarity.

Suggested reviewers

  • yermulnik

📜 Recent review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c949dd2 and 7e1dd4f.

📒 Files selected for processing (11)
  • .pre-commit-config.yaml (2 hunks)
  • ruff.toml (1 hunks)
  • src/pre_commit_terraform/__main__.py (1 hunks)
  • src/pre_commit_terraform/_cli.py (1 hunks)
  • src/pre_commit_terraform/_cli_parsing.py (1 hunks)
  • src/pre_commit_terraform/_cli_subcommands.py (0 hunks)
  • src/pre_commit_terraform/_errors.py (1 hunks)
  • src/pre_commit_terraform/_types.py (2 hunks)
  • src/pre_commit_terraform/terraform_docs_replace.py (3 hunks)
  • tests/pytest/_cli_test.py (4 hunks)
  • tests/pytest/terraform_docs_replace_test.py (8 hunks)
💤 Files with no reviewable changes (1)
  • src/pre_commit_terraform/_cli_subcommands.py
✅ Files skipped from review due to trivial changes (6)
  • src/pre_commit_terraform/main.py
  • src/pre_commit_terraform/_types.py
  • ruff.toml
  • src/pre_commit_terraform/_cli_parsing.py
  • src/pre_commit_terraform/_cli.py
  • src/pre_commit_terraform/_errors.py
⏰ Context from checks skipped due to timeout of 90000ms (12)
  • GitHub Check: 🧪 Tests / pytest@🐍3.9@windows-2025
  • GitHub Check: 🧪 Tests / pytest@🐍3.13@windows-2025
  • GitHub Check: 🧪 Tests / pytest@🐍3.13@macos-14
  • GitHub Check: 🧪 Tests / pytest@🐍3.13@macos-13
  • GitHub Check: 🧪 Tests / pytest@🐍3.12@windows-2025
  • GitHub Check: 🧪 Tests / pytest@🐍3.12@macos-14
  • GitHub Check: 🧪 Tests / pytest@🐍3.12@macos-13
  • GitHub Check: 🧪 Tests / pytest@🐍3.11@windows-2025
  • GitHub Check: 🧪 Tests / pytest@🐍3.10@windows-2025
  • GitHub Check: 🧹 Linters / pre-commit@🐍3.13@ubuntu-latest
  • GitHub Check: 🧪 Tests / pytest@🐍3.10@macos-14
  • GitHub Check: pre-commit
🔇 Additional comments (17)
.pre-commit-config.yaml (3)

50-63: Dockerfile Linter: Verify Beta Release and Ignore Rules
The new Hadolint configuration is clear and well documented with specific ignore rules. However, note that the rev is set to v2.12.1-beta. Please confirm that using a beta release for Dockerfile linting is acceptable for your production needs. Also, if these ignore rules have particular rationale beyond the inline comments, consider adding more detailed documentation.


97-111: Bash Linter: Confirm Formatting Arguments
The newly added Bash linter configuration—with hooks for shfmt and shellcheck—appears to enforce consistent shell script formatting using the specified arguments. Please ensure that these arguments (-l, -i '2', -ci, -sr, -w) fully align with your project's style guidelines. It may be beneficial to add contextual documentation if these options require further explanation.


112-122: Python Linters: Introduction of Ruff and Ruff-Format Hooks
The addition of the Ruff pre-commit hooks, including the --fix flag for auto-correction, is a strong move toward improving code quality consistently. Make sure the related ruff.toml (if present) is up-to-date with these settings so that the linting and formatting behavior remains predictable across the project.

tests/pytest/terraform_docs_replace_test.py (5)

3-5: Well-structured import organization.

Separating imports from the same module and properly documenting why certain imports use noqa directives enhances readability and shows good practices for dealing with linting exceptions.


28-28: Cleaner string concatenation.

Good simplification of the regex pattern by concatenating into a single line without changing functionality.


37-65: Good parametrization format update.

Changing from tuple to list format for test parameters improves consistency with Python style guidelines while maintaining the same functionality.


85-85: Improved assertion readability.

Changing to assert invoke_cli_app(parsed_cli_args) == ReturnCode.OK follows the conventional "actual == expected" pattern, making test failures more intuitive to understand.


119-120: Well-documented security override.

Good practice to explain why shell=True is necessary with an inline comment and to use noqa: S604 to acknowledge the security implication.

src/pre_commit_terraform/terraform_docs_replace.py (5)

1-7: Clear module documentation and import organization.

The improved module docstring and well-organized imports with appropriate noqa comments clearly document the purpose of the module and explain exceptions to linting rules.


53-58: Enhanced function documentation.

The added detailed docstring for invoke_cli_app properly documents the function's purpose and return type, which improves code readability and maintainability.


59-65: Well-documented warning usage.

Good use of noqa: B028 to explain why the warning is implemented this way, with appropriate context about why a user warning is used.


69-72: Improved file extension checking.

The refactored conditional is more concise and readable while properly documenting with noqa: PTH120 why a potentially problematic function is retained.


76-92: Better command construction pattern.

The refactored approach for building the command arguments is more structured and maintainable, with good documentation about why shell=True is necessary in this context.

tests/pytest/_cli_test.py (4)

3-5: Consistent import style.

The import organization follows the same pattern as in other files, promoting consistency throughout the codebase.


53-59: Appropriate use of @staticmethod.

Good addition of @staticmethod decorators to methods that don't use instance data, making the code more accurate and following Python best practices.


88-89: Enhanced error handling readability.

Assigning the error message to a variable before raising the exception improves code clarity and maintainability.


97-97: Better regex pattern notation.

Using raw string notation (r"...") for the regex pattern is good practice and helps avoid unintentional escape sequence interpretations.

✨ Finishing Touches
  • 📝 Generate Docstrings

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

codecov bot commented Feb 27, 2025

Codecov Report

Attention: Patch coverage is 98.71795% with 1 line in your changes missing coverage. Please review.

Project coverage is 97.05%. Comparing base (c949dd2) to head (7e1dd4f).

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/pre_commit_terraform/terraform_docs_replace.py 96.66% 1 Missing ⚠️

❌ Your patch check has failed because the patch coverage (98.71%) is below the target coverage (100.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #831      +/-   ##
==========================================
+ Coverage   96.38%   97.05%   +0.67%     
==========================================
  Files          10       10              
  Lines         249      272      +23     
  Branches        7        6       -1     
==========================================
+ Hits          240      264      +24     
+ Misses          9        8       -1     
Flag Coverage Δ
CI-GHA 97.05% <98.71%> (+0.67%) ⬆️
MyPy 92.07% <93.24%> (+1.01%) ⬆️
OS-Linux 97.05% <98.71%> (+0.67%) ⬆️
OS-Windows 100.00% <100.00%> (ø)
OS-macOS 100.00% <100.00%> (ø)
Py-3.10.11 100.00% <100.00%> (ø)
Py-3.10.16 100.00% <100.00%> (ø)
Py-3.11.11 100.00% <100.00%> (ø)
Py-3.11.9 100.00% <100.00%> (ø)
Py-3.12.8 100.00% <100.00%> (ø)
Py-3.12.9 100.00% <100.00%> (ø)
Py-3.13.1 100.00% <100.00%> (ø)
Py-3.13.2 97.05% <98.71%> (+0.67%) ⬆️
Py-3.9.13 100.00% <100.00%> (ø)
Py-3.9.21 100.00% <100.00%> (ø)
VM-macos-13 100.00% <100.00%> (ø)
VM-macos-14 100.00% <100.00%> (ø)
VM-ubuntu-24.04 100.00% <100.00%> (ø)
VM-ubuntu-latest 92.07% <93.24%> (+1.01%) ⬆️
VM-windows-2025 100.00% <100.00%> (ø)
pytest 100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@yermulnik

This comment was marked as resolved.

@MaxymVlasov

This comment was marked as resolved.

@yermulnik

This comment was marked as resolved.

Comment on lines +15 to +16
"D213", # multi-line-summary-second-line. Incompatible with multi-line-summary-first-line (D212)
"D203", # one-blank-line-before-class. Incompatible with no-blank-line-before-class (D211)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a convention setting per https://docs.astral.sh/ruff/settings/#lint_pydocstyle_convention. I think if you add the following, it'll automatically disable the incompatible rules:

[lint.pydocstyle]
convention = "pep257"

"CPY001", # Missing copyright notice at top of file
"D213", # multi-line-summary-second-line. Incompatible with multi-line-summary-first-line (D212)
"D203", # one-blank-line-before-class. Incompatible with no-blank-line-before-class (D211)
"INP001", # We use namespace packages in this project
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tried setting the config option instead? https://docs.astral.sh/ruff/settings/#namespace-packages

namespace-packages = ["src/pre_commit_terraform/"]

@@ -65,35 +64,37 @@ def invoke_cli_app(self, parsed_cli_args: Namespace) -> ReturnCodeType:
[CustomCmdStub()],
)

assert ReturnCode.ERROR == invoke_cli_app(['sentinel'])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Urgh..

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol. Didn't know that ruff able to replace code in this way

self, subcommand_parser: ArgumentParser,
) -> None:
@staticmethod
def populate_argument_parser(subcommand_parser: ArgumentParser) -> None: # noqa: ARG004
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would need a justification comment, then.

But wait, a second… Why are you even changing this to a @staticmethod? You shouldn't do that. This is a stub for the interface that exists elsewhere, the signatures must match exactly.

@@ -24,8 +25,7 @@ def test_arg_parser_populated() -> None:
def test_check_is_deprecated() -> None:
"""Verify that `replace-docs` shows a deprecation warning."""
deprecation_msg_regex = (
r'^`terraform_docs_replace` hook is DEPRECATED\.'
'For migration.*$'
r'^`terraform_docs_replace` hook is DEPRECATED\.' + 'For migration.*$'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't concatenate static strings in runtime. we talked about this already..


check_call_mock.assert_called_once_with(expected_cmd, shell=True)
assert invoke_cli_app(parsed_cli_args) == ReturnCode.ERROR
# We call cli tools, of course we use shell=True
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which is a terrible, dangerous idea and should be avoided instead of ignored…

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That pre-commit hooks which wrap external CLI tools. They are wrappers. How do you recommend to make wrappers to simplify tools execution and add some extra functional on top of tools, w/o calling underling tool?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Call them without the dangerous shell=True argument.

@webknjaz
Copy link
Contributor

4 PRs:

unrelated is a separate PR. blame ignore is a separate PR. Plus I noticed reshuffling logic which is another one. So it's like 6 PRs.

@webknjaz
Copy link
Contributor

Also, plan for adding https://wemake-python-styleguide.rtfd.io on top after all this.

from subprocess import CalledProcessError
from argparse import ArgumentParser
from argparse import Namespace
from subprocess import CalledProcessError # noqa: S404. We invoke cli tools
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Subprocess interaction should probably be quarantined in a dedicated module where these noqas would be allowed and nowhere else. That module should implement wrappers that the rest of the project would call. This design allows keeping all potentially dangerous code in one place that you can be extra careful about.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants