fix: Remove sneaky unicode characters that break a regex and console outputs on Windows #6987

gukoff · 2025-01-31T12:54:05Z

User description

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Description

There were 2 almost unnoticeable Unicode characters that caused 2 separate problems:

Regex for ANSI escaping contained the wrong dash, making the regex incorrect.
Message about the file parsing errors failed to be displayed when running on a Windows agent on Azure DevOps:

##[error]2025-01-31 08:10:40,467 [MainThread  ] [ERROR]  Exception traceback:
##[error]Traceback (most recent call last):
##[error]  File "checkov\main.py", line 571, in run
##[error]    exit_codes.append(self.print_results(
##[error]                      ^^^^^^^^^^^^^^^^^^^
##[error]  File "checkov\main.py", line 848, in print_results
##[error]    return runner_registry.print_reports(
##[error]           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
##[error]  File "checkov\common\runners\runner_registry.py", line 643, in print_reports
##[error]    self.save_output_to_file(
##[error]  File "checkov\common\runners\runner_registry.py", line 289, in save_output_to_file
##[error]    file_path.write_text(data)
##[error]  File "pathlib.py", line 1048, in write_text
##[error]  File "encodings\cp1252.py", line 19, in encode
##[error]UnicodeEncodeError: 'charmap' codec can't encode character '\u05bf' in position 156311: character maps to <undefined>

Related: #6416

Checklist:

I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have added tests that prove my feature, policy, or fix is effective and works
New and existing tests pass locally with my changes

Generated description

Below is a concise technical summary of the changes proposed in this PR:

Removes an invisible Unicode character (Hebrew Point Rafe) from error messages and comments. Updates regular expression for ANSI escape sequence matching. Improves readability of comments in Kubernetes-related files.

Topic

Details

Error Message Fix

Removes an invisible Unicode character from error messages to fix display issues on Windows

Modified files (2)

checkov/common/output/report.py
checkov/common/runners/runner_registry.py

Latest Contributors(2)

User	Commit	Date
lirshindalman	fix-terraform_plan-run...	December 01, 2024
[email protected]	feat-arm-unsupported-m...	October 15, 2024

Comment Cleanup

Improves readability of comments in Kubernetes-related files and serverless utilities

Modified files (3)

checkov/serverless/utils.py
checkov/terraform/checks/resource/kubernetes/HostPort.py
checkov/kubernetes/checks/resource/k8s/HostPort.py

Latest Contributors(2)

User	Commit	Date
omriyoffe-panw	feat-serverless-Server...	December 18, 2024
[email protected]...	feat-terraform-include...	November 07, 2022

This pull request is reviewed by Baz. Join @gukoff and the rest of your team on (Baz).

gukoff · 2025-02-03T11:54:24Z

checkov/common/output/report.py

@@ -337,7 +337,7 @@ def print_console(

        if not is_quiet:
            for file in self.parsing_errors:
-                output_data += colored(f"Error parsing file {file}ֿ\n", "red")
+                output_data += colored(f"Error parsing file {file}\n", "red")


This line causes problems with outputs, notice a symbol before \n

gukoff · 2025-02-03T11:54:45Z

checkov/common/runners/runner_registry.py

@@ -488,7 +488,7 @@ def print_reports(
            )

            # Remove colors from the cli output
-            ansi_escape = re.compile(r'(?:\x1B[@-_]|[\x80-\x9F])[0–9:;<=>?]*[ -/]*[@-~]')
+            ansi_escape = re.compile(r'(?:\x1B[@-_]|[\x80-\x9F])[0-9:;<=>?]*[ -/]*[@-~]')


This line makes a wrong regex, notice a wrong dash

gukoff · 2025-02-03T11:55:59Z

checkov/kubernetes/checks/resource/k8s/HostPort.py

@@ -9,7 +9,7 @@ def __init__(self) -> None:
        """
        https://kubernetes.io/docs/concepts/configuration/overview/

-        Don’t specify a hostPort for a Pod unless it is absolutely necessary.
+        Don't specify a hostPort for a Pod unless it is absolutely necessary.


This line was changed by the script I made to go over this repo to find bad unicode characters.
I left it in the PR just because unicode seems unnecessary here. There's no bug in this line.

gukoff · 2025-02-03T11:56:03Z

checkov/serverless/utils.py

@@ -54,7 +54,7 @@ def get_scannable_file_paths(root_folder: str | None = None, excluded_paths: lis
        return files_list

    for root, d_names, f_names in os.walk(root_folder):
-        # Don't walk in to "node_modules" directories under the root folder. If –for some reason–
+        # Don't walk in to "node_modules" directories under the root folder. If for some reason


This line was changed by the script I made to go over this repo to find bad unicode characters.
I left it in the PR just because unicode seems unnecessary here. There's no bug in this line.

gukoff · 2025-02-03T11:56:05Z

checkov/terraform/checks/resource/kubernetes/HostPort.py

@@ -9,7 +9,7 @@ def __init__(self):
        """
               https://kubernetes.io/docs/concepts/configuration/overview/

-               Don’t specify a hostPort for a Pod unless it is absolutely necessary.
+               Don't specify a hostPort for a Pod unless it is absolutely necessary.


This line was changed by the script I made to go over this repo to find bad unicode characters.
I left it in the PR just because unicode seems unnecessary here. There's no bug in this line.

Remove a sneaky unicode character

9d294d2

gukoff requested a deployment to scan-security January 31, 2025 12:54 — with GitHub Actions Waiting

Fix unicode dash

aad9972

gukoff requested a deployment to scan-security January 31, 2025 13:29 — with GitHub Actions Waiting

Konstantin Gukov added 2 commits January 31, 2025 14:30

Remove unicode single quote

b68fba5

Remove dashes from comment

3843e22

gukoff requested a deployment to scan-security January 31, 2025 13:35 — with GitHub Actions Waiting

gukoff changed the title ~~fix: Remove a sneaky Hebrew Point Rafe character that breaks console outputs on Windows~~ fix: Remove a sneaky unicode characters that break a regex and a console outputs on Windows Jan 31, 2025

gukoff changed the title ~~fix: Remove a sneaky unicode characters that break a regex and a console outputs on Windows~~ fix: Remove sneaky unicode characters that break a regex and console outputs on Windows Jan 31, 2025

gukoff commented Feb 3, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: Remove sneaky unicode characters that break a regex and console outputs on Windows #6987

fix: Remove sneaky unicode characters that break a regex and console outputs on Windows #6987

gukoff commented Jan 31, 2025 •

edited

Loading

gukoff Feb 3, 2025

gukoff Feb 3, 2025

gukoff Feb 3, 2025

gukoff Feb 3, 2025

gukoff Feb 3, 2025

fix: Remove sneaky unicode characters that break a regex and console outputs on Windows #6987

Are you sure you want to change the base?

fix: Remove sneaky unicode characters that break a regex and console outputs on Windows #6987

Conversation

gukoff commented Jan 31, 2025 • edited Loading

User description

Description

Checklist:

Generated description

gukoff Feb 3, 2025

Choose a reason for hiding this comment

gukoff Feb 3, 2025

Choose a reason for hiding this comment

gukoff Feb 3, 2025

Choose a reason for hiding this comment

gukoff Feb 3, 2025

Choose a reason for hiding this comment

gukoff Feb 3, 2025

Choose a reason for hiding this comment

gukoff commented Jan 31, 2025 •

edited

Loading