Add --unstable flag

CHANGES.md

@@ -34,6 +34,14 @@ changes:
 - Fix incorrect formatting of certain async statements (#3609)
 - Allow combining `# fmt: skip` with other comments (#3959)
 
+There are already a few improvements in the `--preview` style, which are slated for the
+2025 stable style. Try them out and
+[share your feedback](https://github.com/psf/black/issues). In the past, the preview
+style has included some features that we were not able to stabilize. This year, we're
+adding a separate `--unstable` style for features with known problems. Now, the
+`--preview` style only includes features that we actually expect to make it into next
+year's stable style.
+
 ### Stable style
 
 <!-- Changes that affect Black's stable style -->
@@ -53,6 +61,7 @@ release:
 
 <!-- Changes that affect Black's preview style -->
 
+- Add `--unstable` style (#4096)
 - Format module docstrings the same as class and function docstrings (#4095)
 - Fix crash when using a walrus in a dictionary (#4155)
 - Fix unnecessary parentheses when wrapping long dicts (#4135)

docs/faq.md

@@ -41,9 +41,10 @@ other tools, such as `# noqa`, may be moved by _Black_. See below for more detai
 Stable. _Black_ aims to enforce one style and one style only, with some room for
 pragmatism. See [The Black Code Style](the_black_code_style/index.md) for more details.
 
-Starting in 2022, the formatting output will be stable for the releases made in the same
-year (other than unintentional bugs). It is possible to opt-in to the latest formatting
-styles, using the `--preview` flag.
+Starting in 2022, the formatting output is stable for the releases made in the same year
+(other than unintentional bugs). At the beginning of every year, the first release will
+make changes to the stable style. It is possible to opt in to the latest formatting
+styles using the `--preview` flag.
 
 ## Why is my file not formatted?
 

docs/the_black_code_style/current_style.md

@@ -449,6 +449,12 @@ file that are not enforced yet but might be in a future version of the formatter
 _Black_ will normalize line endings (`\n` or `\r\n`) based on the first line ending of
 the file.
 
+### Form feed characters
+
+_Black_ will retain form feed characters on otherwise empty lines at the module level.
+Only one form feed is retained for a group of consecutive empty lines. Where there are
+two empty lines in a row, the form feed is placed on the second line.
+
 ## Pragmatism
 
 Early versions of _Black_ used to be absolutist in some respects. They took after its

docs/the_black_code_style/future_style.md

@@ -1,117 +1,38 @@
 # The (future of the) Black code style
 
-```{warning}
-Changes to this document often aren't tied and don't relate to releases of
-_Black_. It's recommended that you read the latest version available.
-```
-
-## Using backslashes for with statements
-
-[Backslashes are bad and should be never be used](labels/why-no-backslashes) however
-there is one exception: `with` statements using multiple context managers. Before Python
-3.9 Python's grammar does not allow organizing parentheses around the series of context
-managers.
-
-We don't want formatting like:
-
-```py3
-with make_context_manager1() as cm1, make_context_manager2() as cm2, make_context_manager3() as cm3, make_context_manager4() as cm4:
-    ...  # nothing to split on - line too long
-```
-
-So _Black_ will, when we implement this, format it like this:
-
-```py3
-with \
-     make_context_manager1() as cm1, \
-     make_context_manager2() as cm2, \
-     make_context_manager3() as cm3, \
-     make_context_manager4() as cm4 \
-:
-    ...  # backslashes and an ugly stranded colon
-```
-
-Although when the target version is Python 3.9 or higher, _Black_ uses parentheses
-instead in `--preview` mode (see below) since they're allowed in Python 3.9 and higher.
-
-An alternative to consider if the backslashes in the above formatting are undesirable is
-to use {external:py:obj}`contextlib.ExitStack` to combine context managers in the
-following way:
-
-```python
-with contextlib.ExitStack() as exit_stack:
-    cm1 = exit_stack.enter_context(make_context_manager1())
-    cm2 = exit_stack.enter_context(make_context_manager2())
-    cm3 = exit_stack.enter_context(make_context_manager3())
-    cm4 = exit_stack.enter_context(make_context_manager4())
-    ...
-```
-
-(labels/preview-style)=
-
 ## Preview style
 
 Experimental, potentially disruptive style changes are gathered under the `--preview`
 CLI flag. At the end of each year, these changes may be adopted into the default style,
 as described in [The Black Code Style](index.md). Because the functionality is
 experimental, feedback and issue reports are highly encouraged!
 
-### Improved string processing
-
-_Black_ will split long string literals and merge short ones. Parentheses are used where
-appropriate. When split, parts of f-strings that don't need formatting are converted to
-plain strings. User-made splits are respected when they do not exceed the line length
-limit. Line continuation backslashes are converted into parenthesized strings.
-Unnecessary parentheses are stripped. The stability and status of this feature is
-tracked in [this issue](https://github.com/psf/black/issues/2188).
-
-### Improved line breaks
-
-For assignment expressions, _Black_ now prefers to split and wrap the right side of the
-assignment instead of left side. For example:
-
-```python
-some_dict[
-    "with_a_long_key"
-] = some_looooooooong_module.some_looooooooooooooong_function_name(
-    first_argument, second_argument, third_argument
-)
-```
-
-will be changed to:
+In the past, the preview style included some features with known bugs, so that we were
+unable to move these features to the stable style. Therefore, such features are now
+moved to the `--unstable` style. All features in the `--preview` style are expected to
+make it to next year's stable style; features in the `--unstable` style will be
+stabilized only if issues with them are fixed. If bugs are discovered in a `--preview`
+feature, it is demoted to the `--unstable` style.
 
-```python
-some_dict["with_a_long_key"] = (
-    some_looooooooong_module.some_looooooooooooooong_function_name(
-        first_argument, second_argument, third_argument
-    )
-)
-```
+Currently, the following features are included in the preview style:
 
-### Improved parentheses management
+- `hex_codes_in_unicode_sequences`: normalize casing of Unicode escape characters in
+  strings
+- `unify_docstring_detection`: fix inconsistencies in whether certain strings are
+  detected as docstrings
+- `hug_parens_with_braces_and_square_brackets`: more compact formatting of nested
+  brackets ([see below](labels/hug-parens))
 
-For dict literals with long values, they are now wrapped in parentheses. Unnecessary
-parentheses are now removed. For example:
+The unstable style additionally includes the following features:
 
-```python
-my_dict = {
-    "a key in my dict": a_very_long_variable
-    * and_a_very_long_function_call()
-    / 100000.0,
-    "another key": (short_value),
-}
-```
+- `string_processing`: split long string literals and related changes
+  ([see below](labels/string-processing))
+- `wrap_long_dict_values_in_parens`: add parentheses to long values in dictionaries
+  ([see below](labels/wrap-long-dict-values))
+- `multiline_string_handling`: more compact formatting of expressions involving
+  multiline strings ([see below](labels/multiline-string-handling))
 
-will be changed to:
-
-```python
-my_dict = {
-    "a key in my dict": (
-        a_very_long_variable * and_a_very_long_function_call() / 100000.0
-    ),
-    "another key": short_value,
-}
-```
+(labels/hug-parens)=
 
 ### Improved multiline dictionary and list indentation for sole function parameter
 
@@ -185,6 +106,46 @@ foo(
 )
 ```
 
+(labels/string-processing)=
+
+### Improved string processing
+
+_Black_ will split long string literals and merge short ones. Parentheses are used where
+appropriate. When split, parts of f-strings that don't need formatting are converted to
+plain strings. User-made splits are respected when they do not exceed the line length
+limit. Line continuation backslashes are converted into parenthesized strings.
+Unnecessary parentheses are stripped. The stability and status of this feature is
+tracked in [this issue](https://github.com/psf/black/issues/2188).
+
+(labels/wrap-long-dict-values)=
+
+### Improved parentheses management in dicts
+
+For dict literals with long values, they are now wrapped in parentheses. Unnecessary
+parentheses are now removed. For example:
+
+```python
+my_dict = {
+    "a key in my dict": a_very_long_variable
+    * and_a_very_long_function_call()
+    / 100000.0,
+    "another key": (short_value),
+}
+```
+
+will be changed to:
+
+```python
+my_dict = {
+    "a key in my dict": (
+        a_very_long_variable * and_a_very_long_function_call() / 100000.0
+    ),
+    "another key": short_value,
+}
+```
+
+(labels/multiline-string-handling)=
+
 ### Improved multiline string handling
 
 _Black_ is smarter when formatting multiline strings, especially in function arguments,
@@ -297,13 +258,51 @@ s = (  # Top comment
 )
 ```
 
-=======
+## Potential future changes
+
+This section lists changes that we may want to make in the future, but that aren't
+implemented yet.
+
+### Using backslashes for with statements
+
+[Backslashes are bad and should be never be used](labels/why-no-backslashes) however
+there is one exception: `with` statements using multiple context managers. Before Python
+3.9 Python's grammar does not allow organizing parentheses around the series of context
+managers.
+
+We don't want formatting like:
+
+```py3
+with make_context_manager1() as cm1, make_context_manager2() as cm2, make_context_manager3() as cm3, make_context_manager4() as cm4:
+    ...  # nothing to split on - line too long
+```
+
+So _Black_ will, when we implement this, format it like this:
+
+```py3
+with \
+     make_context_manager1() as cm1, \
+     make_context_manager2() as cm2, \
+     make_context_manager3() as cm3, \
+     make_context_manager4() as cm4 \
+:
+    ...  # backslashes and an ugly stranded colon
+```
+
+Although when the target version is Python 3.9 or higher, _Black_ uses parentheses
+instead in `--preview` mode (see below) since they're allowed in Python 3.9 and higher.
 
-### Form feed characters
+An alternative to consider if the backslashes in the above formatting are undesirable is
+to use {external:py:obj}`contextlib.ExitStack` to combine context managers in the
+following way:
 
-_Black_ will now retain form feed characters on otherwise empty lines at the module
-level. Only one form feed is retained for a group of consecutive empty lines. Where
-there are two empty lines in a row, the form feed will be placed on the second line.
+```python
+with contextlib.ExitStack() as exit_stack:
+    cm1 = exit_stack.enter_context(make_context_manager1())
+    cm2 = exit_stack.enter_context(make_context_manager2())
+    cm3 = exit_stack.enter_context(make_context_manager3())
+    cm4 = exit_stack.enter_context(make_context_manager4())
+    ...
+```
 
-_Black_ already retained form feed literals inside a comment or inside a string. This
-remains the case.
+(labels/preview-style)=

docs/the_black_code_style/index.md

@@ -42,9 +42,11 @@ _Black_:
   enabled by newer Python language syntax as well as due to improvements in the
   formatting logic.
 
-- The `--preview` flag is exempt from this policy. There are no guarantees around the
-  stability of the output with that flag passed into _Black_. This flag is intended for
-  allowing experimentation with the proposed changes to the _Black_ code style.
+- The `--preview` and `--unstable` flags are exempt from this policy. There are no
+  guarantees around the stability of the output with these flags passed into _Black_.
+  They are intended for allowing experimentation with proposed changes to the _Black_
+  code style. The `--preview` style at the end of a year should closely match the stable
+  style for the next year, but we may always make changes.
 
 Documentation for both the current and future styles can be found:
 

docs/usage_and_configuration/black_as_a_server.md

@@ -62,6 +62,9 @@ The headers controlling how source code is formatted are:
 - `X-Preview`: corresponds to the `--preview` command line flag. If present and its
   value is not an empty string, experimental and potentially disruptive style changes
   will be used.
+- `X-Unstable`: corresponds to the `--unstable` command line flag. If present and its
+  value is not an empty string, experimental style changes that are known to be buggy
+  will be used.
 - `X-Fast-Or-Safe`: if set to `fast`, `blackd` will act as _Black_ does when passed the
   `--fast` command line flag.
 - `X-Python-Variant`: if set to `pyi`, `blackd` will act as _Black_ does when passed the

docs/usage_and_configuration/the_basics.md

@@ -144,9 +144,18 @@ magic trailing comma is ignored.
 
 #### `--preview`
 
-Enable potentially disruptive style changes that may be added to Black's main
-functionality in the next major release. Read more about
-[our preview style](labels/preview-style).
+Enable potentially disruptive style changes that we expect to add to Black's main
+functionality in the next major release. Use this if you want a taste of what next
+year's style will look like.
+
+Read more about [our preview style](labels/preview-style).
+
+#### `--unstable`
+
+Enable all style changes in `--preview`, plus additional changes that we would like to
+make eventually, but that have known issues that need to be fixed before they can move
+back to the `--preview` style. Use this if you want to experiment with these changes and
+help fix issues with them.
 
 (labels/exit-code)=
 

pyproject.toml

@@ -16,10 +16,11 @@ extend-exclude = '''
   | profiling
 )/
 '''
-# We use preview style for formatting Black itself. If you
-# want stable formatting across releases, you should keep
-# this off.
-preview = true
+# We use the unstable style for formatting Black itself. If you
+# want bug-free formatting, you should keep this off. If you want
+# stable formatting across releases, you should also keep `preview = true`
+# (which is implied by this flag) off.
+unstable = true
 
 # Build system information and other project-specific configuration below.
 # NOTE: You don't need this in your own Black configuration.

src/black/__init__.py

@@ -290,7 +290,7 @@ def validate_regex(
     "--experimental-string-processing",
     is_flag=True,
     hidden=True,
-    help="(DEPRECATED and now included in --preview) Normalize string literals.",
+    help="(DEPRECATED and now included in --unstable) Normalize string literals.",
 )
 @click.option(
     "--preview",
@@ -300,6 +300,15 @@ def validate_regex(
         " functionality in the next major release."
     ),
 )
+@click.option(
+    "--unstable",
+    is_flag=True,
+    help=(
+        "Enable potentially disruptive style changes that have known bugs or are not"
+        " currently expected to make it into the stable style Black's next major"
+        " release. Implies --preview."
+    ),
+)
 @click.option(
     "--check",
     is_flag=True,
@@ -492,6 +501,7 @@ def main(  # noqa: C901
     skip_magic_trailing_comma: bool,
     experimental_string_processing: bool,
     preview: bool,
+    unstable: bool,
     quiet: bool,
     verbose: bool,
     required_version: Optional[str],
@@ -580,6 +590,7 @@ def main(  # noqa: C901
         magic_trailing_comma=not skip_magic_trailing_comma,
         experimental_string_processing=experimental_string_processing,
         preview=preview,
+        unstable=unstable,
         python_cell_magics=set(python_cell_magics),
     )