Merge remote-tracking branch 'public/main' into idehook-configcache

CHANGES.md

@@ -10,11 +10,34 @@ This document is intended for Spotless developers.
 We adhere to the [keepachangelog](https://keepachangelog.com/en/1.0.0/) format (starting after version `1.27.0`).
 
 ## [Unreleased]
+### Added
+* APIs to support linting. (implemented in [#2148](https://github.com/diffplug/spotless/pull/2148) and [#2149](https://github.com/diffplug/spotless/pull/2149))
+  * Spotless is still primarily a formatter, not a linter. But when formatting fails, it's more flexible to model those failures as lints so that the formatting can continue and ideally we can also capture the line numbers causing the failure.
+  * `Lint` models a single change. A `FormatterStep` can create a lint by:
+    * throwing an exception during formatting, ideally `throw Lint.atLine(127, "code", "Well what happened was...")`
+    * or by implementing the `List<Lint> lint(String content, File file)` method to create multiple of them
+* Support for line ending policy `PRESERVE` which just takes the first line ending of every given file as setting (no matter if `\n`, `\r\n` or `\r`) ([#2304](https://github.com/diffplug/spotless/pull/2304))
+### Changes
+* **BREAKING** Moved `PaddedCell.DirtyState` to its own top-level class with new methods. ([#2148](https://github.com/diffplug/spotless/pull/2148))
+  * **BREAKING** Removed `isClean`, `applyTo`, and `applyToAndReturnResultIfDirty` from `Formatter` because users should instead use `DirtyState`.
+### Fixed
+* `ktlint` steps now read from the `string` instead of the `file` so they don't clobber earlier steps. (fixes [#1599](https://github.com/diffplug/spotless/issues/1599))
+
+## [3.0.0.BETA3] - 2024-10-15
+### Added
+* Support for `rdf` ([#2261](https://github.com/diffplug/spotless/pull/2261))
+* Support for `buf` on maven plugin ([#2291](https://github.com/diffplug/spotless/pull/2291))
+* `ConfigurationCacheHack` so we can support Gradle's configuration cache and remote build cache at the same time. ([#2298](https://github.com/diffplug/spotless/pull/2298) fixes [#2168](https://github.com/diffplug/spotless/issues/2168))
 ### Changed
 * Support configuring the Equo P2 cache. ([#2238](https://github.com/diffplug/spotless/pull/2238))
-
 * Add explicit support for JSONC / CSS via biome, via the file extensions `.css` and `.jsonc`. 
   ([#2259](https://github.com/diffplug/spotless/pull/2259))
+* Bump default `buf` version to latest `1.24.0` -> `1.44.0`. ([#2291](https://github.com/diffplug/spotless/pull/2291))
+* Bump default `google-java-format` version to latest `1.23.0` -> `1.24.0`. ([#2294](https://github.com/diffplug/spotless/pull/2294))
+* Bump default `jackson` version to latest `2.17.2` -> `2.18.0`. ([#2279](https://github.com/diffplug/spotless/pull/2279))
+* Bump default `cleanthat` version to latest `2.21` -> `2.22`. ([#2296](https://github.com/diffplug/spotless/pull/2296))
+### Fixed
+* Java import order, ignore duplicate group entries. ([#2293](https://github.com/diffplug/spotless/pull/2293))
 
 ## [3.0.0.BETA2] - 2024-08-25
 ### Changed
@@ -62,6 +85,8 @@ We adhere to the [keepachangelog](https://keepachangelog.com/en/1.0.0/) format (
 * **BREAKING** Remove `JarState.getMavenCoordinate(String prefix)`. ([#1945](https://github.com/diffplug/spotless/pull/1945))
 * **BREAKING** Replace `PipeStepPair` with `FenceStep`. ([#1954](https://github.com/diffplug/spotless/pull/1954))
 * **BREAKING** Fully removed `Rome`, use `Biome` instead. ([#2119](https://github.com/diffplug/spotless/pull/2119))
+* **BREAKING** Moved `PaddedCell.DirtyState` to its own top-level class with new methods. ([#2148](https://github.com/diffplug/spotless/pull/2148))
+  * **BREAKING** Removed `isClean`, `applyTo`, and `applyToAndReturnResultIfDirty` from `Formatter` because users should instead use `DirtyState`.
 
 ## [2.45.0] - 2024-01-23
 ### Added

CONTRIBUTING.md

@@ -10,21 +10,20 @@ In order to use and combine `FormatterStep`, you first create a `Formatter`, whi
 
 - an encoding
 - a list of `FormatterStep`
-- a line endings policy (`LineEnding.GIT_ATTRIBUTES` is almost always the best choice)
+- a line endings policy (`LineEnding.GIT_ATTRIBUTES_FAST_ALLSAME` is almost always the best choice)
 
-Once you have an instance of `Formatter`, you can call `boolean isClean(File)`, or `void applyTo(File)` to either check or apply formatting to a file.  Spotless will then:
+Once you have an instance of `Formatter`, you can call `DirtyState.of(Formatter, File)`. Under the hood, Spotless will:
 
 - parse the raw bytes into a String according to the encoding
 - normalize its line endings to `\n`
 - pass the unix string to each `FormatterStep` one after the other
+- check for idempotence problems, and repeatedly apply the steps until the [result is stable](PADDEDCELL.md). 
 - apply line endings according to the policy
 
 You can also use lower-level methods like `String compute(String unix, File file)` if you'd like to do lower-level processing.
 
 All `FormatterStep` implement `Serializable`, `equals`, and `hashCode`, so build systems that support up-to-date checks can easily and correctly determine if any actions need to be taken.
 
-Spotless also provides `PaddedCell`, which makes it easy to diagnose and correct idempotence problems.
-
 ## Project layout
 
 For the folders below in monospace text, they are published on MavenCentral at the coordinate `com.diffplug.spotless:spotless-${FOLDER_NAME}`.  The other folders are dev infrastructure.
@@ -39,15 +38,16 @@ For the folders below in monospace text, they are published on MavenCentral at t
 
 ## How to add a new FormatterStep
 
-The easiest way to create a FormatterStep is `FormatterStep createNeverUpToDate(String name, FormatterFunc function)`, which you can use like this:
+The easiest way to create a FormatterStep is to just create `class FooStep implements FormatterStep`. It has one abstract method which is the formatting function, and you're ready to tinker. To work with the build plugins, this class will need to
 
-```java
-FormatterStep identityStep = FormatterStep.createNeverUpToDate("identity", unixStr -> unixStr)
-```
+- implement equality and hashcode
+- support lossless roundtrip serialization
+
+You can use `StepHarness` (if you don't care about the `File` argument) or `StepHarnessWithFile` to test. The harness will roundtrip serialize your step, check that it's equal to itself, and then perform all tests on the roundtripped step.
 
-This creates a step which will fail up-to-date checks (it is equal only to itself), and will use the function you passed in to do the formatting pass.
+## Implementing equality in terms of serialization
 
-To create a step which can handle up-to-date checks properly, use the method `<State extends Serializable> FormatterStep create(String name, State state, Function<State, FormatterFunc> stateToFormatter)`.  Here's an example:
+Spotless has infrastructure which uses the serialized form of your step to implement equality for you. Here is an example:
 
 ```java
 public final class ReplaceStep {
@@ -62,10 +62,10 @@ public final class ReplaceStep {
   private static final class State implements Serializable {
     private static final long serialVersionUID = 1L;
 
-    private final CharSequence target;
-    private final CharSequence replacement;
+    private final String target;
+    private final String replacement;
 
-    State(CharSequence target, CharSequence replacement) {
+    State(String target, String replacement) {
       this.target = target;
       this.replacement = replacement;
     }
@@ -82,8 +82,6 @@ The `FormatterStep` created above implements `equals` and `hashCode` based on th
 Oftentimes, a rule's state will be expensive to compute. `EclipseFormatterStep`, for example, depends on a formatting file.  Ideally, we would like to only pay the cost of the I/O needed to load that file if we have to - we'd like to create the FormatterStep now but load its state lazily at the last possible moment.  For this purpose, each of the `FormatterStep.create` methods has a lazy counterpart.  Here are their signatures:
 
 ```java
-FormatterStep createNeverUpToDate    (String name, FormatterFunc function                  )
-FormatterStep createNeverUpToDateLazy(String name, Supplier<FormatterFunc> functionSupplier)
 FormatterStep create    (String name, State state                  , Function<State, FormatterFunc> stateToFormatter)
 FormatterStep createLazy(String name, Supplier<State> stateSupplier, Function<State, FormatterFunc> stateToFormatter)
 ```
@@ -95,13 +93,12 @@ Here's a checklist for creating a new step for Spotless:
 - [ ] Class name ends in Step, `SomeNewStep`.
 - [ ] Class has a public static method named `create` that returns a `FormatterStep`.
 - [ ] Has a test class named `SomeNewStepTest` that uses `StepHarness` or `StepHarnessWithFile` to test the step.
-  - [ ] Start with `StepHarness.forStep(myStep).supportsRoundTrip(false)`, and then add round trip support as described in the next section. 
 - [ ] Test class has test methods to verify behavior.
 - [ ] Test class has a test method `equality()` which tests equality using `StepEqualityTester` (see existing methods for examples).
 
 ### Serialization roundtrip
 
-In order to support Gradle's configuration cache, all `FormatterStep` must be round-trip serializable. This is a bit tricky because step equality is based on the serialized form of the state, and `transient` is used to take absolute paths out of the equality check. To make this work, roundtrip compatible steps actually have *two* states:
+In order to support Gradle's configuration cache, all `FormatterStep` must be round-trip serializable. This is a bit tricky because step equality is based on the serialized form of the state, and `transient` can be used to take absolute paths out of the equality check. To make this work, roundtrip compatible steps can actually have *two* states:
 
 - `RoundtripState` which must be roundtrip serializable but has no equality constraints
   - `FileSignature.Promised` for settings files and `JarState.Promised` for the classpath 
@@ -139,6 +136,15 @@ There are many great formatters (prettier, clang-format, black, etc.) which live
 
 Because of Spotless' up-to-date checking and [git ratcheting](https://github.com/diffplug/spotless/tree/main/plugin-gradle#ratchet), Spotless actually doesn't have to call formatters very often, so even an expensive shell call for every single invocation isn't that bad.  Anything that works is better than nothing, and we can always speed things up later if it feels too slow (but it probably won't).
 
+## Lints
+
+Spotless is primarily a formatter, not a linter. But, if something goes wrong during formatting, it's better to model that as a lint with line numbers rather than just a naked exception. There are two ways to go about this:
+
+- at any point during the formatting process, you can throw a `Lint.atLine(int line, ...)` exception. This will be caught and turned into a lint.
+- or you can override the `default List<Lint> lint(String content, File file)` method. This method will only run if the step did not already throw an exception.
+
+Don't go lint crazy! By default, all lints are build failures. Users have to suppress them explicitly if they want to continue.
+
 ## How to add a new plugin for a build system
 
 The gist of it is that you will have to:

README.md

@@ -103,6 +103,7 @@ lib('npm.TsFmtFormatterStep')                    +'{{yes}}       | {{yes}}
 lib('pom.SortPomStep')                           +'{{yes}}       | {{yes}}      | {{no}}       | {{no}}  |',
 lib('protobuf.BufStep')                          +'{{yes}}       | {{no}}       | {{no}}       | {{no}}  |',
 lib('python.BlackStep')                          +'{{yes}}       | {{no}}       | {{no}}       | {{no}}  |',
+lib('rdf.RdfFormatterStep')                      +'{{no}}        | {{yes}}      | {{no}}       | {{no}}  |',
 lib('scala.ScalaFmtStep')                        +'{{yes}}       | {{yes}}      | {{yes}}      | {{no}}  |',
 lib('shell.ShfmtStep')                           +'{{yes}}       | {{yes}}      | {{no}}       | {{no}}  |',
 lib('sql.DBeaverSQLFormatterStep')               +'{{yes}}       | {{yes}}      | {{yes}}      | {{no}}  |',
@@ -157,6 +158,7 @@ lib('yaml.JacksonYamlStep')                      +'{{yes}}       | {{yes}}
 | [`pom.SortPomStep`](lib/src/main/java/com/diffplug/spotless/pom/SortPomStep.java) | :+1:       | :+1:      | :white_large_square:       | :white_large_square:  |
 | [`protobuf.BufStep`](lib/src/main/java/com/diffplug/spotless/protobuf/BufStep.java) | :+1:       | :white_large_square:       | :white_large_square:       | :white_large_square:  |
 | [`python.BlackStep`](lib/src/main/java/com/diffplug/spotless/python/BlackStep.java) | :+1:       | :white_large_square:       | :white_large_square:       | :white_large_square:  |
+| [`rdf.RdfFormatterStep`](lib/src/main/java/com/diffplug/spotless/rdf/RdfFormatterStep.java) | :white_large_square:        | :+1:      | :white_large_square:       | :white_large_square:  |
 | [`scala.ScalaFmtStep`](lib/src/main/java/com/diffplug/spotless/scala/ScalaFmtStep.java) | :+1:       | :+1:      | :+1:      | :white_large_square:  |
 | [`shell.ShfmtStep`](lib/src/main/java/com/diffplug/spotless/shell/ShfmtStep.java) | :+1:       | :+1:      | :white_large_square:       | :white_large_square:  |
 | [`sql.DBeaverSQLFormatterStep`](lib/src/main/java/com/diffplug/spotless/sql/DBeaverSQLFormatterStep.java) | :+1:       | :+1:      | :+1:      | :white_large_square:  |

gradle.properties

@@ -29,6 +29,7 @@ VER_SLF4J=[1.6,2.0[
 # Used in multiple places
 VER_DURIAN=1.2.0
 VER_JGIT=6.10.0.202406032230-r
-VER_JUNIT=5.11.0
+VER_JUNIT=5.11.3
 VER_ASSERTJ=3.26.3
-VER_MOCKITO=5.13.0
+VER_MOCKITO=5.14.2
+VER_SELFIE=2.4.1

gradle/changelog.gradle

@@ -26,11 +26,11 @@ spotlessChangelog {
 	runAfterPush "gh release create ${kind}/{{version}} --title '${releaseTitle} v{{version}}' --notes-from-tag"
 
 	if (kind == 'gradle') {
-		forceNextVersion '7.0.0.BETA2'
+		forceNextVersion '7.0.0.BETA3'
 	} else if (kind == 'maven') {
-		forceNextVersion '2.44.0.BETA2'
+		forceNextVersion '2.44.0.BETA3'
 	} else {
-		forceNextVersion '3.0.0.BETA2'
+		forceNextVersion '3.0.0.BETA3'
 	}
 }
 

gradle/special-tests.gradle

@@ -18,7 +18,14 @@ tasks.withType(Test).configureEach {
 			maxFailures = 10
 		}
 	}
-
+	// selfie https://selfie.dev/jvm/get-started#gradle
+	environment project.properties.subMap([
+		"selfie"
+	]) // optional, see "Overwrite everything" below
+	inputs.files(fileTree("src/test") {
+		// optional, improves up-to-date checking
+		include "**/*.ss"
+	})
 	// https://docs.gradle.org/8.8/userguide/performance.html#execute_tests_in_parallel
 	maxParallelForks = Runtime.runtime.availableProcessors().intdiv(2) ?: 1
 }

lib-extra/build.gradle

@@ -7,7 +7,7 @@ version = rootProject.spotlessChangelog.versionNext
 apply from: rootProject.file('gradle/java-setup.gradle')
 apply from: rootProject.file('gradle/java-publish.gradle')
 
-String VER_SOLSTICE = '1.7.7'
+String VER_SOLSTICE = '1.8.0'
 dependencies {
 	api projects.lib
 	// misc useful utilities

lib-extra/src/main/java/com/diffplug/spotless/extra/EquoBasedStepBuilder.java

@@ -116,7 +116,7 @@ public FormatterStep build() {
 			}
 			var classpath = new ArrayList<File>();
 			var mavenDeps = new ArrayList<String>();
-			mavenDeps.add("dev.equo.ide:solstice:1.7.7");
+			mavenDeps.add("dev.equo.ide:solstice:1.8.0");
 			mavenDeps.add("com.diffplug.durian:durian-swt.os:4.2.0");
 			mavenDeps.addAll(query.getJarsOnMavenCentral());
 			classpath.addAll(mavenProvisioner.provisionWithTransitives(false, mavenDeps));

lib-extra/src/main/java/com/diffplug/spotless/extra/integration/DiffMessageFormatter.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2016-2023 DiffPlug
+ * Copyright 2016-2024 DiffPlug
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -58,15 +58,17 @@ interface CleanProvider {
 	}
 
 	private static class CleanProviderFormatter implements CleanProvider {
+		private final Path rootDir;
 		private final Formatter formatter;
 
-		CleanProviderFormatter(Formatter formatter) {
+		CleanProviderFormatter(Path rootDir, Formatter formatter) {
+			this.rootDir = Objects.requireNonNull(rootDir);
 			this.formatter = Objects.requireNonNull(formatter);
 		}
 
 		@Override
 		public Path getRootDir() {
-			return formatter.getRootDir();
+			return rootDir;
 		}
 
 		@Override
@@ -123,8 +125,8 @@ public Builder runToFix(String runToFix) {
 			return this;
 		}
 
-		public Builder formatter(Formatter formatter) {
-			this.formatter = new CleanProviderFormatter(formatter);
+		public Builder formatter(Path rootDir, Formatter formatter) {
+			this.formatter = new CleanProviderFormatter(rootDir, formatter);
 			return this;
 		}
 
@@ -244,8 +246,8 @@ private String diff(File file) throws IOException {
 	 * look like if formatted using the given formatter. Does not end with any newline
 	 * sequence (\n, \r, \r\n). The key of the map entry is the 0-based line where the first difference occurred.
 	 */
-	public static Map.Entry<Integer, String> diff(Formatter formatter, File file) throws IOException {
-		return diff(new CleanProviderFormatter(formatter), file);
+	public static Map.Entry<Integer, String> diff(Path rootDir, Formatter formatter, File file) throws IOException {
+		return diff(new CleanProviderFormatter(rootDir, formatter), file);
 	}
 
 	private static Map.Entry<Integer, String> diff(CleanProvider formatter, File file) throws IOException {

lib-extra/src/test/java/com/diffplug/spotless/extra/groovy/GrEclipseFormatterStepSpecialCaseTest.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2023 DiffPlug
+ * Copyright 2023-2024 DiffPlug
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.

lib/build.gradle

@@ -80,7 +80,7 @@ dependencies {
 
 	// GLUE CODE (alphabetic order please)
 	// cleanthat
-	String VER_CLEANTHAT='2.21'
+	String VER_CLEANTHAT='2.22'
 	cleanthatCompileOnly "io.github.solven-eu.cleanthat:java:$VER_CLEANTHAT"
 	compatCleanthat2Dot1CompileAndTestOnly "io.github.solven-eu.cleanthat:java:$VER_CLEANTHAT"
 	// diktat old supported version 1.x
@@ -93,11 +93,11 @@ dependencies {
 	gherkinCompileOnly 'io.cucumber:gherkin-utils:9.0.0'
 	gherkinCompileOnly 'org.slf4j:slf4j-api:2.0.16'
 	// googleJavaFormat
-	googleJavaFormatCompileOnly 'com.google.googlejavaformat:google-java-format:1.23.0'
+	googleJavaFormatCompileOnly 'com.google.googlejavaformat:google-java-format:1.24.0'
 	// gson
 	gsonCompileOnly 'com.google.code.gson:gson:2.11.0'
 	// jackson
-	String VER_JACKSON='2.17.2'
+	String VER_JACKSON='2.18.0'
 	jacksonCompileOnly "com.fasterxml.jackson.core:jackson-databind:$VER_JACKSON"
 	jacksonCompileOnly "com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:$VER_JACKSON"
 	// ktfmt

lib/src/compatDiktat1Dot2Dot5/java/com/diffplug/spotless/glue/diktat/compat/DiktatCompat1Dot2Dot5Adapter.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2023 DiffPlug
+ * Copyright 2023-2024 DiffPlug
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -77,9 +77,7 @@ public String format(final File file, final String content, final boolean isScri
 				false,
 				new EditorConfigOverride(),
 				false));
-
-		DiktatReporting.reportIfRequired(errors, LintError::getLine, LintError::getCol, LintError::getDetail);
-
+		DiktatReporting.reportIfRequired(errors, LintError::getLine, LintError::getRuleId, LintError::getDetail);
 		return result;
 	}
 }

lib/src/compatDiktat2Dot0Dot0/java/com/diffplug/spotless/glue/diktat/compat/DiktatCompat2Dot0Dot0Adapter.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2023 DiffPlug
+ * Copyright 2023-2024 DiffPlug
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -49,7 +49,7 @@ public DiktatCompat2Dot0Dot0Adapter(@Nullable File configFile) {
 	public String format(File file, String content, boolean isScript) {
 		errors.clear();
 		String result = processor.fix(content, file.toPath(), formatterCallback);
-		DiktatReporting.reportIfRequired(errors, DiktatError::getLine, DiktatError::getCol, DiktatError::getDetail);
+		DiktatReporting.reportIfRequired(errors, DiktatError::getLine, DiktatError::getRuleId, DiktatError::getDetail);
 		return result;
 	}