Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce @JsonIgnoreUnknownKeys annotation #2874

Merged
merged 3 commits into from
Dec 9, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/basic-serialization.md
Original file line number Diff line number Diff line change
Expand Up @@ -411,8 +411,8 @@ Attempts to explicitly specify its value in the serial format, even if the speci
value is equal to the default one, produces the following exception.

```text
Exception in thread "main" kotlinx.serialization.json.internal.JsonDecodingException: Unexpected JSON token at offset 42: Encountered an unknown key 'language' at path: $.name
Use 'ignoreUnknownKeys = true' in 'Json {}' builder to ignore unknown keys.
Exception in thread "main" kotlinx.serialization.json.internal.JsonDecodingException: Encountered an unknown key 'language' at offset 42 at path: $
Use 'ignoreUnknownKeys = true' in 'Json {}' builder or '@JsonIgnoreUnknownKeys' annotation to ignore unknown keys.
```

<!--- TEST LINES_START -->
Expand Down
96 changes: 68 additions & 28 deletions docs/json.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ In this chapter, we'll walk through features of [JSON](https://www.json.org/json
* [Pretty printing](#pretty-printing)
* [Lenient parsing](#lenient-parsing)
* [Ignoring unknown keys](#ignoring-unknown-keys)
* [Ignoring unknown keys per class](#ignoring-unknown-keys-per-class)
* [Alternative Json names](#alternative-json-names)
* [Encoding defaults](#encoding-defaults)
* [Explicit nulls](#explicit-nulls)
Expand Down Expand Up @@ -164,6 +165,44 @@ Project(name=kotlinx.serialization)

<!--- TEST -->

### Ignoring unknown keys per class

Sometimes, for cleaner and safer API, it is desirable to ignore unknown properties only for specific classes.
In that case, you can use [JsonIgnoreUnknownKeys] annotation on such classes while leaving global [ignoreUnknownKeys][JsonBuilder.ignoreUnknownKeys] setting
turned off:

```kotlin
@OptIn(ExperimentalSerializationApi::class) // JsonIgnoreUnknownKeys is an experimental annotation for now
@Serializable
@JsonIgnoreUnknownKeys
data class Outer(val a: Int, val inner: Inner)

@Serializable
data class Inner(val x: String)

fun main() {
// 1
println(Json.decodeFromString<Outer>("""{"a":1,"inner":{"x":"value"},"unknownKey":42}"""))
println()
// 2
println(Json.decodeFromString<Outer>("""{"a":1,"inner":{"x":"value","unknownKey":"unknownValue"}}"""))
}
```

> You can get the full code [here](../guide/example/example-json-04.kt).

Line (1) decodes successfully despite "unknownKey" in `Outer`, because annotation is present on the class.
However, line (2) throws `SerializationException` because there is no "unknownKey" property in `Inner`:

```text
Outer(a=1, inner=Inner(x=value))

Exception in thread "main" kotlinx.serialization.json.internal.JsonDecodingException: Encountered an unknown key 'unknownKey' at offset 29 at path: $.inner
Use 'ignoreUnknownKeys = true' in 'Json {}' builder or '@JsonIgnoreUnknownKeys' annotation to ignore unknown keys.
```

<!--- TEST LINES_START-->

### Alternative Json names

It's not a rare case when JSON fields are renamed due to a schema version change.
Expand All @@ -184,7 +223,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-04.kt).
> You can get the full code [here](../guide/example/example-json-05.kt).

As you can see, both `name` and `title` Json fields correspond to `name` property:

Expand Down Expand Up @@ -222,7 +261,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-05.kt).
> You can get the full code [here](../guide/example/example-json-06.kt).

It produces the following output which encodes all the property values including the default ones:

Expand Down Expand Up @@ -261,7 +300,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-06.kt).
> You can get the full code [here](../guide/example/example-json-07.kt).

As you can see, `version`, `website` and `description` fields are not present in output JSON on the first line.
After decoding, the missing nullable property `website` without a default values has received a `null` value,
Expand Down Expand Up @@ -319,7 +358,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-07.kt).
> You can get the full code [here](../guide/example/example-json-08.kt).

The invalid `null` value for the `language` property was coerced into the default value:

Expand Down Expand Up @@ -348,7 +387,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-08.kt).
> You can get the full code [here](../guide/example/example-json-09.kt).

Despite that we do not have `Color.pink` and `Color.purple` colors, `decodeFromString` function returns successfully:

Expand Down Expand Up @@ -384,7 +423,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-09.kt).
> You can get the full code [here](../guide/example/example-json-10.kt).

The map with structured keys gets represented as JSON array with the following items: `[key1, value1, key2, value2,...]`.

Expand Down Expand Up @@ -415,7 +454,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-10.kt).
> You can get the full code [here](../guide/example/example-json-11.kt).

This example produces the following non-stardard JSON output, yet it is a widely used encoding for
special values in JVM world:
Expand Down Expand Up @@ -449,7 +488,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-11.kt).
> You can get the full code [here](../guide/example/example-json-12.kt).

In combination with an explicitly specified [SerialName] of the class it provides full
control over the resulting JSON object:
Expand Down Expand Up @@ -506,7 +545,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-12.kt).
> You can get the full code [here](../guide/example/example-json-13.kt).

As you can see, discriminator from the `Base` class is used:

Expand Down Expand Up @@ -543,7 +582,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-13.kt).
> You can get the full code [here](../guide/example/example-json-14.kt).

Note that it would be impossible to deserialize this output back with kotlinx.serialization.

Expand Down Expand Up @@ -579,7 +618,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-14.kt).
> You can get the full code [here](../guide/example/example-json-15.kt).

It affects serial names as well as alternative names specified with [JsonNames] annotation, so both values are successfully decoded:

Expand Down Expand Up @@ -612,7 +651,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-15.kt).
> You can get the full code [here](../guide/example/example-json-16.kt).

As you can see, both serialization and deserialization work as if all serial names are transformed from camel case to snake case:

Expand Down Expand Up @@ -710,7 +749,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-16.kt)
> You can get the full code [here](../guide/example/example-json-17.kt)

```text
{"base64Input":"Zm9vIHN0cmluZw=="}
Expand Down Expand Up @@ -752,7 +791,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-17.kt).
> You can get the full code [here](../guide/example/example-json-18.kt).

A `JsonElement` prints itself as a valid JSON:

Expand Down Expand Up @@ -795,7 +834,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-18.kt).
> You can get the full code [here](../guide/example/example-json-19.kt).

The above example sums `votes` in all objects in the `forks` array, ignoring the objects that have no `votes`:

Expand Down Expand Up @@ -835,7 +874,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-19.kt).
> You can get the full code [here](../guide/example/example-json-20.kt).

As a result, you get a proper JSON string:

Expand Down Expand Up @@ -864,7 +903,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-20.kt).
> You can get the full code [here](../guide/example/example-json-21.kt).

The result is exactly what you would expect:

Expand Down Expand Up @@ -910,7 +949,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-21.kt).
> You can get the full code [here](../guide/example/example-json-22.kt).

Even though `pi` was defined as a number with 30 decimal places, the resulting JSON does not reflect this.
The [Double] value is truncated to 15 decimal places, and the String is wrapped in quotes - which is not a JSON number.
Expand Down Expand Up @@ -951,7 +990,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-22.kt).
> You can get the full code [here](../guide/example/example-json-23.kt).

`pi_literal` now accurately matches the value defined.

Expand Down Expand Up @@ -991,7 +1030,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-23.kt).
> You can get the full code [here](../guide/example/example-json-24.kt).

The exact value of `pi` is decoded, with all 30 decimal places of precision that were in the source JSON.

Expand All @@ -1014,7 +1053,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-24.kt).
> You can get the full code [here](../guide/example/example-json-25.kt).

```text
Exception in thread "main" kotlinx.serialization.json.internal.JsonEncodingException: Creating a literal unquoted value of 'null' is forbidden. If you want to create JSON null literal, use JsonNull object, otherwise, use JsonPrimitive
Expand Down Expand Up @@ -1090,7 +1129,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-25.kt).
> You can get the full code [here](../guide/example/example-json-26.kt).

The output shows that both cases are correctly deserialized into a Kotlin [List].

Expand Down Expand Up @@ -1142,7 +1181,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-26.kt).
> You can get the full code [here](../guide/example/example-json-27.kt).

You end up with a single JSON object, not an array with one element:

Expand Down Expand Up @@ -1187,7 +1226,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-27.kt).
> You can get the full code [here](../guide/example/example-json-28.kt).

See the effect of the custom serializer:

Expand Down Expand Up @@ -1260,7 +1299,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-28.kt).
> You can get the full code [here](../guide/example/example-json-29.kt).

No class discriminator is added in the JSON output:

Expand Down Expand Up @@ -1312,7 +1351,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-29.kt).
> You can get the full code [here](../guide/example/example-json-30.kt).

`BasicProject` will be printed to the output:

Expand Down Expand Up @@ -1406,7 +1445,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-30.kt).
> You can get the full code [here](../guide/example/example-json-31.kt).

This gives you fine-grained control on the representation of the `Response` class in the JSON output:

Expand Down Expand Up @@ -1471,7 +1510,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-31.kt).
> You can get the full code [here](../guide/example/example-json-32.kt).

```text
UnknownProject(name=example, details={"type":"unknown","maintainer":"Unknown","license":"Apache 2.0"})
Expand Down Expand Up @@ -1517,6 +1556,7 @@ The next chapter covers [Alternative and custom formats (experimental)](formats.
[JsonBuilder.prettyPrint]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/-json-builder/pretty-print.html
[JsonBuilder.isLenient]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/-json-builder/is-lenient.html
[JsonBuilder.ignoreUnknownKeys]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/-json-builder/ignore-unknown-keys.html
[JsonIgnoreUnknownKeys]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/-json-ignore-unknown-keys/index.html
[JsonNames]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/-json-names/index.html
[JsonBuilder.useAlternativeNames]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/-json-builder/use-alternative-names.html
[JsonBuilder.encodeDefaults]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/-json-builder/encode-defaults.html
Expand Down
1 change: 1 addition & 0 deletions docs/serialization-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ Once the project is set up, we can start serializing some classes.
* <a name='pretty-printing'></a>[Pretty printing](json.md#pretty-printing)
* <a name='lenient-parsing'></a>[Lenient parsing](json.md#lenient-parsing)
* <a name='ignoring-unknown-keys'></a>[Ignoring unknown keys](json.md#ignoring-unknown-keys)
* <a name='ignoring-unknown-keys-per-class'></a>[Ignoring unknown keys per class](json.md#ignoring-unknown-keys-per-class)
* <a name='alternative-json-names'></a>[Alternative Json names](json.md#alternative-json-names)
* <a name='encoding-defaults'></a>[Encoding defaults](json.md#encoding-defaults)
* <a name='explicit-nulls'></a>[Explicit nulls](json.md#explicit-nulls)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
/*
* Copyright 2017-2024 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license.
*/

package kotlinx.serialization.json

import kotlinx.serialization.Serializable
import kotlinx.serialization.test.checkSerializationException
import kotlin.test.Test
import kotlin.test.assertContains
import kotlin.test.assertContentEquals
import kotlin.test.assertEquals

class JsonIgnoreKeysTest : JsonTestBase() {
val ignoresKeys = Json(default) { ignoreUnknownKeys = true }

@Serializable
class Outer(val a: Int, val inner: Inner)

@Serializable
@JsonIgnoreUnknownKeys
class Inner(val x: String)

@Test
fun testIgnoresKeyWhenGlobalSettingNotSet() = parametrizedTest { mode ->
val jsonString = """{"a":1,"inner":{"x":"value","unknownKey":"unknownValue"}}"""
val result = default.decodeFromString<Outer>(jsonString, mode)
assertEquals(1, result.a)
assertEquals("value", result.inner.x)
}

@Test
fun testThrowsWithoutAnnotationWhenGlobalSettingNotSet() = parametrizedTest { mode ->
val jsonString = """{"a":1,"inner":{"x":"value","unknownKey":"unknownValue"}, "b":2}"""
checkSerializationException({
default.decodeFromString<Outer>(jsonString, mode)
}) { msg ->
assertContains(
msg,
if (mode == JsonTestingMode.TREE) "Encountered an unknown key 'b' at element: \$\n"
else "Encountered an unknown key 'b' at offset 59 at path: \$\n"
)
}
}

@Test
fun testIgnoresBothKeysWithGlobalSetting() = parametrizedTest { mode ->
val jsonString = """{"a":1,"inner":{"x":"value","unknownKey":"unknownValue"}, "b":2}"""
val result = ignoresKeys.decodeFromString<Outer>(jsonString, mode)
assertEquals(1, result.a)
assertEquals("value", result.inner.x)
}
}
7 changes: 7 additions & 0 deletions formats/json/api/kotlinx-serialization-json.api
Original file line number Diff line number Diff line change
Expand Up @@ -262,6 +262,13 @@ public final class kotlinx/serialization/json/JsonEncoder$DefaultImpls {
public static fun shouldEncodeElementDefault (Lkotlinx/serialization/json/JsonEncoder;Lkotlinx/serialization/descriptors/SerialDescriptor;I)Z
}

public abstract interface annotation class kotlinx/serialization/json/JsonIgnoreUnknownKeys : java/lang/annotation/Annotation {
}

public synthetic class kotlinx/serialization/json/JsonIgnoreUnknownKeys$Impl : kotlinx/serialization/json/JsonIgnoreUnknownKeys {
public fun <init> ()V
}

public final class kotlinx/serialization/json/JsonKt {
public static final fun Json (Lkotlinx/serialization/json/Json;Lkotlin/jvm/functions/Function1;)Lkotlinx/serialization/json/Json;
public static synthetic fun Json$default (Lkotlinx/serialization/json/Json;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lkotlinx/serialization/json/Json;
Expand Down
Loading