[Doc] SchemaDefinition must be used to create Avro schema in a certain case

[BewareMyPower]

What issue do you find in Pulsar docs?

Currently Pulsar docs only shows a simple way to create Avro schema. See https://pulsar.apache.org/docs/next/schema-understand#usage.

Given the following User class:

@Builder
@AllArgsConstructor
@NoArgsConstructor
public static class User {
    String name;
    int age;
}

We can create an producer with Avro schema like:

client.newProducer(Schema.AVRO(User.class))

However, if the class has a nullable field with a default value like:

    @Builder
    @AllArgsConstructor
    @NoArgsConstructor
    public static class User {
        @AvroDefault("\"user\"")
        String name;
        int age;
    }

We must create the Avro schema via a SchemaDefinition and disable the alwaysAllowNull field like:

SchemaDefinition schemaDefinition = SchemaDefinition.builder().withPojo(User.class).withAlwaysAllowNull(false).build();
client.newProducer(Schema.AVRO(schemaDefinition))

Otherwise, the schema info would be invalid. Run following test to reproduce:

    @Test
    public void test() throws Exception {
        var topic = "test-topic";
        try (var client = PulsarClient.builder().serviceUrl(brokerUrl.toString()).build()) {
            client.newProducer(Schema.AVRO(User.class))
                    .topic(topic)
                    .create();
            client.newProducer(Schema.AVRO(User.class))
                    .topic(topic)
                    .create();
        }
    }

The 2nd creation of the producer will fail with following error messages:

2022-05-18T20:37:42,013 - ERROR - [mock-pulsar-bk-OrderedExecutor-0-0:ServerCnx@1292] - Try add schema failed, remote address /127.0.0.1:62344, topic persistent://public/default/test-topic, producerId 1
java.util.concurrent.CompletionException: org.apache.avro.AvroTypeException: Invalid default for field name: "user" not a ["null","string"]
	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315) ~[?:?]
        ...

What is your suggestion?

1. Add examples about how to create Avro schema with a SchemaDefinition.
2. Note that this way must be used when the class has a nullable field with a default value.

Do you have any references?

https://pulsar.apache.org/docs/next/schema-understand#usage

Would you like to fix this issue?

Yes

Note

• I have researched my question.

[BewareMyPower]

In addition, the description of Changes allowed column in Schema compatibility check strategy is wrong.

Optional fields, which are decorated with @AvroDefault annotation, can be added or deleted for all strategies other than ALWAYS_INCOMPATIBLE. The description "Modify optional fields" is confused and ambiguous and should be removed.

[github-actions]

The issue had no activity for 30 days, mark with Stale label.

[momo-jun]

@congbobo184 @liangyepianzhou can you pls take a look at this issue of schema docs? I guess it hasn't been addressed. Any chance we can take care of it through this doc improvement PR?

[github-actions]

The issue had no activity for 30 days, mark with Stale label.