Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Use message ordering enabled property that comes with streaming pull responses #1851

Merged
merged 65 commits into from
Jan 5, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
3c6897c
samples: schema evolution
kamalaboulhosn Jan 23, 2023
7ea48d9
samples: schema evolution
kamalaboulhosn Jan 23, 2023
3438da5
Merge branch 'googleapis:main' into master
kamalaboulhosn Jan 23, 2023
4e5c026
Format fixes
kamalaboulhosn Jan 23, 2023
032a92f
Merge branch 'master' of https://github.com/kamalaboulhosn/java-pubsub
kamalaboulhosn Jan 23, 2023
8a81408
Fix documentation for field.
kamalaboulhosn Jan 23, 2023
90800ac
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Jan 26, 2023
a88377c
Add back in working asserts
kamalaboulhosn Jan 26, 2023
6a741ea
Merge branch 'master' of https://github.com/kamalaboulhosn/java-pubsub
kamalaboulhosn Jan 26, 2023
9a07bbd
Formatting fixes
kamalaboulhosn Jan 26, 2023
df27d57
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Jan 30, 2023
7d1c561
Version/delete fixes
kamalaboulhosn Feb 14, 2023
d917065
Merge branch 'master' of https://github.com/kamalaboulhosn/java-pubsub
kamalaboulhosn Feb 14, 2023
646afa7
samples: schema evolution
kamalaboulhosn Jan 23, 2023
e0cce5a
samples: schema evolution
kamalaboulhosn Jan 23, 2023
51deeb0
Format fixes
kamalaboulhosn Jan 23, 2023
a53b564
Fix documentation for field.
kamalaboulhosn Jan 23, 2023
f1224e6
Add back in working asserts
kamalaboulhosn Jan 26, 2023
de55b05
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Jan 26, 2023
503339f
Formatting fixes
kamalaboulhosn Jan 26, 2023
cbd910c
Version/delete fixes
kamalaboulhosn Feb 14, 2023
7d15680
Merge branch 'master' of https://github.com/kamalaboulhosn/java-pubsub
kamalaboulhosn Feb 23, 2023
1c7e840
samples: Schema evolution (#1499)
kamalaboulhosn Feb 23, 2023
f5d8b39
Merge branch 'googleapis:main' into master
kamalaboulhosn Feb 23, 2023
6195b1c
Minor fixes for comments
kamalaboulhosn Feb 23, 2023
24cc817
Merge branch 'master' of https://github.com/kamalaboulhosn/java-pubsub
kamalaboulhosn Feb 23, 2023
18e9c3b
samples: Schema evolution (#1499)
kamalaboulhosn Feb 23, 2023
37a35cb
Merge branch 'schema-evolution-examples' of https://github.com/google…
kamalaboulhosn Feb 23, 2023
79b0514
Merge branch 'schema-evolution-examples' into master
kamalaboulhosn Feb 23, 2023
b1d2309
Fix rollback example
kamalaboulhosn Mar 10, 2023
3f74191
Formatting
kamalaboulhosn Mar 10, 2023
8b1288a
Formatting and wording fixes
kamalaboulhosn Mar 10, 2023
5057f17
Add new schemas to test directory
kamalaboulhosn Mar 10, 2023
ba38121
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Mar 10, 2023
9dff92e
Samples: Fix exception handling
kamalaboulhosn Mar 11, 2023
30b63a8
Merge branch 'master' of https://github.com/kamalaboulhosn/java-pubsub
kamalaboulhosn Mar 11, 2023
13cbbf9
Merge remote-tracking branch 'upstream/main'
kamalaboulhosn Mar 11, 2023
3185a3e
fix: Set x-goog-request-params for streaming pull request
kamalaboulhosn Mar 14, 2023
3b1f4d9
Revert "fix: Set x-goog-request-params for streaming pull request"
kamalaboulhosn Mar 14, 2023
9c576e8
Revert "Revert "fix: Set x-goog-request-params for streaming pull req…
kamalaboulhosn Mar 14, 2023
f68af6e
Merge remote-tracking branch 'upstream/main'
kamalaboulhosn Mar 14, 2023
817a128
Merge branch 'googleapis:main' into master
kamalaboulhosn Jun 26, 2023
c3a5725
Merge branch 'googleapis:main' into master
kamalaboulhosn Jul 25, 2023
dea5c9c
Thread example
kamalaboulhosn Aug 31, 2023
c58f5a0
Add examples for limited and unlimited exeuctors
kamalaboulhosn Sep 12, 2023
6d8186f
Add back missing semicolon
kamalaboulhosn Sep 12, 2023
307019a
Revert changes to original async example
kamalaboulhosn Sep 12, 2023
aed305a
Revert changes to original async example
kamalaboulhosn Sep 12, 2023
11cddae
Add examples of different threading models
kamalaboulhosn Aug 31, 2023
47328ef
Merge branch 'ML_experiments' of https://github.com/kamalaboulhosn/ja…
kamalaboulhosn Sep 12, 2023
079acc4
Make variables final to conform to style.
kamalaboulhosn Sep 12, 2023
af96f23
Fix catches
kamalaboulhosn Sep 12, 2023
11fda48
Fix ids
kamalaboulhosn Sep 12, 2023
2e58cd5
Fix naming
kamalaboulhosn Sep 12, 2023
5a435fa
Merge pull request #2 from kamalaboulhosn/ML_experiments
kamalaboulhosn Sep 13, 2023
a1394c6
Revert "Merge pull request #2 from kamalaboulhosn/ML_experiments"
kamalaboulhosn Oct 20, 2023
d2f52cf
Set blunderbuss config to auto-assign issues and PRs
kamalaboulhosn Oct 20, 2023
ffccdd3
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Oct 23, 2023
f2a4ce6
fix: Swap writer and reader schema to correct places in sample
kamalaboulhosn Dec 14, 2023
b245c1b
Merge branch 'main' of https://github.com/kamalaboulhosn/java-pubsub
kamalaboulhosn Dec 14, 2023
d220539
Merge branch 'main' of https://github.com/kamalaboulhosn/java-pubsub
kamalaboulhosn Dec 14, 2023
f109199
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Dec 15, 2023
386865a
fix: Use message ordering enabled property that comes with streaming …
kamalaboulhosn Dec 19, 2023
a941bb1
Merge remote-tracking branch 'upstream/main'
kamalaboulhosn Dec 19, 2023
265e453
Merge branch 'googleapis:main' into main
kamalaboulhosn Jan 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
samples: Schema evolution (#1499)
* samples: schema evolution

* samples: schema evolution

* Format fixes

* Fix documentation for field.

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* Add back in working asserts

* Formatting fixes

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* Version/delete fixes

* samples: schema evolution

* samples: schema evolution

* Format fixes

* Fix documentation for field.

* Add back in working asserts

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* Formatting fixes

* Version/delete fixes

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
  • Loading branch information
kamalaboulhosn and gcf-owl-bot[bot] committed Feb 23, 2023
commit 18e9c3bde7181f7f24afc87e60ebfd05eb80e368
9 changes: 9 additions & 0 deletions README.md

Large diffs are not rendered by default.

69 changes: 69 additions & 0 deletions samples/snippets/src/main/java/pubsub/CommitAvroSchemaExample.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
/*
* Copyright 2023 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package pubsub;

// [START pubsub_commit_avro_schema]

import com.google.api.gax.rpc.AlreadyExistsException;
import com.google.cloud.pubsub.v1.SchemaServiceClient;
import com.google.pubsub.v1.ProjectName;
import com.google.pubsub.v1.Schema;
import com.google.pubsub.v1.SchemaName;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;

public class CommitAvroSchemaExample {

public static void main(String... args) throws Exception {
// TODO(developer): Replace these variables before running the sample.
String projectId = "your-project-id";
String schemaId = "your-schema-id";
String avscFile = "path/to/an/avro/schema/file/(.avsc)/formatted/in/json";

commitAvroSchemaExample(projectId, schemaId, avscFile);
}

public static Schema commitAvroSchemaExample(String projectId, String schemaId, String avscFile)
throws IOException {

ProjectName projectName = ProjectName.of(projectId);
SchemaName schemaName = SchemaName.of(projectId, schemaId);

// Read an Avro schema file formatted in JSON as a string.
String avscSource = new String(Files.readAllBytes(Paths.get(avscFile)));

try (SchemaServiceClient schemaServiceClient = SchemaServiceClient.create()) {

Schema schema =
schemaServiceClient.commitSchema(
schemaName.toString(),
Schema.newBuilder()
.setName(schemaName.toString())
.setType(Schema.Type.AVRO)
.setDefinition(avscSource)
.build());

System.out.println("Committed a schema using an Avro schema:\n" + schema);
return schema;
} catch (AlreadyExistsException e) {
System.out.println(schemaName + "already exists.");
return null;
}
}
}
// [END pubsub_commit_avro_schema]
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
/*
* Copyright 2023 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package pubsub;

// [START pubsub_commit_proto_schema]

import com.google.api.gax.rpc.AlreadyExistsException;
import com.google.cloud.pubsub.v1.SchemaServiceClient;
import com.google.pubsub.v1.ProjectName;
import com.google.pubsub.v1.Schema;
import com.google.pubsub.v1.SchemaName;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;

public class CommitProtoSchemaExample {

public static void main(String... args) throws Exception {
// TODO(developer): Replace these variables before running the sample.
String projectId = "your-project-id";
String schemaId = "your-schema-id";
String protoFile = "path/to/a/proto/file/(.proto)/formatted/in/protocol/buffers";

commitProtoSchemaExample(projectId, schemaId, protoFile);
}

public static Schema commitProtoSchemaExample(String projectId, String schemaId, String protoFile)
throws IOException {

ProjectName projectName = ProjectName.of(projectId);
SchemaName schemaName = SchemaName.of(projectId, schemaId);

// Read a proto file as a string.
String protoSource = new String(Files.readAllBytes(Paths.get(protoFile)));

try (SchemaServiceClient schemaServiceClient = SchemaServiceClient.create()) {

Schema schema =
schemaServiceClient.commitSchema(
schemaName.toString(),
Schema.newBuilder()
.setName(schemaName.toString())
.setType(Schema.Type.PROTOCOL_BUFFER)
.setDefinition(protoSource)
.build());

System.out.println("Committed a schema using a protobuf schema:\n" + schema);
return schema;
} catch (AlreadyExistsException e) {
System.out.println(schemaName + "already exists.");
return null;
}
}
}
// [END pubsub_commit_proto_schema]
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ public static void main(String... args) throws Exception {
createAvroSchemaExample(projectId, schemaId, avscFile);
}

public static void createAvroSchemaExample(String projectId, String schemaId, String avscFile)
public static Schema createAvroSchemaExample(String projectId, String schemaId, String avscFile)
throws IOException {

ProjectName projectName = ProjectName.of(projectId);
Expand All @@ -60,8 +60,10 @@ public static void createAvroSchemaExample(String projectId, String schemaId, St
schemaId);

System.out.println("Created a schema using an Avro schema:\n" + schema);
return schema;
} catch (AlreadyExistsException e) {
System.out.println(schemaName + "already exists.");
return null;
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ public static void main(String... args) throws Exception {
createProtoSchemaExample(projectId, schemaId, protoFile);
}

public static void createProtoSchemaExample(String projectId, String schemaId, String protoFile)
public static Schema createProtoSchemaExample(String projectId, String schemaId, String protoFile)
throws IOException {

ProjectName projectName = ProjectName.of(projectId);
Expand All @@ -60,8 +60,10 @@ public static void createProtoSchemaExample(String projectId, String schemaId, S
schemaId);

System.out.println("Created a schema using a protobuf schema:\n" + schema);
return schema;
} catch (AlreadyExistsException e) {
System.out.println(schemaName + "already exists.");
return null;
}
}
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
/*
* Copyright 2023 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package pubsub;

// [START pubsub_create_topic_with_schema_revisions]

import com.google.api.gax.rpc.AlreadyExistsException;
import com.google.cloud.pubsub.v1.TopicAdminClient;
import com.google.pubsub.v1.Encoding;
import com.google.pubsub.v1.SchemaName;
import com.google.pubsub.v1.SchemaSettings;
import com.google.pubsub.v1.Topic;
import com.google.pubsub.v1.TopicName;
import java.io.IOException;

public class CreateTopicWithSchemaRevisionsExample {

public static void main(String... args) throws Exception {
// TODO(developer): Replace these variables before running the sample.
String projectId = "your-project-id";
String topicId = "your-topic-id";
// Use an existing schema.
String schemaId = "your-schema-id";
// Choose either BINARY or JSON message serialization in this topic.
Encoding encoding = Encoding.BINARY;
// Set the minimum and maximum revsion ID
String firstRevisionId = "your-revision-id";
String lastRevisionId = "your-revision-id";

createTopicWithSchemaRevisionsExample(
projectId, topicId, schemaId, firstRevisionId, lastRevisionId, encoding);
}

public static void createTopicWithSchemaRevisionsExample(
String projectId,
String topicId,
String schemaId,
String firstRevisionid,
String lastRevisionId,
Encoding encoding)
throws IOException {
TopicName topicName = TopicName.of(projectId, topicId);
SchemaName schemaName = SchemaName.of(projectId, schemaId);

SchemaSettings schemaSettings =
SchemaSettings.newBuilder()
.setSchema(schemaName.toString())
.setFirstRevisionId(firstRevisionid)
.setLastRevisionId(lastRevisionId)
.setEncoding(encoding)
.build();

try (TopicAdminClient topicAdminClient = TopicAdminClient.create()) {

Topic topic =
topicAdminClient.createTopic(
Topic.newBuilder()
.setName(topicName.toString())
.setSchemaSettings(schemaSettings)
.build());

System.out.println("Created topic with schema: " + topic.getName());
} catch (AlreadyExistsException e) {
System.out.println(schemaName + "already exists.");
}
}
}
// [END pubsub_create_topic_with_schema_revisions]
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
/*
* Copyright 2023 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package pubsub;

// [START pubsub_delete_schema_revision]

import com.google.api.gax.rpc.NotFoundException;
import com.google.cloud.pubsub.v1.SchemaServiceClient;
import com.google.pubsub.v1.DeleteSchemaRevisionRequest;
import com.google.pubsub.v1.SchemaName;
import java.io.IOException;

public class DeleteSchemaRevisionExample {

public static void main(String... args) throws Exception {
// TODO(developer): Replace these variables before running the sample.
String projectId = "your-project-id";
String schemaId = "your-schema-id@your-revision-id";

deleteSchemaRevisionExample(projectId, schemaId);
}

public static void deleteSchemaRevisionExample(String projectId, String schemaId)
throws IOException {
SchemaName schemaName = SchemaName.of(projectId, schemaId);

try (SchemaServiceClient schemaServiceClient = SchemaServiceClient.create()) {

DeleteSchemaRevisionRequest request =
DeleteSchemaRevisionRequest.newBuilder().setName(schemaName.toString()).build();

schemaServiceClient.deleteSchemaRevision(request);

System.out.println("Deleted a schema revision:" + schemaName);

} catch (NotFoundException e) {
System.out.println(schemaName + "not found.");
}
}
}
// [END pubsub_delete_schema_revision]
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
/*
* Copyright 2023 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package pubsub;

// [START pubsub_get_schema_revision]

import com.google.api.gax.rpc.NotFoundException;
import com.google.cloud.pubsub.v1.SchemaServiceClient;
import com.google.pubsub.v1.Schema;
import com.google.pubsub.v1.SchemaName;
import java.io.IOException;

public class GetSchemaRevisionExample {

public static void main(String... args) throws Exception {
// TODO(developer): Replace these variables before running the sample.
String projectId = "your-project-id";
String schemaId = "your-schema-id@your-schema-revision";

getSchemaRevisionExample(projectId, schemaId);
}

public static void getSchemaRevisionExample(String projectId, String schemaId)
throws IOException {
SchemaName schemaName = SchemaName.of(projectId, schemaId);

try (SchemaServiceClient schemaServiceClient = SchemaServiceClient.create()) {

Schema schema = schemaServiceClient.getSchema(schemaName);

System.out.println("Got a schema:\n" + schema);

} catch (NotFoundException e) {
System.out.println(schemaName + "not found.");
}
}
}
// [END pubsub_get_schema_revision]
Loading