Skip to content

Commit

Permalink
Automl GA (#1729)
Browse files Browse the repository at this point in the history
* Add Natural Language GA Samples

* Add Undeploy Model

* Add Vision Classification Samples

* Add Vision Object Detection Samples

* Lint

* Update samples based on feedback: remove unhelpful descriptions, add wrapper to env variables in tests

* Add comment to explain why the create model call doesn't wait for compeletion

* Update comment wording

* Update method comment, only check env var for tests before class, remove javadoc comments
  • Loading branch information
nnegrey authored Nov 15, 2019
1 parent 9b6008c commit 81ff240
Show file tree
Hide file tree
Showing 51 changed files with 3,506 additions and 23 deletions.
47 changes: 40 additions & 7 deletions automl/cloud-client/README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,25 @@
# AutoML Samples

<a href="https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/java-docs-samples&page=editor&open_in_editor=vision/beta/cloud-client/README.md">
<a href="https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/java-docs-samples&page=editor&open_in_editor=automl/cloud-client/README.md">
<img alt="Open in Cloud Shell" src ="http://gstatic.com/cloudssh/images/open-btn.png"></a>


This directory contains samples for the [Google Cloud AutoML APIs](https://cloud.google.com/automl/) - [docs](https://cloud.google.com/automl/docs/)

We highly reccommend that you refer to the official documentation pages:
<!--* AutoML Natural Language
* AutoML Natural Language
* [Classification](https://cloud.google.com/natural-language/automl/docs)
* [Entity Extraction](https://cloud.google.com/natural-language/automl/entity-analysis/docs)
* [Sentiment Analysis](https://cloud.google.com/natural-language/automl/sentiment/docs) -->
* [Sentiment Analysis](https://cloud.google.com/natural-language/automl/sentiment/docs)
* [AutoML Translation](https://cloud.google.com/translate/automl/docs)
<!--* AutoML Video Intelligence
* [Classification](https://cloud.google.com/video-intelligence/automl/docs)
* [Object Tracking](https://cloud.google.com/video-intelligence/automl/object-tracking/docs)
* [Object Tracking](https://cloud.google.com/video-intelligence/automl/object-tracking/docs) -->
* AutoML Vision
* [Classification](https://cloud.google.com/vision/automl/docs)
* [Edge](https://cloud.google.com/vision/automl/docs/edge-quickstart)
<!--* [Edge](https://cloud.google.com/vision/automl/docs/edge-quickstart) -->
* [Object Detection](https://cloud.google.com/vision/automl/object-detection/docs)
* [AutoML Tables](https://cloud.google.com/automl-tables/docs)-->
<!--* [AutoML Tables](https://cloud.google.com/automl-tables/docs)-->

This API is part of the larger collection of Cloud Machine Learning APIs.

Expand Down Expand Up @@ -60,6 +60,9 @@ small section of code to print out the `metadata` field.
* [Get Model](src/main/java/com/example/automl/)
* [Get Model Evaluation](src/main/java/com/example/automl/GetModelEvaluation.java)
* [Delete Model](src/main/java/com/example/automl/DeleteModel.java)
* [Deploy Model](src/main/java/com/example/automl/DeployModel.java) - Not supported by Translation
* [Uneploy Model](src/main/java/com/example/automl/UndeployModel.java) - Not supported by Translation


### Operation Management
* [List Operation Statuses](src/main/java/com/example/automl/ListOperationStatus.java)
Expand All @@ -69,4 +72,34 @@ small section of code to print out the `metadata` field.
### Translation
* [Translate Create Dataset](src/main/java/com/example/automl/TranslateCreateDataset.java)
* [Translate Create Model](src/main/java/com/example/automl/TranslateCreateModel.java)
* [Translate Predict](src/main/java/com/example/automl/TranslatePredict.java)
* [Translate Predict](src/main/java/com/example/automl/TranslatePredict.java)

### Natural Language Entity Extraction
* [Entity Extraction Create Dataset](src/main/java/com/example/automl/LanguageEntityExtractionCreateDataset.java)
* [Entity Extraction Create Model](src/main/java/com/example/automl/LanguageEntityExtractionCreateModel.java)
* [Entity Extraction Predict](src/main/java/com/example/automl/LanguageEntityExtractionPredict.java)
* [Entity Extraction Batch Predict](src/main/java/com/example/automl/LanguageBatchPredict.java)

### Natural Language Sentiment Analysis
* [Sentiment Analysis Create Dataset](src/main/java/com/example/automl/LanguageSentimentAnalysisCreateDataset.java)
* [Sentiment Analysis Create Model](src/main/java/com/example/automl/LanguageSentimentAnalysisCreateModel.java)
* [Sentiment Analysis Predict](src/main/java/com/example/automl/LanguageSentimentAnalysisPredict.java)

### Natural Language Text Classification
* [Text Classification Create Dataset](src/main/java/com/example/automl/LanguageTextClassificationCreateDataset.java)
* [Text Classification Create Model](src/main/java/com/example/automl/LanguageTextClassificationCreateModel.java)
* [Text Classification Predict](src/main/java/com/example/automl/LanguageTextClassificationPredict.java)

### Vision Classification
* [Classification Create Dataset](src/main/java/com/example/automl/VisionClassificationCreateDataset.java)
* [Classification Create Model](src/main/java/com/example/automl/VisionClassificationCreateModel.java)
* [Classification Predict](src/main/java/com/example/automl/VisionClassificationPredict.java)
* [Classification Batch Predict](src/main/java/com/example/automl/VisionBatchPredict.java)
* [Deploy Node Count](src/main/java/com/example/automl/VisionClassificationDeployModelNodeCount.java)

### Vision Object Detection
* [Object Detection Create Dataset](src/main/java/com/example/automl/VisionObjectDetectionCreateDataset.java)
* [Object Detection Create Model](src/main/java/com/example/automl/VisionObjectDetectionCreateModel.java)
* [Object Detection Predict](src/main/java/com/example/automl/VisionObjectDetectionPredict.java)
* [Object Detection Batch Predict](src/main/java/com/example/automl/VisionBatchPredict.java)
* [Deploy Node Count](src/main/java/com/example/automl/VisionObjectDetectionDeployModelNodeCount.java)
2 changes: 1 addition & 1 deletion automl/cloud-client/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-automl</artifactId>
<version>0.114.1-beta</version>
<version>0.115.0-beta</version>
</dependency>
<!-- [END automl_java_dependencies] -->
<dependency>
Expand Down
Binary file added automl/cloud-client/resources/salad.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added automl/cloud-client/resources/test.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ static void deleteModel() throws IOException, ExecutionException, InterruptedExc
deleteModel(projectId, modelId);
}

// Get a model
// Delete a model
static void deleteModel(String projectId, String modelId)
throws IOException, ExecutionException, InterruptedException {
// Initialize client that will be used to send requests. This client only needs to be created
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
/*
* Copyright 2019 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.automl;

// [START automl_deploy_model]
import com.google.api.gax.longrunning.OperationFuture;
import com.google.cloud.automl.v1.AutoMlClient;
import com.google.cloud.automl.v1.DeployModelRequest;
import com.google.cloud.automl.v1.ModelName;
import com.google.cloud.automl.v1.OperationMetadata;
import com.google.protobuf.Empty;

import java.io.IOException;
import java.util.concurrent.ExecutionException;

class DeployModel {

static void deployModel() throws IOException, ExecutionException, InterruptedException {
// TODO(developer): Replace these variables before running the sample.
String projectId = "YOUR_PROJECT_ID";
String modelId = "YOUR_MODEL_ID";
deployModel(projectId, modelId);
}

// Deploy a model for prediction
static void deployModel(String projectId, String modelId)
throws IOException, ExecutionException, InterruptedException {
// Initialize client that will be used to send requests. This client only needs to be created
// once, and can be reused for multiple requests. After completing all of your requests, call
// the "close" method on the client to safely clean up any remaining background resources.
try (AutoMlClient client = AutoMlClient.create()) {
// Get the full path of the model.
ModelName modelFullId = ModelName.of(projectId, "us-central1", modelId);
DeployModelRequest request =
DeployModelRequest.newBuilder().setName(modelFullId.toString()).build();
OperationFuture<Empty, OperationMetadata> future = client.deployModelAsync(request);

future.get();
System.out.println("Model deployment finished");
}
}
}
// [END automl_deploy_model]
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ static void exportDataset() throws IOException, ExecutionException, InterruptedE
exportDataset(projectId, datasetId, gcsUri);
}

// Export a dataset
// Export a dataset to a GCS bucket
static void exportDataset(String projectId, String datasetId, String gcsUri)
throws IOException, ExecutionException, InterruptedException {
// Initialize client that will be used to send requests. This client only needs to be created
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,12 @@

package com.example.automl;

// [START automl_language_entity_extraction_get_dataset]
// [START automl_language_sentiment_analysis_get_dataset]
// [START automl_language_text_classification_get_dataset]
// [START automl_translate_get_dataset]

// [START automl_vision_classification_get_dataset]
// [START automl_vision_object_detection_get_dataset]
import com.google.cloud.automl.v1.AutoMlClient;
import com.google.cloud.automl.v1.Dataset;
import com.google.cloud.automl.v1.DatasetName;
Expand Down Expand Up @@ -52,17 +56,60 @@ static void getDataset(String projectId, String datasetId) throws IOException {
String retrievedDatasetId = names[names.length - 1];
System.out.format("Dataset id: %s\n", retrievedDatasetId);
System.out.format("Dataset display name: %s\n", dataset.getDisplayName());
System.out.println("Dataset create time:");
System.out.format("\tseconds: %s\n", dataset.getCreateTime().getSeconds());
System.out.format("\tnanos: %s\n", dataset.getCreateTime().getNanos());
// [END automl_language_sentiment_analysis_get_dataset]
// [END automl_language_text_classification_get_dataset]
// [END automl_translate_get_dataset]
// [END automl_vision_classification_get_dataset]
// [END automl_vision_object_detection_get_dataset]
System.out.format(
"Text extraction dataset metadata: %s\n", dataset.getTextExtractionDatasetMetadata());
// [END automl_language_entity_extraction_get_dataset]

// [START automl_language_sentiment_analysis_get_dataset]
System.out.format(
"Text sentiment dataset metadata: %s\n", dataset.getTextSentimentDatasetMetadata());
// [END automl_language_sentiment_analysis_get_dataset]

// [START automl_language_text_classification_get_dataset]
System.out.format(
"Text classification dataset metadata: %s\n",
dataset.getTextClassificationDatasetMetadata());
// [END automl_language_text_classification_get_dataset]

// [START automl_translate_get_dataset]
System.out.println("Translation dataset metadata:");
System.out.format(
"\tSource language code: %s\n",
dataset.getTranslationDatasetMetadata().getSourceLanguageCode());
System.out.format(
"\tTarget language code: %s\n",
dataset.getTranslationDatasetMetadata().getTargetLanguageCode());
System.out.println("Dataset create time:");
System.out.format("\tseconds: %s\n", dataset.getCreateTime().getSeconds());
System.out.format("\tnanos: %s\n", dataset.getCreateTime().getNanos());
// [END automl_translate_get_dataset]

// [START automl_vision_classification_get_dataset]
System.out.format(
"Image classification dataset metadata: %s\n",
dataset.getImageClassificationDatasetMetadata());
// [END automl_vision_classification_get_dataset]

// [START automl_vision_object_detection_get_dataset]
System.out.format(
"Image object detection dataset metadata: %s\n",
dataset.getImageObjectDetectionDatasetMetadata());
// [START automl_language_entity_extraction_get_dataset]
// [START automl_language_sentiment_analysis_get_dataset]
// [START automl_language_text_classification_get_dataset]
// [START automl_translate_get_dataset]
// [START automl_vision_classification_get_dataset]
}
}
}
// [END automl_language_entity_extraction_get_dataset]
// [END automl_language_sentiment_analysis_get_dataset]
// [END automl_language_text_classification_get_dataset]
// [END automl_translate_get_dataset]
// [END automl_vision_classification_get_dataset]
// [END automl_vision_object_detection_get_dataset]
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
/*
* Copyright 2019 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.automl;

// [START automl_language_batch_predict]
import com.google.api.gax.longrunning.OperationFuture;
import com.google.cloud.automl.v1.BatchPredictInputConfig;
import com.google.cloud.automl.v1.BatchPredictOutputConfig;
import com.google.cloud.automl.v1.BatchPredictRequest;
import com.google.cloud.automl.v1.BatchPredictResult;
import com.google.cloud.automl.v1.GcsDestination;
import com.google.cloud.automl.v1.GcsSource;
import com.google.cloud.automl.v1.ModelName;
import com.google.cloud.automl.v1.OperationMetadata;
import com.google.cloud.automl.v1.PredictionServiceClient;

import java.io.IOException;
import java.util.concurrent.ExecutionException;

class LanguageBatchPredict {

static void batchPredict() throws IOException, ExecutionException, InterruptedException {
// TODO(developer): Replace these variables before running the sample.
String projectId = "YOUR_PROJECT_ID";
String modelId = "YOUR_MODEL_ID";
String inputUri = "gs://YOUR_BUCKET_ID/path_to_your_input_file.jsonl";
String outputUri = "gs://YOUR_BUCKET_ID/path_to_save_results/";
batchPredict(projectId, modelId, inputUri, outputUri);
}

static void batchPredict(String projectId, String modelId, String inputUri, String outputUri)
throws IOException, ExecutionException, InterruptedException {
// Initialize client that will be used to send requests. This client only needs to be created
// once, and can be reused for multiple requests. After completing all of your requests, call
// the "close" method on the client to safely clean up any remaining background resources.
try (PredictionServiceClient client = PredictionServiceClient.create()) {
// Get the full path of the model.
ModelName name = ModelName.of(projectId, "us-central1", modelId);
GcsSource gcsSource = GcsSource.newBuilder().addInputUris(inputUri).build();
BatchPredictInputConfig inputConfig =
BatchPredictInputConfig.newBuilder().setGcsSource(gcsSource).build();
GcsDestination gcsDestination =
GcsDestination.newBuilder().setOutputUriPrefix(outputUri).build();
BatchPredictOutputConfig outputConfig =
BatchPredictOutputConfig.newBuilder().setGcsDestination(gcsDestination).build();
BatchPredictRequest request =
BatchPredictRequest.newBuilder()
.setName(name.toString())
.setInputConfig(inputConfig)
.setOutputConfig(outputConfig)
.build();

OperationFuture<BatchPredictResult, OperationMetadata> future =
client.batchPredictAsync(request);

System.out.println("Waiting for operation to complete...");
BatchPredictResult response = future.get();
System.out.println("Batch Prediction results saved to specified Cloud Storage bucket.");
}
}
}
// [END automl_language_batch_predict]
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
/*
* Copyright 2019 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.automl;

// [START automl_language_entity_extraction_create_dataset]
import com.google.api.gax.longrunning.OperationFuture;
import com.google.cloud.automl.v1.AutoMlClient;
import com.google.cloud.automl.v1.Dataset;
import com.google.cloud.automl.v1.LocationName;
import com.google.cloud.automl.v1.OperationMetadata;
import com.google.cloud.automl.v1.TextExtractionDatasetMetadata;

import java.io.IOException;
import java.util.concurrent.ExecutionException;

class LanguageEntityExtractionCreateDataset {

static void createDataset() throws IOException, ExecutionException, InterruptedException {
// TODO(developer): Replace these variables before running the sample.
String projectId = "YOUR_PROJECT_ID";
String displayName = "YOUR_DATASET_NAME";
createDataset(projectId, displayName);
}

// Create a dataset
static void createDataset(String projectId, String displayName)
throws IOException, ExecutionException, InterruptedException {
// Initialize client that will be used to send requests. This client only needs to be created
// once, and can be reused for multiple requests. After completing all of your requests, call
// the "close" method on the client to safely clean up any remaining background resources.
try (AutoMlClient client = AutoMlClient.create()) {
// A resource that represents Google Cloud Platform location.
LocationName projectLocation = LocationName.of(projectId, "us-central1");

TextExtractionDatasetMetadata metadata = TextExtractionDatasetMetadata.newBuilder().build();
Dataset dataset =
Dataset.newBuilder()
.setDisplayName(displayName)
.setTextExtractionDatasetMetadata(metadata)
.build();
OperationFuture<Dataset, OperationMetadata> future =
client.createDatasetAsync(projectLocation, dataset);

Dataset createdDataset = future.get();

// Display the dataset information.
System.out.format("Dataset name: %s\n", createdDataset.getName());
// To get the dataset id, you have to parse it out of the `name` field. As dataset Ids are
// required for other methods.
// Name Form: `projects/{project_id}/locations/{location_id}/datasets/{dataset_id}`
String[] names = createdDataset.getName().split("/");
String datasetId = names[names.length - 1];
System.out.format("Dataset id: %s\n", datasetId);
}
}
}
// [END automl_language_entity_extraction_create_dataset]
Loading

0 comments on commit 81ff240

Please sign in to comment.