-
Notifications
You must be signed in to change notification settings - Fork 229
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Python Hello World sample. [DO NOT MERGE] #173
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,67 @@ | ||
# Cloud Bigtable Hello World | ||
|
||
This is a simple application that demonstrates using the [Google Cloud Client | ||
Library][gcloud-python] to connect to and interact with Cloud Bigtable. | ||
|
||
[gcloud-python]: https://github.com/GoogleCloudPlatform/gcloud-python | ||
|
||
|
||
## Provision a cluster | ||
|
||
Follow the instructions in the [user documentation](https://cloud.google.com/bigtable/docs/creating-cluster) | ||
to create a Google Cloud Platform project and Cloud Bigtable cluster if necessary. | ||
You'll need to reference your project ID, zone and cluster ID to run the application. | ||
|
||
|
||
## Run the application | ||
|
||
First, set your [Google Application Default Credentials](https://developers.google.com/identity/protocols/application-default-credentials) | ||
|
||
Install the dependencies with pip. | ||
|
||
``` | ||
$ pip install -r requirements.txt | ||
``` | ||
|
||
Run the application. Replace the command-line parameters with values for your cluster. | ||
|
||
``` | ||
$ python hello.py -p my-project -c my-cluster -z us-central1-c | ||
``` | ||
|
||
You will see output resembling the following: | ||
|
||
``` | ||
Create table Hello-Bigtable-1234 | ||
Write some greetings to the table | ||
Scan for all greetings: | ||
greeting0: Hello World! | ||
greeting1: Hello Cloud Bigtable! | ||
greeting2: Hello HappyBase! | ||
Delete table Hello-Bigtable-1234 | ||
``` | ||
|
||
## Understanding the code | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Doesn't this stuff usually go in the docs? Do you think it's worthwhile to duplicate this here or would it be better to link to the documentation page that describes this sample? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'd prefer to see this stuff in the README as well as the docs. Code should always have everything you need, IMHO. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think "everything you need" is satisfied by a link to the tutorial for this sample, yeah? Otherwise we risk the "official" tutorial in our docs and this version of the tutorial drifting. If this does not yet have a companion documentation tutorial, then this can remain. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yea - I prefer duplication as sometimes different ways of saying the same thing makes it easier to get. (And when I'm working, I prefer to look at my IDE and not a web page. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fair enough, I'm okay with it as long as it stays relatively minimal. |
||
|
||
The [hello.py](hello.py) application uses the [Google Cloud Bigtable HappyBase | ||
package][Bigtable HappyBase], an implementation of the [HappyBase][HappyBase] | ||
library, to make calls to Cloud Bigtable. It demonstrates several basic | ||
concepts of working with Cloud Bigtable via this API: | ||
|
||
[Bigtable HappyBase]: https://googlecloudplatform.github.io/gcloud-python/stable/happybase-package.html | ||
[HappyBase]: http://happybase.readthedocs.io/en/latest/index.html | ||
|
||
- Creating a [Connection][HappyBase Connection] to a Cloud Bigtable | ||
[Cluster][Cluster API]. | ||
- Using the [Connection][HappyBase Connection] interface to create, disable and | ||
delete a [Table][HappyBase Table]. | ||
- Using the Connection to get a Table. | ||
- Using the Table to write rows via a [put][HappyBase Table Put] and scan | ||
across multiple rows using [scan][HappyBase Table Scan]. | ||
|
||
[Cluster API]: https://googlecloudplatform.github.io/gcloud-python/stable/bigtable-cluster.html | ||
[HappyBase Connection]: https://googlecloudplatform.github.io/gcloud-python/stable/happybase-connection.html | ||
[HappyBase Table]: https://googlecloudplatform.github.io/gcloud-python/stable/happybase-table.html | ||
[HappyBase Table Put]: https://googlecloudplatform.github.io/gcloud-python/stable/happybase-table.html#gcloud.bigtable.happybase.table.Table.put | ||
[HappyBase Table Scan]: https://googlecloudplatform.github.io/gcloud-python/stable/happybase-table.html#gcloud.bigtable.happybase.table.Table.scan | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
#!/usr/bin/env bash | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. License? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please consider using nox/tox and py.test. You'll have to at minimum use py.test in order for this to be merged into python-docs-samples. |
||
|
||
set -e | ||
set -x | ||
|
||
if [ -z "$GCLOUD_PROJECT" ]; then | ||
echo "GCLOUD_PROJECT must be set." | ||
exit 1 | ||
fi | ||
|
||
if [ -z "$BIGTABLE_CLUSTER" ]; then | ||
echo "BIGTABLE_CLUSTER must be set." | ||
exit 1 | ||
fi | ||
|
||
if [ -z "$BIGTABLE_ZONE" ]; then | ||
echo "BIGTABLE_ZONE must be set." | ||
exit 1 | ||
fi | ||
|
||
OUTPUT=$(mktemp) | ||
VENV_DIR="${OUTPUT}_venv" | ||
virtualenv $VENV_DIR | ||
function finish { | ||
rm "$OUTPUT" | ||
rm -rf "$VENV_DIR" | ||
} | ||
trap finish EXIT | ||
|
||
source "${VENV_DIR}/bin/activate" | ||
pip install -r requirements.txt | ||
|
||
python hello.py \ | ||
-p "$GCLOUD_PROJECT" \ | ||
-c "$BIGTABLE_CLUSTER" \ | ||
-z "$BIGTABLE_ZONE" \ | ||
| tee "$OUTPUT" | ||
grep 'Create table Hello-Bigtable' "$OUTPUT" | ||
grep 'Delete table Hello-Bigtable' "$OUTPUT" | ||
grep 'greeting0: Hello World!' "$OUTPUT" | ||
grep 'greeting1: Hello Cloud Bigtable!' "$OUTPUT" | ||
grep 'greeting2: Hello HappyBase!' "$OUTPUT" | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,114 @@ | ||
#!/usr/bin/env python | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. newline between shebang and license. |
||
# Copyright 2016 Google Inc. | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
|
||
"""hello.py demonstrates how to connect to Cloud Bigtable and run some basic | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Don't use the name of the script here (it can change). Just "Demonstrates how to...." is fine. |
||
operations. | ||
|
||
Prerequisites: | ||
|
||
- Create a Cloud Bigtable cluster. | ||
https://cloud.google.com/bigtable/docs/creating-cluster | ||
- Set your Google Application Default Credentials. | ||
https://developers.google.com/identity/protocols/application-default-credentials | ||
- Set the GCLOUD_PROJECT environment variable to your project ID. | ||
https://support.google.com/cloud/answer/6158840 | ||
""" | ||
|
||
from argparse import ArgumentParser | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Import modules not classes, e.g., |
||
import random | ||
|
||
from gcloud import bigtable | ||
from gcloud.bigtable import happybase | ||
|
||
|
||
TABLE_NAME_FORMAT = 'Hello-Bigtable-{0}' | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the |
||
TABLE_NAME_RANGE = 10000 | ||
COLUMN_FAMILY_NAME = 'cf1' | ||
COLUMN_NAME = 'greeting' | ||
FULL_COLUMN_NAME = '{fam}:{col}'.format( | ||
fam=COLUMN_FAMILY_NAME, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hard to tell on github, but this should be indented 4 spaces. |
||
col=COLUMN_NAME) | ||
|
||
GREETINGS = [ | ||
'Hello World!', | ||
'Hello Cloud Bigtable!', | ||
'Hello HappyBase!', | ||
] | ||
|
||
|
||
def parse_args(): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We typically put all of this in the |
||
"""Parses command-line options.""" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Obvious docstrings can be omitted. |
||
parser = ArgumentParser( | ||
description='A sample application that connects to Cloud' + | ||
' Bigtable.') | ||
parser.add_argument( | ||
'--project', | ||
'-p', | ||
action="store", | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this is the default, leave it off. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. With Bigtable, projects where the Bigtable Cluster is located might not be the same place as the code. (which regularly disappoints me that it isn't optional) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I didn't mean leave off the project argument, just leave off |
||
required=True, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just make it positional instead of a flag. |
||
help='Google Cloud Platform project ID that contains the Cloud' + | ||
' Bigtable cluster.') | ||
parser.add_argument( | ||
'--cluster', | ||
'-c', | ||
action="store", | ||
required=True, | ||
help='ID of the Cloud Bigtable cluster to connect to.') | ||
parser.add_argument( | ||
'--zone', | ||
'-z', | ||
action="store", | ||
required=True, | ||
help='Zone that contains the Cloud Bigtable cluster.') | ||
return parser.parse_args() | ||
|
||
|
||
def main(): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. main should take explicit arguments instead of parsing them. Makes testing easier. For example
|
||
"""Runs the sample application.""" | ||
args = parse_args() | ||
|
||
# Require admin client, because hello.py creates a table. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Don't treat the script as a third-person, it's weird. How about |
||
client = bigtable.Client(project=args.project, admin=True) | ||
with client: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Newline above new blocks, please. |
||
cluster = client.cluster(args.zone, args.cluster) | ||
cluster.reload() | ||
connection = happybase.Connection(cluster=cluster) | ||
|
||
# Select a random table name to prevent conflicts when running tests. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should never let our testing requirements leak into sample scripts. |
||
table_name = TABLE_NAME_FORMAT.format( | ||
random.randrange(TABLE_NAME_RANGE)) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hard to tell on github, but this should only be indented one level above |
||
print('Create table {0}'.format(table_name)) | ||
connection.create_table( | ||
table_name, | ||
{ | ||
COLUMN_FAMILY_NAME: dict() # Use default options. | ||
}) | ||
table = connection.table(table_name) | ||
|
||
print('Write some greetings to the table') | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please use gerunds when describing what the script is doing in output. |
||
for i, value in enumerate(GREETINGS): | ||
row_key = 'greeting{0}'.format(i) | ||
table.put(row_key, {FULL_COLUMN_NAME: value}) | ||
|
||
print('Scan for all greetings:') | ||
for key, row in table.scan(): | ||
print('\t{0}: {1}'.format(key, row[FULL_COLUMN_NAME])) | ||
|
||
print('Delete table {0}'.format(table_name)) | ||
connection.delete_table(table_name) | ||
|
||
|
||
if __name__ == '__main__': | ||
main() |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
enum34==1.1.6 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I know it's common practice to do
|
||
futures==3.0.5 | ||
gcloud==0.14.0 | ||
googleapis-common-protos==1.1.0 | ||
grpcio==0.14.0rc1 | ||
httplib2==0.9.2 | ||
oauth2client==2.1.0 | ||
protobuf==3.0.0b3 | ||
pyasn1==0.1.9 | ||
pyasn1-modules==0.0.8 | ||
rsa==3.4.2 | ||
six==1.10.0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
gcloud sdk?