Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Python Hello World sample. [DO NOT MERGE] #173

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions python/hello-world/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Cloud Bigtable Hello World

This is a simple application that demonstrates using the [Google Cloud Client
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gcloud sdk?

Library][gcloud-python] to connect to and interact with Cloud Bigtable.

[gcloud-python]: https://github.com/GoogleCloudPlatform/gcloud-python


## Provision a cluster

Follow the instructions in the [user documentation](https://cloud.google.com/bigtable/docs/creating-cluster)
to create a Google Cloud Platform project and Cloud Bigtable cluster if necessary.
You'll need to reference your project ID, zone and cluster ID to run the application.


## Run the application

First, set your [Google Application Default Credentials](https://developers.google.com/identity/protocols/application-default-credentials)

Install the dependencies with pip.

```
$ pip install -r requirements.txt
```

Run the application. Replace the command-line parameters with values for your cluster.

```
$ python hello.py -p my-project -c my-cluster -z us-central1-c
```

You will see output resembling the following:

```
Create table Hello-Bigtable-1234
Write some greetings to the table
Scan for all greetings:
greeting0: Hello World!
greeting1: Hello Cloud Bigtable!
greeting2: Hello HappyBase!
Delete table Hello-Bigtable-1234
```

## Understanding the code

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this stuff usually go in the docs? Do you think it's worthwhile to duplicate this here or would it be better to link to the documentation page that describes this sample?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to see this stuff in the README as well as the docs. Code should always have everything you need, IMHO.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think "everything you need" is satisfied by a link to the tutorial for this sample, yeah? Otherwise we risk the "official" tutorial in our docs and this version of the tutorial drifting.

If this does not yet have a companion documentation tutorial, then this can remain.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea - I prefer duplication as sometimes different ways of saying the same thing makes it easier to get. (And when I'm working, I prefer to look at my IDE and not a web page.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough, I'm okay with it as long as it stays relatively minimal.


The [hello.py](hello.py) application uses the [Google Cloud Bigtable HappyBase
package][Bigtable HappyBase], an implementation of the [HappyBase][HappyBase]
library, to make calls to Cloud Bigtable. It demonstrates several basic
concepts of working with Cloud Bigtable via this API:

[Bigtable HappyBase]: https://googlecloudplatform.github.io/gcloud-python/stable/happybase-package.html
[HappyBase]: http://happybase.readthedocs.io/en/latest/index.html

- Creating a [Connection][HappyBase Connection] to a Cloud Bigtable
[Cluster][Cluster API].
- Using the [Connection][HappyBase Connection] interface to create, disable and
delete a [Table][HappyBase Table].
- Using the Connection to get a Table.
- Using the Table to write rows via a [put][HappyBase Table Put] and scan
across multiple rows using [scan][HappyBase Table Scan].

[Cluster API]: https://googlecloudplatform.github.io/gcloud-python/stable/bigtable-cluster.html
[HappyBase Connection]: https://googlecloudplatform.github.io/gcloud-python/stable/happybase-connection.html
[HappyBase Table]: https://googlecloudplatform.github.io/gcloud-python/stable/happybase-table.html
[HappyBase Table Put]: https://googlecloudplatform.github.io/gcloud-python/stable/happybase-table.html#gcloud.bigtable.happybase.table.Table.put
[HappyBase Table Scan]: https://googlecloudplatform.github.io/gcloud-python/stable/happybase-table.html#gcloud.bigtable.happybase.table.Table.scan

43 changes: 43 additions & 0 deletions python/hello-world/e2etest.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
#!/usr/bin/env bash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

License?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please consider using nox/tox and py.test. You'll have to at minimum use py.test in order for this to be merged into python-docs-samples.


set -e
set -x

if [ -z "$GCLOUD_PROJECT" ]; then
echo "GCLOUD_PROJECT must be set."
exit 1
fi

if [ -z "$BIGTABLE_CLUSTER" ]; then
echo "BIGTABLE_CLUSTER must be set."
exit 1
fi

if [ -z "$BIGTABLE_ZONE" ]; then
echo "BIGTABLE_ZONE must be set."
exit 1
fi

OUTPUT=$(mktemp)
VENV_DIR="${OUTPUT}_venv"
virtualenv $VENV_DIR
function finish {
rm "$OUTPUT"
rm -rf "$VENV_DIR"
}
trap finish EXIT

source "${VENV_DIR}/bin/activate"
pip install -r requirements.txt

python hello.py \
-p "$GCLOUD_PROJECT" \
-c "$BIGTABLE_CLUSTER" \
-z "$BIGTABLE_ZONE" \
| tee "$OUTPUT"
grep 'Create table Hello-Bigtable' "$OUTPUT"
grep 'Delete table Hello-Bigtable' "$OUTPUT"
grep 'greeting0: Hello World!' "$OUTPUT"
grep 'greeting1: Hello Cloud Bigtable!' "$OUTPUT"
grep 'greeting2: Hello HappyBase!' "$OUTPUT"

114 changes: 114 additions & 0 deletions python/hello-world/hello.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
#!/usr/bin/env python

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

newline between shebang and license.

# Copyright 2016 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""hello.py demonstrates how to connect to Cloud Bigtable and run some basic

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't use the name of the script here (it can change). Just "Demonstrates how to...." is fine.

operations.

Prerequisites:

- Create a Cloud Bigtable cluster.
https://cloud.google.com/bigtable/docs/creating-cluster
- Set your Google Application Default Credentials.
https://developers.google.com/identity/protocols/application-default-credentials
- Set the GCLOUD_PROJECT environment variable to your project ID.
https://support.google.com/cloud/answer/6158840
"""

from argparse import ArgumentParser

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import modules not classes, e.g., import argparse then use argparse.ArgumentParser.

import random

from gcloud import bigtable
from gcloud.bigtable import happybase


TABLE_NAME_FORMAT = 'Hello-Bigtable-{0}'

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the 0 is only required on python 2.6 and < 3.3, so you can leave it out.

TABLE_NAME_RANGE = 10000
COLUMN_FAMILY_NAME = 'cf1'
COLUMN_NAME = 'greeting'
FULL_COLUMN_NAME = '{fam}:{col}'.format(
fam=COLUMN_FAMILY_NAME,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hard to tell on github, but this should be indented 4 spaces.

col=COLUMN_NAME)

GREETINGS = [
'Hello World!',
'Hello Cloud Bigtable!',
'Hello HappyBase!',
]


def parse_args():

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We typically put all of this in the if __name__ == '__main__ block.

"""Parses command-line options."""

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Obvious docstrings can be omitted.

parser = ArgumentParser(
description='A sample application that connects to Cloud' +
' Bigtable.')
parser.add_argument(
'--project',
'-p',
action="store",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the default, leave it off.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With Bigtable, projects where the Bigtable Cluster is located might not be the same place as the code. (which regularly disappoints me that it isn't optional)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't mean leave off the project argument, just leave off action="store".

required=True,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just make it positional instead of a flag.

help='Google Cloud Platform project ID that contains the Cloud' +
' Bigtable cluster.')
parser.add_argument(
'--cluster',
'-c',
action="store",
required=True,
help='ID of the Cloud Bigtable cluster to connect to.')
parser.add_argument(
'--zone',
'-z',
action="store",
required=True,
help='Zone that contains the Cloud Bigtable cluster.')
return parser.parse_args()


def main():

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

main should take explicit arguments instead of parsing them. Makes testing easier.

For example

def main(project, cluster, zone, ...)

"""Runs the sample application."""
args = parse_args()

# Require admin client, because hello.py creates a table.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't treat the script as a third-person, it's weird. How about # The client must be created with admin=True because it creates a table.

client = bigtable.Client(project=args.project, admin=True)
with client:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Newline above new blocks, please.

cluster = client.cluster(args.zone, args.cluster)
cluster.reload()
connection = happybase.Connection(cluster=cluster)

# Select a random table name to prevent conflicts when running tests.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should never let our testing requirements leak into sample scripts.

table_name = TABLE_NAME_FORMAT.format(
random.randrange(TABLE_NAME_RANGE))

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hard to tell on github, but this should only be indented one level above table_name.

print('Create table {0}'.format(table_name))
connection.create_table(
table_name,
{
COLUMN_FAMILY_NAME: dict() # Use default options.
})
table = connection.table(table_name)

print('Write some greetings to the table')

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use gerunds when describing what the script is doing in output.

for i, value in enumerate(GREETINGS):
row_key = 'greeting{0}'.format(i)
table.put(row_key, {FULL_COLUMN_NAME: value})

print('Scan for all greetings:')
for key, row in table.scan():
print('\t{0}: {1}'.format(key, row[FULL_COLUMN_NAME]))

print('Delete table {0}'.format(table_name))
connection.delete_table(table_name)


if __name__ == '__main__':
main()
12 changes: 12 additions & 0 deletions python/hello-world/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
enum34==1.1.6

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know it's common practice to do pip freeze > requirements.txt, but for the sake of simplicity we always put the minimum set here. I would recommend that you make this:

gcloud[grpc]==0.14.0

futures==3.0.5
gcloud==0.14.0
googleapis-common-protos==1.1.0
grpcio==0.14.0rc1
httplib2==0.9.2
oauth2client==2.1.0
protobuf==3.0.0b3
pyasn1==0.1.9
pyasn1-modules==0.0.8
rsa==3.4.2
six==1.10.0