Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pubsub: api redesign #2380

Merged
merged 67 commits into from
Aug 24, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
1f09a9c
pubsub: api redesign
callmehiphop Jun 12, 2017
e515dc8
funnel requests for projectId interpolation
callmehiphop Jun 20, 2017
3fbdd2f
change Error to TypeError
callmehiphop Jun 21, 2017
db200cc
update license year
callmehiphop Jun 21, 2017
1cca960
refactor batching
callmehiphop Jun 21, 2017
aec9640
added batching for modifyAckDeadline
callmehiphop Jun 21, 2017
6984126
implement message flow control
callmehiphop Jun 21, 2017
cad7d07
remove all message listeners upon sub delete
callmehiphop Jun 21, 2017
73eec6d
add comment explaining error handling
callmehiphop Jun 21, 2017
4088c45
small refactor to inventory management
callmehiphop Jun 22, 2017
e0cd344
store deadline in milliseconds
callmehiphop Jun 26, 2017
f53c47f
update queue settings to add caps
callmehiphop Jun 26, 2017
c6615c4
tweak how we calculate max message cap
callmehiphop Jun 26, 2017
7d13032
update system tests to use message events
callmehiphop Jun 27, 2017
4a9e0e4
refactor publisher to not use queue object
callmehiphop Jun 27, 2017
5d6ed94
create new getSubscriptions streaming method for topics
callmehiphop Jun 27, 2017
543af0f
huge refactor of subscription/message objects
callmehiphop Jun 27, 2017
955d7eb
remove queue object
callmehiphop Jun 27, 2017
5827b04
kill timeout if publish queue fills
callmehiphop Jun 27, 2017
cce7949
optimize in memory message storing
callmehiphop Jun 27, 2017
cdd9803
let gax handle errors
callmehiphop Jun 27, 2017
91485d8
use maxBytes option to do inventory management
callmehiphop Jul 10, 2017
d605fe2
add connection pooling
callmehiphop Jul 10, 2017
1e330d3
manually track publisher queue size
callmehiphop Jul 25, 2017
1bb715b
track publisher callbacks in separate array
callmehiphop Jul 25, 2017
86aa39f
renamed callback to onEndCallback
callmehiphop Jul 25, 2017
85e2471
store histogram values at second level percision
callmehiphop Jul 25, 2017
1c17045
refactor publishing logic
callmehiphop Jul 25, 2017
71df034
remove console logs
callmehiphop Jul 25, 2017
7a2f9c0
refactored connection pool
callmehiphop Jul 26, 2017
e1d1b8c
small refactors based on pr feedback
callmehiphop Jul 26, 2017
d389573
system test updates
callmehiphop Jul 27, 2017
0e3a6ef
update unit tests for index.js
callmehiphop Jul 28, 2017
5e9e939
update unit tests for iam.js
callmehiphop Jul 28, 2017
57f02f1
update unit tests for snapshot.js
callmehiphop Jul 28, 2017
2530761
update unit tests for topic.js
callmehiphop Jul 28, 2017
0d0422a
update unit tests for subscription.js
callmehiphop Jul 28, 2017
2844bf7
add unit tests for publisher.js
callmehiphop Jul 28, 2017
b116fbb
moved message.js code into connection-pool
callmehiphop Jul 28, 2017
017e006
add tests for connection-pool.js
callmehiphop Jul 28, 2017
f2dfd09
add unit tests for histogram.js
callmehiphop Jul 28, 2017
88e75e1
removed unused deps & lint
callmehiphop Jul 28, 2017
768bb86
update top level object documentation
callmehiphop Jul 31, 2017
46b9cd9
update topic documentation
callmehiphop Jul 31, 2017
15273fd
update subscription documentation
callmehiphop Jul 31, 2017
4853923
update snapshot documentation
callmehiphop Jul 31, 2017
e1848e9
add docs for new classes
callmehiphop Jul 31, 2017
0075979
update PubSub examples in READMEs
callmehiphop Jul 31, 2017
cc6ca62
update docs script to ignore internal pubsub classes
callmehiphop Jul 31, 2017
fb068d1
fix documentation typos/errors
callmehiphop Jul 31, 2017
3a367fe
tweaks per PR feedback & full test coverage
callmehiphop Aug 3, 2017
fe1d030
stub out os module
callmehiphop Aug 3, 2017
c00f6b4
add link for push config attributes
callmehiphop Aug 3, 2017
09ef328
test tweaks
callmehiphop Aug 3, 2017
e6cf720
add top level autoPaginate support
callmehiphop Aug 7, 2017
79cf701
made Snapshot#delete cb optional
callmehiphop Aug 7, 2017
73a025c
tweaks based on PR feedback
callmehiphop Aug 15, 2017
bbef999
increase grpc max recieve message size
callmehiphop Aug 16, 2017
403f8fe
refactoring connection pool error handling
callmehiphop Aug 16, 2017
8cea209
small cleanup
callmehiphop Aug 17, 2017
7fa8118
bypass gax for streamingPull rpc
callmehiphop Aug 18, 2017
f95b8c4
tweaks/refactors based on pr feedback + stream retry strategy
callmehiphop Aug 22, 2017
194a03e
adjustments per pr feedback
callmehiphop Aug 23, 2017
9e9bdfa
add get and exists methods
callmehiphop Aug 24, 2017
31ab748
tweak test
callmehiphop Aug 24, 2017
ca6e982
update readmes to list pubsub as beta
callmehiphop Aug 24, 2017
f758d07
move pubsub sample placement
callmehiphop Aug 24, 2017
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 52 additions & 49 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ This client supports the following Google Cloud Platform services at a [General
This client supports the following Google Cloud Platform services at a [Beta](#versioning) quality level:

* [Cloud Natural Language](#cloud-natural-language-beta) (Beta)
* [Cloud Pub/Sub](#cloud-pubsub-alpha) (Beta)
* [Cloud Spanner](#cloud-spanner-beta) (Beta)
* [Cloud Vision](#cloud-vision-beta) (Beta)
* [Google BigQuery](#google-bigquery-beta) (Beta)
Expand All @@ -28,7 +29,6 @@ This client supports the following Google Cloud Platform services at an [Alpha](

* [Cloud Bigtable](#cloud-bigtable-alpha) (Alpha)
* [Cloud DNS](#cloud-dns-alpha) (Alpha)
* [Cloud Pub/Sub](#cloud-pubsub-alpha) (Alpha)
* [Cloud Resource Manager](#cloud-resource-manager-alpha) (Alpha)
* [Cloud Speech](#cloud-speech-alpha) (Alpha)
* [Google Compute Engine](#google-compute-engine-alpha) (Alpha)
Expand Down Expand Up @@ -450,6 +450,57 @@ languageClient.analyzeSentiment({document: document}).then(function(responses) {
```


## Cloud Pub/Sub (Beta)

- [API Documentation][gcloud-pubsub-docs]
- [Official Documentation][cloud-pubsub-docs]

#### Using the Cloud Pub/Sub API module

```
$ npm install --save @google-cloud/pubsub
```

```js
var pubsub = require('@google-cloud/pubsub');
```

#### Authentication

See [Authentication](#authentication).

#### Preview

```js
var pubsubClient = pubsub({
projectId: 'grape-spaceship-123',
keyFilename: '/path/to/keyfile.json'
});

// Reference a topic that has been previously created.
var topic = pubsubClient.topic('my-topic');

// Publish a message to the topic.
var publisher = topic.publisher();
var message = new Buffer('New message!');

publisher.publish(message, function(err, messageId) {});

// Subscribe to the topic.
topic.createSubscription('subscription-name', function(err, subscription) {
// Register listeners to start pulling for messages.
function onError(err) {}
function onMessage(message) {}
subscription.on('error', onError);
subscription.on('message', onMessage);

// Remove listeners to stop pulling for messages.
subscription.removeListener('message', onMessage);
subscription.removeListener('error', onError);
});
```


## Cloud Spanner (Beta)

- [API Documentation][gcloud-spanner-docs]
Expand Down Expand Up @@ -816,54 +867,6 @@ zone.export('/zonefile.zone', function(err) {});
```


## Cloud Pub/Sub (Alpha)

- [API Documentation][gcloud-pubsub-docs]
- [Official Documentation][cloud-pubsub-docs]

#### Using the Cloud Pub/Sub API module

```
$ npm install --save @google-cloud/pubsub
```

```js
var pubsub = require('@google-cloud/pubsub');
```

#### Authentication

See [Authentication](#authentication).

#### Preview

```js
var pubsubClient = pubsub({
projectId: 'grape-spaceship-123',
keyFilename: '/path/to/keyfile.json'
});

// Reference a topic that has been previously created.
var topic = pubsubClient.topic('my-topic');

// Publish a message to the topic.
topic.publish('New message!', function(err) {});

// Subscribe to the topic.
topic.subscribe('subscription-name', function(err, subscription) {
// Register listeners to start pulling for messages.
function onError(err) {}
function onMessage(message) {}
subscription.on('error', onError);
subscription.on('message', onMessage);

// Remove listeners to stop pulling for messages.
subscription.removeListener('message', onMessage);
subscription.removeListener('error', onError);
});
```


## Cloud Resource Manager (Alpha)

- [API Documentation][gcloud-resource-docs]
Expand Down
11 changes: 7 additions & 4 deletions packages/pubsub/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# @google-cloud/pubsub ([Alpha][versioning])
# @google-cloud/pubsub ([Beta][versioning])
> Cloud Pub/Sub Client Library for Node.js

*Looking for more Google APIs than just Pub/Sub? You might want to check out [`google-cloud`][google-cloud].*
Expand All @@ -20,10 +20,13 @@ var pubsub = require('@google-cloud/pubsub')({
var topic = pubsub.topic('my-topic');

// Publish a message to the topic.
topic.publish('New message!', function(err) {});
var publisher = topic.publisher();
var message = new Buffer('New message!');

publisher.publish(message, function(err, messageId) {});

// Subscribe to the topic.
topic.subscribe('subscription-name', function(err, subscription) {
topic.createSubscription('subscription-name', function(err, subscription) {
// Register listeners to start pulling for messages.
function onError(err) {}
function onMessage(message) {}
Expand All @@ -36,7 +39,7 @@ topic.subscribe('subscription-name', function(err, subscription) {
});

// Promises are also supported by omitting callbacks.
topic.publish('New message!').then(function(data) {
publisher.publish(message).then(function(data) {
var messageIds = data[0];
});

Expand Down
4 changes: 2 additions & 2 deletions packages/pubsub/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -54,12 +54,12 @@
"@google-cloud/common": "^0.13.0",
"@google-cloud/common-grpc": "^0.4.0",
"arrify": "^1.0.0",
"async-each": "^1.0.1",
"extend": "^3.0.0",
"google-auto-auth": "^0.7.1",
"google-gax": "^0.13.0",
"google-proto-files": "^0.12.0",
"is": "^3.0.1",
"modelo": "^4.2.0",
"propprop": "^0.3.0",
"uuid": "^3.0.1"
},
"devDependencies": {
Expand Down
Loading