Error in storage - calling transform done when ws.length != 0

[leibale]

/home/leibale/workspace/images-processor/node_modules/gcloud/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:204
    throw new Error('calling transform done when ws.length != 0');
          ^
Error: calling transform done when ws.length != 0
    at done (/home/leibale/workspace/images-processor/node_modules/gcloud/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:204:11)
    at DestroyableTransform.<anonymous> (/home/leibale/workspace/images-processor/node_modules/gcloud/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:139:7)
    at DestroyableTransform.g (events.js:199:16)
    at DestroyableTransform.emit (events.js:129:20)
    at DestroyableTransform.<anonymous> (/home/leibale/workspace/images-processor/node_modules/gcloud/node_modules/stream-forward/index.js:31:26)
    at DestroyableTransform.<anonymous> (/home/leibale/workspace/images-processor/node_modules/gcloud/node_modules/stream-forward/node_modules/on-everything/index.js:12:17)
    at DestroyableTransform.obj.(anonymous function) [as emit] (/home/leibale/workspace/images-processor/node_modules/gcloud/node_modules/stream-forward/node_modules/stubs/index.js:28:10)
    at finishMaybe (/home/leibale/workspace/images-processor/node_modules/gcloud/node_modules/retry-request/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:502:14)
    at endWritable (/home/leibale/workspace/images-processor/node_modules/gcloud/node_modules/retry-request/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:512:3)
    at DestroyableTransform.Writable.end (/home/leibale/workspace/images-processor/node_modules/gcloud/node_modules/retry-request/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:477:5)

i don't know the exact request- but it's from the storage

[stephenplusplus]

Thanks for reporting. If you can zero in on one or more of your gcloud calls that consistently produces this, that would definitely help. But, generally speaking, our streams do need some safeguards to prevent again this.

[leibale]

file.download({
  destination: path
}, function (err) {
// --
/});

this is the request

[leibale]

it's important so.. i will be happy if somebody help me
sorry for my audacity

[stephenplusplus]

Sure, we'll do our best. Your best option for now is probably to downgrade to v0.17.

[leibale]

It also doesn't work

[stephenplusplus]

Ok, roll back once more to 0.16 or the soonest you are able to make the request successfully.

[leibale]

but in v0.16 the maxInProgress doesn't exist

---

maybe it's happened after change in the server

"On June 9th your project XXXXXXXX was affected by an incident which caused elevated error rates for 1 hour 40 minutes. You can read about the incident and the changes we have made to prevent re-occurrence at https://status.cloud.google.com/incident/storage/16025.

We take such incidents seriously and are therefore issuing a credit under the terms of our Service Level Agreement (SLA) as defined at https://cloud.google.com/storage/sla. This credit will be equal to 10% of your June Google Cloud Storage bill. In this case, we will apply this credit to your account automatically, bypassing the normal SLA application procedure, so no action is required on your part.

Yours,
The Google Cloud Storage Team

Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043

You have received this mandatory service announcement to update you about important changes to Google Cloud Platform or your account.

i got this in email from [email protected]

[stephenplusplus]

Can you try removing the node_modules directory and re-installing gcloud? I updated the problematic dependency here: stephenplusplus/stream-forward@11ec2a7

[leibale]

re-installing gcloud from github or from npm?

[stephenplusplus]

From npm, it will automatically pick up the new stream-forward. Just make sure to rm -rf node_modules first. Updating is weird.

[leibale]

now requests not returns (and i am in google vm so it's not internet problem)

[stephenplusplus]

All requests aren't returning? Can you show more of the code you're using?

[leibale]

pub/sub working

file.download({
  destination: path
}, function (err) {
// --
/});

[leibale]

sorry, the code is:

storage.backupBucket.file(message.filename).createReadStream().on('error', callback).pipe(fs.createWriteStream(path)).on('finish', function () {
  console.log('downloaded');
  callback(null, message, path);
});

[stephenplusplus]

Ah, thanks-- I see now! You need to listen for the complete event, instead of finish.

[stephenplusplus]

Oh, I'm sorry, I misread the code. BRB :)

[leibale]

you need help?

[stephenplusplus]

PR with a fix is coming shortly... basically, we destroy the stream. We don't have to do that unless there's an error.

[stephenplusplus]

PR sent! #809