"Cannot sign data without client_email" for getSignedUrl
#360
Comments
|
I think Here's a related issue: #116. |
|
I'm getting it on Cloud Functions, though, which I believe is a service account. |
JustinBeckwith
added
🚨
|
I tried to reproduce it on Cloud Functions, but was able to generate signed URL from the code provided. Default service account in Cloud Functions should have a client_email |
jkwlui
added
the
needs more info
|
Can you provide the error you got when running the code under Functions? |
|
I did something soon after my original post to get it working on Functions. But I'm still getting this error locally. |
|
As @fhinkel mentioned, it wouldn't work if you'd set up your credentials via Please let me know if this works and is an acceptable solution to you. :) |
|
Hi @pikelnys. You will have to specify keyFilename while creating the Storage object if you are deploying locally. keyFilename is not required if you have deployed your code under the same project your Google Cloud Bucket was created. You just have to ensure you have the right access by having a look at your service-account-key. The error says cannot sign without client_email. client_email is available inside the service-account-key.json but was not provided while creating the object hence the error. This error would not have arrived had you deployed your code inside the same project as your Cloud Bucket was. Here is a sample service-account-key.json file. Look for client_email in the file. Steps to Follow. Download the service account key in JSON format and specify its path inside keyFilename. Default service-account-key provided by Google has read-only access. So make sure if you want to write to your bucket you would have to change the permissions inside the service-account. Here is the Hope this resolves your issue. |
|
@ujwalkagrawal, thanks for posting such detailed explanation on service account credentials! @pikelnys we hope this resolves your issue, please re-open if you run into anymore issues :) |
|
Is this possible to authenticate without a keyfile? |
google-cloud-label-sync
bot
added
the
api: storage
The default account does not include `client_email`, which is necessary for signing. Note this for folks. see: #360
|
I was getting "Cannot sign data without I fixed it by setting up a service account and downloading the service account .json credentials file as |
|
Service account isn't good for multi-stage environments and security. This should be reopened. You should be using the exposed environment that has the same values. |
|
BUMP as there isn't seem to be a way to deal with it via |
|
Bump. Firebase emulator should emulate the environment, this appears to be a failure state |
|
Hi @dwanderton thank you for bumping this. In reading your comment and the last few, it appears the feeling is that there is incorrect behavior in the Firebase emulator. If so, I would encourage you to open an issue with the Firebase CLI repository. That repository is at: https://github.com/firebase/firebase-tools |
|
For anyone else that comes across this issue, here is a bit more info for posterity. The GCP client libraries docs use the Storage libary as the example when leveraging ADC locally. The same docs also call out the fact that using service accounts locally is bad practice (see the Note in this section). However, for some reason Google seems to contradict itself with the implementation of the It would be nice if Google would have bothered to call this out in the ADC docs where they use Storage as an example, or better yet, return the relevant instructions in the SigningError message, especially because they seemingly contradict themselves within the different docs on related topics. Hope this helps someone! |
|
Indeed many of the google services use the less secure credentials initialization; you think they’d be on top of this with some simple version updates; but I’m seeing nothing being updated for years and incorrect documentation everywhere. |
|
Hey @timbuckiii and @ollyde , just wanted to note a couple things:
Also, in the future, if you could open a new issue for any problems rather than commenting on a closed one, that would help us respond faster. Thanks! |
|
Thanks @tritone, appreciate the response! |
|
Got the issue and solved it thanks to this solution [email protected] |
|
I don't think you should switch to a json keyfile. I just added an npm command to add the npm install --save-dev json"scripts": {
"login": "gcloud auth application-default login --impersonate-service-account={MY_EMAIL} {EXTRA_ARGUMENTS} && npm run update-client-email",
"update-client-email": "json -I -f ~/.config/gcloud/application_default_credentials.json -e 'this.client_email=\"{MY_EMAIL}\"'"
}I haven't deployed yet to Google Cloud, but I think you should be able to run this login command locally only, and it should just work when deployed to cloud? Edit: added |
|
@howlettga I have tried using the solution you suggest by adding in a The only way I can get it working currently is by passing in a service account key when using locally to generate a Signed URL. |
|
@boarush yes it works for me. I am using a service account intentionally when logging in the application default credentials (using the This error is specific to the missing So whatever authentication method you already use should work. If you can do things like Decode Id Tokens and stuff, but not sign urls because of this error, you simply need to ensure the Ultimately, I don't believe this library is the proper place for this error. It really has to do with the gcloud cli initialization. It might not even be an error, but a missing configuration command. I haven't played around with the |
|
bump. this should be re-opened. |
|
Double bump. There's not a great firebase-admin solution for this when running in the emulator without using a service account json. It's messy, insecure, and inconvenient for testing... It's really easy on the frontend with Firebase Auth. Ex: |
|
I see a few bumps on this old issue so wanted to leave a comment. It seems that the issue most are bumping about is trying to sign using ADC which does not include the If there is a different issue, I would ask that you open a new issue so that I can investigate / resolve as appropriate. |
|
For me it is while using firebase emulator, I just want to be able to sign local storage URLs. Nothing cloud side. Everything just local! |
|
Any workaround? |
@ddelgrosso1 So are you saying this library doesn't work with ADC? |
Sorry to keep adding to this closed issue, especially when i know the team has deemed that this isn't the appropriate place for it, but this is the only thread google returns for pretty much any way I've tried to search on this issue, so sharing for others that end up here until I understand where the appropriate location to discuss this issue is. Adding my "client_email" to the ADC file didn't work for me either. That gives me the error: I hate the service account key solution from a security perspective. Google's own documentation recommends against it. I'll look into "signing with impersonation" tomorrow. :/ |
That is not what I was saying. ADC works fine outside of signing. This library doesn't own or produce the ADC credentials. If the ADC credentials do not contain the |
Yeah I can see that logging in with my admin user doesn't seem to configure the ADC to properly pull the Gaia id. May as well just impersonate a service account that is properly configured |
|
I can't believe it's been six years since I ran into this issue and it's still not resolved. Since then, I found a work around for the issue, the project died, I quit that job, and the company went out of business. |
github-actions
bot
removed
the
needs more info
|
I think impersonating the service account is the answer for local development. It doesn't have the security concerns that using a service account key does. It also has the benefit of using the actual deployed account, so you are confirming it is set up correctly. Was a breeze to do. See: |
|
Nevermind. I thought it wasn't erroring, but that's because it's now no longer getting to the signing code. For me, account impersonation is failing even earlier, when I call I'm getting: If I skip the exists() check, Checking the My hunt for a working solution continues. 🤕
That makes total sense. Have any issues been submitted to anyone else for this? I'd love to take this discussion to someone else's repo instead who could actually fix it. I think you're in the best position to write up what needs to be fixed in which repo and also influence the rest of the googleapis team. In the meantime, you've got a known broken use case and the errors that are bubbling up out of your library don't make sense in the context of your library. Is that a valid bug worth having an open issue for? |
@dinkel-katilyst I found this public issue that was filed against |
@dinkel-katilyst if you just add |
|
@ddelgrosso1 The link you shared doesn't appear to be public. @howlettga adding the Something else I just stumbled upon: it appears that the python version of this library was able to fix what looks like the same issue themselves. I've tried to read it and figure out how, but I don't think my python skills are good enough. Or maybe it's not quite the same issue. Might be worth a look tho. https://stackoverflow.com/a/74116712/4839461 |
Try this link. Please let me know if this does not work. Also updated in my previous comment. |
|
That one worked! |
|
@dinkzilla so then you are resolving the error in this thread... And sounds like you have a separate error initializing the |
|
I wouldn't agree at all that manually editing a config file with a value that gets removed every time i re-auth and doesn't get the feature working would count at all as a resolution of this issue. My code is pretty simple - I'm not doing anything weird during initialization. I've sunk enough of my time into this for now, so I'm just going to download the service account key file, which is a huge bummer but I need to get unblocked on this. Luckily, my client is small enough that they don't have that locked down. That solution definitely isn't available to everyone (and shouldn't be). I'll throw a note in that public issue, but I really think this issue should be reopened here as well. This bug exists in your library. The fact that the bug results from a library that you call and not directly from your own code is irrelevant in my opinion. I would never get away with saying to my client "No, the app I built you doesn't have a bug, the googleapis/nodejs-storage library has a bug. Go talk to them." Expecting your users to understand the relationships between your dependencies and go complain to them instead when you have bugs just doesn't seem fair to me. I really do appreciate how quick you've both responded trying to help me resolve this though. 🫶 |
|
Just realized the previously linked issue was merged in last year and the |
|
Thank you so much @howlettga ! an upgrade made it work for me too! 🕺 Many thanks! |
|
Circling back. I spoke with the teams who own the login / writing of the ADC JSON file. This isn't just a matter of adding a The best course of action is to utilize SA impersonation for signing in these situations. |
|
Jeez. Enabling OIDC has been a massive challenge with the garbage documentation out there. Anyhow, after hours of fighting, here's how I could sign the URL using service account impersonation: 1st. Create the service account: gcloud iam service-accounts create "local-dev-account" \
--description="Local Development Account" \
--project="${YOUR_PROJECT_ID}" \
--display-name="Local Development Account"2nd. Create the gcloud iam roles create "local_dev_role" \
--project="${YOUR_PROJECT_ID}" \
--file="./roles-local.gcp.yml"roles-local.gcp.yml: 3rd. Attach the service account at the project level gcloud projects add-iam-policy-binding ${YOUR_PROJECT_ID} \
--member="serviceAccount:local-dev-account@${YOUR_PROJECT_ID}.iam.gserviceaccount.com" \
--role="projects/${YOUR_PROJECT_ID}/roles/local_dev_role" \
--project="${YOUR_PROJECT_ID}"Note: There's a similar command called 4th. Impersonate the service account: gcloud auth application-default login --impersonate-service-account=local-dev-account@alertdown-staging.iam.gserviceaccount.comIf that didn't work, and you're getting errors about the account not getting impersonated: Attach the gcloud iam service-accounts add-iam-policy-binding \
local-dev-account@${YOUR_PROJECT_ID}.iam.gserviceaccount.com \
--member="user:[email protected]" \
--role="roles/iam.serviceAccountTokenCreator" \
--project="${YOUR_PROJECT_ID}"
gcloud iam service-accounts add-iam-policy-binding \
local-dev-account@${YOUR_PROJECT_ID}.iam.gserviceaccount.com \
--member="user:[email protected]" \
--role="roles/iam.serviceAccountUser" \
--project="${YOUR_PROJECT_ID}"And just for the sake of completeness: Here's the full Storage class wrapper I've created in Node: import {
InternalServerException,
PromiseExceptionResult,
} from '@alertdown/core';
import { Storage as GoogleCloudStorage } from '@google-cloud/storage';
import { Err, Ok } from 'oxide.ts';
const getFileUrl = (bucket: string, filename: string) => {
return `https://storage.googleapis.com/${bucket}/${filename}`;
};
type ConstructorInput = {
bucket: string;
/**
* Development only
*/
projectId?: string;
};
export class Storage {
#client: GoogleCloudStorage;
#bucket: string;
constructor(input: ConstructorInput) {
this.#client = new GoogleCloudStorage({
projectId: input.projectId,
});
this.#bucket = input.bucket;
}
async upload(file: Buffer, filename: string): PromiseExceptionResult<string> {
try {
const bucket = this.#client.bucket(this.#bucket);
const blob = bucket.file(filename);
await blob.save(file, {
metadata: {
contentType: 'image/png',
},
});
return Ok(getFileUrl(this.#bucket, filename));
} catch (error) {
return Err(new InternalServerException(error as Error));
}
}
async resolveUrl(url: string): PromiseExceptionResult<string> {
try {
const parsedUrl = new URL(url);
const filename = parsedUrl.pathname.split('/').pop();
console.log((await this.#client.authClient.getClient()).credentials);
if (!filename) {
return Err(new InternalServerException('Invalid URL format'));
}
const bucket = this.#client.bucket(this.#bucket);
const file = bucket.file(filename);
const [signedUrl] = await file.getSignedUrl({
version: 'v4',
action: 'read',
expires: Date.now() + 15 * 60 * 1000, // 15 minutes
});
return Ok(signedUrl);
} catch (error) {
console.error(`Error resolving URL: ${url}`, error);
return Err(new InternalServerException(error as Error));
}
}
}Note: Additional Notes:
Helpful Commands For Debugging:
gcloud iam roles describe local_dev_role \
--project=${YOUR_PROJECT_ID}
gcloud projects get-iam-policy ${YOUR_PROJECT_ID} \
--filter="bindings.members:local-dev-account@${YOUR_PROJECT_ID}.iam.gserviceaccount.com" \
--format="table(bindings.role)"
gcloud iam roles update local_dev_role \
--project=${YOUR_PROJECT_ID} \
--file=./roles-local.gcp.yml |
|
@superjose thank you so much, it works, btw I think in step 4. you used the actual PROJECT_ID: |
and others
I'm trying to get a signed url from a storage bucket, but am getting a signing error. I have permission to read from the bucket, and have been able to stream its contents fine.
Code:
Error:
Environment details
@google-cloud/storageversion: 1.7.0Edit: I'm getting this in both a local environment (where I authenticate with
gcloud auth application-default login) and while the code is deployed to cloud functions.The text was updated successfully, but these errors were encountered: