feat: support OAuth2 and OpenID Connect in API-based flows

[splaunov]

Related issue(s)

#707

Checklist

• [ x] I have read the contributing guidelines.
• [ x] I have referenced an issue containing the design document if my change
  introduces a new feature.
• [ x] I am following the
  contributing code guidelines.
• [x ] I have read the security policy.
• [ x] I confirm that this pull request does not address a security
  vulnerability. If this pull request addresses a security. vulnerability, I
  confirm that I got green light (please contact
  [email protected]) from the maintainers to push
  the changes.
• I have added tests that prove my fix is effective or that my feature
  works.
• I have added or changed the documentation.

Further Comments

[aeneasr]

Bruh are you for real? Awesome!

Could you please provide a high-level overview of how this feature worked? Preferably with an example for e.g. the Facebook SDK?

[aeneasr]

Awesome, thank you for your contribution! This looks pretty good and I have some ideas how to improve it further :)

[splaunov]

Just submit login flow with

{
  "method": "oidc",
  "porvider": "name it",
  "id_token": "your token"
}

or

{
  "method": "oidc",
  "porvider": "name it",
  "access_token": "your token"
}

in the body.
As simple as that.
Have tested it with google and apple. Not sure about Facebook.

[aeneasr]

@splaunov feel free to re-request a review any time you're ready :)

[aeneasr]

I tried pushing some changes required for merging the PR to your fork & branch, but it appears that I am not allowed to do so 😕

% git push ...
ERROR: Permission to push denied to aeneasr.
fatal: could not read from the remote repository.

Please make sure that you have the correct access rights
and the repository exists.

But the good news is, giving access is easy! ☺️ All you need to do is enable write access for maintainers. Thank you! 😄

If the repository belongs to an organization, please add me for the project as a collaborator!

[aeneasr]

Thank you, this looks great! The CI is failing because some files are formatted incorrectly. To format them, run:

$ make format
$ git commit -a -m "styles: format code"
$ git push

Thank you!

[aeneasr]

Thank you for these great changes, I think we are now on track to get this shipped. I haven't had a lot of time to really look into the details of the implementation yet (currently on vacation) but from a high level it looks very good.

One question I have is how we deal with validation errors? In the browser flow, if a validation error occurrs (e.g. some required field is missing), we show the registration form again and ask the user to fill out their details. How does that work for API flows?

Another question is, how do we deal with detecting whether we need to execute login or registration? How is that information relayed to the application? In the browser flow, we just redirect to the appropriate login (user exists) or registration (user does not exist yet) endpoint.

Another thing we will need to add to this PR is an e2e test for this new feature. We already have e2e tests for browser OIDC flows which you can find here:

• https://github.com/ory/kratos/blob/master/test/e2e/cypress/integration/profiles/oidc/registration
• https://github.com/ory/kratos/tree/master/test/e2e/cypress/integration/profiles/oidc/login
• https://github.com/ory/kratos/tree/master/test/e2e/cypress/integration/profiles/oidc/settings

The profile for mobile apps (API flows) you can find here: https://github.com/ory/kratos/tree/master/test/e2e/cypress/integration/profiles/mobile

It will probably not be trivial to add OIDC tests to the mobile app as the flow works differently (we need a valid ID token) but it is a requirement from our end to get this merged into master, as it is the only way to verify that everything works as expected from end-to-end. The tests will need to cover:

1. Login with an ID Token and a user which exists
2. Login with an ID Token and a user which does not exist -> we end up registering a new user
3. Registration with an ID Token and a user which exists -> we end up signed in
4. Registration with an ID Token and a user which does not exist, and validation errors (e.g. website missing)
5. Successful registration with an ID Token
6. Linking and unlinking OIDC providers in the settings flow
7. Error cases for each flow (login, registration, settings)
   1. Audience mismatch
   2. Signature key mismatch
   3. ID Token generally invalid
   4. Issuer mismatch

[splaunov]

I tried pushing some changes required for merging the PR to your fork & branch, but it appears that I am not allowed to do so 😕

That's strange. I added you as a collaborator some time ago and you've already pushed some changes to our fork, but to another branch.

[codecov]

Codecov Report

Merging #2346 (0db186d) into master (1ed6839) will increase coverage by 0.54%.
The diff coverage is 53.74%.

❗ Current head 0db186d differs from pull request most recent head 57925e5. Consider uploading reports for the commit 57925e5 to get more accurate results

@@            Coverage Diff             @@
##           master    #2346      +/-   ##
==========================================
+ Coverage   75.89%   76.43%   +0.54%     
==========================================
  Files         311      318       +7     
  Lines       19185    17414    -1771     
==========================================
- Hits        14560    13310    -1250     
+ Misses       3486     3158     -328     
+ Partials     1139      946     -193     

Impacted Files	Coverage Δ
selfservice/strategy/oidc/provider_apple.go	0.00% <0.00%> (-20.84%)	⬇️
selfservice/strategy/oidc/provider_config.go	37.20% <ø> (-1.68%)	⬇️
selfservice/strategy/oidc/provider_microsoft.go	0.00% <0.00%> (ø)
selfservice/strategy/oidc/strategy.go	66.35% <ø> (+2.20%)	⬆️
selfservice/strategy/oidc/strategy_settings.go	62.76% <50.00%> (-1.36%)	⬇️
selfservice/strategy/oidc/provider_generic_oidc.go	84.05% <55.00%> (-8.80%)	⬇️
selfservice/strategy/oidc/strategy_login.go	62.00% <60.86%> (-5.54%)	⬇️

... and 336 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[splaunov]

One question I have is how we deal with validation errors? In the browser flow, if a validation error occurrs (e.g. some required field is missing), we show the registration form again and ask the user to fill out their details. How does that work for API flows?
Another question is, how do we deal with detecting whether we need to execute login or registration? How is that information relayed to the application? In the browser flow, we just redirect to the appropriate login (user exists) or registration (user does not exist yet) endpoint.

As for now, there is no registration API flow for oidc. New users are silently provisioned in login flow.
This works fine only if identity has no required traits.

You are right in that we need to support registration flow as well.
To do this, I can move the provisioning code from login to registration flow. So, login flow will return error to mobile app if credentials not found and new registration flow will need to be started by the mobile app in this case.

[splaunov]

Another thing we will need to add to this PR is an e2e test for this new feature. We already have e2e tests for browser OIDC flows which you can find here:

This assumes extending the React Native app. It should get ID Token from Hydra.
I feel like I can't spend enough time for this in nearest future. If someone can help with RN, I would be just happy to work on cypress part.

[aeneasr]

Thank you! I will for sure be able to help when I'm back from vacation in May. I think we need to iron out some of the big questions (registration vs login) before we can finalize it and merge it to upstream. But your work so far has been great! If you want to work on this in the meanwhile please do let me know if there's anything I can help with (e.g. pointers) :))

[klueken]

Hi everyone, is there any planned work to come back and finish this PR? This is a critically important feature that is blocking my project.

I'm not well-versed in Golang, but could try to help with documentation or testing if that helps move this forward.

[aeneasr]

Hey there, I think this is a great PR and feature, and it's on our roadmap and backlog! For us unfortauntely it has not as high of a priority as a couple of other very urgent topics.

[Probotect0r]

@aeneasr Are there any alternative solutions for mobile based OAuth flows while this PR is still in progress?

[hernangonzalez]

Would be very nice if this could be prioritized :)

[splaunov]

Have added support for apps using WebView for oidc logins

[dmakarenko]

@splaunov thank you for pushing this feature forward 🙏

I have a question though. I am not sure how the proposed WebView flow would work for native apps. AFAIK the final redirect often relies on a custom URL scheme (at least on iOS) which passes the control to the application and the only information available to the native app is the URL so any extra details (like credentials) should be passed via query params. Usually mobile apps don't have access to the WebView context (for security and privacy reasons) especially if something like Apple's Authentication Services framework is used

There is an open discussion on that topic here: #2434

It could be that I am missing something here. If so, my apologies in advance.

[David-Wobrock]

Hi all 👋
First of all, thanks a lot for working on this. It seems to me that it will play an important role in the adoption of Kratos :)

Two questions:

1/ About this:

One question I have is how we deal with validation errors? In the browser flow, if a validation error occurrs (e.g. some required field is missing), we show the registration form again and ask the user to fill out their details. How does that work for API flows?
Another question is, how do we deal with detecting whether we need to execute login or registration? How is that information relayed to the application? In the browser flow, we just redirect to the appropriate login (user exists) or registration (user does not exist yet) endpoint.

As for now, there is no registration API flow for oidc. New users are silently provisioned in login flow. This works fine only if identity has no required traits.

You are right in that we need to support registration flow as well. To do this, I can move the provisioning code from login to registration flow. So, login flow will return error to mobile app if credentials not found and new registration flow will need to be started by the mobile app in this case.

As @aeneasr stated, in the current social browser flow, the decision between login and register is made when the callback is called whether the user (or more precisely the OIDC credentials) exist or not.
I'm wondering whether it would make sense to have a similar logic for the API flow.
Basically, somewhere here https://github.com/ory/kratos/pull/2346/files#diff-a847b9b099ed3a4c0c0eed58467442371554463a07f865dcaa3d8f034759f875R233 when we try to log the user in, but it doesn't exist, switch over to the registering logic.

So that we can avoid having the mobile app starting a registration flow when a new user connects through a social provider.

2/ About the method used.
I completely understand that passing the id_token from the mobile app to the server is enough for the server to connect/create a user, as the JWT contains enough information.
Would it also work to take as input the authorization code, so that Kratos uses its client secret to exchange the code for the oauth2 tokens? (and therefore it could re-use the existing social callback logic).
A bit like it is described in this documentation from Google.

[sayoun]

Thanks for this PR, we really need this feature and want to help, can someone list what is missing here so it could be merged ? @aeneasr or @splaunov ?

[splaunov]

Here is what missed to my understanding:

1. We need an end-to-end test. To make it happen, React Native app should be extended first to register/login with pure api and WebView flows using Hydra for OIDC. And then these flows should be covered by cypress tests.
2. A review of my recently added WebView related code needed. @aeneasr

[splaunov]

@splaunov thank you for pushing this feature forward 🙏

I have a question though. I am not sure how the proposed WebView flow would work for native apps. AFAIK the final redirect often relies on a custom URL scheme (at least on iOS) which passes the control to the application and the only information available to the native app is the URL so any extra details (like credentials) should be passed via query params. Usually mobile apps don't have access to the WebView context (for security and privacy reasons) especially if something like Apple's Authentication Services framework is used

There is an open discussion on that topic here: #2434

It could be that I am missing something here. If so, my apologies in advance.

@dmakarenko yes, you're right. I have added the token to query params of the final redirect.
Thanks for pointing it out!

[peturgeorgievv]

Can we merge this, as we really need this functionality in our app?

[chancezeus]

Hi @splaunov (and @aeneasr),

Since I'm interested in having this feature in native apps as well I did some checking on the code changes that are made. Most of them look OK but I have to put some question marks at the "webview" flow, the reason mainly being that it would be all to easy for the dev to not open it in an actual webview but to delegate it to the system browser and then redirecting back using a "custom url scheme". This on it's own would not be a problem if those custom schemes would actually require some form of registration at a central provider but unfortunately (or luckily depending on how you look at it) they don't and thus it'd be possible for any app to hijack the redirect and have access to the session and/or session_token. I would suggest making the flow similar to the OAuth2 with PKCE flow where the app needs to generate a challenge + verifier, first it sends the verifier together with the flow request, opens the browser, get's back a code and then exchanges the code together with the challenge for the actual session and session_token. This makes it more flexible, secure and actually easier (since we don't have to decode the redirect url query string or document body) at the cost of a single extra request.

FYI: I'm not a Go dev but I've been doing web and app development for over a decade and been coding for even longer and in that time I've written a lot of code in a lot of different (programming) languages (and read even more in even more (programming) languages) so although I don't know the specifics of Go, I do have a quite broad "general programming" skill-set and can read it well enough to understand what's happening.

[splaunov]

Hi @chancezeus!
Thank you for the feedback.
I agree in that WebView method is known to have some weeknesses.
But it's handy to have it in the tool chest when the task is to migrate legacy apps to kratos. When app's installed base is huge and we can't upgrade it in one go.
So, I would keep it even if there are more advanced/secure methods to be added in the future.

[chancezeus]

Hi @chancezeus! Thank you for the feedback. I agree in that WebView method is known to have some weeknesses. But it's handy to have it in the tool chest when the task is to migrate legacy apps to kratos. When app's installed base is huge and we can't upgrade it in one go. So, I would keep it even if there are more advanced/secure methods to be added in the future.

I'm not saying we should remove it (especially since we'd need it for some flows where a native SDK wouldn't be available or provide an ID token (like Facebook and/or potentially some generic provider)), I'm just suggesting we should use a code exchange like flow with PKCE similar to OAuth2 to make it more secure so that a malicious party is less likely to hijack the session token.

[chancezeus]

Note for previous remark: you wouldn't even need to put anything in the query, since we could use the flows id as the "code" and the challenge/verifiers are generated (and thus known) and send by the frontend anyway, so you could simply say:

1. If WebView is needed -> app MUST send a PKCE verifier together with the submit login/registration request
2. Kratos saves the PKCE verifier in the flow
3. Kratos returns URL
4. App opens URL (either in actual WebView OR in system browser)
5. WebView flow is run and in the end Kratos redirects to the given webview return url with only the flow id in the query
6. Application detects the return url in webview OR gets (re)opened by system browser
7. Application retrieves PKCE challenge from (secure) storage or memory
8. Application submits the flow again but this time it includes the "challenge" instead of the verifier
9. Kratos checks if the flow is in the correct state, i.e. steps 1 to 5 have completed correctly, if not return the correct error (login cancelled, etc) -> app must restart flow from at least step 1
10. Kratos checks the challenge against the verifier
11. If challenge is incorrect -> invalidate flow (potential session hijack) return error
12. Return session + session_token

=> You could/should even generalize this together with the regular browser flow so it wouldn't matter if you're using browser OR api, but that would be a (major) breaking change...

[crabio]

Hi!
It it very common feature for us, because we are using kratos to authenticate users. But we can't add authentication via Social networks in our mobile app without it.
May we help you with it? Or what is blocking current review process?

[jwmay2012]

Does this PR enable me to use Apple Authentication and Google Sign In libraries to get sign in tokens from the mobile device directly and then pass that token to Kratos over API in exchange for a Kratos auth token? Without webview?

I think I'm understanding that the webview implementation is incomplete (missing PKCE), but the support for API login from a mobile device Apple/Google login token is here?

Ultimately, some form of Apple/Google social login from React Native is critical for our switch from Firebase to Ory/Kratos.

[jwmay2012]

Does this PR enable me to use Apple Authentication and Google Sign In libraries to get sign in tokens from the mobile device directly and then pass that token to Kratos over API in exchange for a Kratos auth token? Without webview?

To answer my own question, yes it does, and it works great!

And thanks @splaunov for the recent update from master. (I had done my own master -> feature/oidc-api update a few weeks ago and was going to offer it up this week, but you beat me to it 😄 )

@aeneasr @ory-team This PR passes all unit tests, builds, and functions well for us. We're running our development stacks from this build. Mobile API social logins are a requirement for us to go live with our switch to Ory, which we have operational with this PR.

Are there any remaining barriers to getting this merged to master?
If webview flow isn't considered complete until PKCE, could we split it out from the rest to get API Social flows merged?

We were aiming to make the switch to Ory/Kratos end of January. Would be preferable to use official images instead of our own Kratos build pipeline.

[kszafran]

BTW, looks like there's a newer version now:

Suggested change

	github.com/coreos/go-oidc/v3 v3.1.0
	github.com/coreos/go-oidc/v3 v3.5.0

[kmherrmann]

We are definitely highly interested to make this part of Kratos out of the box - we're just time constrained on our end. We'll give this a review soon!

[MortalKastor]

Hi!

We're using the code from this PR on our backend, and I'm trying to do a Facebook login on our React Native app.

The issue I have is that the Facebook SDK doesn't give me an ID Token, but an Access Token and I can't get the API to accept it (sending it as access_token fails with Authentication failed because no id_token was returned. Please accept the "openid" permission and try again. (I verified openid was part of the scope passed to the SDK)).

Is there a way to do API Facebook login that's known to work, by any chance?

[rivten]

Just for info, we implemented @MortalKastor's feature with the following patch. I thought we should share it with you all :)

diff --git a/selfservice/strategy/oidc/provider_facebook.go b/selfservice/strategy/oidc/provider_facebook.go
index f875e592..fd9f3fe7 100644
--- a/selfservice/strategy/oidc/provider_facebook.go
+++ b/selfservice/strategy/oidc/provider_facebook.go
@@ -126,3 +126,12 @@ func (g *ProviderFacebook) Claims(ctx context.Context, exchange *oauth2.Token, q
 		Birthdate:         user.BirthDay,
 	}, nil
 }
+
+func (g *ProviderFacebook) ClaimsFromIDToken(ctx context.Context, rawIDToken string) (*Claims, error) {
+	claims, err := g.ProviderGenericOIDC.verifyAndDecodeClaimsWithProvider(ctx, g.ProviderGenericOIDC.p, rawIDToken)
+	if err != nil {
+		// some facebook SDK do not provide idToken after oauth, only an access_token so we need to use it instead
+		return g.Claims(ctx, &oauth2.Token{AccessToken: rawIDToken}, nil)
+	}
+	return claims, err
+}

[hperl]

Hi everybody!

First of all a big thank you to @splaunov for this PR. Based on this work, we revisited the flow to get OIDC working on mobile and ultimately settled on an approach implemented here: #3216.

The main difference between that PR and this is that we continue to let Kratos fetch the identity from the OIDC provider, which also means that we can continue using the more secure authorization code grant.

The app gets the session token as follows:

• When initializing a flow, you can request a session token exchange code (only available for app flows)
• After the user logged in, you can exchange the code for a session token

If you are interested, please take a look at the implementation. I welcome any comments :).

And thanks for being part of the Ory Community ❤️

[aptgetgit]

2/ About the method used. I completely understand that passing the id_token from the mobile app to the server is enough for the server to connect/create a user, as the JWT contains enough information. Would it also work to take as input the authorization code, so that Kratos uses its client secret to exchange the code for the oauth2 tokens? (and therefore it could re-use the existing social callback logic). A bit like it is described in this documentation from Google.

Hey @David-Wobrock, i am working on a same design (google sign in on native app), have you had any progress with this ?
If yes, Would you please share how one should proceed with this design where the ouath2 tokens are required in the back end, ie server side access is required .
Or any workaround you figured out for the same.
Thanks

[David-Wobrock]

Not really @aptgetgit, sorry.

But happy to see this feature to be merged into Kratos 👍 Thank you everyone involved ❤️