Support sending emails through CRM transactional email

[obukhov]

Is your feature request related to a problem? Please describe.

The current implementation only supports sending emails through SMTP using templates bundled with kratos. It would be nice to be able to delegate templating and sending the email to an external service with API call.

Describe the solution you'd like

First of all, there should be a configuration parameter to change strategy between built-in templating and sending via SMTP to an API call. The question is how to implement customization - there are a lot of systems/services out there with different APIs. I was thinking of introducing an intermediate service translating kratos events to a target system API. The communication between kratos and "adapter" service can be implemented in several different ways:

• webhooks - HTTP calls with JSON payload standardized and documented
• gRPC call - similar to the one above but more performant
• queueing a message - but then again there are a lot of queueing systems: Kafka, AMPQ, ...

Not sure what would be the best in this case but in any case, it's possible to also provide some boilerplate project to start with to implement a specific system adapter.

Describe alternatives you've considered

The only possibility I see is to mock the SMTP server as with MailSlurper but then you would also need to pack and unpack event data into the message body that seems to be a nasty workaround.

Additional context

If you can provide some guidance (what you think in general about the idea and which of described options to pick) I may try to prepare a PR with it. I've looked in Sender and Courier services I think the solution can be using sender strategy there.

[zepatrik]

It would make sense to then externalize the current SMTP and templating solution as well. This is kind of required anyway (#1024), and having this intermediate API would make it possible to plug in other systems. What do you think @aeneasr ? We can only support the SMTP+template solution officially and then the community can write adapters as needed.

[aeneasr]

Yes, thank you for writing down this issue! So generally we target services like SparkPost or Mailgun which have SMTP APIs with the current implementation. This does not allow using their template language however - it would need Mailgun/SparkPost specific API calls for that.

Are you planning on using a CRM to handle your verification and recovery mails? Or do you want to generally sync your users with your CRM?

[obukhov]

Hi @aeneasr. Yeah sorry for a bit misleading description. I've meant any email delivery system that allows you to send data (trigger name and set of key-value parameters) and then manage to templates and actual delivery. This can be either services like Mailgun or Sendgrid, but can also be more sophisticated CRM systems. This wasn't about CRM users sync though (also an important but different subject).

I like @zepatrik's idea about also extracting current built-in SMTP handling to a separate service making it possible to replace it with another solution. What do you think? What would be the best way to communicate between those services in your opinion?

[aeneasr]

I like @zepatrik's idea about also extracting current built-in SMTP handling to a separate service making it possible to replace it with another solution. What do you think? What would be the best way to communicate between those services in your opinion?

We will need some type of integration with e.g. SMS providers anyways, so I think it is a possibility. It might make sense to have several strategies in the courier:

• smtp (current)
• twilio (sms)
• json-rpc (any other)

I'm not sure how exactly we would set it up in the config itself. Maybe something like:

courier:
  smtp_url: ...
  rpc_url: ...
  twilio_url: ...
  delivery:
    email:
      smtp_url: ... # OR
      rpc_url: ...
    sms:
      twilio_url: ... # OR
      rpc_url: ...

What do you think?

The problem with rpc is that it will probably require authentication (so how do we get that in there?), custom headers (e.g. setting the Host in the request), and other customizations. Generic RPC is always a bit of a pain.

[zepatrik]

My idea was rather to define an rpc interface that you will have to implement in some kind of intermediary application. It can then forward messages to the upstream API (smtp, mailgun, twilio, matrix, deutsche post, ...) and apply some kind of templating in between. The current smtp courier would be just one of the official couriers, but anyone can write a new one by simply implementing the rpc interface.

[robertlagrant]

Random info: Auth0 solve this sort of problem in their Rules section by having a generic NodeJS you can write code for, and you can add in secret variables to refer to in the code.

[aeneasr]

Yes, that's basically an RPC call to their https://webtask.io infrastructure. I think it has a lot of drawbacks as it is not possible to test, lint or use autocompletion. A generic RPC call could be routed to your TypeScript, Go, Java, ... app which makes all of this much easier as you could even get some SDK in place. Maybe we'll also support GitHub codespace at some point which I think would be terrific!

[github-actions]

I am marking this issue as stale as it has not received any engagement from the community or maintainers in over half a year. That does not imply that the issue has no merit! If you feel strongly about this issue

• open a PR referencing and resolving the issue;
• leave a comment on it and discuss ideas how you could contribute towards resolving it;
• open a new issue with updated details and a plan on resolving the issue.

We are cleaning up issues every now and then, primarily to keep the 4000+ issues in our backlog in check and to prevent maintainer burnout. Burnout in open source maintainership is a widespread and serious issue. It can lead to severe personal and health issues as well as enabling catastrophic attack vectors.

Thank you for your understanding and to anyone who participated in the issue! 🙏✌️

If you feel strongly about this issues and have ideas on resolving it, please comment. Otherwise it will be closed in 30 days!

[aeneasr]

Reopening this

[barnarddt]

Hi Everyone
I created a branch to implement this, please have a look and point anything out before I create a PR.
master...barnarddt:kratos:master
The basic outline is:

• Create a mailer object inside of courier
• Check config if mailer is enabled when dispatching an email
• Have mailer load the same EmailTemplate as the smtp implementation
• Marshall the EmailTemplate values as TemplateValues for mailer
• Have the Jsonnet marshall the TemplateValues as required for the HTTP implementation
• Call the HTTP endpoint as configured in the courier.mailer.request_config exact same codepath as sms

[LowLevelSubmarine]

Thank you for doing the work.
Correct me if i'm wrong, but the current implementation only allows delegating message sending, not templating right?

[barnarddt]

Yeah, so the idea would be to post the template data required to another provider (sendgrid or some CRM) that has all the templating and sending capabilities and just needs the values to plug in

[LowLevelSubmarine]

Sounds like a perfect solution to me

[supercairos]

That sounds cool

We today use a fake SMTP email server (using TS lib: https://www.npmjs.com/package/smtp-server) that then redispatch the Email via an API call to our CRM.

Having an Api would be much more handier than this logic.

[iotanbo]

I've recently set up an email sender with Cloudflare and MailChannels, a simple solution that works well for me.
I'm really looking forward to the Email via an API feature to be implemented.

[aeneasr]

Resolved on master