Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Google Pubsub integration #500

Open
MFAshby opened this issue Apr 7, 2022 · 1 comment
Open

Google Pubsub integration #500

MFAshby opened this issue Apr 7, 2022 · 1 comment

Comments

@MFAshby
Copy link

MFAshby commented Apr 7, 2022

Problem I need to solve

I want to receive change notifications for data in aidbox via Google Pubsub. Using a pubsub system allows us to build decoupled systems which scale better. Using google pubsub reduces operations burden for our company as we don't have to host our own (e.g. kafka or something)

Solution I see

Configuration option in aidbox to publish change notifications for either all resources or a subset to a google pubsub topic. Change notification should contain at least the operation (create, update, delete) and the self-link of the resource (e.g. /fhir/Patient/123, or same but with the full URL)

Alternatives I've considered, but they don't work

Existing changes API allows for polling
[1] but this requires additional glue code to feed that data into a messaging system.

Existing REST hooks API [2] may be configured to push to google pubsub via it's REST API [3] but I can't see an obvious way to transform to the required format for google pubsub, and authentication seems difficult to configure.

Additional context

[1] https://docs.aidbox.app/api-1/reactive-api-and-subscriptions/usdsnapshot-usdwatch-and-usdversions-api
[2] https://docs.aidbox.app/api-1/reactive-api-and-subscriptions/subscriptions-1
[3] https://cloud.google.com/pubsub/docs/reference/rest/v1/projects.topics/publish

@MFAshby
Copy link
Author

MFAshby commented May 23, 2022

Additional to this; it would be really good to consider message ordering when integrating with pubsub. For our use case, it's useful to have resources created in the right order for referential integrity; i.e. our application should receive newly created Organization resources before newly created Practitioner resources, because our application code can be significantly simpler as a result (it can assume the Organization exists)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant