Add redis-pq-demo #78

Jsyro · 2023-05-18T17:19:48Z

Add a configuration that leverages the redis-persistent-queue plugin.

Signed-off-by: Jason Syrotuck <[email protected]>

usingtechnology

Great work putting this together- already used in rudimentary load testing.

BIggest comment is for documentation. If I handed that readme to a green dev and asked them to stand all of it up, they couldn't. And then not without a lot of questions. We should be providing as much context and as many commands for someone to get up and running without jumping through too many hoops. The fewer things they need to figure out and troubleshoot, the better.

redis-pq-demo/docker-compose.redis.yml

redis-pq-demo/docker-compose.yml

usingtechnology · 2023-05-19T17:39:33Z

redis-pq-demo/docker-compose.yml

+      --admin 0.0.0.0 ${MEDIATOR_AGENT_HTTP_ADMIN_PORT} 
+      --admin-api-key ${MEDIATOR_AGENT_ADMIN_API_KEY} 
+      --endpoint ${MEDIATOR_URL_E} wss://${MEDIATOR_URL_E}
+      --plugin redis_queue.v1_0.events 


i think this is a discussion for acapy + plugins, but the way this is built and then referenced does not match up with the documentation in that redis-queue plugin. there it will say that --plugin redis_queue.v1_0. This works and is great but just pointing out that we need to figure out why it is like this here and if that will be the standard going forward. Really great that you got that image built with the plugin and working - super useful!

the redis_queue plugin has 3 components, so I'm wondering if this is specifically referencing the events component of that plugin, but ignoring the other two.

I'm not sure how this would differ from installing the other two. I'm fairly certain we want the events component and not the inbound (relay) or outbound (deliverer) ones.

redis-pq-demo/requirements.txt

redis-pq-demo/README.md

Signed-off-by: Jason Syrotuck <[email protected]>

redis-pq-demo/.env.sample

redis-pq-demo/README.md

usingtechnology · 2023-05-25T17:43:29Z

redis-pq-demo/docker-compose.yml

+
+  mediator:
+    build:
+      context: .


The build for this works (as in it stands up a mediator with redis_queu), but doesn't use ngrok even if you want it.

I'm working to reinstate ngrok easily. I agree this should be part of the demo.

Signed-off-by: Jason Syrotuck <[email protected]>

jleach · 2023-07-17T23:21:07Z

@Jsyro @usingtechnology Checking in on this one. Still a thing?

usingtechnology · 2023-07-17T23:33:44Z

I'll have to look at it again to see if it addressed the issues/concerns I raised. I'll try to carve out some time tomorrow. @Jsyro can comment on whether he had work outstanding on this.

WadeBarnes · 2023-09-06T17:04:10Z

Quick reminder this PR needs to be reviewed so we can update some mediator deployments and get them off the development fork.

usingtechnology · 2023-09-06T18:59:48Z

Follow up - I did check it on Jul 18 and seemed good, was waiting on @Jsyro to chime in on if his work was complete... didn't hear so fell off my radar. My concerns are alleviated.

I believe he is testing it out today to make sure it works alongside other changes done in the meantime.

esune · 2023-09-07T16:53:34Z

@Jsyro please chime-in as soon as possible on whether we can go ahead and merge/deploy this or further changes are needed.

Signed-off-by: Jason Syrotuck <[email protected]>

Jsyro · 2023-09-07T20:59:26Z

I reused my README and had one issue regarding which dockerfile to use. I also had to modify the scripts to put all containers on the same network for local testing because ngrok is blocked by my corporate policies.

The configuration properly used redis and it can be inspected by the redis ui tool.

usingtechnology · 2023-09-07T21:10:15Z

i suppose i can test it out as is (with ngrok). are you documenting the steps to take if you cannot use ngrok?

usingtechnology · 2023-09-07T21:34:52Z

Dockerfile.ngrok has FROM mediator_pq_acapy. mediator_pq_acapy is never built anywhere.

Signed-off-by: Jason Syrotuck <[email protected]>

Jsyro · 2023-09-07T22:55:21Z

The demo steps are cumbersome, but do work as described. Requiring the tester to have python and pip and install dependencies should be looked at in the future. A Simpler example should be fairly easy to make.

esune · 2023-09-07T22:59:00Z

I just ran the tests with @Jsyro and everything seems to work as expected.

@jleach are yuo able to review/approve/merge this one as well?

Signed-off-by: Jason Syrotuck <[email protected]>

usingtechnology · 2023-09-07T23:03:35Z

The demo steps are cumbersome, but do work as described. Requiring the tester to have python and pip and install dependencies should be looked at in the future. A Simpler example should be fairly easy to make.

I was just about to add a ticket to simplify that. It wasn't written for this purpose so you could just swipe it and put it in this repo as an example (use env var or something for the mediator_url, use random ascii or uuid for the names instead of random-word).

perhaps that is better... move the sample in here? seems fragile to expect that script to remain in aca-py.

anyway, since you added in the redis-ui, you should have a quick blurb about where it is, how to connect and what they should see.

WadeBarnes · 2023-09-08T12:02:28Z

@esune, @usingtechnology, @Jsyro, Let me know when this is ready to go. I can approve and merge.

jleach · 2023-09-08T15:17:32Z

redis-pq-demo/docker-compose.redis.yml

+        acapy_default:
+          ipv4_address: 172.28.0.107
+    depends_on:
+      - redis-node-1


Is this allot a nodes for a local / docker instance?

I believe a redi cluster requires a minimum of 6 nodes.

Signed-off-by: Jason Syrotuck <[email protected]>

Jsyro · 2023-09-08T16:17:00Z

I believe i've addressed the last concern by @usingtechnology about what the redis-ui is there for.

I think this is ready for a final review/merge.

usingtechnology

plugin and plugin config are done in two places (yml file and start.sh) - can we limit it to a single location?

also, there are files (local.yml, mediator-with-plugin.yml, provision.yml) that don't appear to be used, can we remove them?

Signed-off-by: Jason Syrotuck <[email protected]>

Jsyro · 2023-09-08T17:25:07Z

@usingtechnology Good catch. Fixed

usingtechnology

👍 👍

Jsyro added 5 commits May 18, 2023 10:18

add redis-py-demo

055cab4

Signed-off-by: Jason Syrotuck <[email protected]>

ledger is set in playground demo, not required

4916cc3

Signed-off-by: Jason Syrotuck <[email protected]>

network definition and read me. use provided redis cluster

a911c98

Signed-off-by: Jason Syrotuck <[email protected]>

separate two parts

0a87fd6

Signed-off-by: Jason Syrotuck <[email protected]>

move network definition to redis, mediator needs to use external

fc4557d

Signed-off-by: Jason Syrotuck <[email protected]>

usingtechnology suggested changes May 19, 2023

View reviewed changes

Jsyro added 3 commits May 19, 2023 13:17

acapy version change has been merged to main, update reference

ba555b5

Signed-off-by: Jason Syrotuck <[email protected]>

don't need to install packages, redis network syntax

5c11b35

Signed-off-by: Jason Syrotuck <[email protected]>

add ngrok service for caddy

5876577

Signed-off-by: Jason Syrotuck <[email protected]>

Jsyro mentioned this pull request May 23, 2023

Test out, update documentation and enable the use of Redis with ACA-Py deployments openwallet-foundation/acapy#2212

Closed

provide actual commands instead of instructions

5cdb624

Signed-off-by: Jason Syrotuck <[email protected]>

usingtechnology suggested changes May 25, 2023

View reviewed changes

Jsyro added 4 commits May 25, 2023 10:59

readme and instruction updates

3319eaa

Signed-off-by: Jason Syrotuck <[email protected]>

add links and explain script

7a00acb

Signed-off-by: Jason Syrotuck <[email protected]>

start with mediator or mediator-ngrok

40d8883

Signed-off-by: Jason Syrotuck <[email protected]>

revert to in container ngrok management

006c530

Signed-off-by: Jason Syrotuck <[email protected]>

swcurran mentioned this pull request Jul 11, 2023

Add SocketDock to Aries Mediator Service to make HA Mediator #82

Open

WadeBarnes mentioned this pull request Sep 6, 2023

Deploy instances of the Aries Mediator Service with persistent queue support bcgov/DITP-DevOps#113

Closed

22 tasks

add redis-ui, expand container names to avoid conflicts

10a36d0

Signed-off-by: Jason Syrotuck <[email protected]>

Jsyro added 3 commits September 7, 2023 15:37

don't need two dockerfiles

fe0097b

Signed-off-by: Jason Syrotuck <[email protected]>

change file reference to match

4dfa29f

Signed-off-by: Jason Syrotuck <[email protected]>

need .env file

9d2af5d

Signed-off-by: Jason Syrotuck <[email protected]>

complete command

e91fe0a

Signed-off-by: Jason Syrotuck <[email protected]>

jleach previously approved these changes Sep 8, 2023

View reviewed changes

explaination of redis ui

ea0dd54

Signed-off-by: Jason Syrotuck <[email protected]>

Jsyro dismissed jleach’s stale review via ea0dd54 September 8, 2023 16:16

usingtechnology suggested changes Sep 8, 2023

View reviewed changes

remove redudant/unused configs

3070b89

Signed-off-by: Jason Syrotuck <[email protected]>

usingtechnology approved these changes Sep 8, 2023

View reviewed changes

WadeBarnes approved these changes Sep 8, 2023

View reviewed changes

WadeBarnes merged commit f6ed271 into openwallet-foundation:main Sep 8, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add redis-pq-demo #78

Add redis-pq-demo #78

Jsyro commented May 18, 2023

usingtechnology left a comment

usingtechnology May 19, 2023

Jsyro May 23, 2023

usingtechnology May 25, 2023

Jsyro May 26, 2023

jleach commented Jul 17, 2023

usingtechnology commented Jul 17, 2023

WadeBarnes commented Sep 6, 2023

usingtechnology commented Sep 6, 2023

esune commented Sep 7, 2023

Jsyro commented Sep 7, 2023

usingtechnology commented Sep 7, 2023

usingtechnology commented Sep 7, 2023

Jsyro commented Sep 7, 2023

esune commented Sep 7, 2023 •

edited

Loading

usingtechnology commented Sep 7, 2023

WadeBarnes commented Sep 8, 2023

jleach Sep 8, 2023

WadeBarnes Sep 8, 2023

Jsyro commented Sep 8, 2023

usingtechnology left a comment

Jsyro commented Sep 8, 2023

usingtechnology left a comment

Add redis-pq-demo #78

Add redis-pq-demo #78

Conversation

Jsyro commented May 18, 2023

usingtechnology left a comment

Choose a reason for hiding this comment

usingtechnology May 19, 2023

Choose a reason for hiding this comment

Jsyro May 23, 2023

Choose a reason for hiding this comment

usingtechnology May 25, 2023

Choose a reason for hiding this comment

Jsyro May 26, 2023

Choose a reason for hiding this comment

jleach commented Jul 17, 2023

usingtechnology commented Jul 17, 2023

WadeBarnes commented Sep 6, 2023

usingtechnology commented Sep 6, 2023

esune commented Sep 7, 2023

Jsyro commented Sep 7, 2023

usingtechnology commented Sep 7, 2023

usingtechnology commented Sep 7, 2023

Jsyro commented Sep 7, 2023

esune commented Sep 7, 2023 • edited Loading

usingtechnology commented Sep 7, 2023

WadeBarnes commented Sep 8, 2023

jleach Sep 8, 2023

Choose a reason for hiding this comment

WadeBarnes Sep 8, 2023

Choose a reason for hiding this comment

Jsyro commented Sep 8, 2023

usingtechnology left a comment

Choose a reason for hiding this comment

Jsyro commented Sep 8, 2023

usingtechnology left a comment

Choose a reason for hiding this comment

esune commented Sep 7, 2023 •

edited

Loading