feat: add event dispatcher #2908

tanns2 · 2023-04-22T11:00:07Z

Background

The output of AutoGPT is logged to the console and prompt. There is no possibility to directly interact with the output. The aim of the event dispatcher is to provide the possibility for sending the events to a defined endpoint. The endpoint can be any microservice or 3rd party application (which then can do further transformations, e.g. enterprise bus). The event dispatcher fires and forgets all data that is put to the console. It can be seen as the enabler for further integration in a more complex application landscape.
Note: As this is a data forwarding, a plugin does not make sense as it does not have any effect on the running instance of AutoGPT.

Changes

The change include a configuration and the event dispatcher itself. The configuration contains all information for the endpoint (protocol, port, host, endpoint) while the event dispatcher just sends the event to the defined endpoint over HTTP POST. The feature is attached to the logs.py typewriter_log method. It forwards the data if the event dispatcher is enabled. The event dispatcher catches every exception that can occur when trying to send the request. It sends the data asynchronosly over a new thread. In case of error and if it is not muted, the error is printed to the console. It never blocks the execution and flow of AutoGPT.

Documentation

The readme.md contains the configuration documentation. Inside the environments, there is a sample config as well. Further, the code is clean and also documented.

Test Plan

Tested with following scenarios:

nothing set up -> nothing is received at endpoint (simple separated HTTP server script)
set up, not enabled -> nothing is received at endpoint (simple separated HTTP server script)
set up, enabled, muted -> HTTP script receives events
set up, no HTTP server running, muted-> AutoGPT works normally, no error output
set up, no HTTP server running, not muted -> error output in the console
set up, enabled, wrong configuration, muted -> AutoGPT works normally, no error output
set up, enabled, wrong configuration, not muted -> error output in the console

PR Quality Checklist

My pull request is atomic and focuses on a single change.
I have thoroughly tested my changes with multiple different prompts.
I have considered potential risks and mitigations for my changes.
I have documented my changes clearly and comprehensively.
I have not snuck in any "extra" small tweaks changes

README.md

ntindle · 2023-04-22T17:13:29Z

autogpt/config/config.py

+        self.event_dispatcher_enabled = os.getenv("EVENT_DISPATCHER_ENABLED", "False")
+        self.event_dispatcher_protocol = os.getenv("EVENT_DISPATCHER_PROTOCOL")
+        self.event_dispatcher_host = os.getenv(
+            "EVENT_DISPATCHER_HOST",
+        )
+        self.event_dispatcher_endpoint = os.getenv("EVENT_DISPATCHER_ENDPOINT")
+        self.event_dispatcher_port = os.getenv("EVENT_DISPATCHER_PORT")
+        self.event_dispatcher_mute = os.getenv("EVENT_DISPATCHER_MUTE", "True")
+


Are defaults needed for the rest of these

ntindle · 2023-04-22T17:14:18Z

autogpt/event_dispatcher.py

+        except Exception as e:  # ensure catching all exceptions
+            if CFG.event_dispatcher_mute == "False":
+                print(f"Error: {e}")


Use the existing debug flag, don't have your own

changed to debug flag

autogpt/event_dispatcher.py

codecov · 2023-04-22T17:17:00Z

Codecov Report

Patch coverage: 66.66% and project coverage change: -9.65 ⚠️

Comparison is base (45f2513) 49.63% compared to head (3529a1c) 39.99%.

❗ Current head 3529a1c differs from pull request most recent head 76dae0e. Consider uploading reports for the commit 76dae0e to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2908      +/-   ##
==========================================
- Coverage   49.63%   39.99%   -9.65%     
==========================================
  Files          64       61       -3     
  Lines        3022     2923      -99     
  Branches      505      479      -26     
==========================================
- Hits         1500     1169     -331     
- Misses       1402     1678     +276     
+ Partials      120       76      -44

Impacted Files	Coverage Δ
autogpt/logs.py	`46.63% <33.33%> (-11.27%)`	⬇️
autogpt/event_dispatcher.py	`64.00% <64.00%> (ø)`
autogpt/config/config.py	`74.83% <100.00%> (-1.33%)`	⬇️

... and 33 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

tanns2 · 2023-04-23T09:32:38Z

@ntindle thanks for your input, I've added your requested changes.

github-actions · 2023-04-24T13:43:31Z

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

github-actions · 2023-04-24T18:12:33Z

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

github-actions · 2023-04-24T22:38:46Z

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

ntindle · 2023-04-25T04:47:23Z

fix merge conflicts and its gtg

codecov · 2023-04-25T04:50:04Z

Codecov Report

Patch coverage: 66.66% and project coverage change: -18.22 ⚠️

Comparison is base (aab79fd) 60.11% compared to head (76dae0e) 41.89%.

❗ Current head 76dae0e differs from pull request most recent head 81f6de7. Consider uploading reports for the commit 81f6de7 to get more accurate results

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #2908       +/-   ##
===========================================
- Coverage   60.11%   41.89%   -18.22%     
===========================================
  Files          69       65        -4     
  Lines        3091     3055       -36     
  Branches      509      509               
===========================================
- Hits         1858     1280      -578     
- Misses       1103     1710      +607     
+ Partials      130       65       -65

Impacted Files	Coverage Δ
autogpt/logs.py	`38.86% <33.33%> (-41.44%)`	⬇️
autogpt/event_dispatcher.py	`64.00% <64.00%> (ø)`
autogpt/config/config.py	`75.64% <100.00%> (-1.19%)`	⬇️

... and 53 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

github-actions · 2023-04-25T06:00:56Z

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

tanns2 · 2023-04-25T06:19:21Z

Hi @ntindle thanks a lot! Fixed, rebased, retested. Looking forward for the merging.

ntindle · 2023-04-25T06:38:18Z

autogpt/event_dispatcher.py

+                pass  # do nothing with the response
+        except Exception as e:  # ensure catching all exceptions
+            if CFG.debug_mode:
+                print(f"[Event Dispatcher] Error: {e}")


Use shared logging library for logging

tanns2 · 2023-04-25T13:45:32Z

@ntindle good point, thanks. Changed, retested and ready for review.

github-actions · 2023-04-26T18:30:14Z

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

vercel · 2023-04-26T18:59:38Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Apr 30, 2023 10:46am

github-actions · 2023-04-26T18:59:47Z

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

github-actions · 2023-04-26T20:23:26Z

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

github-actions · 2023-04-26T20:57:29Z

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

tanns2 · 2023-04-30T10:48:01Z

Hi @ntindle or other maintainers,
I've rebased and pushed again everything to ensure the branch is up to date with a clean commit.
Looking forward for this PR to be merged.

github-actions · 2023-05-01T12:09:37Z

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

p-i- · 2023-05-05T00:55:06Z

This is a mass message from the AutoGPT core team.
Our apologies for the ongoing delay in processing PRs.
This is because we are re-architecting the AutoGPT core!

For more details (and for infor on joining our Discord), please refer to:
https://github.com/Significant-Gravitas/Auto-GPT/wiki/Architecting

Pwuts · 2023-06-14T16:43:36Z

@ntindle bump, still wanna merge this? Will need a bit of conflict resolution but nothing too bad.

Pwuts · 2023-09-08T13:32:25Z

Two remarks:

The custom Logger class has been replaced by the native logging module. The same thing can now be achieved by implementing a custom Handler, just like we did with the TTSHandler.
The proposed functionality should be covered by the Agent Protocol afaik, for which support should be added soon. :)

Pwuts · 2023-09-08T22:13:32Z

Closing this in favor of Agent Protocol implementation:

Support Agent Protocol #5178

github-actions bot added the size/l label Apr 22, 2023

ntindle requested changes Apr 22, 2023

View reviewed changes

ntindle added the enhancement New feature or request label Apr 22, 2023

tanns2 requested a review from ntindle April 23, 2023 09:32

github-actions bot added the conflicts Automatically applied to PRs with merge conflicts label Apr 24, 2023

github-actions bot removed the conflicts Automatically applied to PRs with merge conflicts label Apr 24, 2023

github-actions bot added the conflicts Automatically applied to PRs with merge conflicts label Apr 24, 2023

ntindle previously approved these changes Apr 25, 2023

View reviewed changes

tanns2 dismissed ntindle’s stale review via 7c9d623 April 25, 2023 06:00

tanns2 force-pushed the add-event-dispatcher branch from 76dae0e to 7c9d623 Compare April 25, 2023 06:00

github-actions bot removed the conflicts Automatically applied to PRs with merge conflicts label Apr 25, 2023

tanns2 force-pushed the add-event-dispatcher branch from 7c9d623 to a22ed12 Compare April 25, 2023 06:12

tanns2 requested a review from ntindle April 25, 2023 06:19

ntindle requested changes Apr 25, 2023

View reviewed changes

tanns2 requested a review from ntindle April 25, 2023 13:45

github-actions bot added the conflicts Automatically applied to PRs with merge conflicts label Apr 26, 2023

github-actions bot removed the conflicts Automatically applied to PRs with merge conflicts label Apr 26, 2023

vercel bot temporarily deployed to Preview April 26, 2023 19:00 Inactive

github-actions bot added the conflicts Automatically applied to PRs with merge conflicts label Apr 26, 2023

github-actions bot removed the conflicts Automatically applied to PRs with merge conflicts label Apr 26, 2023

vercel bot temporarily deployed to Preview April 27, 2023 07:06 Inactive

feat: add event dispatcher

81f6de7

tanns2 force-pushed the add-event-dispatcher branch from e158836 to 81f6de7 Compare April 30, 2023 10:45

vercel bot temporarily deployed to Preview April 30, 2023 10:46 Inactive

github-actions bot added the conflicts Automatically applied to PRs with merge conflicts label May 1, 2023

Pwuts closed this Sep 8, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add event dispatcher #2908

feat: add event dispatcher #2908

tanns2 commented Apr 22, 2023

ntindle Apr 22, 2023

tanns2 Apr 23, 2023

ntindle Apr 22, 2023

tanns2 Apr 23, 2023

codecov bot commented Apr 22, 2023 •

edited

Loading

tanns2 commented Apr 23, 2023 •

edited

Loading

github-actions bot commented Apr 24, 2023

github-actions bot commented Apr 24, 2023

github-actions bot commented Apr 24, 2023

ntindle commented Apr 25, 2023

codecov bot commented Apr 25, 2023 •

edited

Loading

github-actions bot commented Apr 25, 2023

tanns2 commented Apr 25, 2023

ntindle Apr 25, 2023

tanns2 commented Apr 25, 2023 •

edited

Loading

github-actions bot commented Apr 26, 2023

vercel bot commented Apr 26, 2023 •

edited

Loading

github-actions bot commented Apr 26, 2023

github-actions bot commented Apr 26, 2023

github-actions bot commented Apr 26, 2023

tanns2 commented Apr 30, 2023

github-actions bot commented May 1, 2023

p-i- commented May 5, 2023

Pwuts commented Jun 14, 2023

Pwuts commented Sep 8, 2023

Pwuts commented Sep 8, 2023

feat: add event dispatcher #2908

feat: add event dispatcher #2908

Conversation

tanns2 commented Apr 22, 2023

Background

Changes

Documentation

Test Plan

PR Quality Checklist

ntindle Apr 22, 2023

Choose a reason for hiding this comment

tanns2 Apr 23, 2023

Choose a reason for hiding this comment

ntindle Apr 22, 2023

Choose a reason for hiding this comment

tanns2 Apr 23, 2023

Choose a reason for hiding this comment

codecov bot commented Apr 22, 2023 • edited Loading

Codecov Report

tanns2 commented Apr 23, 2023 • edited Loading

github-actions bot commented Apr 24, 2023

github-actions bot commented Apr 24, 2023

github-actions bot commented Apr 24, 2023

ntindle commented Apr 25, 2023

codecov bot commented Apr 25, 2023 • edited Loading

Codecov Report

github-actions bot commented Apr 25, 2023

tanns2 commented Apr 25, 2023

ntindle Apr 25, 2023

Choose a reason for hiding this comment

tanns2 commented Apr 25, 2023 • edited Loading

github-actions bot commented Apr 26, 2023

vercel bot commented Apr 26, 2023 • edited Loading

github-actions bot commented Apr 26, 2023

github-actions bot commented Apr 26, 2023

github-actions bot commented Apr 26, 2023

tanns2 commented Apr 30, 2023

github-actions bot commented May 1, 2023

p-i- commented May 5, 2023

Pwuts commented Jun 14, 2023

Pwuts commented Sep 8, 2023

Pwuts commented Sep 8, 2023

codecov bot commented Apr 22, 2023 •

edited

Loading

tanns2 commented Apr 23, 2023 •

edited

Loading

codecov bot commented Apr 25, 2023 •

edited

Loading

tanns2 commented Apr 25, 2023 •

edited

Loading

vercel bot commented Apr 26, 2023 •

edited

Loading