fix: supervisor not producing desired output #121
Conversation
| Use the technical information provided by the technical expert to create a compelling product description. | ||
| You are skilled at writing catchy product descriptions making customers to instantly fall in love with the product. | ||
| You always answer why they should buy the product, how it will make their life better, | ||
| and what emotions it will evoke. |
There was a problem hiding this comment.
Agent doesn't know who provides the input, so I rephrased it a bit.
I also moved "requirements" from workflow description to agent, as this is "how" that applies to that one specifically
|
|
||
| Here are some ground rules to follow: | ||
| - Include one sentence summary for each project. | ||
| Research the "https://github.com/trending/typescript" page. |
There was a problem hiding this comment.
Removed "using fire crawl tool", as the how is handled by responsible agent.
| @@ -20,7 +20,7 @@ const { firecrawl } = createFireCrawlTool({ | |||
| const webCrawler = agent({ | |||
| description: ` | |||
| You are skilled at browsing Web pages. | |||
| You are saving the documents to vector store for later usage. | |||
| You can save the documents to Vector store for later usage. | |||
There was a problem hiding this comment.
Changed to expressing capabilities instead of orders
| You can search Vector Store to find relevant informations and create reports based on it | ||
| Based on the information from Vector Store you can compile a comprehensive report. | ||
| You're famous of beautiful Markdown formatting. | ||
| You can create a comprehensive report based on the information from Vector store. |
There was a problem hiding this comment.
Combined lines 42 and 43 into one, as they were very close to each other
| `, | ||
| knowledge: ` | ||
| Behaviour: | ||
| - be professional and kind, |
There was a problem hiding this comment.
Moved this to agent description, as it describes nurse capabilities more than universal knowledge to all agents
| - medications, | ||
| - allergies, | ||
| - surgeries |
There was a problem hiding this comment.
Removed number of things to focus on. Previously, due to how this workflow was structured, it would ask up to 5 questions FOR EACH area, making it hard to complete this without having to answer ~10-15 questions.
Decreasing number of areas of focus and increasing number of questions allows the system to both ask comprehensive set of questions, and also do follow-ups.
| @@ -9,15 +9,15 @@ const personalizedActivityPlanner = agent({ | |||
| description: ` | |||
| You are skilled at creating personalized itineraries that cater to | |||
| the specific preferences and demographics of travelers. | |||
| Your goal is to research and find cool things to do at the destination, | |||
| Your research and find cool things to do at the destination, | |||
There was a problem hiding this comment.
Removed goal as this should go to workflow description.
| @@ -65,7 +64,9 @@ const researchTripWorkflow = workflow({ | |||
| - Hotel location: Main Square, Wrocław | |||
| - Flight information: Flight AA123, arriving on 2023-12-15 | |||
| - How long is the trip: 7 days | |||
| `, | |||
|
|
|||
| Consider flights confirmed. | |||
There was a problem hiding this comment.
To make comprehensive report, it wants to confirm flights, which it doesn't have APIs. So to avoid erroring from time to time, we make this explicit.
There was a problem hiding this comment.
Also, replaced word literary with plan.
|
Also @pkarw I have issues with Github Trending example - content of website too large exceeded context window |
|
Great job @grabbou, thanks for this effort 🙏 I've checked the changes and suggested only one minor change |
This PR started as a quick work to clean-up examples, but it quickly revealed a bug in supervisor that I decided to fix, and test.
Motivation
First of all, let me start with cosmetic changes to examples and motivations behind them. I wanted to make sure the examples use the following rules:
Workflow:
Agent:
I have provided inline comments alongside examples to point out real examples.
Bug
Then, I quickly discovered the bug, that supervisor is often returning
nulland not producing desired output, despite finishing all the steps.I console.logged reasoning and here's what I found:
or
As you can see, supervisor was thinking the workflow was completed when all the steps were finished, returning
null. This is somewhat true, but also, the last and final task should always be to produce the desired output.I slightly changed the workflow request to include generating output as a last step.
Examples
Medical Surevy Output
Surprise Trip
Wikipedia
Ecommerce
Template
supervisor
└─▶ librarian Working on: Analyze the photo of the library and list all the books in the library.
supervisor
└─▶ librarian Working on: functions.visionTool
supervisor
└─▶ librarian Waiting for tools: visionTool
supervisor
└─▶ librarian Processing tool response...
supervisor Looking for next task...
supervisor
└─▶ resourcePlanner Looking for best agent...
supervisor
└─▶ webmaster Working on: Find the best template to use for the website from the available templates in the "/Users/grabbou/Repositories/dead-simple-ai-agent/example/assets" directory.
supervisor
└─▶ webmaster Waiting for tools: listFilesFromDirectory
supervisor
└─▶ webmaster Processing tool response...
supervisor Looking for next task...
supervisor
└─▶ resourcePlanner Looking for best agent...
supervisor
└─▶ webmaster Working on: Copy the template 'book_library_template.html' to '/Users/grabbou/Repositories/dead-simple-ai-agent/example/assets/library.html'.
supervisor
└─▶ webmaster Waiting for tools: readFile
supervisor
└─▶ webmaster Processing tool response...
supervisor
└─▶ webmaster Waiting for tools: saveFile
supervisor
└─▶ webmaster Processing tool response...
supervisor Looking for next task...
supervisor
└─▶ resourcePlanner Looking for best agent...
supervisor
└─▶ webmaster Working on: Replace the content of '/Users/grabbou/Repositories/dead-simple-ai-agent/example/assets/library.html' with the list of books.
supervisor
└─▶ webmaster Waiting for tools: readFile
supervisor
└─▶ webmaster Processing tool response...
supervisor
└─▶ webmaster Working on: Replace placeholder content with actual book list
supervisor
└─▶ webmaster Waiting for tools: saveFile
supervisor
└─▶ webmaster Processing tool response...
supervisor Looking for next task...
supervisor Looking for next task...
The list of books has been successfully incorporated into the HTML template and saved to '/Users/grabbou/Repositories/dead-simple-ai-agent/example/assets/library.html'.