[Docs]: Chain of thought prompting usability isn't documented

[prrao87]

The existing chain of thought prompting method in the docs isn't very concise or readable from a prompt perspective - the user has to type out this long and drawn out string as part of the BAML function.

Instead, it would be simpler to extract out the CoT prompt into a template_string and call that from inside the main function.

Something like this based on my chat on Discord (maybe there's a better way to write it than the example shown?)

template_string CoT() #"
    Before you output the JSON, please explain your
    reasoning step-by-step. Here is an example on how to do this:
    'If we think step by step we can see that ...
     therefore the output JSON is:
    {
      ... the json schema ...
    }'
"#

function MyFunction() {
    ...
    ...
   {{ CoT("..") }}
}

Having some form of abstracted way to do this that separates out the CoT concerns from the rest of the prompt would be nice, and should be documented to help users do CoT prompting more elegantly.

[ansu86d]

I am able to get some basic CoT by adding it as an attribute to the output class

Adding CoT property to Output object

class Receipt { establishment_name string date string @description("ISO8601 formatted date") total int @description("The total amount of the receipt") currency string items Item[] @description("The items on the receipt") chain_of_thought string[] @description("Explain your reasoning step by step") }

Response:

{ "establishment_name": "Brasa Rotisserie", "date": "2024-01-27T20:50:00", "total": 23, "currency": "USD", "items": [ { "name": "Roasted Pork & Over Easy Egg Bowl", "price": 15.25, "quantity": 1 }, { "name": "Indeed Mex Honey Lite", "price": 6.5, "quantity": 1 } ], "chain_of_thought": [ "The receipt is from Brasa Rotisserie.", "The date and time on the receipt is 1/27/24 at 8:50 PM, which translates to the ISO8601 format as 2024-01-27T20:50:00.", "The total amount is listed as $23.88. USD is inferred as the currency because the establishment is located in Minnesota, USA.", "Two items are listed: 'Roasted Pork & Over Easy Egg Bowl' priced at $15.25 and 'Indeed Mex Honey Lite' priced at $6.50. No additional quantity information is provided, so quantity is assumed to be 1 by default.", "Subtotal is correctly calculated as $21.75, which matches the sum of the item prices (15.25 + 6.50).", "The tax is $2.13, making the total $23.88." ] }

Prompt Fiddle : https://www.promptfiddle.com/BAML-Examples-RsZnY

This might not be an elegant solution by glueing it to the output object , but the fact that just adding a new property with the description with no changes to the 'prompt' section highlights the ability to extract reasoning in a structured manner

@prrao87 @hellovai @aaronvg

[prrao87]

This is great, thanks a ton!