Support multiturn history

[chenmoneygithub]

We are letting users pass in the convesation history in DSPy program with this PR. From the LLM's viewpoint, conversation history is appended after the few-shot examples, please see code demo and example LLM history below.

Please note that DSPy doesn't help capture the conversation history, but you can easily capture it with custom code (as reflected in the code demo). Will write a tutorial in a followup PR.

import logging

import dspy
from dspy.datasets.gsm8k import GSM8K, gsm8k_metric

# Set up the LM.
dspy.settings.configure(lm=dspy.LM("openai/gpt-4o-mini"))


# Load math questions from the GSM8K dataset.
gsm8k = GSM8K()
gsm8k_trainset, gsm8k_devset = gsm8k.train[:10], gsm8k.dev[:10]


class CoT(dspy.Module):
    def __init__(self):
        super().__init__()
        self.prog = dspy.ChainOfThought("question -> answer")

    def forward(self, question, **kwargs):
        return self.prog(question=question, **kwargs)


cot = CoT()

# Set up the optimizer: we want to "bootstrap" (i.e., self-generate) 4-shot examples of our CoT program.
config = dict(max_bootstrapped_demos=4, max_labeled_demos=4, max_rounds=5)
optimizer = dspy.BootstrapFewShot(metric=gsm8k_metric, **config)
optimized_cot = optimizer.compile(CoT(), trainset=gsm8k_trainset)


outputs = optimized_cot(question="Seriously though, what is 2 + 2?")
history = [{"question": "Seriously though, what is 2 + 2?", **outputs}]
print(ptimized_cot(question="Seriously though, what is 2 + 2?", conversation_history=history))

Running dspy.inspect_history(n=1):

[2025-02-25T13:44:37.986061]

System message:

Your input fields are:
1. `question` (str)

Your output fields are:
1. `reasoning` (str)
2. `answer` (str)

All interactions will be structured in the following way, with the appropriate values filled in.

[[ ## question ## ]]
{question}

[[ ## reasoning ## ]]
{reasoning}

[[ ## answer ## ]]
{answer}

[[ ## completed ## ]]

In adhering to this structure, your objective is: 
        Given the fields `question`, produce the fields `answer`.


User message:

[[ ## question ## ]]
The result from the 40-item Statistics exam Marion and Ella took already came out. Ella got 4 incorrect answers while Marion got 6 more than half the score of Ella. What is Marion's score?

Respond with the corresponding output fields, starting with the field `[[ ## reasoning ## ]]`, then `[[ ## answer ## ]]`, and then ending with the marker for `[[ ## completed ## ]]`.


Assistant message:

[[ ## reasoning ## ]]
Ella got 4 incorrect answers on a 40-item exam, which means she got 40 - 4 = 36 correct answers. Marion's score is 6 more than half of Ella's score. Half of Ella's score is 36 / 2 = 18. Therefore, Marion's score is 18 + 6 = 24.

[[ ## answer ## ]]
24

[[ ## completed ## ]]


{truncated_few_shot_examples}


User message:

[[ ## question ## ]]
Seriously though, what is 2 + 2?

Respond with the corresponding output fields, starting with the field `[[ ## reasoning ## ]]`, then `[[ ## answer ## ]]`, and then ending with the marker for `[[ ## completed ## ]]`.


Assistant message:

[[ ## reasoning ## ]]
The question asks for the sum of 2 and 2. To find this, we simply add the two numbers together: 2 + 2 = 4.

[[ ## answer ## ]]
4

[[ ## completed ## ]]


User message:

[[ ## question ## ]]
are you sure that's correct?

Respond with the corresponding output fields, starting with the field `[[ ## reasoning ## ]]`, then `[[ ## answer ## ]]`, and then ending with the marker for `[[ ## completed ## ]]`.


Response:

[[ ## reasoning ## ]]
Yes, I am sure that the calculation is correct. The sum of 2 and 2 is a basic arithmetic operation. When you add 2 + 2, the result is indeed 4. This is a fundamental mathematical fact.

[[ ## answer ## ]]
Yes, the answer is correct: 2 + 2 = 4.

[[ ## completed ## ]]