Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add exmaple of gpt-4o vision #1104

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions examples/openai-gpt4o-vision-example/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# OpenAI GPT-4o Vision Example

Welcome to this cheerful example of using OpenAI's GPT-4o vision model with Go! 🎉

This project demonstrates how to interact with OpenAI's powerful language model using the `langchaingo` library. It's a fantastic way to explore the capabilities of AI in your Go applications!

## What This Example Does

This example does some really cool stuff:

1. 🤖 It sets up a connection to OpenAI's GPT-4o model.
2. 💬 It prepares a conversation with the AI, telling it to act as a Bird expert.
3. 🗨️ It asks the AI a question about a what bird is it?
4. 🌊 It streams the AI's response in real-time, printing it to the console.

## How It Works

The magic happens in the `main()` function:

1. We create a new OpenAI client, specifically requesting the "gpt-4o" model.
2. We set up our conversation content, including a system message and a user question and image url.
3. We generate content from the AI, using a streaming function to print the response as it's received.

## Running the Example

To run this example, you'll need to:

1. Make sure you have Go installed on your system.
2. Set up your OpenAI API credentials (usually as environment variables).
3. Run the Go program!

## Have Fun!

This example is a great starting point for building more complex applications with AI. Feel free to modify the question, add more context, or expand on the functionality. The possibilities are endless! 🚀

Happy coding, and enjoy exploring the world of AI with Go! 😄
13 changes: 13 additions & 0 deletions examples/openai-gpt4o-vision-example/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
module github.com/tmc/langchaingo/examples/openai-gpt4o-example

go 1.22.0

toolchain go1.22.1

require github.com/tmc/langchaingo v0.1.13-pre.0

require (
github.com/dlclark/regexp2 v1.10.0 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/pkoukk/tiktoken-go v0.1.6 // indirect
)
22 changes: 22 additions & 0 deletions examples/openai-gpt4o-vision-example/go.sum
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dlclark/regexp2 v1.10.0 h1:+/GIL799phkJqYW+3YbOd8LCcbHzT0Pbo8zl70MHsq0=
github.com/dlclark/regexp2 v1.10.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/pkoukk/tiktoken-go v0.1.6 h1:JF0TlJzhTbrI30wCvFuiw6FzP2+/bR+FIxUdgEAcUsw=
github.com/pkoukk/tiktoken-go v0.1.6/go.mod h1:9NiV+i9mJKGj1rYOT+njbv+ZwA/zJxYdewGl6qVatpg=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/tmc/langchaingo v0.1.13-pre.0 h1:bmeNREQX433Ys4gggx5AYnJxP/tZX7/vTTMAZbMnbeQ=
github.com/tmc/langchaingo v0.1.13-pre.0/go.mod h1:EeervIv/DNYhSfQSMaql20wMFvhgF7lDaVaatp8lVPw=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
sigs.k8s.io/yaml v1.3.0 h1:a2VclLzOGrwOHDiV8EfBGhvjHvP46CtW5j6POvhYGGo=
sigs.k8s.io/yaml v1.3.0/go.mod h1:GeOyir5tyXNByN85N/dRIT9es5UQNerPYEKK56eTBm8=
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
package main

import (
"context"
"fmt"
"log"

"github.com/tmc/langchaingo/llms"
"github.com/tmc/langchaingo/llms/openai"
)

func main() {
llm, err := openai.New(openai.WithModel("gpt-4o"))
if err != nil {
log.Fatal(err)
}
ctx := context.Background()

content := []llms.MessageContent{
llms.TextParts(llms.ChatMessageTypeSystem, "You are a Birds expert"),
{
Role: llms.ChatMessageTypeHuman,
Parts: []llms.ContentPart{
llms.ImageURLContent{
URL: "https://upload.wikimedia.org/wikipedia/commons/3/39/Brown-Falcon%2C-Vic%2C-3.1.2008.jpg",
// OpenAI GPT Vision API detail parameter explanation:
// - Controls the fidelity of image understanding: "low," "high," or "auto".
// - "auto" (default): Chooses between "low" and "high" based on input image size.
// - "low": Processes a 512x512 low-res image with 85 tokens for faster responses and fewer input tokens.
// - "high": Analyzes the low-res image (85 tokens) and adds detailed crops (170 tokens per tile) for higher fidelity.
Detail: "auto",
},
llms.TextPart("what bird is it?"),
},
},
}

completion, err := llm.GenerateContent(ctx, content, llms.WithStreamingFunc(func(ctx context.Context, chunk []byte) error {
fmt.Print(string(chunk))
return nil
}))
if err != nil {
log.Fatal(err)
}
_ = completion
}
Loading