Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: added langchains LLM #66

Merged
merged 22 commits into from
Jul 20, 2023
Merged

Conversation

jjmachan
Copy link
Member

@jjmachan jjmachan commented Jul 19, 2023

fixes #46
fixes #53

You can now use langchains LLM abstraction to access all the LLM endpoints langchain supports.

eg

from langchain.chat_models import ChatOpenAI

gpt4 = ChatOpenAI(model_name="gpt-4")
gpt4.generate_prompt(prompts=[prompts])

# init a new Metric with llm
cr = ContextRelevancy(llm=gpt4)
cr.init_model()
result = cr.score(ds.select(range(4)))

result["context_relavency"]
# [0.46687018871307373, 0.1532887363433838,0.17359847468989234, 0.17340516530234237]

We're also now using OpenAI's chat models as default which brings a 10x decrease in cost. also using gpt-3.5-turbo-16k as default for even bigger context size

@jjmachan jjmachan temporarily deployed to unit_tests July 19, 2023 16:51 — with GitHub Actions Inactive
@jjmachan jjmachan temporarily deployed to unit_tests July 19, 2023 16:51 — with GitHub Actions Inactive
@jjmachan jjmachan temporarily deployed to unit_tests July 19, 2023 16:51 — with GitHub Actions Inactive
@jjmachan jjmachan temporarily deployed to unit_tests July 19, 2023 16:51 — with GitHub Actions Inactive
@jjmachan jjmachan temporarily deployed to unit_tests July 19, 2023 16:51 — with GitHub Actions Inactive
@jjmachan jjmachan temporarily deployed to unit_tests July 19, 2023 16:51 — with GitHub Actions Inactive
@jjmachan jjmachan temporarily deployed to unit_tests July 19, 2023 16:51 — with GitHub Actions Inactive
@jjmachan jjmachan temporarily deployed to unit_tests July 19, 2023 16:51 — with GitHub Actions Inactive
@jjmachan jjmachan temporarily deployed to unit_tests July 19, 2023 16:51 — with GitHub Actions Inactive
@jjmachan jjmachan temporarily deployed to unit_tests July 19, 2023 16:51 — with GitHub Actions Inactive
@jjmachan jjmachan temporarily deployed to unit_tests July 19, 2023 17:27 — with GitHub Actions Inactive
@shahules786 shahules786 merged commit b5770f0 into explodinggradients:main Jul 20, 2023
@jjmachan jjmachan deleted the feat/llm branch July 20, 2023 09:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

InvalidRequestError when context size to too big Expose LLM
2 participants