-
Notifications
You must be signed in to change notification settings - Fork 46
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(llm-observability): metric and feedback methods #365
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
This PR adds new LLM observability methods to capture metrics and user feedback across all PostHog JS Lite sub-libraries, with appropriate version bumps following semantic versioning.
- Added
captureTraceFeedback
method inposthog-core/src/index.ts
to track user feedback with$ai_feedback
events - Added
captureTraceMetric
method inposthog-core/src/index.ts
to track metrics with$ai_metric
events - Implemented automatic string conversion for numeric/boolean values in both methods for consistent property types
- Added comprehensive test coverage in
posthog-core/test/posthog.ai.spec.ts
verifying feedback/metric capture and type conversions - Bumped minor versions across all sub-libraries (posthog-node: 4.5.0, posthog-react-native: 3.7.0, posthog-web: 3.3.0)
5 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile
Size Change: +539 B (+0.41%) Total Size: 131 kB
ℹ️ View Unchanged
|
missing changelogs and the versioning might be outdated already, but otherwise LGTM |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
left a comment before merging, LGTM
739a0e4
to
3c44aeb
Compare
Problem
It should be straightforward to capture a LLM metric or feedback.
Changes
@k11kirky please check the naming. I thought about
captureAIFeedback
, but I don't like three capital letters in a row.captureFeedback
andcaptureMetric
are probably too generic.Release info Sub-libraries affected
Bump level
Libraries affected
Changelog notes
captureTraceMetric
andcaptureTraceFeedback
methods for LLM Observability.