Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(llm-observability): metric and feedback methods #365

Merged
merged 5 commits into from
Feb 20, 2025

Conversation

skoob13
Copy link
Contributor

@skoob13 skoob13 commented Jan 31, 2025

Problem

It should be straightforward to capture a LLM metric or feedback.

Changes

  • Add methods to capture user feedback and metrics.
  • Convert values like numbers and booleans to strings to unify the property type.

@k11kirky please check the naming. I thought about captureAIFeedback, but I don't like three capital letters in a row. captureFeedback and captureMetric are probably too generic.

Release info Sub-libraries affected

Bump level

  • Major
  • Minor
  • Patch

Libraries affected

  • All of them
  • posthog-web
  • posthog-node
  • posthog-react-native

Changelog notes

  • Added captureTraceMetric and captureTraceFeedback methods for LLM Observability.

@skoob13 skoob13 requested review from Twixes and k11kirky January 31, 2025 14:51
@skoob13 skoob13 requested a review from marandaneto as a code owner January 31, 2025 14:51
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

This PR adds new LLM observability methods to capture metrics and user feedback across all PostHog JS Lite sub-libraries, with appropriate version bumps following semantic versioning.

  • Added captureTraceFeedback method in posthog-core/src/index.ts to track user feedback with $ai_feedback events
  • Added captureTraceMetric method in posthog-core/src/index.ts to track metrics with $ai_metric events
  • Implemented automatic string conversion for numeric/boolean values in both methods for consistent property types
  • Added comprehensive test coverage in posthog-core/test/posthog.ai.spec.ts verifying feedback/metric capture and type conversions
  • Bumped minor versions across all sub-libraries (posthog-node: 4.5.0, posthog-react-native: 3.7.0, posthog-web: 3.3.0)

5 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile

Copy link

github-actions bot commented Jan 31, 2025

Size Change: +539 B (+0.41%)

Total Size: 131 kB

Filename Size Change
posthog-react-native/lib/posthog-core/src/index.js 10.3 kB +103 B (+1.01%)
posthog-web/lib/index.cjs.js 18.3 kB +218 B (+1.21%)
posthog-web/lib/index.esm.js 18.2 kB +218 B (+1.21%)
ℹ️ View Unchanged
Filename Size
posthog-node/lib/index.cjs.js 31 kB
posthog-node/lib/index.esm.js 31 kB
posthog-react-native/lib/posthog-core/src/eventemitter.js 1.08 kB
posthog-react-native/lib/posthog-core/src/lz-string.js 1.42 kB
posthog-react-native/lib/posthog-core/src/types.js 420 B
posthog-react-native/lib/posthog-core/src/utils.js 870 B
posthog-react-native/lib/posthog-core/src/vendor/uuidv7.js 2.04 kB
posthog-react-native/lib/posthog-react-native/index.js 485 B
posthog-react-native/lib/posthog-react-native/src/autocapture.js 1.8 kB
posthog-react-native/lib/posthog-react-native/src/frameworks/wix-navigation.js 651 B
posthog-react-native/lib/posthog-react-native/src/hooks/useFeatureFlag.js 437 B
posthog-react-native/lib/posthog-react-native/src/hooks/useFeatureFlags.js 362 B
posthog-react-native/lib/posthog-react-native/src/hooks/useLifecycleTracker.js 416 B
posthog-react-native/lib/posthog-react-native/src/hooks/useNavigationTracker.js 628 B
posthog-react-native/lib/posthog-react-native/src/hooks/usePostHog.js 249 B
posthog-react-native/lib/posthog-react-native/src/legacy.js 810 B
posthog-react-native/lib/posthog-react-native/src/native-deps.js 1.31 kB
posthog-react-native/lib/posthog-react-native/src/optional/OptionalAsyncStorage.js 183 B
posthog-react-native/lib/posthog-react-native/src/optional/OptionalExpoApplication.js 215 B
posthog-react-native/lib/posthog-react-native/src/optional/OptionalExpoDevice.js 211 B
posthog-react-native/lib/posthog-react-native/src/optional/OptionalExpoFileSystem.js 224 B
posthog-react-native/lib/posthog-react-native/src/optional/OptionalExpoLocalization.js 216 B
posthog-react-native/lib/posthog-react-native/src/optional/OptionalReactNativeDeviceInfo.js 220 B
posthog-react-native/lib/posthog-react-native/src/optional/OptionalReactNativeLocalize.js 169 B
posthog-react-native/lib/posthog-react-native/src/optional/OptionalReactNativeNavigation.js 218 B
posthog-react-native/lib/posthog-react-native/src/optional/OptionalReactNativeNavigationWix.js 222 B
posthog-react-native/lib/posthog-react-native/src/optional/OptionalSessionReplay.js 231 B
posthog-react-native/lib/posthog-react-native/src/posthog-rn.js 4.42 kB
posthog-react-native/lib/posthog-react-native/src/PostHogContext.js 210 B
posthog-react-native/lib/posthog-react-native/src/PostHogProvider.js 1.57 kB
posthog-react-native/lib/posthog-react-native/src/storage.js 1.09 kB
posthog-react-native/lib/posthog-react-native/src/types.js 90 B
posthog-react-native/lib/posthog-react-native/src/version.js 123 B

compressed-size-action

@marandaneto
Copy link
Member

missing changelogs and the versioning might be outdated already, but otherwise LGTM

Copy link
Member

@marandaneto marandaneto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left a comment before merging, LGTM

@skoob13 skoob13 force-pushed the feat/metrics-methods branch from 739a0e4 to 3c44aeb Compare February 20, 2025 11:57
@skoob13 skoob13 merged commit 05154c8 into main Feb 20, 2025
4 checks passed
@skoob13 skoob13 deleted the feat/metrics-methods branch February 20, 2025 11:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants