feat: implement hooks

[jxnl]

Add Hooks System to Instructor Library

This PR introduces a new Hooks system to the Instructor library, providing a flexible way to intercept and handle events during the completion and parsing process.

Changes in instructor/hooks.py:

• Implemented a Hooks class with the following features:
  • Support for multiple event types (HookName enum)
  • Methods to register (on), unregister (off), and clear hooks
  • Event emission methods for various stages of the completion process
  • Type hints and docstrings for improved code readability and maintainability

Changes in docs/concepts/hooks.md:

• Added comprehensive documentation for the Hooks system, including:
  • Overview of the Hooks concept
  • Detailed explanation of supported hook events
  • Examples of registering, emitting, and removing hooks
  • A complete example demonstrating hook usage for logging and debugging
  • Best practices for using hooks effectively

Key Features:

1. Event-driven architecture for intercepting various stages of the completion process
2. Type-safe hook registration and emission
3. Flexible error handling in hook handlers
4. Clear documentation with examples for easy integration

Impact:

This addition will allow users of the Instructor library to:

• Implement custom logging and debugging
• Add custom behavior at different stages of the API interaction
• Enhance error handling and monitoring capabilities
• Create more modular and maintainable code when working with language models

---

Important

Introduces a Hooks system in the Instructor library for managing event hooks during the completion process, with integration, documentation, and tests.

• Hooks System:
  • Adds Hooks class in instructor/hooks.py for managing event hooks.
  • Supports events: completion:kwargs, completion:response, completion:error, completion:last_attempt, parse:error.
  • Methods to register (on), unregister (off), and emit events.
• Integration:
  • Adds hooks parameter to Instructor and AsyncInstructor classes in instructor/client.py.
  • Updates retry_sync and retry_async in instructor/retry.py to emit hooks during retries.
• Documentation:
  • Adds docs/concepts/hooks.md with detailed usage and examples.
• Testing:
  • Adds test_concepts.py for testing hook examples in documentation.

^{This description was created by for d31ae49. It will automatically update as commits are pushed.}

[ellipsis-dev]

❌ Changes requested. Reviewed everything up to b95235b in 45 seconds

More details

• Looked at 981 lines of code in 7 files
• Skipped 0 files when reviewing.
• Skipped posting 8 drafted comments based on config settings.

1. instructor/hooks.py:1

• Draft comment:
  Typo in PR title: 'implimennt' should be 'implement'.
• Reason this comment was not posted:
  Confidence changes required: 10%
  The PR introduces a new Hooks system, but there is a minor typo in the PR title.

2. instructor/hooks.py:109

• Draft comment:
  Consider using the logging module instead of warnings.warn for better control over logging levels and consistency with other logging practices.
• Reason this comment was not posted:
  Confidence changes required: 30%
  The Hooks class in instructor/hooks.py uses warnings.warn to notify about exceptions in handlers. This is a good practice for debugging, but it might be better to log these warnings using the logging module for consistency and better control over logging levels.

3. instructor/hooks.py:118

• Draft comment:
  Consider using the logging module instead of warnings.warn for better control over logging levels and consistency with other logging practices.
• Reason this comment was not posted:
  Confidence changes required: 30%
  The Hooks class in instructor/hooks.py uses warnings.warn to notify about exceptions in handlers. This is a good practice for debugging, but it might be better to log these warnings using the logging module for consistency and better control over logging levels.

4. instructor/hooks.py:127

• Draft comment:
  Consider using the logging module instead of warnings.warn for better control over logging levels and consistency with other logging practices.
• Reason this comment was not posted:
  Confidence changes required: 30%
  The Hooks class in instructor/hooks.py uses warnings.warn to notify about exceptions in handlers. This is a good practice for debugging, but it might be better to log these warnings using the logging module for consistency and better control over logging levels.

5. instructor/hooks.py:136

• Draft comment:
  Consider using the logging module instead of warnings.warn for better control over logging levels and consistency with other logging practices.
• Reason this comment was not posted:
  Confidence changes required: 30%
  The Hooks class in instructor/hooks.py uses warnings.warn to notify about exceptions in handlers. This is a good practice for debugging, but it might be better to log these warnings using the logging module for consistency and better control over logging levels.

6. instructor/hooks.py:145

• Draft comment:
  Consider using the logging module instead of warnings.warn for better control over logging levels and consistency with other logging practices.
• Reason this comment was not posted:
  Confidence changes required: 30%
  The Hooks class in instructor/hooks.py uses warnings.warn to notify about exceptions in handlers. This is a good practice for debugging, but it might be better to log these warnings using the logging module for consistency and better control over logging levels.

7. instructor/retry.py:235

• Draft comment:
  Consider using the logging module instead of warnings.warn for better control over logging levels and consistency with other logging practices.
• Reason this comment was not posted:
  Confidence changes required: 30%
  The Hooks class in instructor/hooks.py uses warnings.warn to notify about exceptions in handlers. This is a good practice for debugging, but it might be better to log these warnings using the logging module for consistency and better control over logging levels.

8. instructor/retry.py:324

• Draft comment:
  Consider using the logging module instead of warnings.warn for better control over logging levels and consistency with other logging practices.
• Reason this comment was not posted:
  Confidence changes required: 30%
  The Hooks class in instructor/hooks.py uses warnings.warn to notify about exceptions in handlers. This is a good practice for debugging, but it might be better to log these warnings using the logging module for consistency and better control over logging levels.

Workflow ID: wflow_v5gdRFnu5txoyZci

---

Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[ellipsis-dev]

The new docs/concepts/hooks.md file should be added to the mkdocs.yml to ensure it is included in the documentation.

[ellipsis-dev]

👍 Looks good to me! Incremental review on 4582b71 in 14 seconds

More details

• Looked at 144 lines of code in 1 files
• Skipped 0 files when reviewing.
• Skipped posting 4 drafted comments based on config settings.

1. instructor/hooks.py:131

• Draft comment:
  Use try-except to handle the case where the handler is not found in the list to avoid ValueError.
• Reason this comment was not posted:
  Confidence changes required: 80%
  The off method should handle the case where the handler is not found in the list to avoid a ValueError.

2. instructor/hooks.py:81

• Draft comment:
  Add comments to explain the error handling mechanism when exceptions occur in handlers. This will improve code readability and maintainability.
• Reason this comment was not posted:
  Confidence changes required: 50%
  The code is missing comments for complex sections, especially where exceptions are handled and warnings are issued. This would help in understanding the error handling mechanism better.

3. instructor/hooks.py:29

• Draft comment:
  Use more descriptive variable names for _handlers to improve code readability. This applies to other similar instances as well.
• Reason this comment was not posted:
  Confidence changes required: 50%
  The code uses a consistent naming pattern for methods, which is good. However, the variable names could be more descriptive in some places.

4. instructor/hooks.py:77

• Draft comment:
  Consider refactoring the repeated exception handling logic in emit methods to adhere to the DRY principle. This applies to other emit methods as well.
• Reason this comment was not posted:
  Confidence changes required: 50%
  The code should be DRY, and there are repeated patterns in the exception handling across different emit methods.

Workflow ID: wflow_DyIy8rC6BAFsG1bd

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[ellipsis-dev]

👍 Looks good to me! Incremental review on f364bce in 26 seconds

More details

• Looked at 206 lines of code in 4 files
• Skipped 0 files when reviewing.
• Skipped posting 4 drafted comments based on config settings.

1. README.md:83

• Draft comment:
  The hook event names used here (e.g., completion:kwargs, completion:error) should be consistent with the naming conventions used in the instructor/hooks.py file and documentation. Ensure consistency across all examples and documentation.
• Reason this comment was not posted:
  Confidence changes required: 50%
  The PR introduces a new Hooks system, but the README and documentation examples use inconsistent naming for the hook events. This could lead to confusion for users trying to implement the hooks.

2. docs/concepts/hooks.md:134

• Draft comment:
  Ensure that the hook event names (e.g., completion:kwargs, completion:error) are consistent with those used in the instructor/hooks.py file and other documentation. Consistency is key to avoid confusion.
• Reason this comment was not posted:
  Confidence changes required: 50%
  The PR introduces a new Hooks system, but the README and documentation examples use inconsistent naming for the hook events. This could lead to confusion for users trying to implement the hooks.

3. docs/index.md:193

• Draft comment:
  The hook event names used here (e.g., completion:kwargs, completion:error) should be consistent with the naming conventions used in the instructor/hooks.py file and documentation. Ensure consistency across all examples and documentation.
• Reason this comment was not posted:
  Confidence changes required: 50%
  The PR introduces a new Hooks system, but the README and documentation examples use inconsistent naming for the hook events. This could lead to confusion for users trying to implement the hooks.

4. mkdocs.yml:169

• Draft comment:
  The new docs/concepts/hooks.md file is correctly added to the mkdocs.yml file, ensuring it is included in the documentation.
• Reason this comment was not posted:
  Confidence changes required: 0%
  The PR introduces a new Hooks system, and the documentation for it is added in docs/concepts/hooks.md. This new documentation file is correctly added to the mkdocs.yml file under the Concepts section, ensuring it is included in the documentation build.

Workflow ID: wflow_19w1K5jcpDqk3o8c

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[ellipsis-dev]

👍 Looks good to me! Incremental review on c5c5642 in 17 seconds

More details

• Looked at 16 lines of code in 1 files
• Skipped 0 files when reviewing.
• Skipped posting 1 drafted comments based on config settings.

1. docs/concepts/hooks.md:48

• Draft comment:
  Ensure hooks.md is added to mkdocs.yml for documentation.
• Reason this comment was not posted:
  Comment did not seem useful.

Workflow ID: wflow_NMwvWQSmJCTG0CFb

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[ellipsis-dev]

👍 Looks good to me! Incremental review on a33bb16 in 13 seconds

More details

• Looked at 12 lines of code in 1 files
• Skipped 0 files when reviewing.
• Skipped posting 2 drafted comments based on config settings.

1. pyproject.toml:3

• Draft comment:
  The version update to 1.6.0 is appropriate for the new feature addition, following semantic versioning guidelines.
• Reason this comment was not posted:
  Confidence changes required: 0%
  The version update in the pyproject.toml file is consistent with the new feature addition described in the PR. The change from version 1.5.2 to 1.6.0 indicates a minor version update, which is appropriate for adding new functionality without breaking changes.

2. pyproject.toml:3

• Draft comment:
  If the library code has changed, ensure that the documentation and tests are updated accordingly. This includes updating any relevant md files in docs and ensuring they are added to mkdocs.yml.
• Reason this comment was not posted:
  Confidence changes required: 80%
  The version update in the pyproject.toml file indicates a change in the library code, which should be accompanied by updates in documentation and tests.

Workflow ID: wflow_vdUdt37Bwftj9wiM

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[ellipsis-dev]

👍 Looks good to me! Incremental review on d31ae49 in 21 seconds

More details

• Looked at 205 lines of code in 3 files
• Skipped 0 files when reviewing.
• Skipped posting 4 drafted comments based on config settings.

1. instructor/patch.py:20

• Draft comment:
  handle_templating is imported but not used in this file. Consider removing the import statement to clean up the code.
• Reason this comment was not posted:
  Confidence changes required: 10%
  The import statement for handle_templating in instructor/patch.py is unnecessary after the refactor, as the function is no longer used in this file.

2. tests/llm/test_cohere/test_json_schema.py:121

• Draft comment:
  The # type: ignore comment is used here. Ensure this is necessary and not suppressing important type errors.
• Reason this comment was not posted:
  Confidence changes required: 33%
  In tests/llm/test_cohere/test_json_schema.py, the # type: ignore comment is used to suppress type checking. It's important to ensure that this is necessary and not hiding potential issues.

3. tests/dsl/test_partial.py:79

• Draft comment:
  Assertions should include error messages for clarity and debugging purposes. This applies to all assertions in this file.
• Reason this comment was not posted:
  Confidence changes required: 80%
  The assertion statements in the test files lack error messages, which is a violation of the rule that assertions should always have an error message.

4. tests/llm/test_cohere/test_json_schema.py:118

• Draft comment:
  Assertions should include error messages for clarity and debugging purposes. This applies to all assertions in this file.
• Reason this comment was not posted:
  Confidence changes required: 80%
  The assertion statements in the test files lack error messages, which is a violation of the rule that assertions should always have an error message.

Workflow ID: wflow_6dDIwfdJcrsbU1M2

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.