Integrate baserun

[waynehamadi]

Background

Changes

PR Quality Checklist

• I have run the following commands against my code to ensure it passes our linters:

  black . --exclude test.py
  isort .
  mypy .
  autoflake --remove-all-unused-imports --recursive --ignore-init-module-imports --ignore-pass-after-docstring --in-place agbenchmark

[github-actions]

PR Analysis

• 🎯 Main theme: Integration of Baserun
• 📌 Type of PR: Enhancement
• 🧪 Relevant tests added: No
• ✨ Focused PR: Yes, because all changes are related to the integration of Baserun
• 🔒 Security concerns: No, because the PR does not introduce any new security vulnerabilities as it only integrates with an external service using an API key that is securely stored in GitHub secrets.

PR Feedback

• General suggestions: The PR seems to be well-structured and focused on integrating Baserun. However, it would be beneficial to add tests to ensure the new functionality works as expected.

• 🤖 Code feedback:

  • relevant file: agbenchmark/start_benchmark.py
    suggestion: It would be better to handle the case when the 'BASERUN_API_KEY' environment variable is not set. This can be done by raising an exception or providing a default value. [important]
    relevant line: if os.environ.get("BASERUN_API_KEY"):

  • relevant file: .github/workflows/ci.yml
    suggestion: Ensure that the 'BASERUN_API_KEY' secret is properly set in the GitHub secrets. If it's not set, the workflow might fail. [important]
    relevant line: BASERUN_API_KEY: ${{ secrets.BASERUN_API_KEY }}

  • relevant file: pyproject.toml
    suggestion: Ensure that the version of the 'baserun' package is compatible with the other dependencies of the project. [medium]
    relevant line: baserun = "^0.3"

How to use

To invoke the PR-Agent, add a comment using one of the following commands:
/review [-i]: Request a review of your Pull Request. For an incremental review, which only considers changes since the last review, include the '-i' option.
/describe: Modify the PR title and description based on the contents of the PR.
/improve: Suggest improvements to the code in the PR.
/ask <QUESTION>: Pose a question about the PR.

To edit any configuration parameter from 'configuration.toml', add --config_path=new_value
For example: /review --pr_reviewer.extra_instructions="focus on the file: ..."
To list the possible configuration parameters, use the /config command.