Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add ai:models:detach command #18

Merged
merged 7 commits into from
Sep 26, 2024
Merged

feat: add ai:models:detach command #18

merged 7 commits into from
Sep 26, 2024

Conversation

k80bowman
Copy link
Contributor

Description

This PR implements the ai:models:detach command with tests according to the design outlined in the UX design doc.

How to test

Prepare your environment for testing

  • Fetch this branch.
  • Run yarn && yarn build.
  • Set the HEROKU_INFERENCE_ADDON to the production canary add-on via export HEROKU_INFERENCE_ADDON="inference-staging"

Actual testing

  • Verify that help looks ok: ./bin/run ai:models:detach --help
  • Create a test app: heroku apps:create test-cli-plugin-ai
  • Run ./bin/run ai:models:create claude-3-opus -a test-cli-plugin-ai --as opus_model and confirm that the command succeeds, creating the model resource config vars using the alias indicated.
  • Use the add-on name returned for our last model created (it will be called inference-staging-<adjective>-XXXXX) in the following command to create a second attachment to the same model resource: heroku addons:attach <addon-name> -a test-cli-plugin-ai --as opus_again.
  • Run ./bin/run ai:models:detach <add-on name> -a test-cli-plugin-ai, you should get an error saying that there are multiple matching attachments
  • Run ./bin/run ai:models:detach opus_again -a test-cli-plugin-ai, this operation should be successful
  • Run ./bin/run ai:models:detach opus_false -a test-cli-plugin-ai, you should get an error that says "We can’t find a model resource called opus_false. Run 'heroku addons' to see a list of model resources attached to your app."
  • Run ./bin/run ai:models:detach opus_model -a false-test-app, you should get an error that says "We can’t find the false-test-app app. Check your spelling."
  • Remove your test app to get rid of all add-ons: heroku apps:destroy -a test-cli-plugin-ai

SOC2 Compliance

GUS Work Item

@k80bowman k80bowman requested a review from a team as a code owner September 25, 2024 21:01
Copy link
Contributor

@sbosio sbosio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a comment to fix a copy on an error message according to what we discussed and agreed with Sandy, but it looks great! I'm approving.

const resource = error.http.body.resource

if (statusCode === 404 && resource === 'attachment') {
ux.error(`We can’t find a model resource called ${modelResource}. Run 'heroku addons' to see a list of model resources attached to your app.`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We agreed with Sandy to use Run 'heroku ai:models:info -a <appname' to see a list..., because it's the command expected to return all model resources according to the requested UX.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok. That change must not have made it into the design doc. Thanks.

await runCommand(Cmd, ['--app', 'myapp', 'model-123'])

expect(stdout.output).to.equal('')
expect(stderr.output).to.contain('Detaching model-123 to model from myapp... done\n')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using the stripAnsi helper might help with the differences between local and CI.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried that here and it didn't work because depending on the connection you can get those messages multiple times. So I left it as is, but I did use the stripAnsi helper on another test below where the message got longer.

@k80bowman k80bowman merged commit 1518f10 into main Sep 26, 2024
6 checks passed
@k80bowman k80bowman deleted the k80/ai-models-detach branch September 26, 2024 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants