Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the signature validation tests in sentence-transformers suite #9895

Merged
merged 1 commit into from
Oct 11, 2023

Conversation

BenWilson2
Copy link
Member

@BenWilson2 BenWilson2 commented Oct 10, 2023

Open in GitHub Codespaces

Related Issues/PRs

#xxx

What changes are proposed in this pull request?

Adjust the signature validation within sentence-transformers test suite to conform to changes made in #9874

How is this PR tested?

  • Existing unit/integration tests
  • New unit/integration tests
  • Manual tests

Does this PR require documentation update?

  • No. You can skip the rest of this section.
  • Yes. I've updated:
    • Examples
    • API references
    • Instructions

Release Notes

Is this a user-facing change?

  • No. You can skip the rest of this section.
  • Yes. Give a description of this change to be included in the release notes for MLflow users.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

  • area/artifacts: Artifact stores and artifact logging
  • area/build: Build and test infrastructure for MLflow
  • area/docs: MLflow documentation pages
  • area/examples: Example code
  • area/gateway: AI Gateway service, Gateway client APIs, third-party Gateway integrations
  • area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
  • area/models: MLmodel format, model serialization/deserialization, flavors
  • area/recipes: Recipes, Recipe APIs, Recipe configs, Recipe Templates
  • area/projects: MLproject format, project running backends
  • area/scoring: MLflow Model server, model deployment tools, Spark UDFs
  • area/server-infra: MLflow Tracking server backend
  • area/tracking: Tracking Service, tracking client APIs, autologging

Interface

  • area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
  • area/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
  • area/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registry
  • area/windows: Windows support

Language

  • language/r: R APIs and clients
  • language/java: Java APIs and clients
  • language/new: Proposals for new client languages

Integrations

  • integrations/azure: Azure and Azure ML integrations
  • integrations/sagemaker: SageMaker integrations
  • integrations/databricks: Databricks integrations

How should the PR be classified in the release notes? Choose one:

  • rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
  • rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
  • rn/feature - A new user-facing feature worth mentioning in the release notes
  • rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
  • rn/documentation - A user-facing documentation change worth mentioning in the release notes

@github-actions
Copy link

Documentation preview for e2f6f1a will be available here when this CircleCI job completes successfully.

More info

@github-actions github-actions bot added area/build Build and test infrastructure for MLflow rn/none List under Small Changes in Changelogs. labels Oct 10, 2023
@BenWilson2
Copy link
Member Author

@serena-ruan please confirm that the behavior change reflected in this fix matches the behavior that you were intending in #9874 .
Previous behavior prior to that PR was to coerce the signature to a pandas DF mapping of columns from a List[str] input.

Question: Does the change in #9874 work as expected for model serving use cases with a defined schema for the following conditions:

  1. Model is saved with an inferred signature based on a List[str] input example
  2. Model is saved with default signature and a pd.DataFrame representation is submitted to serving (with different split formats)

@BenWilson2 BenWilson2 added the enable-dev-tests Enables cross-version tests for dev versions label Oct 10, 2023
@serena-ruan
Copy link
Collaborator

@serena-ruan please confirm that the behavior change reflected in this fix matches the behavior that you were intending in #9874 . Previous behavior prior to that PR was to coerce the signature to a pandas DF mapping of columns from a List[str] input.

Question: Does the change in #9874 work as expected for model serving use cases with a defined schema for the following conditions:

  1. Model is saved with an inferred signature based on a List[str] input example
  2. Model is saved with default signature and a pd.DataFrame representation is submitted to serving (with different split formats)

@BenWilson2 Yes, this fix matches the expected behavior in #9874, thanks! My test case contains a serving example for 1. For 2, it seems irrelevant to this change as the change only affects how we infer the signature?

Copy link
Collaborator

@serena-ruan serena-ruan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@BenWilson2
Copy link
Member Author

@serena-ruan thank you for the validation :D

@BenWilson2 BenWilson2 merged commit 3cd359a into mlflow:master Oct 11, 2023
@BenWilson2 BenWilson2 deleted the fix-sentence-transformers-sig branch October 11, 2023 00:12
BenWilson2 added a commit that referenced this pull request Oct 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/build Build and test infrastructure for MLflow enable-dev-tests Enables cross-version tests for dev versions rn/none List under Small Changes in Changelogs.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants