Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Good first issue] MPNet TensorFlow Integration tests #9956

Closed
LysandreJik opened this issue Feb 2, 2021 · 0 comments · Fixed by #9979
Closed

[Good first issue] MPNet TensorFlow Integration tests #9956

LysandreJik opened this issue Feb 2, 2021 · 0 comments · Fixed by #9979
Labels
Good First Issue TensorFlow Anything TensorFlow Tests Related to tests

Comments

@LysandreJik
Copy link
Member

The TensorFlow implementation of the MPNet model currently has no integration tests. This is problematic as the behavior can diverge without being noticed.

The test_modeling_tf_mpnet.py file should be updated to include integration testing.

An example of a good modeling integration test is visible in the test_modeling_tf_bert.py#L365-L387 file:

@require_tf
class TFBertModelIntegrationTest(unittest.TestCase):
@slow
def test_inference_masked_lm(self):
model = TFBertForPreTraining.from_pretrained("lysandre/tiny-bert-random")
input_ids = tf.constant([[0, 1, 2, 3, 4, 5]])
output = model(input_ids)[0]
expected_shape = [1, 6, 32000]
self.assertEqual(output.shape, expected_shape)
print(output[:, :3, :3])
expected_slice = tf.constant(
[
[
[-0.05243197, -0.04498899, 0.05512108],
[-0.07444685, -0.01064632, 0.04352357],
[-0.05020351, 0.05530146, 0.00700043],
]
]
)
tf.debugging.assert_near(output[:, :3, :3], expected_slice, atol=1e-4)

Some additional tips:

  • The test must be marked as slow using the @slow decorator, so as to be run daily, and not on every commit of every branch/pull request of this repository.
  • The test must be decorated with the @require_tf decorator so as to only be run in environments using PyTorch.
  • A single test is necessary. If you feel like implementing multiple of these, then sharing the same checkpoint would be ideal so as to reduce download time.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Good First Issue TensorFlow Anything TensorFlow Tests Related to tests
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant