feat: support native_dropout dynamo converter

[zewenli98]

Description

Support native_dropout dynamo converter.

Per the schema, this function returns two tensors. The first is output and the other is mask. For inference, i.e., train is not True or None, whatever the p is, it always returns input and all Trues.

Fixes #2494

Type of change

• New feature (non-breaking change which adds functionality)

Checklist:

• My code follows the style guidelines of this project (You can use the linters)
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas and hacks
• I have made corresponding changes to the documentation
• I have added tests to verify my fix or my feature
• New and existing unit tests pass locally with my changes
• I have added the relevant labels to my PR in so that relevant reviewers are notified

[chohk88]

Could you please explain the reason for dividing the test cases into inputs = [torch.randn(input_shape)] and inputs = []?

[zewenli98]

If passing in inputs = [torch.randn(input_shape)], the input will be an ITensor. If passing the input directly into the pytorch function, the input will be a normal tensor.

[chohk88]

LGTM. The code looks good to merge.

I have one more question about input types for my understanding. Usually, we handle inputs as input: TRTTensor and only consider ITensor input. However, sometimes we use input_val: Union[TRTTensor, torch.Tensor, np.ndarray] or input: Sequence[Union[TRTTensor, torch.Tensor, np.ndarray]] to handle both ITensor and normal tensor types. Even though the schema doesn't show a difference, how do we distinguish between them?

[zewenli98]

Yeah very good questions.

1. Even though the schema doesn't show a difference, how do we distinguish between them?

From my understanding, ITensor is a concept in TRT so PyTorch schema doesn't distinguish them from the design perspective. (TRT is just one backend among many backends)

2. Usually, we handle inputs as input: TRTTensor and only consider ITensor input. However, sometimes we use input_val: Union[TRTTensor, torch.Tensor, np.ndarray] or input: Sequence[Union[TRTTensor, torch.Tensor, np.ndarray]] to handle both ITensor and normal tensor types.

I think in the entry point the inputs are not promised to be ITensor. Instead, they could be torch.Tensor or np.ndarray. So there are two ways to handle it. 1) use enforce_tensor_types to make it as ITensor 2) manually cast it to ITensor inside of the function. I think in most cases both are fine, but if this function is called by another function that passes in non-ITensor, the first solution will error out.

[chohk88]

I understand better now. Thank you so much for the explanation!