Triton server crashes with a signal 11 when attempting to call nonexistent model inference

[gerasim13]

Steps to reproduce the issue:

1. Call the Triton server using the following code:

import tritonclient.grpc.aio as client
import numpy as np
from IPython import display as disp

async def async_request_iterator(asset_uid: str):
    inputs = []
    outputs = []
    outputs.append(client.InferRequestedOutput('transcriber_result'))
    outputs.append(client.InferRequestedOutput('waveform_result'))
    inputs.append(client.InferInput('asset_uid', [1], 'BYTES'))
    inputs[0].set_data_from_numpy(
        np.asarray(asset_uid.encode('utf_8')).reshape([1])
    )
    yield {
        'model_name': 'any_string_here',
        'inputs': inputs,
        'outputs': outputs
    }

async with client.InferenceServerClient(
    url='0.0.0.0:8887',
    verbose=False
) as triton_client:
    asset_uid = '226e33b5-6b67-409f-9b87-af9727f58ad9'
    async for response, error in triton_client.stream_infer(
        inputs_iterator=async_request_iterator(asset_uid)
    ):
        if error:
            raise error
        transcriber_result = response.as_numpy('transcriber_result')
        waveform_result = response.as_numpy('waveform_result')

Expected behavior:
The server should return an error indicating that the requested model does not exist on the server.

Actual behavior:
The Triton server crashes with a signal 11 error code.

I0521 04:41:02.172857 1 grpc_server.cc:4355] Process for ModelStreamInferHandler, rpc_ok=1, context 0, 0 step START
I0521 04:41:02.172936 1 grpc_server.cc:4348] New request handler for ModelStreamInferHandler, 0
I0521 04:41:02.173046 1 grpc_server.cc:4355] Process for ModelStreamInferHandler, rpc_ok=1, context 0, 0 step READ
I0521 04:41:02.173080 1 model_lifecycle.cc:327] GetModel() 'any_string_here' version -1
Signal (11) received.
I0521 04:41:04.576238 1 sequence_batch_scheduler.cc:882] Reaper: sleeping for 5000000us...
I0521 04:41:05.813510 1 sequence_batch_scheduler.cc:882] Reaper: sleeping for 5000000us...
 0# 0x000055E6C691D739 in tritonserver
 1# 0x00007FCF8C81B090 in /usr/lib/x86_64-linux-gnu/libc.so.6
 2# TRITONSERVER_InferenceRequestId in /opt/tritonserver/lib/libtritonserver.so
 3# 0x000055E6C69534A6 in tritonserver
 4# 0x000055E6C6955035 in tritonserver
 5# 0x00007FCF8CC0CDE4 in /usr/lib/x86_64-linux-gnu/libstdc++.so.6
 6# 0x00007FCF8DCEA609 in /usr/lib/x86_64-linux-gnu/libpthread.so.0
 7# clone in /usr/lib/x86_64-linux-gnu/libc.so.6

System information:

• Operating system: Ubuntu 18.04.6 LTS
• Triton server version: 2.32.0
• CUDA version: 12.1 driver version 530.30.02 with kernel driver version 470.129.06

Additional context:

I have observed that this issue occurs only when attempting to call inferences for nonexistent models. The server works correctly when valid models are specified.

Please let me know if you have any questions.

[dyastremsky]

Thank you for reporting this. We've filed a ticket to investigate.

[nnshah1]

Thanks again for reporting and the detailed reproduction instructions.

This commit should take care of the issue: #5842