One byte of death #6781

ligurio · 2022-01-13T13:53:52Z

Bug description

A clear and concise description of what the bug is.

OS: Linux
OS Version: Ubuntu 20.04 x86_64
Architecture: amd64

Tarantool 2.10.0-beta2-37-g9d43b842b
Target: Linux-x86_64-Debug
Build options: cmake . -DCMAKE_INSTALL_PREFIX=/usr/local -DENABLE_BACKTRACE=ON
Compiler: /usr/bin/clang /usr/bin/clang++

Steps to reproduce

In Tarantool's console:

box.cfg{listen=3303}
require('net.box').connect('3303'):call('\x8a')

Actual behavior

Tarantool is dead.

Expected behavior

Tarantool is alive.

The text was updated successfully, but these errors were encountered:

This patch adds a missed check for `lua_yaml_encode` return value, similar to the check in `console_dump_plain`. Now Lua error is raised if YAML encoding failed for any reason (e.g. OOM or formatting error). Before: ``` tarantool> box.error.new(box.error.ILLEGAL_PARAMS, '\x80') ~/test$ echo $? 0 ~/test$ ``` After: ``` tarantool> box.error.new(box.error.ILLEGAL_PARAMS, '\x80') --- - error: 'console: an exception occurred when formatting the output: expected SCALAR, SEQUENCE-START, MAPPING-START, or ALIAS' ... tarantool> ``` Part of #6781 Part of #6934 NO_DOC=bugfix NO_TEST=see later commits NO_CHANGELOG=see later commits

Currently the call to internal.format_yaml in output_handlers["yaml"] is covered by pcall only for `status == true`, however the opposite case also can raise an exception. This patch adds the missed exception handling. Two distinct calls are required, because it is not possible to assing variadic arguments (...) to a variable if some of them is nil. If the first call fails, internal.format_yaml will be called for the second time. Hopefully, it will not fail during formatting of the error message received from libyaml. Before: ``` tarantool> require('net.box').self:call('\x80') LuajitError: builtin/box/console.lua:710: expected SCALAR, SEQUENCE-START, MAPPING-START, or ALIAS fatal error, exiting the event loop ~/test$ echo $? 0 ~/test$ ``` After: ``` tarantool> require('net.box').self:call('\x80') --- - error: 'console: an exception occurred when formatting the output: expected SCALAR, SEQUENCE-START, MAPPING-START, or ALIAS' ... tarantool> ``` Part of #6781 Part of #6934 NO_DOC=bugfix NO_TEST=see later commits NO_CHANGELOG=see later commits

lua_encode can raise an exception (e.g. from luaL_checkfield) or return `nil, err` result. Handle it in lbox_console_format_lua, similar to lbox_console_format_yaml. When fixed, it will expose another issue - nil value can not be serialized: ``` tarantool> \set output lua true; tarantool> 1, nil, 2 1, {error = "console: exception while formatting the output: \"serializer: unexpected data (nd.field.size 0 nd.field.type 5)\""}, 2; tarantool> ``` Fix this too. Part of #6781 Part of #6934 NO_DOC=bugfix NO_TEST=not a visible change NO_CHANGELOG=not a visible change

Currently the call to internal.format_lua in format_lua_value is covered by pcall only for `status == true`, however the opposite case also can raise an exception. This patch adds the missed exception handling. Part of #6781 Part of #6934 NO_DOC=bugfix NO_CHANGELOG=minor bug NO_TEST=not possible to trigger format_lua failure for !status

If MP_ERROR contains an error message with invalid UTF-8 sequences, encode it in Base64 to obtain printable string. Closes #6781 Closes #6934 NO_DOC=bugfix

This patch adds a missed check for `lua_yaml_encode` return value, similar to the check in `console_dump_plain`. Now Lua error is raised if YAML encoding failed for any reason (e.g. OOM or formatting error). Before: ``` tarantool> box.error.new(box.error.ILLEGAL_PARAMS, '\x80') ~/test$ echo $? 0 ~/test$ ``` After: ``` tarantool> box.error.new(box.error.ILLEGAL_PARAMS, '\x80') --- - error: 'console: an exception occurred when formatting the output: expected SCALAR, SEQUENCE-START, MAPPING-START, or ALIAS' ... tarantool> ``` Part of #6781 Part of #6934 NO_DOC=bugfix NO_TEST=see later commits NO_CHANGELOG=see later commits (cherry picked from commit c84cdc7)

Currently the call to internal.format_yaml in output_handlers["yaml"] is covered by pcall only for `status == true`, however the opposite case also can raise an exception. This patch adds the missed exception handling. Two distinct calls are required, because it is not possible to assing variadic arguments (...) to a variable if some of them is nil. If the first call fails, internal.format_yaml will be called for the second time. Hopefully, it will not fail during formatting of the error message received from libyaml. Before: ``` tarantool> require('net.box').self:call('\x80') LuajitError: builtin/box/console.lua:710: expected SCALAR, SEQUENCE-START, MAPPING-START, or ALIAS fatal error, exiting the event loop ~/test$ echo $? 0 ~/test$ ``` After: ``` tarantool> require('net.box').self:call('\x80') --- - error: 'console: an exception occurred when formatting the output: expected SCALAR, SEQUENCE-START, MAPPING-START, or ALIAS' ... tarantool> ``` Part of #6781 Part of #6934 NO_DOC=bugfix NO_TEST=see later commits NO_CHANGELOG=see later commits (cherry picked from commit 405bfe4)

lua_encode can raise an exception (e.g. from luaL_checkfield) or return `nil, err` result. Handle it in lbox_console_format_lua, similar to lbox_console_format_yaml. When fixed, it will expose another issue - nil value can not be serialized: ``` tarantool> \set output lua true; tarantool> 1, nil, 2 1, {error = "console: exception while formatting the output: \"serializer: unexpected data (nd.field.size 0 nd.field.type 5)\""}, 2; tarantool> ``` Fix this too. Part of #6781 Part of #6934 NO_DOC=bugfix NO_TEST=not a visible change NO_CHANGELOG=not a visible change (cherry picked from commit 69852e1)

Currently the call to internal.format_lua in format_lua_value is covered by pcall only for `status == true`, however the opposite case also can raise an exception. This patch adds the missed exception handling. Part of #6781 Part of #6934 NO_DOC=bugfix NO_CHANGELOG=minor bug NO_TEST=not possible to trigger format_lua failure for !status (cherry picked from commit e17515d)

If MP_ERROR contains an error message with invalid UTF-8 sequences, encode it in Base64 to obtain printable string. Closes #6781 Closes #6934 NO_DOC=bugfix (cherry picked from commit 8f799cd)

This patch adds a missed check for `lua_yaml_encode` return value, similar to the check in `console_dump_plain`. Now Lua error is raised if YAML encoding failed for any reason (e.g. OOM or formatting error). Before: ``` tarantool> box.error.new(box.error.ILLEGAL_PARAMS, '\x80') ~/test$ echo $? 0 ~/test$ ``` After: ``` tarantool> box.error.new(box.error.ILLEGAL_PARAMS, '\x80') --- - error: 'console: an exception occurred when formatting the output: expected SCALAR, SEQUENCE-START, MAPPING-START, or ALIAS' ... tarantool> ``` Part of tarantool#6781 Part of tarantool#6934 NO_DOC=bugfix NO_TEST=see later commits NO_CHANGELOG=see later commits

Currently the call to internal.format_yaml in output_handlers["yaml"] is covered by pcall only for `status == true`, however the opposite case also can raise an exception. This patch adds the missed exception handling. Two distinct calls are required, because it is not possible to assing variadic arguments (...) to a variable if some of them is nil. If the first call fails, internal.format_yaml will be called for the second time. Hopefully, it will not fail during formatting of the error message received from libyaml. Before: ``` tarantool> require('net.box').self:call('\x80') LuajitError: builtin/box/console.lua:710: expected SCALAR, SEQUENCE-START, MAPPING-START, or ALIAS fatal error, exiting the event loop ~/test$ echo $? 0 ~/test$ ``` After: ``` tarantool> require('net.box').self:call('\x80') --- - error: 'console: an exception occurred when formatting the output: expected SCALAR, SEQUENCE-START, MAPPING-START, or ALIAS' ... tarantool> ``` Part of tarantool#6781 Part of tarantool#6934 NO_DOC=bugfix NO_TEST=see later commits NO_CHANGELOG=see later commits

lua_encode can raise an exception (e.g. from luaL_checkfield) or return `nil, err` result. Handle it in lbox_console_format_lua, similar to lbox_console_format_yaml. When fixed, it will expose another issue - nil value can not be serialized: ``` tarantool> \set output lua true; tarantool> 1, nil, 2 1, {error = "console: exception while formatting the output: \"serializer: unexpected data (nd.field.size 0 nd.field.type 5)\""}, 2; tarantool> ``` Fix this too. Part of tarantool#6781 Part of tarantool#6934 NO_DOC=bugfix NO_TEST=not a visible change NO_CHANGELOG=not a visible change

Currently the call to internal.format_lua in format_lua_value is covered by pcall only for `status == true`, however the opposite case also can raise an exception. This patch adds the missed exception handling. Part of tarantool#6781 Part of tarantool#6934 NO_DOC=bugfix NO_CHANGELOG=minor bug NO_TEST=not possible to trigger format_lua failure for !status

If MP_ERROR contains an error message with invalid UTF-8 sequences, encode it in Base64 to obtain printable string. Closes tarantool#6781 Closes tarantool#6934 NO_DOC=bugfix

Examples of IPROTO decoding issues: tarantool#3900, tarantool#1928, tarantool#6781. Patch adds a number of fuzzing tests that covers IPROTO decoding: - xrow_decode_auth - xrow_decode_begin - xrow_decode_call - xrow_decode_dml - xrow_decode_error - xrow_decode_id - xrow_decode_raft - xrow_decode_sql - xrow_decode_watch - xrow_greeting_decode - xrow_header_decode NO_DOC=testing NO_CHANGELOG=testing

Examples of IPROTO decoding issues: tarantool#3900, tarantool#1928, tarantool#6781. Patch adds a number of fuzzing tests that covers IPROTO decoding: - xrow_decode_auth - xrow_decode_begin - xrow_decode_call - xrow_decode_dml - xrow_decode_error - xrow_decode_id - xrow_decode_raft - xrow_decode_sql - xrow_decode_watch - xrow_greeting_decode NO_DOC=testing NO_CHANGELOG=testing

Examples of IPROTO decoding issues: tarantool#3900, tarantool#1928, tarantool#6781. Patch adds a number of fuzzing tests that covers IPROTO decoding: - xrow_decode_auth - xrow_decode_begin - xrow_decode_call - xrow_decode_dml - xrow_decode_id - xrow_decode_raft - xrow_decode_sql - xrow_decode_watch - xrow_greeting_decode NO_DOC=testing NO_CHANGELOG=testing

Examples of IPROTO decoding issues: #3900, #1928, #6781. Patch adds a number of fuzzing tests that covers IPROTO decoding: - xrow_decode_auth - xrow_decode_begin - xrow_decode_call - xrow_decode_dml - xrow_decode_id - xrow_decode_raft - xrow_decode_sql - xrow_decode_watch - xrow_greeting_decode NO_DOC=testing NO_CHANGELOG=testing

Examples of IPROTO decoding issues: #3900, #1928, #6781. Patch adds a number of fuzzing tests that covers IPROTO decoding: - xrow_decode_auth - xrow_decode_begin - xrow_decode_call - xrow_decode_dml - xrow_decode_id - xrow_decode_raft - xrow_decode_sql - xrow_decode_watch - xrow_greeting_decode NO_DOC=testing NO_CHANGELOG=testing (cherry picked from commit 46cacf3)

Examples of IPROTO decoding issues: tarantool#3900, tarantool#1928, tarantool#6781. Patch adds a number of fuzzing tests that covers IPROTO decoding: - xrow_decode_auth - xrow_decode_begin - xrow_decode_call - xrow_decode_dml - xrow_decode_id - xrow_decode_raft - xrow_decode_sql - xrow_decode_watch - xrow_greeting_decode NO_DOC=testing NO_CHANGELOG=testing (cherry picked from commit 46cacf3)

Examples of IPROTO decoding issues: #3900, #1928, #6781. Patch adds a number of fuzzing tests that covers IPROTO decoding: - xrow_decode_auth - xrow_decode_begin - xrow_decode_call - xrow_decode_dml - xrow_decode_id - xrow_decode_raft - xrow_decode_sql - xrow_decode_watch - xrow_greeting_decode NO_DOC=testing NO_CHANGELOG=testing (cherry picked from commit 46cacf3)

Examples of IPROTO decoding issues: tarantool#3900, tarantool#1928, tarantool#6781. Patch adds a number of fuzzing tests that covers IPROTO decoding: - xrow_decode_auth - xrow_decode_begin - xrow_decode_call - xrow_decode_dml - xrow_decode_id - xrow_decode_raft - xrow_decode_sql - xrow_decode_watch - xrow_greeting_decode NO_DOC=testing NO_CHANGELOG=testing (cherry picked from commit 46cacf3) (cherry picked from commit 28ac932)

ligurio added the bug Something isn't working label Jan 13, 2022

kyukhin added the crash label Jan 14, 2022

This comment has been minimized.

Sign in to view

kyukhin added the teamC label Jan 21, 2022

alyapunov assigned Gumix Apr 18, 2022

alyapunov added the 2sp label Apr 18, 2022

Gumix mentioned this issue Apr 26, 2022

Fix Tarantool terminations on non-printable error messages #7085

Merged

locker closed this as completed in #7085 Jul 14, 2022

locker pushed a commit that referenced this issue Jul 14, 2022

lua-yaml: encode malformed error messages in base64

8f799cd

If MP_ERROR contains an error message with invalid UTF-8 sequences, encode it in Base64 to obtain printable string. Closes #6781 Closes #6934 NO_DOC=bugfix

locker added this to the 2.10.1 milestone Jul 14, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

One byte of death #6781

One byte of death #6781

ligurio commented Jan 13, 2022

This comment has been minimized.

One byte of death #6781

One byte of death #6781

Comments

ligurio commented Jan 13, 2022

This comment has been minimized.