Server crashing adding tags

[germainm]

sist2 version:2.8.3

Platform Linux x86-64 (wsl 2):

Elasticsearch version: 7.5.1

Command with arguments: web ~/src.idx/Loaded index: src.idx

Starting web server @ http://192.168.117.107:4090
[7F86A4E6C600] [2020-09-16 21:15:46] [INFO store.c] Updated mdb mapsize to 32768 bytes
[7F86A4E6C600] [2020-09-16 21:15:46] [WARNING serve.c] ElasticSearch error during query (500)
[7F86A4E6C600] [2020-09-16 21:15:46] [WARNING serve.c] {
        "error":        {
                "root_cause":   [{
                                "type": "char_conversion_exception",
                                "reason":       "Invalid UTF-32 character 0xb9acb8c8 (above 0x0010ffff) at char #2, byte #11)"
                        }],
                "type": "char_conversion_exception",
                "reason":       "Invalid UTF-32 character 0xb9acb8c8 (above 0x0010ffff) at char #2, byte #11)"
        },
        "status":       500
}
=================================================================
==1675==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x7ffea61c3360 in thread T0
    #0 0x7f86a3d7c7a8 in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xde7a8)
    #1 0x5610b7f9f325 in ev_router /home/germain/sist2/sist2/src/web/serve.c:695
    #2 0x5610b80309c8 in mg_call /home/germain/vcpkg/buildtrees/mongoose/src/6.15-61160b4ed7.clean/mongoose.c:2462
    #3 0x5610b80376d3 in mg_http_handler /home/germain/vcpkg/buildtrees/mongoose/src/6.15-61160b4ed7.clean/mongoose.c:6627
    #4 0x5610b80309c8 in mg_call /home/germain/vcpkg/buildtrees/mongoose/src/6.15-61160b4ed7.clean/mongoose.c:2462
    #5 0x5610b8030cb9 in mg_if_poll /home/germain/vcpkg/buildtrees/mongoose/src/6.15-61160b4ed7.clean/mongoose.c:2526
    #6 0x5610b8034165 in mg_mgr_handle_conn /home/germain/vcpkg/buildtrees/mongoose/src/6.15-61160b4ed7.clean/mongoose.c:4059
    #7 0x5610b8034cb6 in mg_socket_if_poll /home/germain/vcpkg/buildtrees/mongoose/src/6.15-61160b4ed7.clean/mongoose.c:4290
    #8 0x5610b80312c3 in mg_mgr_poll /home/germain/vcpkg/buildtrees/mongoose/src/6.15-61160b4ed7.clean/mongoose.c:2667
    #9 0x5610b7f9fb84 in serve /home/germain/sist2/sist2/src/web/serve.c:718
    #10 0x5610b7f6db77 in sist2_web /home/germain/sist2/sist2/src/main.c:380
    #11 0x5610b7f66aa8 in main /home/germain/sist2/sist2/src/main.c:516
    #12 0x7f86a28b7b96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
    #13 0x5610b7f671a9 in _start (/home/germain/sist2/sist2/sist2_debug+0x1e61a9)

Address 0x7ffea61c3360 is located in stack of thread T0
SUMMARY: AddressSanitizer: bad-free (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xde7a8) in __interceptor_free
==1675==ABORTING

[germainm]

seem your software use utf32 but elasticsearch can't do nothing with bad encoding except return 500

[germainm]

I debugged the server and the script to update tags in es is incorrectly encoded:

it is sent like this : { "scH{ ( U { "source": "if(ctx._sour " {ctx._so ( U Z XN @ @ h( p tag.add( " ag)", ( U Z H{ ( U ` " "tag" : " ] ( U ( U Z

so, I think the string is encoded differently because of the way the code written. hexview of file servce.c show that the string is ascii (seem to be 1 char, 1 byte) My C is far. I'm python guy. can you help ?

[simon987]

Yes I'll debug this whenever I get the chance

[germainm]

it has nothing to do with encoding. es server is normally utf8 and I think your code don't encode any char * and normal ascii.
the es request work with web_post but not with web_post_async . I guess the async code currupt that memory

only a guess

[simon987]

Hi, please let me know if you still have this error in v2.8.4, thank you

[germainm]

no, tried with some index. it seems to work fine now