About transferring PCRE2 compilation results in shared memory.

[Yoitsu-Holo]

Hello! I am currently working on a project that requires the use of the PCRE2 library for string extraction tasks. Since this project involves network packet forwarding, the performance and latency requirements are very high.

I would like to compile the regular expression patterns in one process and, after the PCRE2 automaton compilation is complete (including JIT compilation), pass all relevant information directly to my data forwarding process so that it can use it directly without spending additional time on compilation.

Currently, my solution is to create a memory pool using shared memory in the compilation process (hereinafter referred to as Process A) and map it to my address space. At the same time, I register the memory pool management functions in the PCRE2 context. After completing the pattern compilation and JIT compilation, I notify the data forwarding process (hereinafter referred to as Process B) of the starting address of this memory.

After Process B receives the message, it opens the shared memory and maps it to the same address space location, then begins executing the PCRE2 matching operations. However, it always crashes and throws a segmentation fault.

Here is the stack trace information I obtained using GDB:

#0  0x00007ff8c3893010 in ?? ()
#1  0x00007fb0e2353cb8 in jit_machine_stack_exec () from /lib64/libpcre2-8.so.0
#2  0x00007fb0e237bff6 in pcre2_jit_match_8 () from /lib64/libpcre2-8.so.0
#3  0x00007fb0e237d155 in pcre2_match_8 () from /lib64/libpcre2-8.so.0
#...

Could you please advise me on how to resolve this issue? Or do I have to perform JIT compilation again in the execution process, and cannot simply compile in one process and execute in another through shared memory?

Thank you very much for your help!

[zherczeg]

In theory, your idea is possible. However, you need to share executable memory, and I am not sure operating systems like (support/allow) that. The first thing you need to check that the shared memory has executable permissions. If the crash happens at the first instruction of the jit code (gdb disassembly can show that), it is likely that you have no permission for that. mprotect may help on Linux.

[zherczeg]

Btw, the defaullt allocator may allocate multiple memory chunks. You might need a custom allocator which satisfies all your needs:
https://github.com/zherczeg/sljit/tree/master/sljit_src/allocator_src

[Yoitsu-Holo]

Thank you very much for your reply. I am currently using my own memory pool to handle all the memory allocations. I will continue to try based on your suggestions. Thank you!

In theory, your idea is possible. However, you need to share executable memory, and I am not sure operating systems like (support/allow) that. The first thing you need to check that the shared memory has executable permissions. If the crash happens at the first instruction of the jit code (gdb disassembly can show that), it is likely that you have no permission for that. mprotect may help on Linux.

[NWilson]

Does the JIT generate position-independent-code? If the memory is mapped into two different processes, the addresses of the jump targets won't be the same.

[zherczeg]

No. I suspect he maps the code to the same address.