code relocation generating different memory layout cause user mode not working #17038
Comments
wentongwu
added
bug
|
From above map file, we can find there is an additional " .text.sedi_pm_pmu2nvic_isr.__stub" in the process of building priv_stacks_prebuilt.elf, but there is no this __stub for building zephyr_prebuilt.elf. So the random __stub generated by compiler causes the different size of .text section in the two process, and then different start address of .rodata, .noinit, .data sections(maybe sometimes the mpu align can cause some sections have same start address, but not always). So in priv_stacks_hash.c, the hash key(stack address) is not the final stack address. |
|
@andrewboie PR #17042 is a workaround for this issue , and so far I haven't found the reason why compiler generate this __stub. If found it, I will remove this workaround. |
Code relocation causes comipler randomly generates _stub that will cause different memory layout in the magic building process for user mode. And the different memory layout will generate wrong hash table in priv_stacks_hash.c and it will cause user mode doesn't work. Align sram .text section when userspace and code relocation enabled to generate same .rodata, .data and .bss sections in sram region. User can adjust Kconfig RELOCATION_SRAM_TEXT_ALIGN_SIZE to make less wasting memory. Fixes: zephyrproject-rtos#17038. Signed-off-by: Wentong Wu <[email protected]>
|
Ok, let's move the discussion here. The extra stubs are comprised of veneer functions, because the distance between caller and callee is too far, linker will automatically generate and insert veneer functions. Look at the map file again, The map file of building priv_stacks_prebuilt.elf is: The map file of building zephyr_prebuilt.elf is: The map file of building zephyr.elf is: From above fragments, we know even though the stubs call different names (.text.sedi_pm_pmu2nvic_isr.__stub, .priv_stacks.text.__stub, .kobject_data.text.__stub), but they contain the same veneer functions and have same size(0x68 Bytes), so it seems linker will put the linker stubs(veneer functions) at the end of .text sections(right after the last instruction), and because priv_stacks-text.ld and kobject-text.ld hold enough memory to contain these linker stubs. But when generating priv_stacks_prebuilt.elf, these linker stubs don't been put in these holding memory, so it will cause the different memory layout. So my current idea will be like below, and checked the three maps with below change, they have same memory layout. I will verify below change when get the setup at Monday. If @andrewboie @nashif @jocelyn-li have any concerns about below changes, please let me know. Update: I do the test with this change, code relocation for user mode works well. |
|
@andrewboie Another issue I want to talk with you is that when code relocation enabled, there will be some other regions to hold kernel objects(like k_mutex, k_sem, etc), but in elf_helper.py, it will do the address check like below code, only in SRAM region, so kernel objects in other regions will be discarded. There is a method which is to add symbols in these regions to mark the start and end of the memory containing kernel objects, and then add the check here. But I want to know if I delete below check logic, is there any other side effect? Because, run time we will check if there is hash key in the hash table, even there is no below check, other intend and unintended non-kernel object address will also be dropped when these addresses being passed to kernel space. If anything wrong, please let me know. |
When code relocation enabled, there will be serval regions holding text. And then there will be function call between these .text regions, when distance between caller and callee is too far, linker will automatically generate and insert veneer functions. And these veneer functions will be located right after the last instruction in the .text region by the linker. So these code will be put in the memory reserved for priv_stacks text and kobject text if they don't consume all the reserved memory. Or the veneer functions will be put before the reserved memory if there isn't code in the reserved memory. And then in the user mode building process, there will be different memory layout and it will cause usr mode not working. And moving the memory reserved for priv_stacks text and kobject text at the beginning of .text will avoid above problem. The detailed analysis for this issue can be found on Github issue zephyrproject-rtos#17038. Fixes: zephyrproject-rtos#17038. Signed-off-by: Wentong Wu <[email protected]>
When code relocation enabled, there will be serval regions holding text. And then there will be function call between these .text regions, when distance between caller and callee is too far, linker will automatically generate and insert veneer functions. And these veneer functions will be located right after the last instruction in the .text region by the linker. So these code will be put in the memory reserved for priv_stacks text and kobject text if they don't consume all the reserved memory. Or the veneer functions will be put before the reserved memory if there isn't code in the reserved memory. And then in the user mode building process, there will be different memory layout and it will cause usr mode not working. And moving the memory reserved for priv_stacks text and kobject text at the beginning of .text will avoid above problem. The detailed analysis for this issue can be found on Github issue #17038. Fixes: #17038. Signed-off-by: Wentong Wu <[email protected]>
When code relocation enabled, there will be serval regions holding text. And then there will be function call between these .text regions, when distance between caller and callee is too far, linker will automatically generate and insert veneer functions. And these veneer functions will be located right after the last instruction in the .text region by the linker. So these code will be put in the memory reserved for priv_stacks text and kobject text if they don't consume all the reserved memory. Or the veneer functions will be put before the reserved memory if there isn't code in the reserved memory. And then in the user mode building process, there will be different memory layout and it will cause usr mode not working. And moving the memory reserved for priv_stacks text and kobject text at the beginning of .text will avoid above problem. The detailed analysis for this issue can be found on Github issue zephyrproject-rtos#17038. Fixes: zephyrproject-rtos#17038. Signed-off-by: Wentong Wu <[email protected]>
When code relocation enabled, there will be serval regions holding text. And then there will be function call between these .text regions, when distance between caller and callee is too far, linker will automatically generate and insert veneer functions. And these veneer functions will be located right after the last instruction in the .text region by the linker. So these code will be put in the memory reserved for priv_stacks text and kobject text if they don't consume all the reserved memory. Or the veneer functions will be put before the reserved memory if there isn't code in the reserved memory. And then in the user mode building process, there will be different memory layout and it will cause usr mode not working. And moving the memory reserved for priv_stacks text and kobject text at the beginning of .text will avoid above problem. The detailed analysis for this issue can be found on Github issue #17038. Fixes: #17038. Signed-off-by: Wentong Wu <[email protected]>
Describe the bug
In the magic building process for user mode, there are different memory layout as below log show.
The map file of building zephyr_prebuilt.elf is:
And ztest_thread_stack located in 0x60030000 from below log and we can also find it from map file.
gen_kobject_list.py: symbol 'ztest_thread_stack' at 0x60030000 contains 1 object(s)The map file of building priv_stacks_prebuilt.elf is:
And ztest_thread_stack located in 0x60050000 from below log and we can also find it from map file.
gen_priv_stacks.py: symbol 'ztest_thread_stack' at 0x60050000 contains 1 object(s)So that difference will cause the wrong hash table in priv_stacks_hash.c because of the different .rodata .noinit .data sections in the same one sram region and it will cause the user mode doesn't work.
The text was updated successfully, but these errors were encountered: