bpf: Add kprobe multi link #58

Adding support to have priv pointer in swap callback function. Following the initial change on cmp callback functions [1] and adding SWAP_WRAPPER macro to identify sort call of sort_r. Cc: Rasmus Villemoes <[email protected]> [1] 4333fb9 ("media: lib/sort.c: implement sort() variant taking context argument") Signed-off-by: Jiri Olsa <[email protected]> Reviewed-by: Masami Hiramatsu <[email protected]>

Adding new link type BPF_LINK_TYPE_KPROBE_MULTI that attaches kprobe program through fprobe API. The fprobe API allows to attach probe on multiple functions at once very fast, because it works on top of ftrace. On the other hand this limits the probe point to the function entry or return. The kprobe program gets the same pt_regs input ctx as when it's attached through the perf API. Adding new attach type BPF_TRACE_KPROBE_MULTI that allows attachment kprobe to multiple function with new link. User provides array of addresses or symbols with count to attach the kprobe program to. The new link_create uapi interface looks like: struct { __aligned_u64 syms; __aligned_u64 addrs; __u32 cnt; __u32 flags; } kprobe_multi; The flags field allows single BPF_TRACE_KPROBE_MULTI bit to create return multi kprobe. Signed-off-by: Masami Hiramatsu <[email protected]> Signed-off-by: Jiri Olsa <[email protected]>

Adding support to call bpf_get_func_ip helper from kprobe programs attached by multi kprobe link. Signed-off-by: Jiri Olsa <[email protected]>

Adding support to inline it on x86, because it's single load instruction. Signed-off-by: Jiri Olsa <[email protected]>

Adding support to call bpf_get_attach_cookie helper from kprobe programs attached with kprobe multi link. The cookie is provided by array of u64 values, where each value is paired with provided function address or symbol with the same array index. Suggested-by: Andrii Nakryiko <[email protected]> Signed-off-by: Jiri Olsa <[email protected]>

Add bpf trampoline support for arm64. Most of the logic is the same as x86. Tested on raspberry pi 4b and qemu with KASLR disabled (avoid long jump), result: #9 /1 bpf_cookie/kprobe:OK #9 /2 bpf_cookie/multi_kprobe_link_api:FAIL #9 /3 bpf_cookie/multi_kprobe_attach_api:FAIL #9 /4 bpf_cookie/uprobe:OK #9 /5 bpf_cookie/tracepoint:OK #9 /6 bpf_cookie/perf_event:OK #9 /7 bpf_cookie/trampoline:OK #9 /8 bpf_cookie/lsm:OK #9 bpf_cookie:FAIL #18 /1 bpf_tcp_ca/dctcp:OK #18 /2 bpf_tcp_ca/cubic:OK #18 /3 bpf_tcp_ca/invalid_license:OK #18 /4 bpf_tcp_ca/dctcp_fallback:OK #18 /5 bpf_tcp_ca/rel_setsockopt:OK #18 bpf_tcp_ca:OK #51 /1 dummy_st_ops/dummy_st_ops_attach:OK #51 /2 dummy_st_ops/dummy_init_ret_value:OK #51 /3 dummy_st_ops/dummy_init_ptr_arg:OK #51 /4 dummy_st_ops/dummy_multiple_args:OK #51 dummy_st_ops:OK #55 fentry_fexit:OK #56 fentry_test:OK #57 /1 fexit_bpf2bpf/target_no_callees:OK #57 /2 fexit_bpf2bpf/target_yes_callees:OK #57 /3 fexit_bpf2bpf/func_replace:OK #57 /4 fexit_bpf2bpf/func_replace_verify:OK #57 /5 fexit_bpf2bpf/func_sockmap_update:OK #57 /6 fexit_bpf2bpf/func_replace_return_code:OK #57 /7 fexit_bpf2bpf/func_map_prog_compatibility:OK #57 /8 fexit_bpf2bpf/func_replace_multi:OK #57 /9 fexit_bpf2bpf/fmod_ret_freplace:OK #57 fexit_bpf2bpf:OK #58 fexit_sleep:OK #59 fexit_stress:OK #60 fexit_test:OK #67 get_func_args_test:OK #68 get_func_ip_test:OK #104 modify_return:OK #237 xdp_bpf2bpf:OK bpf_cookie/multi_kprobe_link_api and bpf_cookie/multi_kprobe_attach_api failed due to lack of multi_kprobe on arm64. Signed-off-by: Xu Kuohai <[email protected]> Acked-by: Song Liu <[email protected]>

Add bpf trampoline support for arm64. Most of the logic is the same as x86. Tested on raspberry pi 4b and qemu with KASLR disabled (avoid long jump), result: #9 /1 bpf_cookie/kprobe:OK #9 /2 bpf_cookie/multi_kprobe_link_api:FAIL #9 /3 bpf_cookie/multi_kprobe_attach_api:FAIL #9 /4 bpf_cookie/uprobe:OK #9 /5 bpf_cookie/tracepoint:OK #9 /6 bpf_cookie/perf_event:OK #9 /7 bpf_cookie/trampoline:OK #9 /8 bpf_cookie/lsm:OK #9 bpf_cookie:FAIL #18 /1 bpf_tcp_ca/dctcp:OK #18 /2 bpf_tcp_ca/cubic:OK #18 /3 bpf_tcp_ca/invalid_license:OK #18 /4 bpf_tcp_ca/dctcp_fallback:OK #18 /5 bpf_tcp_ca/rel_setsockopt:OK #18 bpf_tcp_ca:OK #51 /1 dummy_st_ops/dummy_st_ops_attach:OK #51 /2 dummy_st_ops/dummy_init_ret_value:OK #51 /3 dummy_st_ops/dummy_init_ptr_arg:OK #51 /4 dummy_st_ops/dummy_multiple_args:OK #51 dummy_st_ops:OK #55 fentry_fexit:OK #56 fentry_test:OK #57 /1 fexit_bpf2bpf/target_no_callees:OK #57 /2 fexit_bpf2bpf/target_yes_callees:OK #57 /3 fexit_bpf2bpf/func_replace:OK #57 /4 fexit_bpf2bpf/func_replace_verify:OK #57 /5 fexit_bpf2bpf/func_sockmap_update:OK #57 /6 fexit_bpf2bpf/func_replace_return_code:OK #57 /7 fexit_bpf2bpf/func_map_prog_compatibility:OK #57 /8 fexit_bpf2bpf/func_replace_multi:OK #57 /9 fexit_bpf2bpf/fmod_ret_freplace:OK #57 fexit_bpf2bpf:OK #58 fexit_sleep:OK #59 fexit_stress:OK #60 fexit_test:OK #67 get_func_args_test:OK #68 get_func_ip_test:OK #104 modify_return:OK #237 xdp_bpf2bpf:OK bpf_cookie/multi_kprobe_link_api and bpf_cookie/multi_kprobe_attach_api failed due to lack of multi_kprobe on arm64. Signed-off-by: Xu Kuohai <[email protected]> Acked-by: Song Liu <[email protected]> Acked-by: KP Singh <[email protected]>

Since commit 262ca38 ("clk: Stop forwarding clk_rate_requests to the parent"), the clk_rate_request is .. as the title says, not forwarded anymore to the parent: this produces an issue with the MediaTek clock MUX driver during GPU DVFS on MT8195, but not on MT8192 or others. This is because, differently from others, like MT8192 where all of the clocks in the MFG parents tree are of mtk_mux type, but in the parent tree of MT8195's MFG clock, we have one mtk_mux clock and one (clk framework generic) mux clock, like so: names: mfg_bg3d -> mfg_ck_fast_ref -> top_mfg_core_tmp (or) mfgpll types: mtk_gate -> mux -> mtk_mux (or) mtk_pll To solve this issue and also keep the GPU DVFS clocks code working as expected, wire up a .determine_rate() callback for the mtk_mux ops; for that, the standard clk_mux_determine_rate_flags() was used as it was possible to. This commit was successfully tested on MT6795 Xperia M5, MT8173 Elm, MT8192 Spherion and MT8195 Tomato; no regressions were seen. For the sake of some more documentation about this issue here's the trace of it: [ 12.211587] ------------[ cut here ]------------ [ 12.211589] WARNING: CPU: 6 PID: 78 at drivers/clk/clk.c:1462 clk_core_init_rate_req+0x84/0x90 [ 12.211593] Modules linked in: stp crct10dif_ce mtk_adsp_common llc rfkill snd_sof_xtensa_dsp panfrost(+) sbs_battery cros_ec_lid_angle cros_ec_sensors snd_sof_of cros_ec_sensors_core hid_multitouch cros_usbpd_logger snd_sof gpu_sched snd_sof_utils fuse ipv6 [ 12.211614] CPU: 6 PID: 78 Comm: kworker/u16:2 Tainted: G W 6.0.0-next-20221011+ #58 [ 12.211616] Hardware name: Acer Tomato (rev2) board (DT) [ 12.211617] Workqueue: devfreq_wq devfreq_monitor [ 12.211620] pstate: 40400009 (nZcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 12.211622] pc : clk_core_init_rate_req+0x84/0x90 [ 12.211625] lr : clk_core_forward_rate_req+0xa4/0xe4 [ 12.211627] sp : ffff80000893b8e0 [ 12.211628] x29: ffff80000893b8e0 x28: ffffdddf92f9b000 x27: ffff46a2c0e8bc05 [ 12.211632] x26: ffff46a2c1041200 x25: 0000000000000000 x24: 00000000173eed80 [ 12.211636] x23: ffff80000893b9c0 x22: ffff80000893b940 x21: 0000000000000000 [ 12.211641] x20: ffff46a2c1039f00 x19: ffff46a2c1039f00 x18: 0000000000000000 [ 12.211645] x17: 0000000000000038 x16: 000000000000d904 x15: 0000000000000003 [ 12.211649] x14: ffffdddf9357ce48 x13: ffffdddf935e71c8 x12: 000000000004803c [ 12.211653] x11: 00000000a867d7ad x10: 00000000a867d7ad x9 : ffffdddf90c28df4 [ 12.211657] x8 : ffffdddf9357a980 x7 : 0000000000000000 x6 : 0000000000000004 [ 12.211661] x5 : ffffffffffffffc8 x4 : 00000000173eed80 x3 : ffff80000893b940 [ 12.211665] x2 : 00000000173eed80 x1 : ffff80000893b940 x0 : 0000000000000000 [ 12.211669] Call trace: [ 12.211670] clk_core_init_rate_req+0x84/0x90 [ 12.211673] clk_core_round_rate_nolock+0xe8/0x10c [ 12.211675] clk_mux_determine_rate_flags+0x174/0x1f0 [ 12.211677] clk_mux_determine_rate+0x1c/0x30 [ 12.211680] clk_core_determine_round_nolock+0x74/0x130 [ 12.211682] clk_core_round_rate_nolock+0x58/0x10c [ 12.211684] clk_core_round_rate_nolock+0xf4/0x10c [ 12.211686] clk_core_set_rate_nolock+0x194/0x2ac [ 12.211688] clk_set_rate+0x40/0x94 [ 12.211691] _opp_config_clk_single+0x38/0xa0 [ 12.211693] _set_opp+0x1b0/0x500 [ 12.211695] dev_pm_opp_set_rate+0x120/0x290 [ 12.211697] panfrost_devfreq_target+0x3c/0x50 [panfrost] [ 12.211705] devfreq_set_target+0x8c/0x2d0 [ 12.211707] devfreq_update_target+0xcc/0xf4 [ 12.211708] devfreq_monitor+0x40/0x1d0 [ 12.211710] process_one_work+0x294/0x664 [ 12.211712] worker_thread+0x7c/0x45c [ 12.211713] kthread+0x104/0x110 [ 12.211716] ret_from_fork+0x10/0x20 [ 12.211718] irq event stamp: 7102 [ 12.211719] hardirqs last enabled at (7101): [<ffffdddf904ea5a0>] finish_task_switch.isra.0+0xec/0x2f0 [ 12.211723] hardirqs last disabled at (7102): [<ffffdddf91794b74>] el1_dbg+0x24/0x90 [ 12.211726] softirqs last enabled at (6716): [<ffffdddf90410be4>] __do_softirq+0x414/0x588 [ 12.211728] softirqs last disabled at (6507): [<ffffdddf904171d8>] ____do_softirq+0x18/0x24 [ 12.211730] ---[ end trace 0000000000000000 ]--- Fixes: 262ca38 ("clk: Stop forwarding clk_rate_requests to the parent") Signed-off-by: AngeloGioacchino Del Regno <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Stephen Boyd <[email protected]>

When trying to use copy_from_kernel_nofault() to read vsyscall page through a bpf program, the following oops was reported: BUG: unable to handle page fault for address: ffffffffff600000 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page PGD 3231067 P4D 3231067 PUD 3233067 PMD 3235067 PTE 0 Oops: 0000 [#1] PREEMPT SMP PTI CPU: 1 PID: 20390 Comm: test_progs ...... 6.7.0+ #58 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996) ...... RIP: 0010:copy_from_kernel_nofault+0x6f/0x110 ...... Call Trace: <TASK> ? copy_from_kernel_nofault+0x6f/0x110 bpf_probe_read_kernel+0x1d/0x50 bpf_prog_2061065e56845f08_do_probe_read+0x51/0x8d trace_call_bpf+0xc5/0x1c0 perf_call_bpf_enter.isra.0+0x69/0xb0 perf_syscall_enter+0x13e/0x200 syscall_trace_enter+0x188/0x1c0 do_syscall_64+0xb5/0xe0 entry_SYSCALL_64_after_hwframe+0x6e/0x76 </TASK> ...... ---[ end trace 0000000000000000 ]--- The oops happens as follows: A bpf program uses bpf_probe_read_kernel() to read from vsyscall page, bpf_probe_read_kernel() invokes copy_from_kernel_nofault() in turn and then invokes __get_user_asm(). A page fault exception is triggered accordingly, but handle_page_fault() considers the vsyscall page address as a userspace address instead of a kernel space address, so the fix-up set-up by bpf isn't applied. Because the exception happens in kernel space and page fault handling is disabled, page_fault_oops() is invoked and an oops happens. Fix it by disallowing vsyscall page read for copy_from_kernel_nofault(). Originally-from: Thomas Gleixner <[email protected]> Reported-by: [email protected] Closes: https://lore.kernel.org/bpf/CAG48ez06TZft=ATH1qh2c5mpS5BT8UakwNkzi6nvK5_djC-4Nw@mail.gmail.com Reported-by: xingwei lee <[email protected]> Closes: https://lore.kernel.org/bpf/CABOYnLynjBoFZOf3Z4BhaZkc5hx_kHfsjiW+UWLoB=w33LvScw@mail.gmail.com Signed-off-by: Hou Tao <[email protected]>

When trying to use copy_from_kernel_nofault() to read vsyscall page through a bpf program, the following oops was reported: BUG: unable to handle page fault for address: ffffffffff600000 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page PGD 3231067 P4D 3231067 PUD 3233067 PMD 3235067 PTE 0 Oops: 0000 [#1] PREEMPT SMP PTI CPU: 1 PID: 20390 Comm: test_progs ...... 6.7.0+ #58 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996) ...... RIP: 0010:copy_from_kernel_nofault+0x6f/0x110 ...... Call Trace: <TASK> ? copy_from_kernel_nofault+0x6f/0x110 bpf_probe_read_kernel+0x1d/0x50 bpf_prog_2061065e56845f08_do_probe_read+0x51/0x8d trace_call_bpf+0xc5/0x1c0 perf_call_bpf_enter.isra.0+0x69/0xb0 perf_syscall_enter+0x13e/0x200 syscall_trace_enter+0x188/0x1c0 do_syscall_64+0xb5/0xe0 entry_SYSCALL_64_after_hwframe+0x6e/0x76 </TASK> ...... ---[ end trace 0000000000000000 ]--- It seems the occurrence of oops depends on SMAP feature of CPU. It happens as follow: a bpf program uses bpf_probe_read_kernel() to read from vsyscall page, bpf_probe_read_kernel() invokes copy_from_kernel_nofault() in turn and then invokes __get_user_asm(). Because the vsyscall page address is not readable for kernel space, a page fault exception is triggered accordingly, handle_page_fault() considers the vsyscall page address as a userspace address instead of a kernel space address, so the fix-up set-up by bpf isn't applied. Because the CPU has SMAP feature and the access happens in kernel mode, so page_fault_oops() is invoked and an oops happens. If these is no SMAP feature, the fix-up set-up by bpf will be applied and copy_from_kernel_nofault() will return -EFAULT instead. Considering handle_page_fault() has already considered the vsyscall page address as a userspace address, fix the problem by disallowing vsyscall page read for copy_from_kernel_nofault(). Originally-by: Thomas Gleixner <[email protected]> Reported-by: [email protected] Closes: https://lore.kernel.org/bpf/CAG48ez06TZft=ATH1qh2c5mpS5BT8UakwNkzi6nvK5_djC-4Nw@mail.gmail.com Reported-by: xingwei lee <[email protected]> Closes: https://lore.kernel.org/bpf/CABOYnLynjBoFZOf3Z4BhaZkc5hx_kHfsjiW+UWLoB=w33LvScw@mail.gmail.com Signed-off-by: Hou Tao <[email protected]>

When trying to use copy_from_kernel_nofault() to read vsyscall page through a bpf program, the following oops was reported: BUG: unable to handle page fault for address: ffffffffff600000 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page PGD 3231067 P4D 3231067 PUD 3233067 PMD 3235067 PTE 0 Oops: 0000 [#1] PREEMPT SMP PTI CPU: 1 PID: 20390 Comm: test_progs ...... 6.7.0+ #58 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996) ...... RIP: 0010:copy_from_kernel_nofault+0x6f/0x110 ...... Call Trace: <TASK> ? copy_from_kernel_nofault+0x6f/0x110 bpf_probe_read_kernel+0x1d/0x50 bpf_prog_2061065e56845f08_do_probe_read+0x51/0x8d trace_call_bpf+0xc5/0x1c0 perf_call_bpf_enter.isra.0+0x69/0xb0 perf_syscall_enter+0x13e/0x200 syscall_trace_enter+0x188/0x1c0 do_syscall_64+0xb5/0xe0 entry_SYSCALL_64_after_hwframe+0x6e/0x76 </TASK> ...... ---[ end trace 0000000000000000 ]--- The oops is triggered when: 1) A bpf program uses bpf_probe_read_kernel() to read from the vsyscall page and invokes copy_from_kernel_nofault() which in turn calls __get_user_asm(). 2) Because the vsyscall page address is not readable from kernel space, a page fault exception is triggered accordingly. 3) handle_page_fault() considers the vsyscall page address as a user space address instead of a kernel space address. This results in the fix-up setup by bpf not being applied and a page_fault_oops() is invoked due to SMAP. Considering handle_page_fault() has already considered the vsyscall page address as a userspace address, fix the problem by disallowing vsyscall page read for copy_from_kernel_nofault(). Originally-by: Thomas Gleixner <[email protected]> Reported-by: [email protected] Closes: https://lore.kernel.org/bpf/CAG48ez06TZft=ATH1qh2c5mpS5BT8UakwNkzi6nvK5_djC-4Nw@mail.gmail.com Reported-by: xingwei lee <[email protected]> Closes: https://lore.kernel.org/bpf/CABOYnLynjBoFZOf3Z4BhaZkc5hx_kHfsjiW+UWLoB=w33LvScw@mail.gmail.com Signed-off-by: Hou Tao <[email protected]>

When trying to use copy_from_kernel_nofault() to read vsyscall page through a bpf program, the following oops was reported: BUG: unable to handle page fault for address: ffffffffff600000 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page PGD 3231067 P4D 3231067 PUD 3233067 PMD 3235067 PTE 0 Oops: 0000 [#1] PREEMPT SMP PTI CPU: 1 PID: 20390 Comm: test_progs ...... 6.7.0+ #58 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996) ...... RIP: 0010:copy_from_kernel_nofault+0x6f/0x110 ...... Call Trace: <TASK> ? copy_from_kernel_nofault+0x6f/0x110 bpf_probe_read_kernel+0x1d/0x50 bpf_prog_2061065e56845f08_do_probe_read+0x51/0x8d trace_call_bpf+0xc5/0x1c0 perf_call_bpf_enter.isra.0+0x69/0xb0 perf_syscall_enter+0x13e/0x200 syscall_trace_enter+0x188/0x1c0 do_syscall_64+0xb5/0xe0 entry_SYSCALL_64_after_hwframe+0x6e/0x76 </TASK> ...... ---[ end trace 0000000000000000 ]--- The oops is triggered when: 1) A bpf program uses bpf_probe_read_kernel() to read from the vsyscall page and invokes copy_from_kernel_nofault() which in turn calls __get_user_asm(). 2) Because the vsyscall page address is not readable from kernel space, a page fault exception is triggered accordingly. 3) handle_page_fault() considers the vsyscall page address as a user space address instead of a kernel space address. This results in the fix-up setup by bpf not being applied and a page_fault_oops() is invoked due to SMAP. Considering handle_page_fault() has already considered the vsyscall page address as a userspace address, fix the problem by disallowing vsyscall page read for copy_from_kernel_nofault(). Originally-by: Thomas Gleixner <[email protected]> Reported-by: [email protected] Closes: https://lore.kernel.org/bpf/CAG48ez06TZft=ATH1qh2c5mpS5BT8UakwNkzi6nvK5_djC-4Nw@mail.gmail.com Reported-by: xingwei lee <[email protected]> Closes: https://lore.kernel.org/bpf/CABOYnLynjBoFZOf3Z4BhaZkc5hx_kHfsjiW+UWLoB=w33LvScw@mail.gmail.com Signed-off-by: Hou Tao <[email protected]> Reviewed-by: Sohil Mehta <[email protected]>

When trying to use copy_from_kernel_nofault() to read vsyscall page through a bpf program, the following oops was reported: BUG: unable to handle page fault for address: ffffffffff600000 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page PGD 3231067 P4D 3231067 PUD 3233067 PMD 3235067 PTE 0 Oops: 0000 [#1] PREEMPT SMP PTI CPU: 1 PID: 20390 Comm: test_progs ...... 6.7.0+ #58 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996) ...... RIP: 0010:copy_from_kernel_nofault+0x6f/0x110 ...... Call Trace: <TASK> ? copy_from_kernel_nofault+0x6f/0x110 bpf_probe_read_kernel+0x1d/0x50 bpf_prog_2061065e56845f08_do_probe_read+0x51/0x8d trace_call_bpf+0xc5/0x1c0 perf_call_bpf_enter.isra.0+0x69/0xb0 perf_syscall_enter+0x13e/0x200 syscall_trace_enter+0x188/0x1c0 do_syscall_64+0xb5/0xe0 entry_SYSCALL_64_after_hwframe+0x6e/0x76 </TASK> ...... ---[ end trace 0000000000000000 ]--- The oops is triggered when: 1) A bpf program uses bpf_probe_read_kernel() to read from the vsyscall page and invokes copy_from_kernel_nofault() which in turn calls __get_user_asm(). 2) Because the vsyscall page address is not readable from kernel space, a page fault exception is triggered accordingly. 3) handle_page_fault() considers the vsyscall page address as a user space address instead of a kernel space address. This results in the fix-up setup by bpf not being applied and a page_fault_oops() is invoked due to SMAP. Considering handle_page_fault() has already considered the vsyscall page address as a userspace address, fix the problem by disallowing vsyscall page read for copy_from_kernel_nofault(). Originally-by: Thomas Gleixner <[email protected]> Reported-by: [email protected] Closes: https://lore.kernel.org/bpf/CAG48ez06TZft=ATH1qh2c5mpS5BT8UakwNkzi6nvK5_djC-4Nw@mail.gmail.com Reported-by: xingwei lee <[email protected]> Closes: https://lore.kernel.org/bpf/CABOYnLynjBoFZOf3Z4BhaZkc5hx_kHfsjiW+UWLoB=w33LvScw@mail.gmail.com Signed-off-by: Hou Tao <[email protected]> Reviewed-by: Sohil Mehta <[email protected]> Acked-by: Thomas Gleixner <[email protected]>

When trying to use copy_from_kernel_nofault() to read vsyscall page through a bpf program, the following oops was reported: BUG: unable to handle page fault for address: ffffffffff600000 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page PGD 3231067 P4D 3231067 PUD 3233067 PMD 3235067 PTE 0 Oops: 0000 [#1] PREEMPT SMP PTI CPU: 1 PID: 20390 Comm: test_progs ...... 6.7.0+ #58 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996) ...... RIP: 0010:copy_from_kernel_nofault+0x6f/0x110 ...... Call Trace: <TASK> ? copy_from_kernel_nofault+0x6f/0x110 bpf_probe_read_kernel+0x1d/0x50 bpf_prog_2061065e56845f08_do_probe_read+0x51/0x8d trace_call_bpf+0xc5/0x1c0 perf_call_bpf_enter.isra.0+0x69/0xb0 perf_syscall_enter+0x13e/0x200 syscall_trace_enter+0x188/0x1c0 do_syscall_64+0xb5/0xe0 entry_SYSCALL_64_after_hwframe+0x6e/0x76 </TASK> ...... ---[ end trace 0000000000000000 ]--- The oops is triggered when: 1) A bpf program uses bpf_probe_read_kernel() to read from the vsyscall page and invokes copy_from_kernel_nofault() which in turn calls __get_user_asm(). 2) Because the vsyscall page address is not readable from kernel space, a page fault exception is triggered accordingly. 3) handle_page_fault() considers the vsyscall page address as a user space address instead of a kernel space address. This results in the fix-up setup by bpf not being applied and a page_fault_oops() is invoked due to SMAP. Considering handle_page_fault() has already considered the vsyscall page address as a userspace address, fix the problem by disallowing vsyscall page read for copy_from_kernel_nofault(). Originally-by: Thomas Gleixner <[email protected]> Reported-by: [email protected] Closes: https://lore.kernel.org/bpf/CAG48ez06TZft=ATH1qh2c5mpS5BT8UakwNkzi6nvK5_djC-4Nw@mail.gmail.com Reported-by: xingwei lee <[email protected]> Closes: https://lore.kernel.org/bpf/CABOYnLynjBoFZOf3Z4BhaZkc5hx_kHfsjiW+UWLoB=w33LvScw@mail.gmail.com Signed-off-by: Hou Tao <[email protected]> Reviewed-by: Sohil Mehta <[email protected]> Acked-by: Thomas Gleixner <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>

Right now, the BPF_ADD and BPF_ADD | BPF_FETCH cases are tested twice: #55/u atomic BPF_ADD access through non-pointer OK #55/p atomic BPF_ADD access through non-pointer OK #56/u atomic BPF_ADD | BPF_FETCH access through non-pointer OK #56/p atomic BPF_ADD | BPF_FETCH access through non-pointer OK #57/u atomic BPF_ADD access through non-pointer OK #57/p atomic BPF_ADD access through non-pointer OK #58/u atomic BPF_ADD | BPF_FETCH access through non-pointer OK #58/p atomic BPF_ADD | BPF_FETCH access through non-pointer OK Reviewed-by: Josh Don <[email protected]> Signed-off-by: Peilin Ye <[email protected]>

kernel-patches-bot added bpf-next new V1 labels Feb 22, 2022

kernel-patches-bot force-pushed the series/616820=>bpf-next branch from e3e574c to b5b6fb6 Compare February 22, 2022 22:41

kernel-patches-bot force-pushed the series/616820=>bpf-next branch from b5b6fb6 to 6c2f025 Compare February 22, 2022 23:24

kernel-patches-bot force-pushed the series/616820=>bpf-next branch from 6c2f025 to 38461e3 Compare February 23, 2022 22:01

kernel-patches-bot force-pushed the series/616820=>bpf-next branch from 38461e3 to 80b1219 Compare February 23, 2022 23:10

kernel-patches-bot force-pushed the series/616820=>bpf-next branch from 80b1219 to 3ee5df7 Compare February 23, 2022 23:26

kernel-patches-bot force-pushed the series/616820=>bpf-next branch from 3ee5df7 to d3affb4 Compare February 24, 2022 01:39

kernel-patches-bot force-pushed the series/616820=>bpf-next branch from d3affb4 to 825c192 Compare February 25, 2022 21:01

kernel-patches-bot force-pushed the series/616820=>bpf-next branch from 825c192 to 9c7549d Compare February 28, 2022 12:57

kernel-patches-bot force-pushed the series/616820=>bpf-next branch from 9c7549d to 202cdc4 Compare February 28, 2022 13:26

kernel-patches-bot force-pushed the series/616820=>bpf-next branch from 202cdc4 to 2a140c5 Compare February 28, 2022 15:18

Nobody and others added 6 commits February 28, 2022 07:37

adding ci files

d451f52

bpf: Add bpf_get_func_ip kprobe helper for multi kprobe link

4123135

Adding support to call bpf_get_func_ip helper from kprobe programs attached by multi kprobe link. Signed-off-by: Jiri Olsa <[email protected]>

bpf: Add support to inline bpf_get_func_ip helper on x86

cd48357

Adding support to inline it on x86, because it's single load instruction. Signed-off-by: Jiri Olsa <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bpf: Add kprobe multi link #58

bpf: Add kprobe multi link #58

kernel-patches-bot commented Feb 22, 2022

kernel-patches-bot commented Feb 22, 2022

kernel-patches-bot commented Feb 22, 2022

kernel-patches-bot commented Feb 22, 2022

kernel-patches-bot commented Feb 23, 2022

kernel-patches-bot commented Feb 23, 2022

kernel-patches-bot commented Feb 23, 2022

kernel-patches-bot commented Feb 24, 2022

kernel-patches-bot commented Feb 25, 2022

kernel-patches-bot commented Feb 28, 2022

kernel-patches-bot commented Feb 28, 2022

kernel-patches-bot commented Feb 28, 2022

bpf: Add kprobe multi link #58

bpf: Add kprobe multi link #58

Conversation

kernel-patches-bot commented Feb 22, 2022

kernel-patches-bot commented Feb 22, 2022

kernel-patches-bot commented Feb 22, 2022

kernel-patches-bot commented Feb 22, 2022

kernel-patches-bot commented Feb 23, 2022

kernel-patches-bot commented Feb 23, 2022

kernel-patches-bot commented Feb 23, 2022

kernel-patches-bot commented Feb 24, 2022

kernel-patches-bot commented Feb 25, 2022

kernel-patches-bot commented Feb 28, 2022

kernel-patches-bot commented Feb 28, 2022

kernel-patches-bot commented Feb 28, 2022