Add an e2e test to verify the --comm option works fine

[hengyoush]

Is your feature request related to a problem? Please describe.
Add an e2e test to verify the --comm (filter by process name) option works fine

Describe the solution you'd like

1. Add a bash script file in testdata/ which contains the main test logic. (like this one: test_docker_filter_by_pid.sh), including two cases:

   1. target process start , then kyanos start , expected kyanos can watch the target process's network.
   2. kyanos start, then target process start, expected kyanos can watch the target process's network.

2. Add a step in workflow : .github/workflows/test.yml which executes the test script file in 1.

[spencercjh]

Can a simple Python HTTP server be started directly (not in a Docker container) in this e2e test?

sudo ./kyanos watch -d http --comm python3

I need to simulate my local operation in the test.

AM I RIGHT?

[spencercjh]

You haven't started this work yet, have you?

[hengyoush]

You haven't started this work yet, have you?

No, I haven't.

Can a simple Python HTTP server be started directly (not in a Docker container) in this e2e test?

ok.

@spencercjh Thank you very much, assigned this issue to you.

[spencercjh]

HTTPS may require sending more than two requests over a long connection to be correctly captured 😂. Can this test be simplified by using curl to request any website (via http), and then using --comm curl? Finally check, when grepping, filter for the domain name (e.g., if curl requests baidu, grep for baidu.com).

What do you think? @spencercjh

I looked at the source code and the filtering of processes is done in SetupAgent -> LoadBPF -> setAndValidateParameters, and this call link is only executed once when the program starts. The range of processes filtered is only a snapshot of what is running at the time and does not dynamically select emerging processes as the program runs. This makes it difficult to use --comm to filter curl processes for #212.

It looks like --comm is not designed for one-off processes like curl but server proc. So the second test sub-task is unrealizable now.

kyanos start, then target process start, expected kyanos can watch the target process's network.

I've tested all of the above locally.

Also, the current implementation of --comm is not very user-friendly. It only supports filtering something like the output of ps —o comm, which I could extend to support ps —o cmd.

@hengyoush

[hengyoush]

I looked at the source code and the filtering of processes is done in SetupAgent -> LoadBPF -> setAndValidateParameters, and this call link is only executed once when the program starts. The range of processes filtered is only a snapshot of what is running at the time and does not dynamically select emerging processes as the program runs. This makes it difficult to use --comm to filter curl processes for #212.

Actually, Kyanos knows when a process starts via eBPF tracepoint(sched_process_exec). After a process starts, it checks if the comm parameter matches the process name. If they match, it updates the filterPidMap in the kernel, bpf progs use filterPidMap to filter event by pid.

Look source code at: setAndValidateParameters(line: 292 ~ 309)

[spencercjh]

I looked at the source code and the filtering of processes is done in SetupAgent -> LoadBPF -> setAndValidateParameters, and this call link is only executed once when the program starts. The range of processes filtered is only a snapshot of what is running at the time and does not dynamically select emerging processes as the program runs. This makes it difficult to use --comm to filter curl processes for #212.

Actually, Kyanos knows when a process starts via eBPF tracepoint(sched_process_exec). After a process starts, it checks if the comm parameter matches the process name. If they match, it updates the filterPidMap in the kernel, bpf progs use filterPidMap to filter event by pid.

Look source code at: setAndValidateParameters(line: 292 ~ 309)

Looks like something went wrong with my local test. I'll carry on.

[hengyoush]

Also, the current implementation of --comm is not very user-friendly. It only supports filtering something like the output of ps —o comm, which I could extend to support ps —o cmd.

Nice.

[spencercjh]

I looked at the source code and the filtering of processes is done in SetupAgent -> LoadBPF -> setAndValidateParameters, and this call link is only executed once when the program starts. The range of processes filtered is only a snapshot of what is running at the time and does not dynamically select emerging processes as the program runs. This makes it difficult to use --comm to filter curl processes for #212.

Actually, Kyanos knows when a process starts via eBPF tracepoint(sched_process_exec). After a process starts, it checks if the comm parameter matches the process name. If they match, it updates the filterPidMap in the kernel, bpf progs use filterPidMap to filter event by pid.
Look source code at: setAndValidateParameters(line: 292 ~ 309)

Looks like something went wrong with my local test. I'll carry on.

The HTTPS protocol cannot be used when testing with curl. When using HTTP, we can see HTTP responses with 301.

[hengyoush]

The first case "target process start, then kyanos start, expected kyanos can watch the target process's network." , you may not be able to use curl as the 'curl' process will terminate very quickly. Therefore, a server process might be needed for testing. @spencercjh

[spencercjh]

The first case "target process start, then kyanos start, expected kyanos can watch the target process's network." , you may not be able to use curl as it will terminate very quickly. Therefore, a server process might be needed for testing. @spencercjh

Thanks for the explanation. 😭 🙏