Skip to content
This repository has been archived by the owner on Apr 20, 2021. It is now read-only.

Guess offsets #9

Merged
merged 3 commits into from
Jan 10, 2017
Merged

Guess offsets #9

merged 3 commits into from
Jan 10, 2017

Conversation

iaguis
Copy link
Contributor

@iaguis iaguis commented Dec 16, 2016

Instead of requiring a different ebpf object file per kernel version /
distro, we try to guess the offsets of the fields we access in the eBPF
program.

We do so by creating connections to a server we start, trying to read
through struct sock, and comparing the obtained values with the expected
ones.

Requires https://github.com/kinvolk/tcptracer-bpf/pull/15

@iaguis iaguis changed the title Guess offsets [WIP] Guess offsets Dec 16, 2016
@alban
Copy link
Contributor

alban commented Dec 16, 2016

@schu @alepuccetti: The changes in bpf/bpf.go would go to iovisor/gobpf but the changes in main.go would go to Scope.


releaseStr := strings.Trim(utsnameStr(buf.Release[:]), "\x00")

kernelVersionStr := strings.Split(releaseStr, "-")[0]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iaguis This does not work in the rkt kernel (rkt with the kvm flavor) because releaseStr is "4.8.6rkt-v1":

# uname -r
4.8.6rkt-v1

Symptoms:

strconv.ParseInt: parsing "6rkt": invalid syntax

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tested with rkt/rkt#3487

Copy link
Contributor

@schu schu Dec 20, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't break on my Debian box but the result doesn't reflect the current kernel version it seems: uname -r is 4.8.0-2-amd64 (kernel version 264192) whereas the actual version seems to be 4.8.11 (264203) and the prog cannot be loaded..

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix in rkt about localversion got merged rkt/rkt#3489

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@schu are you testing the correct branch of tcptracer-bpf? https://github.com/kinvolk/tcptracer-bpf/pull/15

Copy link
Contributor

@schu schu Dec 20, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alban I'm testing with dummy.c from iovisor/gobpf#6

@alban
Copy link
Contributor

alban commented Dec 17, 2016

Under the rkt (flavor kvm) kernel, this guesses the offsets correctly most of the time, but sometimes it freezes. dmesg says:

nf_conntrack: nf_conntrack: table full, dropping packet

We should add a timeout, so it does not wait for too long for the connection to succeed.

Also, this code is not supposed to do many connections simultaneously, so this should not be a problem for conntrack. We might not close the connection correctly before testing the next offset.

@schu
Copy link
Contributor

schu commented Dec 22, 2016

The changes in bpf/bpf.go would go to iovisor/gobpf

As gobpf now includes our elf package, the bpf part can be a separate pull request against iovisor/gobpf.

@iaguis
Copy link
Contributor Author

iaguis commented Jan 6, 2017

Also, this code is not supposed to do many connections simultaneously, so this should not be a problem for conntrack. We might not close the connection correctly before testing the next offset.

Maybe we fill it up because the closed connections stay in TIME_WAIT or such?

@iaguis
Copy link
Contributor Author

iaguis commented Jan 6, 2017

Maybe we fill it up because the closed connections stay in TIME_WAIT or such?

I just tried with rkt-kvm and the connections stay in TIME_WAIT even after the program exits:

root@rkt-46288066-3418-44b3-8854-a1e9980466d5:/# conntrack -C
17

*** run ./gobpf-elf-loader ./ebpf.o

root@rkt-46288066-3418-44b3-8854-a1e9980466d5:/# conntrack -C
858

*** stop ./gobpf-elf-loader

root@rkt-46288066-3418-44b3-8854-a1e9980466d5:/# conntrack -L | grep TIME_WAIT | wc -l
conntrack v1.4.2 (conntrack-tools): 854 flow entries have been shown.
845

@@ -298,6 +298,13 @@ func guessOffsets(b *bpf.BPFKProbePerf) error {
if err != nil {
return fmt.Errorf("error: %v", err)
}

if tcpConn, ok := conn.(*net.TCPConn); ok {
tcpConn.SetLinger(0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A comment would be nice to explain why

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@iaguis iaguis force-pushed the iaguis/guess-offsets branch 2 times, most recently from 852adce to d1ccfcc Compare January 6, 2017 15:12
@iaguis iaguis changed the title [WIP] Guess offsets Guess offsets Jan 10, 2017
Instead of requiring a different ebpf object file per kernel version /
distro, we try to guess the offsets of the fields we access in the eBPF
program.

We do so by creating connections to a server we start, trying to read
through struct sock, and comparing the obtained values with the expected
ones.
We make a lot of connections when we get the offsets and they can
potentially fill the conntrack table because the state the connections
remain in when they get closed is TIME_WAIT and it stays like that for a
while (around 2 minutes).

We set SO_LINGER to 0 so the connection state after closing is CLOSE
instead. In this way, they will disappear from the conntrack table after
around 10 seconds.
@iaguis iaguis force-pushed the iaguis/guess-offsets branch from d1ccfcc to ee7be00 Compare January 10, 2017 16:07
@iaguis
Copy link
Contributor Author

iaguis commented Jan 10, 2017

kinvolk/tcptracer-bpf#15 is merged. Let's merge this.

@iaguis iaguis merged commit 20812f6 into master Jan 10, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants