Segfault with Netflow Lite

[cardigliano]

Moved from ntop/ntopng#3432

nprobe --nflite 2055:16 -i none -n 10.1.11.34:9996
[...]
18/Feb/2020 09:36:04 [engine.c:1785] WARNING: Internal error: NULL head for index 66329 [num_runs: 1][thread_id: 0]

gdb --args nprobe --nflite 2055:16 -i none -n 10.1.11.34:9996
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-115.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/...
Reading symbols from /usr/local/bin/nprobe...(no debugging symbols found)...done.
(gdb) run
Starting program: /usr/local/bin/nprobe --nflite 2055:16 -i none -n 10.1.11.34:9996
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
18/Feb/2020 12:39:58 [plugin.c:177] No plugins found in ./plugins
18/Feb/2020 12:39:58 [plugin.c:185] Loading 23 plugins [.so] from /usr/local/lib/nprobe/plugins
18/Feb/2020 12:39:58 [plugin.c:873] Unable to enable plugin DHCP Protocol [/etc/nprobe.license.dhcp]: Missing license file
18/Feb/2020 12:39:58 [plugin.c:873] Unable to enable plugin Diameter Protocol [/etc/nprobe.license.diameter]: Missing license file
18/Feb/2020 12:39:58 [plugin.c:873] Unable to enable plugin DNS/LLMNR Protocol [/etc/nprobe.license]: Missing license file
18/Feb/2020 12:39:58 [plugin.c:873] Unable to enable plugin Export Plugin [/etc/nprobe.license.export]: Missing license file
18/Feb/2020 12:39:58 [plugin.c:873] Unable to enable plugin FTP Protocol [/etc/nprobe.license.ftp]: Missing license file
18/Feb/2020 12:39:58 [plugin.c:873] Unable to enable plugin GTPv0 Signaling Protocol [/etc/nprobe.license.gtpv0]: Missing license file
18/Feb/2020 12:39:58 [plugin.c:873] Unable to enable plugin GTPv1 Signaling Protocol [/etc/nprobe.license.gtpv1]: Missing license file
18/Feb/2020 12:39:58 [plugin.c:873] Unable to enable plugin GTPv2 Signaling Protocol [/etc/nprobe.license.gtpv2]: Missing license file
18/Feb/2020 12:39:59 [plugin.c:873] Unable to enable plugin HTTP Protocol [/etc/nprobe.license]: Missing license file
18/Feb/2020 12:39:59 [plugin.c:873] Unable to enable plugin IMAP Protocol [/etc/nprobe.license.email]: Missing license file
18/Feb/2020 12:39:59 [plugin.c:873] Unable to enable plugin POP3 Protocol [/etc/nprobe.license.email]: Missing license file
18/Feb/2020 12:39:59 [plugin.c:873] Unable to enable plugin Radius Protocol [/etc/nprobe.license.radius]: Missing license file
18/Feb/2020 12:39:59 [plugin.c:873] Unable to enable plugin RTP Plugin [/etc/nprobe.license.voip]: Missing license file
18/Feb/2020 12:39:59 [plugin.c:873] Unable to enable plugin SIP Plugin [/etc/nprobe.license.voip]: Missing license file
18/Feb/2020 12:39:59 [plugin.c:873] Unable to enable plugin SMTP Protocol [/etc/nprobe.license.email]: Missing license file
18/Feb/2020 12:39:59 [nprobe.c:4530] Valid nProbe Pro license found
18/Feb/2020 12:39:59 [nprobe.c:6478] IMPORTANT: Enabling NflitePlugin will also enable IP address forging, thus
18/Feb/2020 12:39:59 [nprobe.c:6479] IMPORTANT: flows appear as they were sent from the NflitePlugin-enabled switch
18/Feb/2020 12:39:59 [nprobe.c:4020] Exporting flows towards 10.1.11.34:9996 using UDP
18/Feb/2020 12:39:59 [nprobe.c:6578] WARNING: The output interfaceId is set to 0: did you forget to use -Q perhaps ?
18/Feb/2020 12:39:59 [nprobe.c:6581] WARNING: The input interfaceId is set to 0: did you forget to use -u perhaps ?
18/Feb/2020 12:39:59 [nprobe.c:6668] Welcome to Pro nProbe v.8.7.200218 ($Revision: 6753 $) for x86_64-unknown-linux-gnu with native PF_RING acceleration
18/Feb/2020 12:39:59 [nprobe.c:6679] Running on CentOS Linux release 7.7.1908 (Core)
18/Feb/2020 12:39:59 [nprobe.c:6690] [LICENSE] nProbe SystemId: 689ED9C776066B16
18/Feb/2020 12:39:59 [nprobe.c:6761] Sample rate [packet: 1][flow collection/export: 1/1]
18/Feb/2020 12:39:59 [modbusPlugin.c:104] [MODBUS] Idle flow timeout set to 120 sec
18/Feb/2020 12:39:59 [nflitePlugin.c:879] ERROR: Flow collector port 2055/IPv6 already in use ? [Address family not supported by protocol/97]: disabling collection over IPv6
[New Thread 0x7fffec65c700 (LWP 35757)]
[New Thread 0x7fffebe5b700 (LWP 35758)]
[New Thread 0x7fffeb65a700 (LWP 35759)]
[New Thread 0x7fffeae59700 (LWP 35760)]
[New Thread 0x7fffea658700 (LWP 35761)]
[New Thread 0x7fffe9e57700 (LWP 35762)]
[New Thread 0x7fffe9656700 (LWP 35763)]
[New Thread 0x7fffe8e55700 (LWP 35764)]
[New Thread 0x7fffe8654700 (LWP 35765)]
[New Thread 0x7fffe7e53700 (LWP 35766)]
[New Thread 0x7fffe7652700 (LWP 35767)]
[New Thread 0x7fffe6e51700 (LWP 35768)]
[New Thread 0x7fffe6650700 (LWP 35769)]
[New Thread 0x7fffe5e4f700 (LWP 35770)]
[New Thread 0x7fffe564e700 (LWP 35771)]
[New Thread 0x7fffe4e4d700 (LWP 35772)]
18/Feb/2020 12:39:59 [nflitePlugin.c:903] [NFLite] Listening on port range 2055-2070 (16)
18/Feb/2020 12:39:59 [nprobe.c:9589] Welcome to nProbe v.8.7.200218 for x86_64-unknown-linux-gnu
18/Feb/2020 12:39:59 [nprobe.c:8546] You selected v9/IPFIX without specifying a template (-T).
18/Feb/2020 12:39:59 [nprobe.c:8547] The default template will be used
18/Feb/2020 12:39:59 [nprobe.c:8554] Using NetFlow Packet Payload Len: 1472
18/Feb/2020 12:39:59 [nprobe.c:8589] Flow export type: unidirectional flows
18/Feb/2020 12:39:59 [plugin.c:1309] 1 plugin(s) enabled
18/Feb/2020 12:39:59 [nprobe.c:9024] Each flow is 71 bytes long
18/Feb/2020 12:39:59 [nprobe.c:9025] The # flows per packet has been set to 19
18/Feb/2020 12:39:59 [nprobe.c:9028] IP TOS is ignored
18/Feb/2020 12:39:59 [nprobe.c:9056] Non IPv4/v6 traffic is discarded according to the template
18/Feb/2020 12:39:59 [nprobe.c:9872] Flows ASs will not be computed (no GeoDB files loaded)
18/Feb/2020 12:39:59 [nprobe.c:9977] Not capturing packet from interface (collector mode)
18/Feb/2020 12:39:59 [plugin.c:961] Enabling plugin Netflow-Lite Plugin
[New Thread 0x7fffe464c700 (LWP 35774)]
[New Thread 0x7fffe3e4b700 (LWP 35775)]
18/Feb/2020 12:39:59 [export.c:543] Using TLV as serialization format
18/Feb/2020 12:39:59 [nprobe.c:10239] nProbe started successfully

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe9e57700 (LWP 35762)]
0x00007ffff79262b4 in walkHashList () from /usr/local/lib/libnprobe-8.7.200218.so
Missing separate debuginfos, use: debuginfo-install nprobe-8.7.200218-6753.x86_64
(gdb) bt
#0 0x00007ffff79262b4 in walkHashList () from /usr/local/lib/libnprobe-8.7.200218.so
#1 0x00007ffff7930590 in idleThreadTask () from /usr/local/lib/libnprobe-8.7.200218.so
#2 0x00007ffff7929aea in processFlowPacket () from /usr/local/lib/libnprobe-8.7.200218.so
#3 0x000000000041989f in ?? ()
#4 0x000000000041a1cd in decodePacket ()
#5 0x00007fffed7d6c13 in readNFlitePackets () from /usr/local/lib/nprobe/plugins/libnflitePlugin-8.7.200218.so
#6 0x00007ffff646ae65 in start_thread () from /lib64/libpthread.so.0
#7 0x00007ffff489188d in clone () from /lib64/libc.so.6
(gdb)

[cardigliano]

@christianbj87 are you able to provide a PCAP file (with the nflite nProbe is processing)to reproduce this issue? Thank you.

[christianbj87]

@cardigliano Hi Alfredo, can you guide me please?

[cardigliano]

@christianbj87 are you able to run tcpdump on the system where nprobe is running?
Something like "tcpdump -ni any -w nflite.pcap port 2055" should work.
Please drop me a direct email at cardigliano at ntop.org

[christianbj87]

@cardigliano Done, emailed.

[cardigliano]

@christianbj87 a new (dev/nightly) package will be available in ~1h, it includes a patch that (likely) fixes this, please let me know.