Process.get_connections(): provide an argument to filter for connection types

[giampaolo]

From g.rodola on October 06, 2011 17:39:57

As of right now Process.get_connections() returns a list for all TCP and UDP 
connections opened by process, for both IPv4 and IPv6 address families.

One might be interested in filtering those results to, say, collect TCP 
connections only, or TCP + IPv6 only.

There's someone who already solved this problem in a fork of psutil by 
introducing a "kind" parameter accepting a string: 
https://github.com/elventear/psutil/commit/f2821283a411e52876384ad4aa73cb4db4973532
 The solution looks pretty nice to me in contrast to introducing something like 
two separate "family" and "type" arguments requiring to use socket.AF_* and 
socket.SOCK_* constants.

The possible solutions for the "kind" parameter would be the following:

        inet            IPv4 and IPv6
        inet4           IPv4
        inet6           IPv6
        tcp             TCP
        tcp4            TCP over IPv4
        tcp6            TCP over IPv6
        udp             UDP
        udp4            UDP over IPv4
        udp6            UDP over IPv6
        all             the sum of all the possible families and protocols

"kind" parameter should default to "inet", so that we remain compatible with 
the current implementation returning TCP + UDP + IPv4 + IPv6.
This would also be compatible when we'll add support for UNIX sockets ( issue 216 ). 

Example usage:

>>> p = psutil.Process(os.getpid())
>>> p.get_connections(kind="tcp")
[...]
>>> p.get_connections(kind="inet6")
[...]
>>>

Original issue: http://code.google.com/p/psutil/issues/detail?id=217

[giampaolo]

From [email protected] on October 06, 2011 08:45:59

Sounds like a good idea, only thing I'd suggest is using a different name for 
the parameter instead of "kind". Some alternative suggestions: 

* filter
* family
* proto (protocol)

[giampaolo]

From g.rodola on October 06, 2011 08:55:14

We can't use "family" and "proto" because the filter we're going to apply is 
for both of them (e.g. you can use kind="tcp4" to filter for TCP *protocol* AND 
IPv4 *family* in one shot).

-1 about calling this "filter" because we already have a python builtin with 
the same name.

[giampaolo]

From [email protected] on October 06, 2011 09:19:29

ok, how about "type" or "conn_type" then :)

[giampaolo]

From g.rodola on October 06, 2011 09:28:44

"family", "type" and "proto" are all blacklisted names because they recall the 
arguments passed to the socket constructor: 
http://docs.python.org/library/socket.html#socket.socket Generally speaking, 
all 3 of them have a precise meaning in the socket world, and the less we 
conflict with them, the more unambiguous we are.

Is there a particular reason why you don't like "kind"?

[giampaolo]

From [email protected] on October 06, 2011 09:38:48

"kind" just isn't very descriptive to me of what it does, I would prefer 
conn_type or something that indicates specifically what the parameter is for. 
Feel free to come up with other ideas, I was just throwing a few things out 
there off the top of my head :)

[giampaolo]

From g.rodola on October 06, 2011 15:21:20

Isn't "kind" a synonym of "type"?
Again, "conn_type" is ambiguous because it conflicts with "type".
If we exclude "family"/"type" and their variants I personally can't think of 
any other valid substitute (but as you know English is not my native language =)).

[giampaolo]

From [email protected] on October 06, 2011 16:45:39

The word "kind" has a number of different meanings in English, so it wouldn't 
be my ideal choice for something like a parameter name. For example, it can 
mean "gentle" so it could indicate an option to use a less aggressive 
algorithm. I would prefer something that's as clear as possible and does not 
have multiple meanings. How about conn_filter for example? It indicates that 
the parameter is a filter for the connections which is clear, and does not 
conflict with any built-ins or other meanings within the socket module.

[giampaolo]

From g.rodola on October 07, 2011 04:57:00

What we would really need here, if "type" wasn't a blacklisted name because of 
its correlation with sockets, is "type".
I think the most obvious thing do to is look for a synonym of "type" and 
between all the available choices ( 
http://dictionary.reverso.net/english-synonyms/type ) "kind" looks like the 
better one to me.

It is not perfect, but with the exception of "gentle"/"courteous" both "type" 
and "kind" have almost the exact same meaning, identifying a certain character, 
genre or category.
It appears clear to me what "kind" means in the context of something called 
"get_connections", because connections can't obviously be "gentle".
I mean, this is "get_connections()", not "get_mate()" or "get_husband()", right?  =)

Now, I'm not a native English speaker, but here: 
http://difference-between.com/english-language/kind-sort-and-type/ ..."kind" is 
described as a "named category, but not necessarily precise", which fits well 
with what we're returning: a certain types/kinds of mixed combinations which 
are not strictly related to each other such as tcp+ipv4, udp+ipv6, inet6, unix, etc.

As for "conn_filter":
- it is a repetition of "connection", which is already the name of the function
- it is longer
- it contains a "_", which we've been avoiding so far (see: "percpu", 
"perdisk", "pernic")

[giampaolo]

From [email protected] on October 12, 2011 16:03:15

I have a slightly different implementation of get_connections().  Right now, 
get_connections is broken on all but Linux because the OS-specific providers 
weren't updated to have the kind field and then use it.  My refactoring, 
attached, handles the connection kind filtering in 
psutil/__init__.get_connections(kind) much like how we handle totals for 
disk/network I/O counters.  So the provider's get_connections() will return all 
connections and then they are filtered in the get_connections(kind) 
implementation in __init__.py.

Attachment: get_connections_refactor.diff

[giampaolo]

From g.rodola on October 13, 2011 03:45:49

Point is getting all connections first (and create a nameduplte for each one of 
them) is less performant. 
This is particularly slow on Linux, where instead of calling a kernel function, 
we read different files in python.
Same for FreeBSD where this is done by parsing lsof output.

I think it's better to apply filter at a lower level (the C extension module in 
case of OSX and Windows).
For OSX I was working on this (uncommitted) patch which passes the desired 
families/types as arguments for the underlying C function. It's not very nice though.

Attachment: osx.patch

[giampaolo]

From [email protected] on October 13, 2011 09:14:20

Well, my purpose was to fix the builds so we can verify other things are 
working properly.  We could just use my approach and make enhancements to make 
it more performant in the meantime instead of having a broken psutil.  As for 
your patch, I'll look at it shortly.

[giampaolo]

From g.rodola on October 13, 2011 09:16:02

Feel free to commit your patch if it's for fixing tests. We can remove it later.

[giampaolo]

From [email protected] on October 13, 2011 09:23:40

Per our conversation, I've committed my patch as part of r1147 .  We can put a 
plan in place to move the implementation from Python to C where necessary, or 
for all platforms.  At least with this new patch trunk builds no longer fail on 
non-Linux and we can safely add new features and fix bugs due to the test suite 
running properly.

[giampaolo]

From [email protected] on October 13, 2011 09:26:30

I realize now that you wanted to not change _pslinux.py.  I will address this.

[giampaolo]

From g.rodola on October 13, 2011 11:52:54

Labels: Progress-3in4

[giampaolo]

From g.rodola on October 21, 2011 16:44:17

Labels: -Milestone-0.3.1

[giampaolo]

From g.rodola on October 21, 2011 16:45:27

Labels: Milestone-0.4.0

[giampaolo]

From g.rodola on October 22, 2011 08:51:02

Fixed for FreeBSD in r1186 .

Status: FixedInSVN
Labels: -Progress-3in4 Progress-4in4

[giampaolo]

From g.rodola on October 28, 2011 20:44:14

Status: Fixed

[giampaolo]

From g.rodola on March 02, 2013 04:04:17

Updated csets after the SVN -> Mercurial migration: r1147 == revision 
2e5e54fe37bd r1186 == revision 3b284f40b531