Implement psutil_proc_exe for NetBSD #1534

krytarowski · 2019-06-12T12:57:44Z

Keep fallback for versions <8.0 without KERN_PROC_PATHNAME.

psutil/_psbsd.py

giampaolo · 2019-06-12T14:59:40Z

Mmm... on a second thought I am not sure I understand why this change is necessary. Isn't pure-python return os.readlink("/proc/%s/exe" % self.pid) supposed to work on all NetBSD vesions?

krytarowski · 2019-06-12T15:00:35Z

/proc is optional for Linux compat only.

giampaolo · 2019-06-12T15:05:53Z

Uh? No, Linux is not involved. This is (Net)BSD specific code which is not shared with the Linux implementation.

krytarowski · 2019-06-12T15:06:28Z

/proc on NetBSD is kept for Linux compat only.

krytarowski · 2019-06-12T15:09:25Z

All native applications shall use sysctl, mounting /proc shall be restricted to running emulated linux binaries only. In practice this is sometimes violated, but that is abnormal state.

krytarowski · 2019-06-12T15:15:02Z

And regarding pid 0, it has to be removed from the list of processes on NetBSD. If some operation for it works, it's rather accident.

psutil/arch/netbsd/specific.c

giampaolo · 2019-06-12T15:57:01Z

All native applications shall use sysctl, mounting /proc shall be restricted to running emulated linux binaries only. In practice this is sometimes violated, but that is abnormal state.

Are you saying /proc on NetBSD is broken, deprecated or something? I'm merely trying to understand the rationale here as there are 2 other functions using it.

And regarding pid 0, it has to be removed from the list of processes on NetBSD. If some operations for it works, it's rather accident.

Nope, PID 0 is a legitimate PID. The kernel itself returns it and so does psutil.pids() function. Most importantly, there are processes having PID 0 as parent, meaning both ppid() and parent/s() methods return it. Those can be used to implement tools like pstree, where the concept of the "lowest" PID is important.

krytarowski · 2019-06-12T16:03:44Z

Are you saying /proc on NetBSD is broken, deprecated or something? I'm merely trying to understand the rationale here as there are 2 other functions using it.

It's not a native interface. It's a compat layer for Linux binaries as NetBSD can run native Linux programs as is. The same applies to FreeBSD. OpenBSD doesn't support compat and deleted /proc completely.

I don't want to go into historical notes as it was.. a replacement for ptrace(2) back in the days. That property is long gone.

In short it shall not be used in modern NetBSD software. We can tell that it is formally deprecated, as adding there !Linux interfaces need special core approval. Depending on /proc mounted on NetBSD is at most a compat fallback. It's not needed to be mounted for native programs (and it shall be unmounted).

krytarowski · 2019-06-12T16:09:38Z

Nope, PID 0 is a legitimate PID. The kernel itself returns it and so does psutil.pids() function. Most importantly, there are processes having PID 0 as parent, meaning both ppid() and parent/s() methods return it. Those can be used to implement tools like pstree, where the concept of the "lowest" PID is important.

I question this. The lowest meaningful PID that can be a parent for other processes is pid1 (/sbin/init). You cannot reparent anything to pid0 (possibly an exception is pid1).

Usability of pid0 as process is probably restricted to programs like ps(1) or top(1) to a little bit introspect into the kernel... but I don't know the original rationale.

krytarowski · 2019-06-12T16:16:15Z

there are 2 other functions using it.

I intend to switch psutil to native interfaces for all calls that are applicable for NetBSD.

krytarowski · 2019-06-12T18:50:09Z

After some discussions, I think we should add missing interfaces for pid0 to return somewhat fake values for queries that in general make no sense for kernel anyway.. this is better than erroring.

giampaolo · 2019-06-13T04:43:09Z

I see no error. Running as limited user:

>>> pp(psutil.Process(0).as_dict())
{'cmdline': [],
 'connections': [],
 'cpu_percent': 0.0,
 'cpu_times': pcputimes(user=0.0, system=8.595674, children_user=0.0, children_system=0.0),
 'create_time': 1553961708.000003,
 'cwd': '/',
 'exe': '',
 'gids': pgids(real=0, effective=0, saved=0),
 'io_counters': pio(read_count=13, write_count=6061, read_bytes=-1, write_bytes=-1),
 'memory_full_info': pmem(rss=7639040, vms=0, text=0, data=0, stack=0),
 'memory_info': pmem(rss=7639040, vms=0, text=0, data=0, stack=0),
 'memory_percent': 0.3694949707078652,
 'name': 'system',
 'nice': 0,
 'num_ctx_switches': pctxsw(voluntary=305175, involuntary=0),
 'num_fds': 236,
 'num_threads': 39,
 'open_files': [popenfile(path='', fd=8),
                popenfile(path='', fd=8),
                popenfile(path='', fd=8),
                popenfile(path='', fd=8),
                popenfile(path='', fd=3),
                popenfile(path='', fd=10),
                popenfile(path='', fd=11),
                popenfile(path='', fd=11),
                popenfile(path='', fd=12),
                popenfile(path='', fd=13),
                popenfile(path='', fd=14),
                popenfile(path='', fd=15),
                popenfile(path='', fd=16),
                popenfile(path='', fd=17),
                popenfile(path='', fd=3)],
 'pid': 0,
 'ppid': 0,
 'status': 'running',
 'terminal': None,
 'threads': [pthread(id=50, user_time=2.884953, system_time=2.884953),
             pthread(id=49, user_time=2.387705, system_time=2.387705),
             pthread(id=48, user_time=2.051632, system_time=2.051632),
             pthread(id=47, user_time=1.5644019999999998, system_time=1.5644019999999998),
             pthread(id=46, user_time=4.2e-05, system_time=4.2e-05),
             pthread(id=45, user_time=1.579319, system_time=1.579319),
             pthread(id=44, user_time=1.396202, system_time=1.396202),
             pthread(id=43, user_time=4e-06, system_time=4e-06),
             pthread(id=39, user_time=0.0, system_time=0.0),
             pthread(id=38, user_time=0.0, system_time=0.0),
             pthread(id=37, user_time=0.003768, system_time=0.003768),
             pthread(id=36, user_time=0.005629, system_time=0.005629),
             pthread(id=27, user_time=0.044605, system_time=0.044605),
             pthread(id=26, user_time=0.0, system_time=0.0),
             pthread(id=25, user_time=0.0, system_time=0.0),
             pthread(id=24, user_time=0.0, system_time=0.0),
             pthread(id=23, user_time=0.0, system_time=0.0),
             pthread(id=22, user_time=4.6e-05, system_time=4.6e-05),
             pthread(id=21, user_time=0.0, system_time=0.0),
             pthread(id=20, user_time=0.0, system_time=0.0),
             pthread(id=19, user_time=0.0, system_time=0.0),
             pthread(id=18, user_time=0.002833, system_time=0.002833),
             pthread(id=17, user_time=0.0, system_time=0.0),
             pthread(id=16, user_time=0.070223, system_time=0.070223),
             pthread(id=15, user_time=0.0, system_time=0.0),
             pthread(id=14, user_time=0.0, system_time=0.0),
             pthread(id=13, user_time=0.0, system_time=0.0),
             pthread(id=12, user_time=0.0, system_time=0.0),
             pthread(id=11, user_time=0.04194, system_time=0.04194),
             pthread(id=10, user_time=0.033575, system_time=0.033575),
             pthread(id=9, user_time=0.029574, system_time=0.029574),
             pthread(id=8, user_time=0.0, system_time=0.0),
             pthread(id=7, user_time=0.000146, system_time=0.000146),
             pthread(id=6, user_time=0.0, system_time=0.0),
             pthread(id=5, user_time=1e-06, system_time=1e-06),
             pthread(id=4, user_time=0.01054, system_time=0.01054),
             pthread(id=3, user_time=0.011838, system_time=0.011838),
             pthread(id=2, user_time=0.0, system_time=0.0),
             pthread(id=1, user_time=0.08237, system_time=0.08237)],
 'uids': puids(real=0, effective=0, saved=0),
 'username': 'root'}
>>>

On other platforms certain methods will fail with AccessDenied, which is fine. ps, top and other tools report it, psutil merely does the same. PID 0 is an old topic which was analyzed many times on all platforms already. Unless a clear wrong value is returned by kernel there is nothing to fix here, including when kernel returns a "null" value, such as an empty cmdline.

krytarowski · 2019-06-13T14:07:59Z

After some more research there is no way to determine exe name for pid0. Probably better than returning some /nonexistent, return an error from the kernel and handle this in a userland program.

krytarowski · 2019-06-13T14:10:32Z

 'threads': [pthread(id=50, user_time=2.884953, system_time=2.884953),
             pthread(id=49, user_time=2.387705, system_time=2.387705),
             pthread(id=48, user_time=2.051632, system_time=2.051632),

pthread does not make sense for kernel.

It is also not that clear for userland as not every threaded program uses POSIX threads, e.g. golang reinvents them.

On NetBSD there are native threads (LWP) and they can be wrapped with libpthread to compose POSIX C-style threads.

giampaolo · 2019-06-13T15:07:28Z

After some more research there is no way to determine exe name for pid0. Probably better than returning some /nonexistent, return an error from the kernel and handle this in a userland program.

What error are you getting?

...as for threads, regardless from PID 0 there are some failing tests (see make test-process), so it is possible that threads() implementation on NetBSD is broken.

krytarowski · 2019-06-13T15:09:34Z

After some more research there is no way to determine exe name for pid0. Probably better than returning some /nonexistent, return an error from the kernel and handle this in a userland program.

What error are you getting?

The kernel can be loaded from other source than filesystem (e.g. passed with qemu -kernel) and then there is no entry. It also does not store it's name on disk (typically /netbsd). This means that it is better to return nothing from the kernel rather than trying to guess.

giampaolo · 2019-06-13T15:11:34Z

OK but what python exception are you seeing if you query PID 0?

krytarowski · 2019-06-13T15:15:07Z

#1534 (comment)

giampaolo · 2019-06-13T15:18:41Z

As I said AccessDenied is legitimate.

giampaolo · 2019-06-14T09:40:10Z

It turns out this approach was already tried and in fact there is even a commented code doing what you did in here (but I forgot):

psutil/psutil/arch/netbsd/specific.c

Line 153 in 6bb5a30

// XXX: This is no longer used as per

Your patch differs in that it does not dynamically determine the size via sysctl.

giampaolo · 2019-06-14T09:42:07Z

Did you run tests? Do you get the same failure reported in #557 (comment)?

krytarowski · 2019-06-14T10:47:13Z

I will update my code tonight.

Keep fallback for versions <8.0 without KERN_PROC_PATHNAME.

krytarowski · 2019-06-15T13:57:49Z

I don't know how to run tests. I'm just checking new functions manually.

$ python2.7 setup.py build 
running build
running build_py
creating build
creating build/lib.netbsd-8.99.45-amd64-2.7
creating build/lib.netbsd-8.99.45-amd64-2.7/psutil
copying psutil/__init__.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil
copying psutil/_common.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil
copying psutil/_compat.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil
copying psutil/_psaix.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil
copying psutil/_psbsd.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil
copying psutil/_pslinux.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil
copying psutil/_psosx.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil
copying psutil/_psposix.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil
copying psutil/_pssunos.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil
copying psutil/_pswindows.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil
creating build/lib.netbsd-8.99.45-amd64-2.7/psutil/tests
copying psutil/tests/__init__.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil/tests
copying psutil/tests/__main__.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil/tests
copying psutil/tests/runner.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil/tests
copying psutil/tests/test_aix.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil/tests
copying psutil/tests/test_bsd.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil/tests
copying psutil/tests/test_connections.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil/tests
copying psutil/tests/test_contracts.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil/tests
copying psutil/tests/test_linux.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil/tests
copying psutil/tests/test_memory_leaks.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil/tests
copying psutil/tests/test_misc.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil/tests
copying psutil/tests/test_osx.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil/tests
copying psutil/tests/test_posix.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil/tests
copying psutil/tests/test_process.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil/tests
copying psutil/tests/test_sunos.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil/tests
copying psutil/tests/test_system.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil/tests
copying psutil/tests/test_unicode.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil/tests
copying psutil/tests/test_windows.py -> build/lib.netbsd-8.99.45-amd64-2.7/psutil/tests
running build_ext
building 'psutil._psutil_bsd' extension
creating build/temp.netbsd-8.99.45-amd64-2.7
creating build/temp.netbsd-8.99.45-amd64-2.7/psutil
creating build/temp.netbsd-8.99.45-amd64-2.7/psutil/arch
creating build/temp.netbsd-8.99.45-amd64-2.7/psutil/arch/netbsd
clang -fno-strict-aliasing -OPT:Olimit=0 -O2 -fwrapv -pthread -I/usr/include -I/usr/pkg/include -DNDEBUG -O2 -fwrapv -pthread -I/usr/include -I/usr/pkg/include -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_BSD=1 -DPSUTIL_VERSION=563 -DPSUTIL_NETBSD=1 -I/usr/pkg/include/python2.7 -c psutil/_psutil_common.c -o build/temp.netbsd-8.99.45-amd64-2.7/psutil/_psutil_common.o
clang -fno-strict-aliasing -OPT:Olimit=0 -O2 -fwrapv -pthread -I/usr/include -I/usr/pkg/include -DNDEBUG -O2 -fwrapv -pthread -I/usr/include -I/usr/pkg/include -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_BSD=1 -DPSUTIL_VERSION=563 -DPSUTIL_NETBSD=1 -I/usr/pkg/include/python2.7 -c psutil/_psutil_posix.c -o build/temp.netbsd-8.99.45-amd64-2.7/psutil/_psutil_posix.o
clang -fno-strict-aliasing -OPT:Olimit=0 -O2 -fwrapv -pthread -I/usr/include -I/usr/pkg/include -DNDEBUG -O2 -fwrapv -pthread -I/usr/include -I/usr/pkg/include -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_BSD=1 -DPSUTIL_VERSION=563 -DPSUTIL_NETBSD=1 -I/usr/pkg/include/python2.7 -c psutil/_psutil_bsd.c -o build/temp.netbsd-8.99.45-amd64-2.7/psutil/_psutil_bsd.o
clang -fno-strict-aliasing -OPT:Olimit=0 -O2 -fwrapv -pthread -I/usr/include -I/usr/pkg/include -DNDEBUG -O2 -fwrapv -pthread -I/usr/include -I/usr/pkg/include -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_BSD=1 -DPSUTIL_VERSION=563 -DPSUTIL_NETBSD=1 -I/usr/pkg/include/python2.7 -c psutil/arch/netbsd/specific.c -o build/temp.netbsd-8.99.45-amd64-2.7/psutil/arch/netbsd/specific.o
clang -fno-strict-aliasing -OPT:Olimit=0 -O2 -fwrapv -pthread -I/usr/include -I/usr/pkg/include -DNDEBUG -O2 -fwrapv -pthread -I/usr/include -I/usr/pkg/include -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_BSD=1 -DPSUTIL_VERSION=563 -DPSUTIL_NETBSD=1 -I/usr/pkg/include/python2.7 -c psutil/arch/netbsd/socks.c -o build/temp.netbsd-8.99.45-amd64-2.7/psutil/arch/netbsd/socks.o
clang -shared -L/tmp/pkgsrc-tmp/lang/python27/work/Python-2.7.16 -pthread -L/usr/lib -Wl,-R/usr/lib -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -L/usr/X11R7/lib -Wl,-R/usr/X11R7/lib build/temp.netbsd-8.99.45-amd64-2.7/psutil/_psutil_common.o build/temp.netbsd-8.99.45-amd64-2.7/psutil/_psutil_posix.o build/temp.netbsd-8.99.45-amd64-2.7/psutil/_psutil_bsd.o build/temp.netbsd-8.99.45-amd64-2.7/psutil/arch/netbsd/specific.o build/temp.netbsd-8.99.45-amd64-2.7/psutil/arch/netbsd/socks.o -L/usr/pkg/lib -lkvm -lpython2.7 -o build/lib.netbsd-8.99.45-amd64-2.7/psutil/_psutil_bsd.so
/usr/bin/ld: /usr/pkg/lib/libpython2.7.so: warning: warning: tmpnam() possibly used unsafely, use mkstemp() or mkdtemp()
/usr/bin/ld: /usr/pkg/lib/libpython2.7.so: warning: warning: tempnam() possibly used unsafely, use mkstemp() or mkdtemp()
building 'psutil._psutil_posix' extension
clang -fno-strict-aliasing -OPT:Olimit=0 -O2 -fwrapv -pthread -I/usr/include -I/usr/pkg/include -DNDEBUG -O2 -fwrapv -pthread -I/usr/include -I/usr/pkg/include -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_BSD=1 -DPSUTIL_VERSION=563 -DPSUTIL_NETBSD=1 -I/usr/pkg/include/python2.7 -c psutil/_psutil_common.c -o build/temp.netbsd-8.99.45-amd64-2.7/psutil/_psutil_common.o
clang -fno-strict-aliasing -OPT:Olimit=0 -O2 -fwrapv -pthread -I/usr/include -I/usr/pkg/include -DNDEBUG -O2 -fwrapv -pthread -I/usr/include -I/usr/pkg/include -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_BSD=1 -DPSUTIL_VERSION=563 -DPSUTIL_NETBSD=1 -I/usr/pkg/include/python2.7 -c psutil/_psutil_posix.c -o build/temp.netbsd-8.99.45-amd64-2.7/psutil/_psutil_posix.o
clang -shared -L/tmp/pkgsrc-tmp/lang/python27/work/Python-2.7.16 -pthread -L/usr/lib -Wl,-R/usr/lib -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -L/usr/X11R7/lib -Wl,-R/usr/X11R7/lib build/temp.netbsd-8.99.45-amd64-2.7/psutil/_psutil_common.o build/temp.netbsd-8.99.45-amd64-2.7/psutil/_psutil_posix.o -L/usr/pkg/lib -lpython2.7 -o build/lib.netbsd-8.99.45-amd64-2.7/psutil/_psutil_posix.so
/usr/bin/ld: /usr/pkg/lib/libpython2.7.so: warning: warning: tmpnam() possibly used unsafely, use mkstemp() or mkdtemp()
/usr/bin/ld: /usr/pkg/lib/libpython2.7.so: warning: warning: tempnam() possibly used unsafely, use mkstemp() or mkdtemp()
146 kamil@chieftec /public/psutil $ python2.7 setup.py test  
running test
Searching for ipaddress
Reading https://pypi.org/simple/ipaddress/
Downloading https://files.pythonhosted.org/packages/fc/d0/7fc3a811e011d4b388be48a0e381db8d990042df54aa4ef4599a31d39853/ipaddress-1.0.22-py2.py3-none-any.whl#sha256=64b28eec5e78e7510698f6d4da08800a5c575caa4a286c93d651c5d3ff7b6794
Best match: ipaddress 1.0.22
Processing ipaddress-1.0.22-py2.py3-none-any.whl
Installing ipaddress-1.0.22-py2.py3-none-any.whl to /public/psutil/.eggs

Installed /public/psutil/.eggs/ipaddress-1.0.22-py2.7.egg
Searching for mock
Reading https://pypi.org/simple/mock/
Downloading https://files.pythonhosted.org/packages/05/d2/f94e68be6b17f46d2c353564da56e6fb89ef09faeeff3313a046cb810ca9/mock-3.0.5-py2.py3-none-any.whl#sha256=d157e52d4e5b938c550f39eb2fd15610db062441a9c2747d3dbfa9298211d0f8
Best match: mock 3.0.5
Processing mock-3.0.5-py2.py3-none-any.whl
Installing mock-3.0.5-py2.py3-none-any.whl to /public/psutil/.eggs
writing requirements to /public/psutil/.eggs/mock-3.0.5-py2.7.egg/EGG-INFO/requires.txt

Installed /public/psutil/.eggs/mock-3.0.5-py2.7.egg
running egg_info
creating psutil.egg-info
writing requirements to psutil.egg-info/requires.txt
writing psutil.egg-info/PKG-INFO
writing top-level names to psutil.egg-info/top_level.txt
writing dependency_links to psutil.egg-info/dependency_links.txt
writing manifest file 'psutil.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'psutil.egg-info/SOURCES.txt'
running build_ext
copying build/lib.netbsd-8.99.45-amd64-2.7/psutil/_psutil_bsd.so -> psutil
copying build/lib.netbsd-8.99.45-amd64-2.7/psutil/_psutil_posix.so -> psutil
Traceback (most recent call last):
  File "setup.py", line 350, in <module>
    main()
  File "setup.py", line 346, in main
    setup(**kwargs)
  File "/usr/pkg/lib/python2.7/site-packages/setuptools/__init__.py", line 145, in setup
    return distutils.core.setup(**attrs)
  File "/usr/pkg/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/pkg/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/pkg/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/pkg/lib/python2.7/site-packages/setuptools/command/test.py", line 229, in run
    self.run_tests()
  File "/usr/pkg/lib/python2.7/site-packages/setuptools/command/test.py", line 251, in run_tests
    exit=False,
  File "/usr/pkg/lib/python2.7/unittest/main.py", line 94, in __init__
    self.parseArgs(argv)
  File "/usr/pkg/lib/python2.7/unittest/main.py", line 149, in parseArgs
    self.createTests()
  File "/usr/pkg/lib/python2.7/unittest/main.py", line 158, in createTests
    self.module)
  File "/usr/pkg/lib/python2.7/unittest/loader.py", line 130, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/usr/pkg/lib/python2.7/unittest/loader.py", line 100, in loadTestsFromName
    parent, obj = obj, getattr(obj, part)
AttributeError: 'module' object has no attribute 'get_suite'

krytarowski · 2020-01-29T04:53:12Z

Ping?

giampaolo · 2020-01-29T23:39:02Z

Please refresh me on this. What's wrong with doing os.readlink("/proc/%s/exe" % self.pid) as we currently do? You said:

/proc on NetBSD is kept for Linux compat only.

...but AFAIK /proc is always available on NetBSD. Also, we rely on /proc also for other things (/proc/stat and /proc/meminfo).

krytarowski · 2020-01-30T07:46:06Z

The official interface for prompting this data is sysctl(3). /proc is a legacy interface from BSD4.4 and it happened to have some compat use with Linux, but it is deprecated for BSD software. Ideally we would remove it completely (some other BSDs already did it).

It is available only if we mount it, it's not enabled by default.

krytarowski · 2020-01-30T08:10:44Z

we rely on /proc also for other things (/proc/stat and /proc/meminfo). we need to switch everything to native interfaces, but it needs some collaboration with upstream to do it more actively.

krytarowski · 2020-01-30T08:21:09Z

The only valid use case of /proc today on NetBSD is running Linux prebuilt binary with compat_linux(8)/compat_linux32(8) in operation. In all other cases we use sysctl(8).

iamleot · 2020-01-30T09:20:46Z

Kamil Rytarowski writes:

[...] It is available only if we mount it, it's not enabled by default. [...]

That's not correct: sysinst(8) always adds an entry in fstab and enable it by default (i.e. I would expect it enabled by default on most installations).

krytarowski · 2020-01-30T12:36:23Z

Kamil Rytarowski writes:
[...] It is available only if we mount it, it's not enabled by default. [...]
That's not correct: sysinst(8) always adds an entry in fstab and enable it by default (i.e. I would expect it enabled by default on most installations).

Interesting. I missed it, but the statement about preference on sysctl stands.

krytarowski · 2020-01-30T12:36:43Z

There was also a core@ statement in NetBSD on deprecation of /proc at some point.

giampaolo · 2020-01-30T18:05:51Z

There was also a core@ statement in NetBSD on deprecation of /proc at some point.

I googled for "netbsd procfs deprecation" and similar but nothing relevant came up. I found different references of procfs on NetBSD but nothing that suggests a deprecation. AFAICT /proc on NetBSD currently looks alive and supported.

krytarowski · 2020-01-30T18:13:44Z

I don't know about a public one, but as I worked on ptrace(2) there was a statement from a core member that any extra work with /proc (that is not for compat for linux executables) needs approval. Today new APIs and interfaces are not implemented in /proc.

krytarowski · 2020-01-30T18:15:26Z

I actually removed some /proc functionality already too.

tuxillo · 2020-01-30T18:46:34Z

Generally speaking the BSDs have been trying to move away from procfs for a long time already. And even so if it's kind of supported, meaning that it can be mounted, one should not rely on its presence. In many systems it just won't be mounted by default.

For example FreeBSD doesn't even mount it by default, OpenBSD doesn't , DragonFly BSD still mounts it but does not recommend it (and in fact doesn't even use it during their binary package build).

sysctl is a much more robust interface to extract information and operate some system parameters, it should be used whenever possible in the BSDs.

HTH

giampaolo · 2020-01-30T20:43:32Z

@krytarowski KERN_PROC_PATHNAME is currently used by FreeBSD implementation:

psutil/psutil/arch/freebsd/specific.c

Line 232 in 994c429

psutil_proc_exe(PyObject *self, PyObject *args) {

If you want, you can move that into _psutil_bsd.c so that it's used by NetBSD and FreeBSD (I think it won't work on OpenBSD, but I may be wrong).

Once you do that let's see the test output though, because there is a comment saying KERN_PROC_PATHNAME was used on NetBSD but it caused problems (and hence why we're now relying on /proc).

psutil/psutil/arch/netbsd/specific.c

Line 155 in 994c429

    
           // https://github.com/giampaolo/psutil/pull/557#issuecomment-171912820

krytarowski · 2020-01-30T20:59:35Z

If you want, you can move that into _psutil_bsd.c so that it's used by NetBSD and FreeBSD

Both BSDs use different arguments. I am not motivated to pretend that BSDs are similar as they are not.

Once you do that let's see the test output though, because there is a comment saying KERN_PROC_PATHNAME was used on NetBSD but it caused problems (and hence why we're now relying on /proc).

/proc and sysctl use the same functionality in the kernel. In general this feature was improved internally to be more reliable over few years. There were issues with long filenames or hardsymlink confusion in a path etc.

giampaolo · 2020-01-30T21:21:21Z

Both BSDs use different arguments.

They look the same to me. :-\

giampaolo reviewed Jun 12, 2019

View reviewed changes

psutil/_psbsd.py Show resolved Hide resolved

giampaolo reviewed Jun 12, 2019

View reviewed changes

psutil/arch/netbsd/specific.c Show resolved Hide resolved

Implement psutil_proc_exe for NetBSD

7199fe6

Keep fallback for versions <8.0 without KERN_PROC_PATHNAME.

krytarowski force-pushed the netbsd-3 branch from 39373d0 to 7199fe6 Compare June 15, 2019 13:55

giampaolo added the netbsd label Nov 15, 2020

Implement psutil_proc_exe for NetBSD #1534

Are you sure you want to change the base?

Implement psutil_proc_exe for NetBSD #1534

Conversation

krytarowski commented Jun 12, 2019

giampaolo commented Jun 12, 2019

krytarowski commented Jun 12, 2019

giampaolo commented Jun 12, 2019

krytarowski commented Jun 12, 2019

krytarowski commented Jun 12, 2019

krytarowski commented Jun 12, 2019 • edited Loading

giampaolo commented Jun 12, 2019 • edited Loading

krytarowski commented Jun 12, 2019 • edited Loading

krytarowski commented Jun 12, 2019 • edited Loading

krytarowski commented Jun 12, 2019

krytarowski commented Jun 12, 2019 • edited Loading

giampaolo commented Jun 13, 2019

krytarowski commented Jun 13, 2019 • edited Loading

krytarowski commented Jun 13, 2019 • edited Loading

giampaolo commented Jun 13, 2019 • edited Loading

krytarowski commented Jun 13, 2019 • edited Loading

giampaolo commented Jun 13, 2019

krytarowski commented Jun 13, 2019

giampaolo commented Jun 13, 2019

giampaolo commented Jun 14, 2019

giampaolo commented Jun 14, 2019

krytarowski commented Jun 14, 2019

krytarowski commented Jun 15, 2019

krytarowski commented Jan 29, 2020

giampaolo commented Jan 29, 2020

krytarowski commented Jan 30, 2020

krytarowski commented Jan 30, 2020

krytarowski commented Jan 30, 2020

iamleot commented Jan 30, 2020 via email

krytarowski commented Jan 30, 2020

krytarowski commented Jan 30, 2020

giampaolo commented Jan 30, 2020

krytarowski commented Jan 30, 2020

krytarowski commented Jan 30, 2020

tuxillo commented Jan 30, 2020

giampaolo commented Jan 30, 2020

krytarowski commented Jan 30, 2020

giampaolo commented Jan 30, 2020

krytarowski commented Jun 12, 2019 •

edited

Loading

giampaolo commented Jun 12, 2019 •

edited

Loading

krytarowski commented Jun 12, 2019 •

edited

Loading

krytarowski commented Jun 12, 2019 •

edited

Loading

krytarowski commented Jun 12, 2019 •

edited

Loading

krytarowski commented Jun 13, 2019 •

edited

Loading

krytarowski commented Jun 13, 2019 •

edited

Loading

giampaolo commented Jun 13, 2019 •

edited

Loading

krytarowski commented Jun 13, 2019 •

edited

Loading