Warning message: mixed CGroups versions not supported?

[skertberg]

Does anyone know what this warning message means? I wasn't able to find anything online about it.

> library(future)
> future::plan(multisession, workers = 10)
Warning message:
In getCGroupsRoot(controller = controller) :
  Mixed CGroups versions are not supported: ‘cgroup2’, ‘cgroup’
> availableCores()
system 
    24 

Session information:

> sessionInfo()
R version 4.4.2 (2024-10-31)
Platform: x86_64-pc-linux-gnu
Running under: Arch Linux

Matrix products: default
BLAS:   /usr/lib/libblas.so.3.12.0 
LAPACK: /usr/lib/liblapack.so.3.12.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

time zone: America/New_York
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] future_1.34.0

loaded via a namespace (and not attached):
[1] compiler_4.4.2    parallelly_1.41.0 tools_4.4.2       parallel_4.4.2   
[5] listenv_0.9.1     codetools_0.2-20  digest_0.6.37     globals_0.16.3 

[HenrikBengtsson]

Warning message:
In getCGroupsRoot(controller = controller) :
  Mixed CGroups versions are not supported: ‘cgroup2’, ‘cgroup’

This comes from the parallelly package that future depends on. Specially, it originates from parallelly::availableCores() that plan(multisession) relies on. If you call parallelly::availableCores() alone, you should see the same warning.

Now, why is there a warning?

First, CGroups ("control groups") is a concept in Linux that is used on some systems to limit how much memory and how many CPU cores a software may use. For instance, although the machine might have 48 cores, the system might only allow the current process (here R) to use, say, 8 cores. This is particularly common on systems shared by many users. If you ask parallel::detectCores() on such a system, it would return 48, completely ignoring any CGroups settings. In contrast, parallelly::availableCores() tries to respect also CGroups, so it would return 8 instead. This helps to prevent over-parallelization.

Second, there are two generations of CGroups: CGroups v1 (cgroups) and CGroups v2 (cgroups2). Since quite recently, parallelly supports both CGroups v1 and v2. Most systems have either or, but a few supports both at the same time. I only had access to pure CGroups v1 and pure CGroups v2 systems, so that is what I've managed to implement and test thus far. However, I did add code to detect systems that run both at the same time, and when that happens, it produced the above warning, with the hope that someone would notice it and reach out (just like you did 🙏). Importantly, the current behavior on such systems is to completely ignore any CGroups settings, which is unfortunate. It's on the to-do list to support also this case.

Could you please share more about your system? Is it a personal, local machine, or is this happening on a compute cluster or similar?

Also, could you please share the output of the following two commands:

cat(readLines("/proc/self/cgroup"), sep = "\n")

and

parallelly:::getCGroupsMounts()

That would help be start adding support for the mixed-CGroups-versions case.

[skertberg]

I greatly appreciate your quick and in-depth reply!

As you correctly predicted, calling parallelly::availableCores() generates the same warning.

My system is a personal machine, using Linux 6.12.8.

Here is the output of the two commands (called from R in Konsole):

> cat(readLines("/proc/self/cgroup"), sep = "\n")
1:net_cls:/
0::/user.slice/user-1000.slice/[email protected]/app.slice/app-org.kde.konsole-447505.scope/tab(447517).scope
> parallelly:::getCGroupsMounts()
    device                     mountpoint    type
10 cgroup2                 /sys/fs/cgroup cgroup2
30    none /opt/piavpn/etc/cgroup/net_cls  cgroup
                                                           options dump pass
10 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot    0    0
30                                             rw,relatime,net_cls    0    0

It appears that the issue involves PIA VPN, which mounts cgroup instead of cgroup2.

For what it's worth, the warning did not start appearing until somewhat recently (sometime within the last 3 weeks). I didn't update PIA VPN in that time, though I did update my system and the R packages.

If the current behavior is to ignore CGroups settings, then usage of future shouldn't be affected for machines with simple CGroups settings (like my single-user case), right?

[HenrikBengtsson]

Thanks for these details.

The warning is from there being both cgroups and cgroups2 in the type column of the parallelly:::getCGroupsMounts() output. The warning works as intended.

This will help to come up with a strategy to support this mixed-version as well, but I'll probably need to reach out to you again for more details on your system. Before I can do that, I need some solid time to come up with a plan, so it will be a while before that happens.

For what it's worth, the warning did not start appearing until somewhat recently (sometime within the last 3 weeks). I didn't update PIA VPN in that time, though I did update my system and the R packages.

This makes sense because this warning was added in parallelly 1.41.0 that was released on 2024-12-18.

If the current behavior is to ignore CGroups settings, then usage of future shouldn't be affected for machines with simple CGroups settings (like my single-user case), right?

Correct.

FYI, you can use:

parallelly::availableCores(which = "all")

to see exactly what CPU-limiting settings that parallelly picks from the current machine.

[HenrikBengtsson]

Moved the implementation of the missing support for this to futureverse/parallelly#125.