[Bug] Root folder being checked for free space on Linux when game folder is in special separate partition #3848

VulcanRidr · 2023-06-10T04:02:03Z

Is there an existing issue for this?

I have searched the existing issues

Operating System

Devuan Linux

CKAN Version

1.32.0.23098

Game Version

1.12.4 or 1.12.5 (happened on both)

Did you make any manual changes to your game folder (i.e., not via CKAN)?

No

Describe the bug

This looks similar to #3768, except ckan does not generate exceptions for me, to my knowledge. I started with a clean copy of the GOG version of KSP 1.12.4 and 1.12.5. I tried to install from a .ckan file, and when I did, it told me that I only had 537.8MB of space free, and it needed to download or store 5.6GB of data. Having said that, I have 3.5TB free on this dataset:

[storm@defiant 1.12.4-oldmods]$ df -h .
Filesystem            Size  Used Avail Use% Mounted on
shuttlepod/GOG Games  3.5T   12G  3.5T   1% /home/storm/GOG Games

Steps to reproduce

Open CKAN
Click File -> install from .ckan
Select appropriate .ckan file
follow prompts to load mods from said file

Relevant log output

Not enough space in game folder to install modules!
Need to store 5.6 GiB to /home/storm/GOG Games/1.12.4-oldmods, but only 537.8 MiB is available!
Free up space on that device or change your settings to use another location.

Error during installation!
If the above message indicates a download error, please try again. Otherwise, please open an issue for us to investigate.
If you suspect a metadata problem: https://github.com/KSP-CKAN/NetKAN/issues/new/choose
If you suspect a bug in the client: https://github.com/KSP-CKAN/CKAN/issues/new/choose

The text was updated successfully, but these errors were encountered:

linuxgurugamer · 2023-06-11T01:42:40Z

Might be useful to see the size of ALL the filesystems on the computer, you only showed one

VulcanRidr · 2023-06-11T16:54:59Z

Thanks @linuxgurugamer. I only included the one that ckan seemed to be complaining about. Here is my list, excluding udev and tmpfs partitions:

Filesystem                     Size  Used Avail Use% Mounted on
/dev/mapper/vg00-root          2.0G  1.3G  630M  67% /
/dev/mapper/vg00-usr            15G  7.7G  6.2G  56% /usr
/dev/sda3                      474M  135M  311M  31% /boot
/dev/sdc1                      197M  6.6M  191M   4% /boot/efi
/dev/mapper/vg00-var            15G  2.8G   12G  21% /var
home                            56G   25G   31G  45% /home
shuttlepod/GOG Games           3.5T  5.7G  3.5T   1% /home/storm/GOG Games
shuttlepod                     3.5T   31G  3.5T   1% /home/storm/shuttlepod
shuttlepod/Kerbal              3.5T  1.0M  3.5T   1% /home/storm/shuttlepod/Kerbal

It appears that the problem is not with the game install directory, but rather in /tmp. My /tmp is on /, and I tend to run a skinny /tmp...So I have a question, is it possible to change ckan's temp directory? I have 12GB free on /var/tmp, so that would be a better place for me to point ckan's temp directory...

And, btw, this bug can be closed out.

HebaruSan · 2023-06-11T17:10:51Z

Comparing two DirectoryInfo objects with == may not work as expected, or maybe Mono implemented it wrong:

CKAN/Core/Extensions/IOExtensions.cs

Line 54 in 9f74d97

.Where(dr => dr.RootDirectory == dir

The documentation uses a far more awkward (Windows-only) construction:

https://learn.microsoft.com/en-us/dotnet/api/system.io.directoryinfo?view=netframework-4.7.2

        if (source.FullName.ToLower() == target.FullName.ToLower())

StackExchange consensus agrees:

https://stackoverflow.com/questions/1794025/how-to-check-whether-2-directoryinfo-objects-are-pointing-to-the-same-directory

... though on second thought, that would not explain this issue. The partition is mounted at /home/storm/GOG Games and we're checking /home/storm/GOG Games/1.12.4-oldmods, which would not need the == to work since it would be caught by IsAncestorOf.

VulcanRidr · 2023-06-11T19:33:35Z

So I don't think it is the GOG Games directory at all. I think that (and this is on linux) ckan is trying to use a temporary directory for the download of mods...By default, it is using /tmp. I freed up space on / (where /tmp is located), and have 1GB free. When I tried to install from a .ckan this time it told me it needed to get

Need to store 5.6 GiB to /home/storm/Kerbal/1.12.4-oldmods/game, but only 1,000.5 MiB is available!

$  df /tmp
Filesystem            1K-blocks   Used Available Use% Mounted on
/dev/mapper/vg00-root   2026008 891776   1024544  47% /

Is there a way to change the temp directory for ckan?

HebaruSan · 2023-06-11T19:37:17Z

Here are the three places where CKAN checks for free space:

CKAN/Core/ModuleInstaller.cs

Lines 144 to 147 in 9f74d97

    
           CKANPathUtils.CheckFreeSpace(new DirectoryInfo(ksp.GameDir()), 
        
                                        modsToInstall.Select(m => m.install_size) 
        
                                                     .Sum(), 
        
                                        Properties.Resources.NotEnoughSpaceToInstall);

Game folder

CKAN/Core/ModuleInstaller.cs

Lines 181 to 184 in 9f74d97

    
           CKANPathUtils.CheckFreeSpace(new DirectoryInfo(ksp.GameDir()), 
        
                                        modsToInstall.Select(m => m.install_size) 
        
                                                     .Sum(), 
        
                                        Properties.Resources.NotEnoughSpaceToInstall);

Game folder

CKAN/Core/Net/NetAsyncModulesDownloader.cs

Lines 54 to 56 in 9f74d97

    
           cache.CheckFreeSpace(unique_downloads.Values 
        
               .Select(m => m.download_size) 
        
               .Sum());

CKAN/Core/Net/NetFileCache.cs

Lines 326 to 328 in 9f74d97

    
           CKANPathUtils.CheckFreeSpace(new DirectoryInfo(cachePath), 
        
                                        bytesToStore, 
        
                                        Properties.Resources.NotEnoughSpaceToCache);

Cache folder

It doesn't check the temp folder, because as of #3666, we don't download to a temp folder anymore.

VulcanRidr · 2023-06-11T21:15:19Z

Interesting, because on linux, it seems to be looking at something on the root filesystem, because once I expanded my root filesystem, it stopped complaining. In either case, we can close this bug report...

HebaruSan · 2023-06-11T21:17:42Z

Right, I suspect there's a bug in how the folder path is translated to a partition. .NET/Mono doesn't provide this natively (presumably because it's non-trivial), so we have had to hack it in, and if we can identify what's going wrong in this instance, we can improve it for future versions.

HebaruSan · 2023-06-12T00:31:50Z

... well, that's just great. I added some debug code to probe this in an Ubuntu VM, and DriveInfo.GetDrives() returns a bunch of junk devices from snapd, but no actual mounted filesystems other than the root!❕❗

45568438272	/	/	/
0	/snap/bare/5	/snap/bare/5	/snap/bare/5
0	/snap/core20/1852	/snap/core20/1852	/snap/core20/1852
0	/snap/core18/2745	/snap/core18/2745	/snap/core18/2745
0	/snap/core22/634	/snap/core22/634	/snap/core22/634
0	/snap/core18/2751	/snap/core18/2751	/snap/core18/2751
0	/snap/core20/1891	/snap/core20/1891	/snap/core20/1891
0	/snap/core22/750	/snap/core22/750	/snap/core22/750
0	/snap/firefox/2710	/snap/firefox/2710	/snap/firefox/2710
0	/snap/firefox/2559	/snap/firefox/2559	/snap/firefox/2559
0	/snap/gnome-3-38-2004/137	/snap/gnome-3-38-2004/137	/snap/gnome-3-38-2004/137
0	/snap/gnome-3-38-2004/140	/snap/gnome-3-38-2004/140	/snap/gnome-3-38-2004/140
0	/snap/gnome-42-2204/105	/snap/gnome-42-2204/105	/snap/gnome-42-2204/105
0	/snap/gnome-42-2204/102	/snap/gnome-42-2204/102	/snap/gnome-42-2204/102
0	/snap/gtk-common-themes/1535	/snap/gtk-common-themes/1535	/snap/gtk-common-themes/1535
0	/snap/gtk-common-themes/1514	/snap/gtk-common-themes/1514	/snap/gtk-common-themes/1514
0	/snap/hunspell-dictionaries-1-7-2004/2	/snap/hunspell-dictionaries-1-7-2004/2	/snap/hunspell-dictionaries-1-7-2004/2
0	/snap/snap-store/959	/snap/snap-store/959	/snap/snap-store/959
0	/snap/snapd/19122	/snap/snapd/19122	/snap/snapd/19122
0	/snap/snap-store/638	/snap/snap-store/638	/snap/snap-store/638
0	/snap/snapd-desktop-integration/57	/snap/snapd-desktop-integration/57	/snap/snapd-desktop-integration/57
0	/snap/snapd/19361	/snap/snapd/19361	/snap/snapd/19361
0	/snap/snapd-desktop-integration/83	/snap/snapd-desktop-integration/83	/snap/snapd-desktop-integration/83
45568438272	/var/snap/firefox/common/host-hunspell	/var/snap/firefox/common/host-hunspell	/var/snap/firefox/common/host-hunspell
535801856	/boot/efi	/boot/efi	/boot/efi
/run/user/1000/doc
0	/run/user/1000/gvfs	/run/user/1000/gvfs	/run/user/1000/gvfs

DEBUG CKAN.CKANPathUtils - Dir /media/sf_Kerbal_Space_Program is on drive /

$ df -h
Filesystem            Size  Used Avail Use% Mounted on
tmpfs                 795M  1.6M  794M   1% /run
/dev/sda5              63G   17G   43G  29% /
tmpfs                 3.9G  8.0K  3.9G   1% /dev/shm
tmpfs                 5.0M  4.0K  5.0M   1% /run/lock
/dev/sda1             511M  4.0K  511M   1% /boot/efi
Kerbal_Space_Program  5.5T  2.8T  2.8T  50% /media/sf_Kerbal_Space_Program
CKAN                  5.5T  2.8T  2.8T  50% /media/sf_CKAN
tmpfs                 795M  128K  795M   1% /run/user/1000

It never crossed my mind that Mono's implementation of this could be that completely and totally and shockingly broken and unusable and useless. I originally tested this on my single filesystem dual boot setup on Linux and just assumed that GetDrives would, you know, get the drives.

I guess we have to turn the disk space check off completely for non-Windows platforms, since it can't actually be done. 😭

(... and in case anyone else goes hunting through the API, Directory.GetLogicalDrives() returns the same list of garbage.)

HebaruSan · 2023-06-12T01:18:38Z

GetDrives calls GetLogicalDrives:

https://github.com/mono/mono/blob/771947925a512a91e1ac88eb463ec845cafc2807/mcs/class/corlib/System.IO/DriveInfo.cs#L149-L159

GetLogicalDrives is implemented in C, with goto and lots of magic numbers and pointer manipulations (no wonder it doesn't work 🙄🤷):

https://github.com/mono/mono/blob/771947925a512a91e1ac88eb463ec845cafc2807/mono/metadata/icall.c#L7342-L7404
https://github.com/mono/mono/blob/771947925a512a91e1ac88eb463ec845cafc2807/mono/metadata/w32file-unix.c#L3986-L4256

It attempts to parse /proc/self/mountinfo, which contains the desired entry in this format:

1210 28 0:49 / /media/sf_Kerbal_Space_Program rw,nodev,relatime shared:610 - vboxsf Kerbal_Space_Program rw,iocharset=utf8,uid=0,gid=998,dmode=0770,fmode=0770,tag=VBoxAutomounter

So somewhere in that C code, that entry gets filtered out. Presumably on one of the ignore_entry = TRUE lines.
... I think it's the final one; the if/else if/... checks are hard-coded to include /, then they allow entries of type overlay or aufs or fuse or nfs, and then they just discard everything else! Astoundingly brain-dead.

VulcanRidr · 2023-06-12T02:02:41Z

Right, that is ubuntu...One reason I don't run Ubuntu. I hate snaps. You also, I see, have everything except /boot/efi in a single filesystem. That may be a better way to go for something like this, but I segregate my filesystems out of habit. Now, having said that, I think that either mono or ckan uses /tmp as a quick place to stash during downloads, then it actually caches the mods in the .local/share/CKAN/downloads directory.

HebaruSan · 2023-06-12T02:05:03Z

You also, I see, have everything except /boot/efi in a single filesystem

No, that VM has more mounted filesystems than that, courtesy of VirtualBox shared folders:

Kerbal_Space_Program  5.5T  2.8T  2.8T  50% /media/sf_Kerbal_Space_Program
CKAN                  5.5T  2.8T  2.8T  50% /media/sf_CKAN

That's what I was testing. /media/sf_Kerbal_Space_Program should have been included in the list returned by GetDrives but was not.

HebaruSan · 2023-06-12T02:05:36Z

As far as I know, /tmp plays no role in this whatsoever. If you have a reason for thinking that, please share it.

HebaruSan added Linux Issues specific for Linux Needs more info We need more info labels Jun 11, 2023

HebaruSan added Bug Something is not working as intended Core (ckan.dll) Issues affecting the core part of CKAN Mono Issues specific for Mono and removed Needs more info We need more info labels Jun 11, 2023

HebaruSan changed the title ~~[Bug]: ckan falsely claims insufficient space to install mods~~ [Bug] Root folder being checked on Linux when game folder is in special separate partition Jun 11, 2023

HebaruSan changed the title ~~[Bug] Root folder being checked on Linux when game folder is in special separate partition~~ [Bug] Root folder being checked for free space on Linux when game folder is in special separate partition Jun 11, 2023

VulcanRidr closed this as completed Jun 11, 2023

HebaruSan reopened this Jun 11, 2023

HebaruSan mentioned this issue Jun 12, 2023

Stop trying to check free space on Mono #3850

Merged

HebaruSan closed this as completed in #3850 Jun 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug] Root folder being checked for free space on Linux when game folder is in special separate partition #3848

[Bug] Root folder being checked for free space on Linux when game folder is in special separate partition #3848

VulcanRidr commented Jun 10, 2023 •

edited by HebaruSan

Loading

linuxgurugamer commented Jun 11, 2023

VulcanRidr commented Jun 11, 2023 •

edited by HebaruSan

Loading

HebaruSan commented Jun 11, 2023 •

edited

Loading

VulcanRidr commented Jun 11, 2023 •

edited by HebaruSan

Loading

HebaruSan commented Jun 11, 2023 •

edited

Loading

VulcanRidr commented Jun 11, 2023

HebaruSan commented Jun 11, 2023

HebaruSan commented Jun 12, 2023 •

edited

Loading

HebaruSan commented Jun 12, 2023 •

edited

Loading

VulcanRidr commented Jun 12, 2023

HebaruSan commented Jun 12, 2023 •

edited

Loading

HebaruSan commented Jun 12, 2023

[Bug] Root folder being checked for free space on Linux when game folder is in special separate partition #3848

[Bug] Root folder being checked for free space on Linux when game folder is in special separate partition #3848

Comments

VulcanRidr commented Jun 10, 2023 • edited by HebaruSan Loading

Is there an existing issue for this?

Operating System

CKAN Version

Game Version

Did you make any manual changes to your game folder (i.e., not via CKAN)?

Describe the bug

Steps to reproduce

Relevant log output

linuxgurugamer commented Jun 11, 2023

VulcanRidr commented Jun 11, 2023 • edited by HebaruSan Loading

HebaruSan commented Jun 11, 2023 • edited Loading

VulcanRidr commented Jun 11, 2023 • edited by HebaruSan Loading

HebaruSan commented Jun 11, 2023 • edited Loading

VulcanRidr commented Jun 11, 2023

HebaruSan commented Jun 11, 2023

HebaruSan commented Jun 12, 2023 • edited Loading

HebaruSan commented Jun 12, 2023 • edited Loading

VulcanRidr commented Jun 12, 2023

HebaruSan commented Jun 12, 2023 • edited Loading

HebaruSan commented Jun 12, 2023

VulcanRidr commented Jun 10, 2023 •

edited by HebaruSan

Loading

VulcanRidr commented Jun 11, 2023 •

edited by HebaruSan

Loading

HebaruSan commented Jun 11, 2023 •

edited

Loading

VulcanRidr commented Jun 11, 2023 •

edited by HebaruSan

Loading

HebaruSan commented Jun 11, 2023 •

edited

Loading

HebaruSan commented Jun 12, 2023 •

edited

Loading

HebaruSan commented Jun 12, 2023 •

edited

Loading

HebaruSan commented Jun 12, 2023 •

edited

Loading