[Bug]: quota S3 storage wrong WITH Q&D FIX!

[mrAceT]

⚠️ This issue respects the following points: ⚠️

• This is a bug, not a question or a configuration/webserver/proxy issue.
• This issue is not already reported on Github (I've searched it).
• Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
• Nextcloud Server is running on 64bit capable CPU, PHP and OS.
• I agree to follow Nextcloud's Code of Conduct.

Bug description

Primary storage S3

It worked fine, but after a while (again....) new users get a fixed quota of 2 GB and no matter what I try I can't get that changed

Steps to reproduce

1. create user
2. change quota
3. refresh page....

Expected behavior

a 100% working quota system for S3 ;)

Installation method

Community Web installer on a VPS or web space

Operating system

RHEL/CentOS

PHP engine version

PHP 7.4

Web server

Apache (supported)

Database engine version

MySQL

Is this bug present after an update or on a fresh install?

No response

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

• Default user-backend (database)
• LDAP/ Active Directory
• SSO - SAML
• Other

Configuration report

No response

List of activated Apps

Enabled:
  - accessibility: 1.10.0
  - activity: 2.16.0
  - apporder: 0.15.0
  - calendar: 3.5.0
  - circles: 24.0.1
  - cloud_federation_api: 1.7.0
  - comments: 1.14.0
  - contacts: 4.2.2
  - contactsinteraction: 1.5.0
  - dashboard: 7.4.0
  - dav: 1.22.0
  - deck: 1.7.1
  - federatedfilesharing: 1.14.0
  - federation: 1.14.0
  - files: 1.19.0
  - files_external: 1.16.1
  - files_pdfviewer: 2.5.0
  - files_rightclick: 1.3.0
  - files_sharing: 1.16.2
  - files_trashbin: 1.14.0
  - files_versions: 1.17.0
  - files_videoplayer: 1.13.0
  - firstrunwizard: 2.13.0
  - groupquota: 0.1.8
  - keeweb: 0.6.9
  - logreader: 2.9.0
  - lookup_server_connector: 1.12.0
  - mail: 1.13.8
  - nextcloud_announcements: 1.13.0
  - notifications: 2.12.1
  - oauth2: 1.12.0
  - password_policy: 1.14.0
  - photos: 1.6.0
  - privacy: 1.8.0
  - provisioning_api: 1.14.0
  - quota_warning: 1.14.0
  - recommendations: 1.3.0
  - richdocuments: 6.2.0
  - richdocumentscode: 22.5.502
  - serverinfo: 1.14.0
  - settings: 1.6.0
  - sharebymail: 1.14.0
  - spreed: 14.0.5
  - support: 1.7.0
  - survey_client: 1.12.0
  - systemtags: 1.14.0
  - text: 3.5.1
  - theming: 1.15.0
  - theming_customcss: 1.12.0
  - twofactor_backupcodes: 1.13.0
  - updatenotification: 1.14.0
  - user_status: 1.4.0
  - user_usage_report: 1.8.0
  - viewer: 1.8.0
  - weather_status: 1.4.0
  - workflowengine: 2.6.0
Disabled:
  - admin_audit
  - encryption
  - user_ldap

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

No response

Additional info

I have had this problem once before.

In utter dismay I (re)build my migration code to migrate back from S3 to local storage. Then the quota problem was gone..

After massive testing of my (re)built migration tool for migrating from Local > S3 I then decided to return to S3 storage.
This worked for almost exactly two months.. now I have again the exact same problem !?

Because the problem was resolved when migrating from S3 > local storage I expect the problem to reside in the filesystem handling of S3.

The strange thing is:

• when I look at the MySql entry for that user in oc_preferences I see the quota I set (10 Gb)!
• when I do 'occ user:setting [user] files quota' I see the quota I set (10 Gb)!

But when I see the quota in the web interface, no matter if I'm the admin or the user I get 2 GB!
When I install the software for the network drive I get the 2 Gb quota

So effectively I set it to 10GB, it it operates at 2 GB !! Even setting it to unlimited does not change the 2GB setting!?

I have looked at the code.. to keep it a s clean as possible I started debugging via the quota info of the logged in user.. (I set it to 10 GB)

I find: apps/files/ajax/getstoragestats?dir=%2F
then: apps/files/lib/Controller/AjaxController.php: public function getStorageStats
then: apps/files/lib/Helper.php: public static function buildFileStorageStatistics
then: apps/files/lib/Controller/ViewController.php: protected function getStorageInfo
then: lib/private/legacy/OC_Helper.php: public static function getStorageInfo

There I get stuc in this part:
$quota = \OCP\Files\FileInfo::SPACE_UNLIMITED; $mount = $rootInfo->getMountPoint(); $storage = $mount->getStorage(); $sourceStorage = $storage;

I can't find the "getQuota" where it goes wrong.. here:
// TODO: need a better way to get total space from storage if ($sourceStorage->instanceOfStorage('\OC\Files\Storage\Wrapper\Quota')) { /** @var \OC\Files\Storage\Wrapper\Quota $storage */ $quota = $sourceStorage->getQuota(); $logger->warning("getStorageInfo testET3: includeExtStorage:".$includeExtStorage." quota:".$quota,[]); }

Here I get the wrong quota.. can someone tell me where this function is in the code?

Al that darn "getQuota" has to do is return the quota set in oc_preferences where userid is the user and return the configkey value for 'quota'.

Is there anyone who can point me in the right direction?

If needed I'll "hack the system" so that I'll at least get the right quota.. But I need to know where that darn "getQuota" is at (there are quite a few functions of that name in the source code....)

AD: the version is now the latest 24.0.5, but this problem has been here for at least half a year..

[mrAceT]

I do apologize, I see now that I opened a a ticket on the first of august about my previous trouble:
#33263

What to do, merge?

I do would like to respond to my suspicion about opcache. That does seem to delay changing the quota. But now I have tested by turning off opcache and altering the quota. no effect.

Also the problem is now with an other account, not the first account, so there does not seem to be a link with the primary/first (admin) account.

The only odd thing is that, again, the wrong quota value that gets stuc is that 2 Gb value..

[mrAceT]

Since this has been bugging me on & off for over half a year now I have spent the better part of the day figuring this out.. and I have (admitted Quick and Dirty) fixed the problem:

in lib/private/Files/Storage/Wrapper/Quota.php:

added: use OC_Util; (below the other use....)

in public function __construct:
below $this->quota = $parameters['quota']; add:

if (is_object($parameters['storage']->getUser())) {
$this->quotaBKP= OC_Util::getUserQuota($parameters['storage']->getUser());
  if ($this->quotaBKP > 0 ) {
    $this->quota   = $this->quotaBKP;
    $this->quotaBKP= $parameters['quota'];
  }
}

Somewhere (I did not find where...) the parameter[quota] gets wrongly set to 2GB.. I now simply re-read that quota.. It's not pretty but at least my headache about the subject is gone now.

[mrAceT]

I'll re-open it, hoping that a better and more permanent fix will be implemented

[mrAceT]

Oh a small addendum;

Now that I have been digging in the code, in the earlier mentioned function:
lib/private/Files/Storage/Wrapper/Quota.php:

The problem of changed quota's not being seen for a while is because in the public static function getStorageInfo there is a caching call to "ICacheFactory".. when I turn that off, That problem is gone.. is there something not being updated to that cache when the quota is changed?

[CarlSchwan]

Thanks for investigating @mrAceT

That the issue is coming from the cache is pretty weird since the cache should be cleared every 5 min. What cache engine are you using? Redis/memcached/apcu?

Something that I can see is that if we fail to calculate quota for a folder we instead get the quota for the root and save that as quota for a sub folders. This is probably an issue

[mrAceT]

cleared every 5 min

If memory serves me well,, I do think that the wrong/old setting is indeed gone after say 5 minutes.
But that does keep the question valid, why isn't the cache updated upon altering the quota?

What cache engine are you using?

Is 'opcache' an answer to your question?

get the quota for the root

Strange, I purposely set all quota's that might have influence to anything but 2Gb.. so there wasn't any user in the system with a 2Gb quota.. but still that quota was 2Gb!

[mrAceT]

AD: last night I upgraded my Nextcloud to version 24.0.6

As expected the "quota problem" described above returned. I implemented the fix I described above (#34422 (comment)) and after a few minutes the correct quotas were back again (that cashing thing..?).

So my "Q&D fix" for this quota problem also works in version 24.0.6

[phoenixwarrior95]

I've tried to apply the Q&D fix. Running 25.0.2 NC Hub 3.

The construct function is as follows:

       public function __construct($parameters) {
                parent::__construct($parameters);
                $this->quota = $parameters['quota'] ?? null;
                $this->quotaCallback = $parameters['quotaCallback'] ?? null;
                $this->sizeRoot = $parameters['root'] ?? '';
                $this->config = \OC::$server->get(SystemConfig::class);
        }

    I added in the block of code

                public function __construct($parameters) {
                parent::__construct($parameters);
                $this->quota = $parameters['quota'] ?? null;
                if (is_object($parameters['storage']->getUser())) {
                $this->quotaBKP= OC_Util::getUserQuota($parameters['storage']->getUser());
                    if ($this->quotaBKP > 0 ) {
                    $this->quota   = $this->quotaBKP;
                    $this->quotaBKP= $parameters['quota'];
                    }
                 }
                $this->quotaCallback = $parameters['quotaCallback'] ?? null;
                $this->sizeRoot = $parameters['root'] ?? '';
                $this->config = \OC::$server->get(SystemConfig::class);
        }

Sorry for my lack of coding experience, but I do get an internal 500, meaning that I either messed up somewhere in the code or something else happened. The original problem I'm facing is similar to yours, quota on the bottom left (next to Deleted Files) is not updating right away. I have set cron to update every minute instead of 5, and for some reason it doesnt update. However, I found out that everytime I go to the User Settings after doing sudo -u www-data php /var/www/nextcloud/occ groupquota:set Default 50GB and placing different values for the share quota, it updates instantly. So there might be something in the user settings page that updates everytime you go to that page.

In the meantime, can anyone tell me what Im doing wrong in Quota.php? Thanks.

[mrAceT]

I've done the update to 25.0.2 and altered the "Q&D fix" a little bit, this should help you enough:

 		$this->quotaBKP= null;
		if (is_object($this->getWrapperStorage()->getUser())) {
			$this->quotaBKP= OC_Util::getUserQuota($this->getWrapperStorage()->getUser());
			if ($this->quotaBKP > 0 ) {
				$this->quota   = $this->quotaBKP;
				$this->quotaBKP= $parameters['quota'] ?? null;
			}
 		}

[phoenixwarrior95]

Thank you for this! I'm still getting an internal 500 error after saving Quota.php.

Did you do anything else after editing Quota.php?

[mrAceT]

did you reset opcache? (php: opcache_reset();)
did you restart apache/nginx?
did you restart php?

(Buffering is a b*tch..)

Otherwise, restore your original, error gone? I didn't change anything else then that after the upgrade.. can you give me the error?

[phoenixwarrior95]

I did reset opcache. Made a file called flush_cache.php and had this in it:

<?php opcache_reset(); ?>

ran php flush_cache.php and systemctl restart apache2. All seemed fine.

So after changing Quota.php and restarting services, I go to the login page. The moment I log in, Im greeted with:

`
Internal Server Error
The server was unable to complete your request.

If this happens again, please send the technical details below to the server administrator.

More details can be found in the server log.

Technical details
Remote Address: 192.168.8.1
Request ID: pArT0Tjb1kMjHAQ4gMqS
`

I made sure I was logged into my session before changing Quota.php. I went to Admin Settings -> Logging so I could catch the error. Here it is:

{"reqId":"pArT0Tjb1kMjHAQ4gMqS","level":3,"time":"2022-12-14T10:17:37-05:00","remoteAddr":"192.168.8.1","user":"admin","app":"index","method":"POST","url":"/login","message":"call_user_func_array(): Argument #1 ($callback) must be a valid callback, class OC\\Files\\Storage\\LocalRootStorage does not have a method \"getUser\" in file '/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php' line 527","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36","version":"25.0.2.3","exception":{"Exception":"Exception","Message":"call_user_func_array(): Argument #1 ($callback) must be a valid callback, class OC\\Files\\Storage\\LocalRootStorage does not have a method \"getUser\" in file '/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php' line 527","Code":0,"Trace":[{"file":"/var/www/nextcloud/lib/private/AppFramework/App.php","line":172,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/Route/Router.php","line":298,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/var/www/nextcloud/lib/base.php","line":1047,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/var/www/nextcloud/index.php","line":36,"function":"handleRequest","class":"OC","type":"::"}],"File":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","Line":165,"Previous":{"Exception":"TypeError","Message":"call_user_func_array(): Argument #1 ($callback) must be a valid callback, class OC\\Files\\Storage\\LocalRootStorage does not have a method \"getUser\"","Code":0,"Trace":[{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":527,"function":"call_user_func_array"},{"function":"__call","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":527,"function":"call_user_func_array"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Quota.php","line":57,"function":"__call","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/var/www/nextcloud/apps/groupfolders/lib/Mount/GroupFolderStorage.php","line":42,"function":"__construct","class":"OC\\Files\\Storage\\Wrapper\\Quota","type":"->"},{"file":"/var/www/nextcloud/apps/groupfolders/lib/Mount/MountProvider.php","line":230,"function":"__construct","class":"OCA\\GroupFolders\\Mount\\GroupFolderStorage","type":"->"},{"file":"/var/www/nextcloud/apps/groupfolders/lib/Mount/MountProvider.php","line":153,"function":"getMount","class":"OCA\\GroupFolders\\Mount\\MountProvider","type":"->"},{"function":"OCA\\GroupFolders\\Mount\\{closure}","class":"OCA\\GroupFolders\\Mount\\MountProvider","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/apps/groupfolders/lib/Mount/MountProvider.php","line":133,"function":"array_map"},{"file":"/var/www/nextcloud/lib/private/Files/Config/MountProviderCollection.php","line":124,"function":"getMountsForUser","class":"OCA\\GroupFolders\\Mount\\MountProvider","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/SetupManager.php","line":226,"function":"addMountForUser","class":"OC\\Files\\Config\\MountProviderCollection","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/SetupManager.php","line":318,"function":"OC\\Files\\{closure}","class":"OC\\Files\\SetupManager","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/Files/SetupManager.php","line":227,"function":"setupForUserWith","class":"OC\\Files\\SetupManager","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Filesystem.php","line":379,"function":"setupForUser","class":"OC\\Files\\SetupManager","type":"->"},{"file":"/var/www/nextcloud/lib/private/Cache/File.php","line":57,"function":"initMountPoints","class":"OC\\Files\\Filesystem","type":"::"},{"file":"/var/www/nextcloud/lib/private/Cache/File.php","line":178,"function":"getStorage","class":"OC\\Cache\\File","type":"->"},{"file":"/var/www/nextcloud/lib/base.php","line":851,"function":"gc","class":"OC\\Cache\\File","type":"->"},{"function":"{closure}","class":"OC","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/Hooks/EmitterTrait.php","line":106,"function":"call_user_func_array"},{"file":"/var/www/nextcloud/lib/private/Hooks/PublicEmitter.php","line":40,"function":"emit","class":"OC\\Hooks\\BasicEmitter","type":"->"},{"file":"/var/www/nextcloud/lib/private/User/Session.php","line":400,"function":"emit","class":"OC\\Hooks\\PublicEmitter","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/CompleteLoginCommand.php","line":44,"function":"completeLogin","class":"OC\\User\\Session","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/ALoginCommand.php","line":40,"function":"process","class":"OC\\Authentication\\Login\\CompleteLoginCommand","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/LoggedInCheckCommand.php","line":60,"function":"processNextOrFinishSuccessfully","class":"OC\\Authentication\\Login\\ALoginCommand","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/ALoginCommand.php","line":40,"function":"process","class":"OC\\Authentication\\Login\\LoggedInCheckCommand","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/EmailLoginCommand.php","line":70,"function":"processNextOrFinishSuccessfully","class":"OC\\Authentication\\Login\\ALoginCommand","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/ALoginCommand.php","line":40,"function":"process","class":"OC\\Authentication\\Login\\EmailLoginCommand","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/UidLoginCommand.php","line":54,"function":"processNextOrFinishSuccessfully","class":"OC\\Authentication\\Login\\ALoginCommand","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/ALoginCommand.php","line":40,"function":"process","class":"OC\\Authentication\\Login\\UidLoginCommand","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/UserDisabledCheckCommand.php","line":58,"function":"processNextOrFinishSuccessfully","class":"OC\\Authentication\\Login\\ALoginCommand","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/ALoginCommand.php","line":40,"function":"process","class":"OC\\Authentication\\Login\\UserDisabledCheckCommand","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/PreLoginHookCommand.php","line":53,"function":"processNextOrFinishSuccessfully","class":"OC\\Authentication\\Login\\ALoginCommand","type":"->"},{"file":"/var/www/nextcloud/lib/private/Authentication/Login/Chain.php","line":108,"function":"process","class":"OC\\Authentication\\Login\\PreLoginHookCommand","type":"->"},{"file":"/var/www/nextcloud/core/Controller/LoginController.php","line":318,"function":"process","class":"OC\\Authentication\\Login\\Chain","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":225,"function":"tryLogin","class":"OC\\Core\\Controller\\LoginController","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":133,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/App.php","line":172,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/Route/Router.php","line":298,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/var/www/nextcloud/lib/base.php","line":1047,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/var/www/nextcloud/index.php","line":36,"function":"handleRequest","class":"OC","type":"::"}],"File":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","Line":527},"CustomMessage":"--"},"id":"6399e9281b84a"}

If I undo all of the changes in Quota.php, it will go back to working. I log in and its as nothing ever happened.

[mrAceT]

Did you add use OC_Util; ?

[phoenixwarrior95]

namespace OC\Files\Storage\Wrapper;
use OC\Files\Filesystem;
use OC\SystemConfig;
use OCP\Files\Cache\ICacheEntry;
use OCP\Files\FileInfo;
use OCP\Files\Storage\IStorage;
use OC_Util;

class Quota extends Wrapper {
        /** @var callable|null */
        protected $quotaCallback;
        protected ?int $quota;
        protected string $sizeRoot;
        private SystemConfig $config;

        /**
         * @param array $parameters
         */
        public function __construct($parameters) {
                parent::__construct($parameters);
                $this->quota = $parameters['quota'] ?? null;

                $this->quotaBKP= null;
                if (is_object(($this->getWrapperStorage())->getUser())) {
                        $this->quotaBKP= OC_Util::getUserQuota($this->getWrapperStorage()->getUser());
                        if ($this->quotaBKP > 0 ) {
                                $this->quota   = $this->quotaBKP;
                                $this->quotaBKP= $parameters['quota'] ?? null;
                        }
                }
                $this->quotaCallback = $parameters['quotaCallback'] ?? null;
                $this->sizeRoot = $parameters['root'] ?? '';
                $this->config = \OC::$server->get(SystemConfig::class);
        }

This is my Quota.php. I did use OC_Util;

[mrAceT]

Please check #34422 (comment) will update to 25.0.3.. but 25.0.2 still has the bug, so I'm not holding my hopes up..

[mrAceT]

I am now at version 25.0.9 (haven't upgraded to 26 yet)

And I ran into the same problem again. I have (again) performed "my hack" and the problem was fixed.

"that hack/Q&D fix" : #34422 (comment)

for future reference, this is what I have now in lib/private/Files/Storage/Wrapper/Quota.php:

...........
use OCP\Files\FileInfo;
use OCP\Files\Storage\IStorage;

// added by mrAceT
use OC_Util;

class Quota extends Wrapper {
	/** @var callable|null */
	protected $quotaCallback;
	/** @var int|float|null int on 64bits, float on 32bits for bigint */
	protected $quota;
	protected string $sizeRoot;
	private SystemConfig $config;

	/**
	 * @param array $parameters
	 */
	public function __construct($parameters) {
		parent::__construct($parameters);

		$this->quota = $parameters['quota'] ?? null;

		// if statement added by mrAceT
		$this->quotaBKP= null;
		if (is_object($parameters['storage']->getUser())) {
			$this->quotaBKP= OC_Util::getUserQuota($parameters['storage']->getUser());
			if ($this->quotaBKP > 0 ) {
	        		$this->quota   = $this->quotaBKP;
				$this->quotaBKP= $parameters['quota'] ?? null;
			}
		}

		$this->quotaCallback = $parameters['quotaCallback'] ?? null;
		$this->sizeRoot = $parameters['root'] ?? '';
		$this->config = \OC::$server->get(SystemConfig::class);
	}
...........

[mrAceT]

I am now at version 27.1.6 (haven't upgraded to 27.1.7 yet)

Same problem again.. same solution again..

[joshtrichards]

Hi @mrAceT - I'm unable to reproduce this. I also don't see evidence that there have been similar reports.

Are you still experiencing this?

If so:

• What is the output of occ user:info <user>?
• What was the exact (faulty) value reported by your debug logging code?
• Are you, by chance, on a 32-bit system?
• Are you able to reproduce this in a clean installation (e.g. a test install)

[nextcloud-command]

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.