Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Files from chunked uploads are not deleted from S3, no ability to view them #30762

Closed
otherguy opened this issue Jan 19, 2022 · 3 comments
Closed
Labels
0. Needs triage Pending check for reproducibility or if it fits our roadmap bug feature: object storage

Comments

@otherguy
Copy link

As a fairly new user I just set up Nextcloud via Kubernetes with an S3 (compatible) primary backend for file storage. I have only one user and deleted all the welcome files, also from the trash. filelocking.enabled is set to false.

The user interface correctly shows 0B used.

My S3 bucket however, shows over 100 objects with over 400MB usage.

CleanShot 2022-01-19 at 14 37 02@2x

The occ tool gives me no ability to view them and tells me there are no files.

k exec -it nextcloud-666466c988-12345 -- su www-data -s /bin/bash -c "php occ files:scan --all"
Starting scan for user 1 out of 1 (admin)
+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 0       | 0     | 00:00:00     |
+---------+-------+--------------+

Many of the files appear to have exactly the same size but their binary content doesn't tell me what the file is. I assume this is part of a chunked upload that I canceled, because they are referenced in the database as part of uploads/web-file-upload-*/.

CleanShot 2022-01-19 at 14 48 43@2x

I found a related help topic that was never answered.

So the questions are:

  • does Nextcloud, at some point, delete chunks from failed/canceled uploads by itself?
  • if not, how can I view those and delete them manually?
@otherguy otherguy added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels Jan 19, 2022
@solracsf
Copy link
Member

solracsf commented Jan 20, 2022

This is a known issue, not only multipart but other files too.
Take a look at feature: object storage

@otherguy
Copy link
Author

Thanks! No real solution, but I might need to write a script that searches the database for these canceled uploads and deletes the files manually.

This should really be a part of nextcloud!

@otherguy
Copy link
Author

otherguy commented Feb 6, 2022

I have solved this by writing a cleanup cronjob that deletes chunks older than a given time.

The script is public and can be found here: https://github.com/otherguy/nextcloud-cleanup

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0. Needs triage Pending check for reproducibility or if it fits our roadmap bug feature: object storage
Projects
None yet
Development

No branches or pull requests

2 participants