Add copy operation to commit API

[mariosasko]

For hffs, it would be nice to have a CopyOperation in the commit API for efficiently copying repo siblings.

PS: this would also require some changes server-side, so "low priority" for now

[Wauplin]

(cc @Pierrci @SBrandeis for the moon-landing related request)

Issue is related to hffs: huggingface/hffs#1 (comment). I agree on the low priority as a sub-optimal workaround is possible (requires to download/copy a file locally).

[julien-c]

IIRC S3 doesn't support copy for >5GB files (you have to multipart upload) so this would/might be tricky to implement in a more efficient way than downloading and uploading. cc @coyotte508

[coyotte508]

Is it copying within a repo or across repos?

Within a repo, the only thing to copy is a pointer file, so no issue.

[SBrandeis]

IIRC S3 doesn't support copy for >5GB files (you have to multipart upload) so this would/might be tricky to implement in a more efficient way than downloading and uploading

You can, but it requires some extra steps: https://docs.aws.amazon.com/AmazonS3/latest/userguide/CopyingObjectsMPUapi.html

[mariosasko]

@coyotte508 Our use case in hffs is copying within a repo.

[Wauplin]

Now possible thanks to @coyotte508's work in https://github.com/huggingface/moon-landing/pull/4252 and https://github.com/huggingface/moon-landing/pull/4271 (internal links).

EDIT: I mixed up "copy a file" and "rename a file". Renaming is now possible out of the box from moon landing put copying is still not.