Suggestion: Allow multiple servers to back up to the same repo at the same time

[mattgconnor]

Being able to back up multiple servers at the same time (to the same repo) would allow all those servers to share the deduplication benefits. This becomes increasingly useful with hundreds of servers sharing mostly identical operating systems, and would add a tremendous amount of value.

[anarcat]

It seems that this is actually supported already. The downside is that the local cache needs to be rebuilt every time, see this post for more information:

http://librelist.com/browser//attic/2014/11/11/backing-up-multiple-servers-into-a-single-repository/#e96345aa5a3469a87786675d65da492b

I believe this issue can be closed, or should be turned into a documentation issue.

[mattgconnor]

Thank you for the update, you're absolutely right that backing up two servers sequentially works perfectly fine. I meant backing up two servers at the same time bombs out, and because of that I thought it just wasn't supported.

[anarcat]

ah yes, i guess that simultaneous backups are not supported.

[ThomasWaldmann]

@mattgconnor if it "bombs out", can you please show:

• commandline(s) used
• the full traceback
• attic version you use

[ThomasWaldmann]

Backing up 2 servers at the same time to same repository would create a cache coherency issue. The local cache (in RAM, later: on disk) needs to be consistent with the repo state (e.g. what chunks we have in the repo).

[xrat]

@ThomasWaldmann I agree, however I thought that it should be possible to have multiple backups writing to the same repository if we sacrifice some efficiency as we accept that for the backup only these chunks can be re-used that existed when the specific backup started. Also, I guess, it will take some inventory either at the end of a backup or at the start.
Say, we have chunks 1-10 in the repository. Client A and client B are writing to it at the same time. Of course, they'll use these chunks and they'll create new ones. Say, client A uses 1-8 and creates 11-14; client B uses 3-10 and creates 13-17. Maybe they can't use each others chunks, but at the end, at least after an inventory, we should have chunks 1-17.
I don't know much about the inner workings of Attic or Borg. So maybe I am mistaken. In fact, I thought it should be even easier: A chunk is written to the repository as a file using the hash sum as filename. If any other client is about to write the same chunk it doesn't have to since it immediately can detect that this chunk already exists.
Sure, the cache won't always be 100% coherent if multiple processes write at the same time. But at least that shouldn't break anything, or am I wrong?