Upload and Download parallel

[flower1024]

If there are many files to sync and up- and downloads are to do they should happen in parallel.

Expected behaviour

Uploads and Downloads should happen at the same time

Actual behaviour

One file is synced at a time

Steps to reproduce

Change an big file on the server and change a big file on client.
both files need to be syned.

Client configuration

Client version: 2.5.1git

Operating system:
XUbuntu 18.10

OS language:
German

[unix0r]

I also had the feeling, that the upload process of the client to the server was some kind of slow.

The server was in the same network and it was just like some kind of multi threading of the sync process was missing? There were a lot of file to sync (mostly pictures 3-10 MB) and the server side encryption is on.

I could see the network updload went shortly up to ~7 MB/ s, and after that there was a break of a couple of seconds (maybe the encryption process). After that there were the ~7MB/s again and so on.

Felt like it was really uploading file by file and waiting the server answer after the encryption to upload the next file. Would be better to use the network bandwidth completely for Up/Downloading by multithreading file synchronisation process.

Nextcloud server was 14.0.4 or 15, cant remember now and the client 2.5.1.

Any other opinions?

[frittentheke]

This issue is actually also causing uploads of many files via the web browser to be slower than the connection speed would allow. All those round-trip times it takes to start and stop the upload of a single file (apart from the transfer of the bits itself) is wasted / unused bandwidth.

[gyohng]

This issue is actually also causing uploads of many files via the web browser to be slower than the connection speed would allow. All those round-trip times it takes to start and stop the upload of a single file (apart from the transfer of the bits itself) is wasted / unused bandwidth.

Not necessarily true. I'm not aware of how Nextcloud works internally, but considering normal TCP and HTTP upload, it doesn't seem like the metadata of several parallel uploads would create any significant overhead (if any at all).

On the other hand, right now I'm uploading files to a server in Hong Kong from Europe. My uplink is about 20mbps, however the upload speed I receive with Nextcloud's app sync is only 200kb/sec. Doing it in multiple streams with other software, such as aria2c, I can achieve full speed, as there's no capping. It is probable that different TCP sessions take different routes to the destination, thus not being restricted by a single route's bandwidth.

Given the nature of Nextcloud, multithreaded delta synchronisation (even if it's file-based, doesn't need to be like torrents) would be of a great help.

[dniasoff]

As you increase the distance from the server and increase latency, Nextcloud becomes exponentially slower

A solution would be to use multiple concurrent connections as supported by many download managers