Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: 3.15.3 Multiple Repeated Unresolvable Conflicts #7761

Open
5 of 8 tasks
Bockeman opened this issue Jan 16, 2025 · 3 comments
Open
5 of 8 tasks

[Bug]: 3.15.3 Multiple Repeated Unresolvable Conflicts #7761

Bockeman opened this issue Jan 16, 2025 · 3 comments
Labels
bug

Comments

@Bockeman
Copy link

⚠️ Before submitting, please verify the following: ⚠️

Bug description

I am getting repeated "unresolved conflicts". When I select "Click for details" the same file shows multiple conflicts. Furthermore, when I select "[/] All local versions" and "[Resolve conflicts]", more conflicts keep recurring.

The same fault is happening on different files, different sync folders, and different clients (all Windows 10). What appears to be common is that the files affected are "in use" and frequently updated.

The conflicts arise when only one client is active, so the conflict is false, it is a bug somewhere in nextcloud.

Although many files are affected and multiple conflicts arise for any one file, for simplicity, the following
information relates to just one file Inbox with just one conflict on a file which changed on 2025-01-16 in the timeframe between 11:30 and 11:39.

From a linux machine, I am able to access the Windows client via a cifs mount, and an ls -l --full-date reveals:

  -rw-rw-r-- 1 apache apache 4719510 2025-01-16 11:34:15.000000000 +0000 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/Mail/pop3.blueyonder.co.uk/Inbox
  -rw-rw-r-- 1 apache apache 4719510 2025-01-16 11:34:22.649039900 +0000 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/Mail/pop3.blueyonder.co.uk/Inbox (conflicted copy 2025-01-16 113422)
  -rw-r--r-- 1 apache apache 4719510 2025-01-16 11:34:15.000000000 +0000 /srv/gluvol1/vault/bobw/files/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/Mail/pop3.blueyonder.co.uk/Inbox

which are the client file, the conflicted file and the server file respectively.

The client file and server file are identical. The conflicted file differs:

  71151c71151
  < X-Mozilla-Status: 0001
  ---
  > X-Mozilla-Status: 0009

Details from the corresponding "Solve sync conflicts" dialog box

  Inbox
  Local Version			Server Version
  Thu Jan 16 11:34:22:2025	Thu Jan 16 11:34:15:2025
  4.50MiB			4.50MiB

The mtime in the server database is 7 seconds older than the mtime on the server file.

Image

The client log in the stated timeframe contains:

#=#=#=# Syncrun started 2025-01-16T11:33:40Z
#=#=#=#=# Propagation starts 2025-01-16T11:33:42Z (last step: 1337 msec, total: 1337 msec)
11:33:41||Profiles/rdvpq1tu.default/Mail/pop3.blueyonder.co.uk/Inbox|64|2|1737027215|860b186e1aaaffc487648b8dcac0be73|4719510|17786730oc3hd6f2u8cw|13||200|4719510|1737027215|7196e852-4863-433e-865a-3eb7f6e1129d|
11:33:42||Profiles/rdvpq1tu.default/permissions.sqlite|64|1|1737027217|03360863d89d23ecf41ce48f8feaa11a|130048|04504266oc3hd6f2u8cw|13||204|130048|1737026720|dc876a59-2f63-4c4b-92f9-79b40c429455|
#=#=#=# Syncrun finished 2025-01-16T11:33:42Z (last step: 543 msec, total: 1880 msec)
#=#=#=# Syncrun started 2025-01-16T11:33:45Z
#=#=#=#=# Propagation starts 2025-01-16T11:33:46Z (last step: 990 msec, total: 990 msec)
#=#=#=# Syncrun finished 2025-01-16T11:33:46Z (last step: 167 msec, total: 1158 msec)
#=#=#=# Syncrun started 2025-01-16T11:34:18Z
#=#=#=#=# Propagation starts 2025-01-16T11:34:19Z (last step: 717 msec, total: 717 msec)
11:34:19||Profiles/rdvpq1tu.default/Mail/pop3.blueyonder.co.uk/Inbox.msf|64|1|1737027255|668f92cfd3fc55a5338d376546c0e1d0|7775|17482571oc3hd6f2u8cw|13||204|7699|1737027215|bfba898d-07fd-4591-8038-258a68f9489d|
11:34:19||Profiles/rdvpq1tu.default/Mail/pop3.blueyonder.co.uk/Trash.msf|64|1|1737027255|6bb6ed7d88946ea930ea69989a26190f|980268|11171205oc3hd6f2u8cw|13||204|979300|1737023830|723a7244-8626-4153-9c97-0c5fff5492de|
11:34:19||Profiles/rdvpq1tu.default/Mail/pop3.blueyonder.co.uk/Inbox|64|1|1737027255|812ccdd2b6c6fc5c6a4e68e795f1b250|4719510|04504918oc3hd6f2u8cw|13||204|4719510|1737027215|8f180ed7-ab44-4353-8134-1b2c22b52c9a|
11:34:19||Profiles/rdvpq1tu.default/Mail/smart mailboxes-1/Trash.msf|64|1|1737027255|6d2746b1cfcdcf532f1bf9ca5560f4b2|1667|13611975oc3hd6f2u8cw|13||204|1618|1737023830|1fbc6aed-7b92-4a50-b76e-fedc252bba54|
11:34:19||Profiles/rdvpq1tu.default/Mail/smart mailboxes-1/Inbox.msf|64|1|1737027255|aa33aec9a4ee61675d07890acad15b62|2028|13611967oc3hd6f2u8cw|13||204|1990|1737027215|424d7ef9-3dcf-4d21-be13-feac18fa5570|
11:34:23||Profiles/rdvpq1tu.default/Mail/pop3.blueyonder.co.uk/Trash|64|1|1737027255|9af38e77e390ee3f5f81b762c71181e4|138852413|04504919oc3hd6f2u8cw|13||204|138769046|1737023488|6838bdc3-fe97-497f-a5ca-3fff0c9a7932|
#=#=#=# Syncrun finished 2025-01-16T11:34:26Z (last step: 7289 msec, total: 8006 msec)
#=#=#=# Syncrun started 2025-01-16T11:34:31Z
#=#=#=#=# Propagation starts 2025-01-16T11:34:32Z (last step: 1323 msec, total: 1323 msec)
11:34:32||Profiles/rdvpq1tu.default/Mail/pop3.blueyonder.co.uk/Inbox.msf|64|1|1737027262|b5f9b3f2b7024eb7b16a81d4abb88949|8258|17482571oc3hd6f2u8cw|13||204|7775|1737027255|7acfdab3-7647-49f3-bb04-7e764c2ff529|
11:34:32||Profiles/rdvpq1tu.default/Mail/pop3.blueyonder.co.uk/PC.sbd/Norton Symantec.msf|64|1|1737027269|c1aa7acb7cc7bb7ab5efb732b2b5ad8c|166160|11500057oc3hd6f2u8cw|13||204|165197|1737026715|f96f70d4-999e-4db4-98cb-15f43b168c4d|
11:34:33||Profiles/rdvpq1tu.default/Mail/smart mailboxes-1/Inbox.msf|64|1|1737027262|5d776ce1edfa2f14d024b7203dc4ebf0|1398|13611967oc3hd6f2u8cw|13||204|2028|1737027255|330a6b79-1c78-4937-9585-53a7e8678474|
11:34:32||Profiles/rdvpq1tu.default/Mail/pop3.blueyonder.co.uk/PC.sbd/Norton Symantec|64|1|1737027262|3a5a9136dbfcaa1c6a2b7d5d49d87e64|16935828|11500026oc3hd6f2u8cw|13||204|16902187|1737026702|4a4dc003-5d63-4f61-86d0-9d4e655698ce|
11:34:33||Profiles/rdvpq1tu.default/prefs.js|64|1|1737027263|3f61d7775680ee8a372dd5b41e0e25cd|51240|17482511oc3hd6f2u8cw|13||204|51255|1737026921|e90c4c7e-6558-470d-9ec5-927f7d30d2f6|
#=#=#=# Syncrun finished 2025-01-16T11:34:33Z (last step: 979 msec, total: 2302 msec)
#=#=#=# Syncrun started 2025-01-16T11:34:36Z
#=#=#=#=# Propagation starts 2025-01-16T11:34:38Z (last step: 1516 msec, total: 1516 msec)
||Profiles/rdvpq1tu.default/Mail/pop3.blueyonder.co.uk/Inbox|64|2|1736892206|698442629fed2122005c6d38c2dcd314|61947904|17786730oc3hd6f2u8cw|2|We received a different E-Tag for resuming. Retrying next time.|200|4719510|1737027255|23f1aad1-2fdb-46c7-8075-05f2452b6ecc|
#=#=#=# Syncrun finished 2025-01-16T11:34:38Z (last step: 227 msec, total: 1743 msec)
#=#=#=# Syncrun started 2025-01-16T11:34:50Z
#=#=#=#=# Propagation starts 2025-01-16T11:34:51Z (last step: 1344 msec, total: 1344 msec)
#=#=#=# Syncrun finished 2025-01-16T11:34:52Z (last step: 195 msec, total: 1539 msec)
#=#=#=# Syncrun started 2025-01-16T11:35:17Z
#=#=#=#=# Propagation starts 2025-01-16T11:35:18Z (last step: 1318 msec, total: 1318 msec)
11:35:18||Profiles/rdvpq1tu.default/Mail/pop3.blueyonder.co.uk/Inbox|64|2|1737027255|812ccdd2b6c6fc5c6a4e68e795f1b250|4719510|17786730oc3hd6f2u8cw|13||200|4719510|1737027255|1dcdebc1-ca6d-4fe2-807e-da1632658d6d|
11:35:18||Profiles/rdvpq1tu.default/Mail/pop3.blueyonder.co.uk/Inbox.msf|64|1|1737027304|5fb47295bc9cdf82b037151b6770804e|8740|17482571oc3hd6f2u8cw|13||204|8258|1737027262|9ade5f3b-fd48-4ccc-88a4-1283836ec058|
11:35:18||Profiles/rdvpq1tu.default/Mail/pop3.blueyonder.co.uk/PC.sbd/Norton Symantec.msf|64|1|1737027304|d25fe712eb9d48ebdedcc64824347d2b|166204|11500057oc3hd6f2u8cw|13||204|166160|1737027269|f8581162-1ca5-4406-b8ac-d6327381ec3b|
#=#=#=# Syncrun finished 2025-01-16T11:35:18Z (last step: 468 msec, total: 1786 msec)
#=#=#=# Syncrun started 2025-01-16T11:35:21Z
#=#=#=#=# Propagation starts 2025-01-16T11:35:22Z (last step: 1561 msec, total: 1561 msec)
||Profiles/rdvpq1tu.default/Mail/pop3.blueyonder.co.uk/Inbox|64|2|1736892206|698442629fed2122005c6d38c2dcd314|61947904|17786730oc3hd6f2u8cw|2|We received a different E-Tag for resuming. Retrying next time.|200|4719510|1737027255|680a46c5-53e4-46eb-a0ba-13ffb5f67b3f|
#=#=#=# Syncrun finished 2025-01-16T11:35:23Z (last step: 378 msec, total: 1940 msec)
#=#=#=# Syncrun started 2025-01-16T11:35:35Z
#=#=#=#=# Propagation starts 2025-01-16T11:35:36Z (last step: 1355 msec, total: 1355 msec)
#=#=#=# Syncrun finished 2025-01-16T11:35:37Z (last step: 200 msec, total: 1556 msec)
#=#=#=# Syncrun started 2025-01-16T11:35:43Z
#=#=#=#=# Propagation starts 2025-01-16T11:35:44Z (last step: 1230 msec, total: 1230 msec)
#=#=#=# Syncrun finished 2025-01-16T11:35:44Z (last step: 180 msec, total: 1410 msec)

Steps to reproduce

  1. Run nextcloud client as normal
  2. Ensure all other clients are disconnected or with synchronisation paused
  3. Continue accessing files as normal
  4. Observe multiple unrsolved conflicts that keep recurring.

Expected behavior

There should never be any conflicts when only one client is active.

Which files are affected by this bug

Inbox

Operating system

Windows

Which version of the operating system you are running.

Windows 10

Package

Official Windows MSI

Nextcloud Server version

Nextcloud Hub 9 (30.0.4)

Nextcloud Desktop Client version

3.15.3

Is this bug present after an update or on a fresh install?

Updated from a minor version (ex. 3.4.2 to 3.4.4)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

Are you using an external user-backend?

  • Default internal user-backend
  • LDAP/ Active Directory
  • SSO - SAML
  • Other

Nextcloud Server logs

The server log in this timeframe contains:

awk '/2025-01-16T11:3/{gsub(/:/,"\n  :");print " ",$0,"\n"}' /srv/gluvol1/vault/nextcloud.log
  {"reqId"
  :"a2YvtomEuIFRRcKcYiOC","level"
  :2,"time"
  :"2025-01-16T11
  :30
  :01+00
  :00","remoteAddr"
  :"","user"
  :"--","app"
  :"files","method"
  :"","url"
  :"--","message"
  :"User admin still has unscanned files after running background scan, background scan might be stopped prematurely","userAgent"
  :"--","version"
  :"30.0.4.1","data"
  :{"app"
  :"files"}} 

  {"reqId"
  :"EoFnW0VZibRGe3TK9qpo","level"
  :2,"time"
  :"2025-01-16T11
  :35
  :01+00
  :00","remoteAddr"
  :"","user"
  :"--","app"
  :"files","method"
  :"","url"
  :"--","message"
  :"User admin still has unscanned files after running background scan, background scan might be stopped prematurely","userAgent"
  :"--","version"
  :"30.0.4.1","data"
  :{"app"
  :"files"}} 

Both of these messages, I believe, are unrelated the false conflicts.

Additional info

Other conflicts:

Image
Image
@mgallien
Copy link
Collaborator

@Bockeman we will need more information to understand the issue
we would need at least a debug archive that includes new conflicts being reported
we would need the use of the conflict dialog to solve them and new conflicts coming back
we would also need more information on the server side
we would need to know which kind of server storage is being used (external folder, group folder, ...)

@Bockeman
Copy link
Author

Bockeman commented Jan 20, 2025 •
edited
Loading

@mgallien Thanks for taking a look at this.

This issue is driving me crazy. I am trying all sorts of things to ameliorate the problem. I am relying on nextcloud sync'ing various folders: Thunderbird(email), Firefox(browser), and Document(all my work). I am trying all sorts of things like pausing sync when actively working, then resuming when I have finished; but I am still getting conflicts, and worse (see snip below). I've seen Nextcloud download files/folders when I know the local copy is the latest and I'm copying back from backups to keep the local correct. Grrr.
Image

I've also been playing around with the server, attempting to fix warnings. I hope I am not making things worse rather than better.

As requested, some debug logs attached.

Debug Archive 250119.zip
<!-- Failed to upload "Debug Archive 250120.zip" -->

Please could you explain a bit more what you want to see from the conflict dialogs, in addition to those attached above.

Some server side info:
There's nothing special, as far as I am aware.
There's a lot of data and the oc_filecache.ibd is large and there are a lot of files "22TB in use".

ls -l /var/lib/mysql/verivault/oc_filecache.frm /var/lib/mysql/verivault/oc_filecache.ibd
  -rw-rw----. 1 mysql mysql      23137 2025-01-19 22:29 /var/lib/mysql/verivault/oc_filecache.frm
  -rw-rw----. 1 mysql mysql 1153433600 2025-01-19 20:21 /var/lib/mysql/verivault/oc_filecache.ibd

mysql -pXXXXXXX verivault --table -e "select count(*) from oc_filecache;"
  +----------+
  | count(*) |
  +----------+
  |  3680397 |
  +----------+

Storage is effectively local, but is in fact network attached with redundancy (glusterfs), and performance can be poor for loads of small files. I moved the mariadb database files to local disk, and recently upgraded that to SSD to speed up queries. But it does occur to me that looking up anything in a 2.1GB file (oc_filecache.ibd) is going to be slow, even on SSD.

I am trying to understand how Nextcloud handles the timestamp on files to work out which is the newer file that needs to be uploaded or downloaded. I can see the mtime (modified time) on the client and server files. Additionally I can see a different time reported in the nextcloud web file viewer. This is awkward because it reports relative time (n hours ago), and is difficult to spot sub-second differences that may the source of the confustion. Where is this additional time stored? Is it related to the E-Tag differences?

I have a suspicion that Nextcloud uses the present time for newly updated files (it should never do that because the flight time could be significant). BTW, I've checked, all machines are in time sync. Under what circumstances does Nextcloud use wall-clock time as opposed to the file modified time?

Is there any more specific server side info you wish to see?

@Bockeman
Copy link
Author

How do I get to upload the most recent debug archive? Is it too larget at 36MB?
<!-- Failed to upload "Debug Archive 250120.zip" -->

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mgallien @Bockeman