Consistent timeouts when syncing #761

weierophinney · 2020-03-12T21:08:37Z

When I open the app, I consistently receive network timeouts when syncing notes from my server. This means that I am no longer receiving new notes from the server, and notes created on my device are not synced to the server.

To Reproduce

Steps to reproduce the behavior:

Open the app
Wait for the initial sync to complete
Receive timeout error

OR

Open the app
Click the "+" button to create a new note
Create the note
Save the note
Wait for sync to complete
Receive timeout error

Expected behavior

Successful sync.

Stacktrace

I receive the following stacktrace in the app:

App Version: 2.10.3
App Version Code: 81
Files App Version Code: 30100190

---

OS Version: 4.4.153-perf+(00WW_1_400_SP01)
OS API Level: 28
Device: DDV_sprout
Model (and Product): Nokia 7.2 (Daredevil_00WW)

---

com.nextcloud.android.sso.exceptions.UnknownErrorException: Read timed out
	at com.nextcloud.android.sso.api.AidlNetworkRequest.performNetworkRequestV2(AidlNetworkRequest.java:172)
	at com.nextcloud.android.sso.api.NextcloudAPI.performNetworkRequestV2(NextcloudAPI.java:148)
	at it.niedermann.owncloud.notes.persistence.NotesClient.requestServer(NotesClient.java:163)
	at it.niedermann.owncloud.notes.persistence.NotesClient.getNotes(NotesClient.java:98)
	at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.pullRemoteChanges(NoteServerSyncHelper.java:413)
	at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.doInBackground(NoteServerSyncHelper.java:320)
	at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.doInBackground(NoteServerSyncHelper.java:284)
	at android.os.AsyncTask$2.call(AsyncTask.java:333)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:245)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
	at java.lang.Thread.run(Thread.java:764)

Server

Nextcloud-Version: 17.0.3
Nextcloud Notes-Version (server app): 3.1.5
PHP-Version: 7.3.15

I have around 1700 notes on the server.

Smartphone

Nextcloud Notes-Version (android app): 2.10.3
F-Droid or Play Store: Play Store
Device: Nokia 7.2
Android-Version: 9 (Pi)

The text was updated successfully, but these errors were encountered:

stefan-niedermann · 2020-03-13T05:24:57Z

Huh, 1700 Notes... insane 😄

Are you able to grab a logcat? I would love to see the timestamps to check if it is really that long running and therefore closing the connection.

Can you try to delete the storage of the app and perform a full synchronization with a good wifi, so the initial sync completes fast? Next notes should theoretically sync faster because it will only be incremental one not by another.

stefan-niedermann · 2020-03-18T09:27:23Z

bump @weierophinney

weierophinney · 2020-03-18T14:30:23Z

I tried deleting the storage and cache of the app, and then doing a full synchronization; same result.

Click to expand log

App Version: 2.10.3
App Version Code: 81
Files App Version Code: 30100190

---

OS Version: 4.4.153-perf+(00WW_1_400_SP01)
OS API Level: 28
Device: DDV_sprout
Model (and Product): Nokia 7.2 (Daredevil_00WW)

---

com.nextcloud.android.sso.exceptions.UnknownErrorException: Read timed out
    at com.nextcloud.android.sso.api.AidlNetworkRequest.performNetworkRequestV2(AidlNetworkRequest.java:172)
    at com.nextcloud.android.sso.api.NextcloudAPI.performNetworkRequestV2(NextcloudAPI.java:148)
    at it.niedermann.owncloud.notes.persistence.NotesClient.requestServer(NotesClient.java:163)
    at it.niedermann.owncloud.notes.persistence.NotesClient.getNotes(NotesClient.java:98)
    at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.pullRemoteChanges(NoteServerSyncHelper.java:413)
    at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.doInBackground(NoteServerSyncHelper.java:320)
    at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.doInBackground(NoteServerSyncHelper.java:284)
    at android.os.AsyncTask$2.call(AsyncTask.java:333)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:245)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
    at java.lang.Thread.run(Thread.java:764)

Can you provide instructions on how to do a logcat?

stefan-niedermann · 2020-03-19T18:34:46Z

Sure, you can grab any free LogCat-app from F-Droid or the Play Store (for example this one).

Then just start recording in the LogCat-app, do what every you want in the Notes app and after you get the timeout, go back to the LogCat-app and stop recording. The logfile might contain private information, so you better should send it via e-mail at [email protected] instead of attaching it here.

If you have further questions, just let me know.

jdkruzr · 2020-03-26T19:40:54Z

FWIW I have exactly the same error, even after dumping Android app data. QOwnNotes on Mac and Windows still work fine.

App Version: 2.11.3
App Version Code: 2011003
Files App Version Code: 30100190

---

OS Version: 4.14.111-gb7713fd5d6ed-ab5820689(5831595)
OS API Level: 29
Device: flame
Model (and Product): Pixel 4 (flame)

---

com.nextcloud.android.sso.exceptions.UnknownErrorException: Read timed out
	at com.nextcloud.android.sso.api.AidlNetworkRequest.performNetworkRequestV2(AidlNetworkRequest.java:172)
	at com.nextcloud.android.sso.api.NextcloudAPI.performNetworkRequestV2(NextcloudAPI.java:148)
	at it.niedermann.owncloud.notes.persistence.NotesClient.requestServer(NotesClient.java:162)
	at it.niedermann.owncloud.notes.persistence.NotesClient.getNotes(NotesClient.java:98)
	at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.pullRemoteChanges(NoteServerSyncHelper.java:451)
	at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.doInBackground(NoteServerSyncHelper.java:358)
	at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.doInBackground(NoteServerSyncHelper.java:320)
	at android.os.AsyncTask$3.call(AsyncTask.java:378)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:289)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
	at java.lang.Thread.run(Thread.java:919)

stefan-niedermann · 2020-03-26T20:18:56Z

Here also a logcat would be interesting.

How many notes (number of files and complete size in kB / MB) do you have on your clouds?

Bricki85 · 2020-03-29T19:57:12Z

Hi,
I have the same Problem. But I had it with an earlier version also. The only other change was that I updated the next cloud instance to version 18.
I have only like 20-30 Notes about 200kb size.

App Version: 2.11.3
App Version Code: 2011003
Files App Version Code: 30110090

---

OS Version: 4.4.111-17263988(G950FXXS8DTC1)
OS API Level: 28
Device: dreamlte
Model (and Product): SM-G950F (dreamltexx)

---

com.nextcloud.android.sso.exceptions.UnknownErrorException: Read timed out
	at com.nextcloud.android.sso.api.AidlNetworkRequest.performNetworkRequestV2(AidlNetworkRequest.java:172)
	at com.nextcloud.android.sso.api.NextcloudAPI.performNetworkRequestV2(NextcloudAPI.java:148)
	at it.niedermann.owncloud.notes.persistence.NotesClient.requestServer(NotesClient.java:162)
	at it.niedermann.owncloud.notes.persistence.NotesClient.getNotes(NotesClient.java:98)
	at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.pullRemoteChanges(NoteServerSyncHelper.java:451)
	at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.doInBackground(NoteServerSyncHelper.java:358)
	at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.doInBackground(NoteServerSyncHelper.java:320)
	at android.os.AsyncTask$2.call(AsyncTask.java:333)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:245)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
	at java.lang.Thread.run(Thread.java:764)

stefan-niedermann · 2020-04-16T18:43:35Z

Can you all please check if there is a hint about an access in your server logs?

Interesting would be (if one has the possibility to) if you could remove all notes from your server and add them one by one back to your cloud (syncing after each note) and check whether this depends on a single special note or on the pure amount.

Also try switching between mobile data and wifi so we can get a hint if this is related to the network type.

How is the sync with the normal files app? Does it work flawlessly?

danidoni · 2020-07-07T14:05:59Z

Hi, i'm having this problem now.

Can you all please check if there is a hint about an access in your server logs?

Interesting would be (if one has the possibility to) if you could remove all notes from your server and add them one by one back to your cloud (syncing after each note) and check whether this depends on a single special note or on the pure amount.

I can confirm adding notes in small chunks seems to work and not trigger the timeout.

A chunk of 213 files (~290KiB) fails, but small chunks of 50 (89KiB) or 100 files (~160KiB) work fine.

My total amount of notes is 382 (2.8MB), the whole amount does not sync.

Also try switching between mobile data and wifi so we can get a hint if this is related to the network type.

In my case, the network type didn't make any difference.

How is the sync with the normal files app? Does it work flawlessly?

The normal files app works flawlessly.

stefan-niedermann · 2021-05-10T17:14:26Z

Yeah, well. I guess the probably best solution will be to exclude the content property when fetching the Notes list on the very first import. Then a request for each note should be make so we keep the total size of the requests as low as possible (and it has the nice side effect that we can display a progress bar when importing an account).

For following synchronizations this shouldn't matter since we already use pruneBefore to only transfer the notes that actually have been changed.

It will require some work in the ImportAccountActivity and in the MainActivity (for importing further accounts) but should be doable.

korelstar · 2021-05-10T18:00:38Z

I'm working on an enhancement of the API, so that you (as client developer) can define a maximum amount of notes that should be transferred in one response. This is the last feature that I want to add to the API version 1.2 and then I will release the app version 4.1. Using this should be more comfortable. :-)

zemua · 2021-09-24T17:55:52Z

Same issue here with 200+ files, F-Droid version of the app v3.4.11, copying files in chunks solves it.
Hopefully this gets solves and I don't have to do this each time I get the app installed in a device.
Thank you for the good work.

Signed-off-by: Stefan Niedermann <[email protected]>

stefan-niedermann · 2021-09-30T10:41:30Z

So, something is happening here, i just commited the first change in this direction (5801f00) to synchronize notes one by one while importing.

This can (and probably will) of course be extended in the future to use the chunk API provided in the Notes server app 3.4.0. The reason i started to implement it this way is to bring the benefit to all users, regardless which version of the server app is installed. The changes will be shipped in Notes Android 3.4.14.

Adding account did formerly not require any network access. This commit mocks those resources. Signed-off-by: Stefan Niedermann <[email protected]>

stefan-niedermann added the sync file syncing label Mar 13, 2020

stefan-niedermann added the Stale :: Waiting for response label Jun 9, 2020

This comment has been minimized.

Sign in to view

boinappi mentioned this issue Feb 18, 2021

Already existing Notes don't show up after first sync #1067

Closed

korelstar mentioned this issue May 20, 2021

API: allow requesting notes list in chunks nextcloud/notes#720

Merged

3 tasks

teatreeoilchocolate mentioned this issue Jun 21, 2021

Notes created via Android's "Share Link" feature are not synced #1272

Closed

stefan-niedermann mentioned this issue Aug 5, 2021

Unable to synchronize: cryptic "duplicate key" error #1313

Closed

stefan-niedermann added a commit that referenced this issue Sep 30, 2021

#761 Import notes one by one to avoid read timeout for the first sync

5801f00

Signed-off-by: Stefan Niedermann <[email protected]>

stefan-niedermann removed the Waiting for response label Sep 30, 2021

stefan-niedermann added a commit that referenced this issue Sep 30, 2021

#761 Adjust unit tests

cb52736

Adding account did formerly not require any network access. This commit mocks those resources. Signed-off-by: Stefan Niedermann <[email protected]>

stefan-niedermann closed this as completed Jan 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Consistent timeouts when syncing #761

Consistent timeouts when syncing #761

weierophinney commented Mar 12, 2020

stefan-niedermann commented Mar 13, 2020

stefan-niedermann commented Mar 18, 2020

weierophinney commented Mar 18, 2020

stefan-niedermann commented Mar 19, 2020

jdkruzr commented Mar 26, 2020

stefan-niedermann commented Mar 26, 2020

Bricki85 commented Mar 29, 2020 •

edited by stefan-niedermann

Loading

stefan-niedermann commented Apr 16, 2020

danidoni commented Jul 7, 2020 •

edited

Loading

This comment has been minimized.

This comment has been minimized.

stefan-niedermann commented May 10, 2021

korelstar commented May 10, 2021

zemua commented Sep 24, 2021

stefan-niedermann commented Sep 30, 2021

Consistent timeouts when syncing #761

Consistent timeouts when syncing #761

Comments

weierophinney commented Mar 12, 2020

To Reproduce

Expected behavior

Stacktrace

Server

Smartphone

stefan-niedermann commented Mar 13, 2020

stefan-niedermann commented Mar 18, 2020

weierophinney commented Mar 18, 2020

stefan-niedermann commented Mar 19, 2020

jdkruzr commented Mar 26, 2020

stefan-niedermann commented Mar 26, 2020

Bricki85 commented Mar 29, 2020 • edited by stefan-niedermann Loading

stefan-niedermann commented Apr 16, 2020

danidoni commented Jul 7, 2020 • edited Loading

This comment has been minimized.

This comment has been minimized.

stefan-niedermann commented May 10, 2021

korelstar commented May 10, 2021

zemua commented Sep 24, 2021

stefan-niedermann commented Sep 30, 2021

Bricki85 commented Mar 29, 2020 •

edited by stefan-niedermann

Loading

danidoni commented Jul 7, 2020 •

edited

Loading