Report client status

[tobiasKaminsky]

What we agreed on:

• clients check for capability
• clients report only if a problem exists, if not, nothing is sent. This also means that if there was an error and error is gone server will not be informed, but can figure it out by last sending date (older than x days no response, but token active -> everything fine again)
• Android (Report client health android#12009) collects
  • any sync conflicts (with timestamp when it occured)
  • upload with error type:
    • CONFLICT_ERROR
    • UploadResult.MAINTENANCE_MODE
    • UploadResult.CREDENTIAL_ERROR
    • UploadResult.CANNOT_CREATE_FILE
    • UploadResult.SYNC_CONFLICT
    • UploadResult.UNKNOWN
    • UploadResult.FOLDER_ERROR
    • UploadResult.NETWORK_CONNECTION
    • UploadResult.SERVICE_INTERRUPTED
    • UploadResult.VIRUS_DETECTED
• iOS: Report client status ios#2634
• Desktop: Report client status desktop#6124

Brainstorm together how/what/when we can send status/health information from clients to server.
Scenario: company with IT department, that wants to have an overview over faulty clients

Already available to server:

• how many (and which) devices per user
• first "registration"
• last usage

What client could do

• send at fixed time (e.g. once a day) status report
• send only on error (e.g. sync error on Desktop)

Server: nextcloud/server#40160

@AndyScherzinger @jancborchardt

[jancborchardt]

We should also loop in @sorbaugh and/or a relevant engineer here. :)

[tobiasKaminsky]

collect local problems: writing problems, disk full, critical upload/download errors?
- send this with timestamp

[tobiasKaminsky]

Android collects

• any sync conflicts (with timestamp when it occured)
• upload with error type:
  • CONFLICT_ERROR
  • UploadResult.MAINTENANCE_MODE
  • UploadResult.CREDENTIAL_ERROR
  • UploadResult.CANNOT_CREATE_FILE
  • UploadResult.SYNC_CONFLICT
  • UploadResult.UNKNOWN
  • UploadResult.FOLDER_ERROR
  • UploadResult.NETWORK_CONNECTION
  • UploadResult.SERVICE_INTERRUPTED
  • UploadResult.VIRUS_DETECTED
• E2E errors

@camilasan @allexzander @claucambra @mgallien for Desktop
@marinofaggiana @mpivchev for iOS

Can you also add your ideas what you want/can send in case of problems to server?

[alperozturk96]

Server creates table under user table such as user_upload_statistics, user_app_events and expects statistic data (e.g UploadResult, Events).

Client sends UploadResult to server for related remote calls.

Maybe even more when user navigate the media screen we can also send events to server (e.g Events.mediaScreenOpen)

Therefore we can track faulty clients and optionally have a good overview about general app usage. This can be useful for some of our clients. @tobiasKaminsky @AndyScherzinger

[tobiasKaminsky]

For privacy reasons we do not want to track/send successful operations, but only faulty states.

[marinofaggiana]

• E2EE:

• errorE2EENotEnabled
• errorE2EEVersion
• errorE2EEKeyChecksums
• errorE2EEKeyEncodeMetadata
• errorE2EEKeyDecodeMetadata
• errorE2EEKeyVerifySignature
• errorE2EEJSon
• errorE2EELock
• errorE2EEEncryptFile
• errorE2EEEncryptPayloadFile
• errorE2EECounter
• errorE2EEGenerateKey
• errorE2EEEncodedKey
• errorE2EENoUserFound
• errorE2EEKeyCiphertext
• errorE2EEKeyFiledropCiphertext

• FORBIDDEN
• ROSOURCE_NOT_FOUND
• CONNECTION_LOST
• BAD_SERVER_RESPONSE
• CHARACTERS_FORBIDDEN

[marinofaggiana]

Android collects

• any sync conflicts (with timestamp when it occured)

• upload with error type:

  • CONFLICT_ERROR
  • UploadResult.MAINTENANCE_MODE
  • UploadResult.CREDENTIAL_ERROR
  • UploadResult.CANNOT_CREATE_FILE
  • UploadResult.SYNC_CONFLICT
  • UploadResult.UNKNOWN
  • UploadResult.FOLDER_ERROR
  • UploadResult.NETWORK_CONNECTION
  • UploadResult.SERVICE_INTERRUPTED
  • UploadResult.VIRUS_DETECTED

@camilasan @allexzander @claucambra @mgallien for Desktop @marinofaggiana @mpivchev for iOS

Can you also add your ideas what you want/can send in case of problems to server?

@tobiasKaminsky please write the code error near you internal code string (every o.s. use a different system but the same error code)

[marinofaggiana]

Is not clear:

Send the error for single upload where counter > 1 ? (some errors occur for individual uploads others for each upload, so more information should be sent) or if need only an alarm every upload failure >= 1 time should be transmitted, and one successful upload is enough to cancel everything and transmit nothing (no loop, example disk full).

[tobiasKaminsky]

Hm. For now I would say that we send each error category on its own, o.eg.:
CONFLICT_ERROR: 2, timestamp
VIRUS_DETECTED: 1, timestamp

They then can be combined on server, if needed.

[marinofaggiana]

can you write the errors number so all use the same ?

[marinofaggiana]

nextcloud/ios#2637

[allexzander]

I will come up with my ideas for desktop client later. For now, I think we must come up with a standardized way of collecting errors on the server, like have an SQL table with columns we must fill for every client app and also some kind of metadata column for platform-specific info. My idea is we need to have a list of error types(it can be long to include most common scenarios and we can add more error types later. Tobias already mentioned this. I would go even further and also include file full path or whatever unique id the file have such that file full path can later be obtained on the server. That way, we can later have some kind of view in the WebUI where we could sort errors by different columns (error type, timestamp with selection for specific time range, filename, folder type (group/normal/external storage), and also have repeating error type for the same file collected such that it is not displayed 1000 times (1000 rows for the same file for the same error) but instead is displayed as one row with a number of occurrences with the ability to open a detailed view with all 1000 rows when needed).

For example, we have server logs that can be viewed in Web UI in that fashion. This is something I would target for. Then those errors can be exported in the form of a text file or CSV file or something.

UPDATE: As I mentioned in my ideas in corresponding desktop client issue, we may also reduce spam and stop sending the same error for the same file in a given timeframe more then once by keeping track of these errors in the desktop client's local database, but in this case we might lose the ability to have detailed reports on the server.

Also, as mentioned in the desktop client's issue ideas, I would consider sending a client log (only the current log file) to the server in case of sync errors, as client logs are being rotated constantly and important log might get lost which will prevent debugging.

[tobiasKaminsky]

@allexzander can you add your categories/problem names here?

[allexzander]

@tobiasKaminsky

DownloadError

• DownloadError.CONFLICT
• DownloadError.CONFLICT_INVALID_CHARACTERS
• DownloadError.CONFLICT_CASECLASH
• DownloadError.CANNOT_CREATE_FILE
• DownloadError.SERVER_ERROR (with code and message in metadata)
• DownloadError.VIRTUAL_FILE_HYDRATION_FAILURE (with full description/reason in metadata)
• DownloadError.NO_FREE_SPACE

UploadError

• UploadError.SERVER_ERROR (with code and message in metadata)
• UploadError.NO_WRITE_PERMISSIONS
• UploadError.NO_FREE_SPACE