Double encoded URLs #920

fri-K · 2020-04-05T11:36:43Z

The version of zmNinja you are reporting:

1.3.096

The version and OS of ZoneMinder you are using:

1.35.1, Ubuntu Bionic (18.04)

Platform zmNinja is running on

 Nokia 6, Android 9, SGS 3, Android 6

Describe the bug
zmNinja is double encoding URLs when I'm using compiled by myself version (build with build_android.sh script to be exact)

Debug logs

Jan 3, 2020 06:02:02 PM DEBUG getEvents:https://example.com/api/events/index/AlarmFrames >=:1.json?&sort=StartTime&direction=desc&page=1&token=<removed>
Jan 3, 2020 06:02:02 PM DEBUG Setting up carousel watchers
Jan 3, 2020 06:02:03 PM DEBUG ***  Inside native HTTP error for url:https://example.com/api/events/index/AlarmFrames%2520%253E=:1.json?&sort=StartTime&direction=desc&page=1&token=<removed>

Additional context
I made necessary changes to fix this up, but I'm not sure

if you accept the changes
if it's working fine on iOS/desktop

I'll create merge request with those changes.

The text was updated successfully, but these errors were encountered:

welcome · 2020-04-05T11:36:44Z

👋 Thanks for opening your first issue here! If you're reporting a 🐞 bug, please make sure you follow the issue template or I may not respond.

pliablepixels · 2020-04-05T11:44:42Z

Happy to take a look at the changes. I’m curious why it would only double encode when you build from source but not in the version I release to play store.

fri-K · 2020-04-05T12:16:16Z

Please take a look at #921 . I truly have no idea why it double encodes only my compilations, but it's bit frustrating. I hope my changes doesn't broke anything.

pliablepixels · 2020-04-05T12:46:22Z

Thanks. I am a little confused. In the logs you posted:

Jan 3, 2020 06:02:02 PM DEBUG getEvents:https://example.com/api/events/index/AlarmFrames >=:1.json?&sort=StartTime&direction=desc&page=1&token=<removed>
Jan 3, 2020 06:02:02 PM DEBUG Setting up carousel watchers
Jan 3, 2020 06:02:03 PM DEBUG ***  Inside native HTTP error for url:https://example.com/api/events/index/AlarmFrames%2520%253E=:1.json?&sort=StartTime&direction=desc&page=1&token=<removed>

Which part is double encoded? It adds an encoding for a space (%20) which is not double encoding. Double encoding would be encoding an already encoded URI.

pliablepixels · 2020-04-05T12:47:39Z

Oh Interesting, I see AlarmFrames%2520%253E in yours. Hmm, let me read up a bit. Doesn't happen on mine. That url escape is necessary for my system.

fri-K · 2020-04-05T13:43:51Z

There are two parts in this problem, simply changing spaces to + signs in URL should be unharmful for everyone, but changes committed to www/js/app.js may provide some troubles. I can remove changes from app.js if you prefer?

pliablepixels · 2020-04-05T13:46:42Z

I think I know what is going on.
Can you try this:
a) Undo all the space removals (I agree it's no harm, but for now, just keep it)
b) Replace encodeURI with encodeURIComponent

Does that work?

pliablepixels · 2020-04-05T13:52:56Z

It seems to me I need to:
a) Use encodeURI for the URI
b) Use encodeURIComponent for the parameters

https://stackoverflow.com/a/3608791/1361529

fri-K · 2020-04-05T14:03:37Z

Oh, I can test encodeURIComponent but as of second part it seems to be more complicated than simple function replacement.
But, why is this working with your compilations, do you applies some patches before compilation or something?

pliablepixels · 2020-04-05T16:39:17Z

My URI encode is working correctly. I added a logging statement in app.js before encodeURI.
I get:

This is the URL that is passed:

2020-04-05 12:36:04.576370-0400 zmNinja[31284:5515423] Apr 5, 2020 12:36:04 PM DEBUG cordova: got url https://server/zm/api/events/index/AlarmFrames >=:1/MonitorId !=:1/MonitorId !=:5/MonitorId !=:7/MonitorId !=:9/MonitorId !=:11/StartTime <=:2020-04-04 12:34:57/EndTime >=:2020-04-03 12:34:57.json?sort=StartTime&direction=desc&token=<removed>&page=1
2020-04-05 12:36:04.577578-0400 zmNinja[31284:5515423] Apr 5, 2020 12:36:04 PM DEBUG

This is the output of encodeURI:

 cordova: url after encode https://server/zm/api/events/index/AlarmFrames%20%3E=:1/MonitorId%20!=:1/MonitorId%20!=:5/MonitorId%20!=:7/MonitorId%20!=:9/MonitorId%20!=:11/StartTime%20%3C=:2020-04-04%2012:34:57/EndTime%20%3E=:2020-04-03%2012:34:57.json?sort=StartTime&direction=desc&token=<removed>&page=1

pliablepixels · 2020-04-07T11:40:38Z

Okay, let's do a quick test. I think its related to the fact that your encoding may not be UTF8

Open up a chrome browser, and its console log (Inspect source - it will give you a console tab)

Now do this:

url="https://example.com/api/events/index/AlarmFrames >=:1.json?&sort=StartTime&direction=desc&page=1&token=1234"

Then please give me the output of:

encodeURI(url)

and then

encodeURI(decodeURI(url))

fri-K · 2020-04-07T19:55:49Z

Sorry for slow responses, but I made few tests:

New Android app

Apr 7, 2020 02:20:11 PM INFO Entering Portal Main
Apr 7, 2020 02:20:11 PM DEBUG current version: 1.3.097 & available version 1.3.096
(...)
Apr 7, 2020 02:20:18 PM DEBUG cordova: got url https://example.com/api/monitors/index/Type !=:WebSite.json?&token=<removed>
Apr 7, 2020 02:20:18 PM DEBUG cordova: url after encode https://example.com/api/monitors/index/Type%20!=:WebSite.json?&token=<removed>
Apr 7, 2020 02:20:18 PM DEBUG ***  Inside native HTTP error for url:https://example.com/api/monitors/index/Type%2520!=:WebSite.json?&token=<removed>
Apr 7, 2020 02:20:18 PM INFO CACHE: error with http get [object Object]
Apr 7, 2020 02:20:18 PM ERROR Monitor load failed {"status":500,"url":"https://example.com/api/monitors/index/Type%2520!=:WebSite.json?&token=<removed>
Apr 7, 2020 02:20:18 PM DEBUG getMonitorsNow: returning 0 monitors
Apr 7, 2020 02:20:18 PM INFO EventCtrl called with: E/MID=0 playEvent =  false

and test you asked for:

url="https://example.com/api/events/index/AlarmFrames >=:1.json?&sort=StartTime&direction=desc&page=1&token=1234"
"https://example.com/api/events/index/AlarmFrames >=:1.json?&sort=StartTime&direction=desc&page=1&token=1234"
encodeURI(url)
"https://example.com/api/events/index/AlarmFrames%20%3E=:1.json?&sort=StartTime&direction=desc&page=1&token=1234"
encodeURI(decodeURI(url))
"https://example.com/api/events/index/AlarmFrames%20%3E=:1.json?&sort=StartTime&direction=desc&page=1&token=1234"

Or should I use true token?

pliablepixels · 2020-04-07T20:04:22Z

I'm confused. In your latest log you say:

url="https://example.com/api/events/index/AlarmFrames >=:1.json?&sort=StartTime&direction=desc&page=1&token=1234"
"https://example.com/api/events/index/AlarmFrames >=:1.json?&sort=StartTime&direction=desc&page=1&token=1234"
encodeURI(url)
"https://example.com/api/events/index/AlarmFrames%20%3E=:1.json?&sort=StartTime&direction=desc&page=1&token=1234"

But in your first post, the encoding is wrong

Jan 3, 2020 06:02:02 PM DEBUG getEvents:https://example.com/api/events/index/AlarmFrames >=:1.json?&sort=StartTime&direction=desc&page=1&token=<removed>
Jan 3, 2020 06:02:02 PM DEBUG Setting up carousel watchers
Jan 3, 2020 06:02:03 PM DEBUG ***  Inside native HTTP error for url:https://example.com/api/events/index/AlarmFrames%2520%253E=:1.json?&sort=StartTime&direction=desc&page=1&token=<removed>

They both use the same encoding command

fri-K · 2020-04-07T20:22:23Z

maybe I cutted it to much, here is more of it:

Apr 7, 2020 02:20:18 PM INFO EventCtrl called with: E/MID=0 playEvent =  false
Apr 7, 2020 02:20:18 PM DEBUG >>>height of list/scrub set to 330 and 370
Apr 7, 2020 02:20:18 PM INFO Image padding digits reported as 5
Apr 7, 2020 02:20:18 PM DEBUG cache_or_http error:[object Object]
Apr 7, 2020 02:20:18 PM DEBUG cordova: got url https://example.com/api/monitors/index/Type !=:WebSite.json?&token=<removed>
Apr 7, 2020 02:20:18 PM DEBUG cordova: url after encode https://example.com/api/monitors/index/Type%20!=:WebSite.json?&token=<removed>
Apr 7, 2020 02:20:18 PM DEBUG EventSever: sendMessage: received->{"event":"push","data":{"type":"badge","badge":0}}
Apr 7, 2020 02:20:18 PM DEBUG EventSever: Not sending WSS message as event server is off
Apr 7, 2020 02:20:18 PM DEBUG Starting page refresh timer
Apr 7, 2020 02:20:18 PM DEBUG ***  Inside native HTTP error for url:https://example.com/api/monitors/index/Type%2520!=:WebSite.json?&token=<removed>
Apr 7, 2020 02:20:18 PM DEBUG getInitialEvents called
Apr 7, 2020 02:20:18 PM DEBUG Setting up carousel watchers
Apr 7, 2020 02:20:18 PM DEBUG cordova: got url https://example.com/api/events/consoleEvents/1 hour/AlarmFrames >=:1.json?&token=<removed>
Apr 7, 2020 02:20:18 PM DEBUG cordova: url after encode https://example.com/api/events/consoleEvents/1%20hour/AlarmFrames%20%3E=:1.json?&token=<removed>
Apr 7, 2020 02:20:18 PM DEBUG cordova: got url https://example.com/api/events/consoleEvents/1 day/AlarmFrames >=:1.json?&token=<removed>
Apr 7, 2020 02:20:18 PM DEBUG cordova: url after encode https://example.com/api/events/consoleEvents/1%20day/AlarmFrames%20%3E=:1.json?&token=<removed>
Apr 7, 2020 02:20:18 PM DEBUG cordova: got url https://example.com/api/events/consoleEvents/1 week/AlarmFrames >=:1.json?&token=<removed>
Apr 7, 2020 02:20:18 PM DEBUG cordova: url after encode https://example.com/api/events/consoleEvents/1%20week/AlarmFrames%20%3E=:1.json?&token=<removed>
Apr 7, 2020 02:20:18 PM DEBUG cordova: got url https://example.com/api/events/consoleEvents/1 month/AlarmFrames >=:1.json?&token=<removed>
Apr 7, 2020 02:20:18 PM DEBUG cordova: url after encode https://example.com/api/events/consoleEvents/1%20month/AlarmFrames%20%3E=:1.json?&token=<removed>
Apr 7, 2020 02:20:18 PM DEBUG Does login need to hear the wizard? false
Apr 7, 2020 02:20:18 PM DEBUG Cancelling page reload timer
Apr 7, 2020 02:20:18 PM DEBUG EventCtrl: Deregistering resize listener
Apr 7, 2020 02:20:19 PM DEBUG ***  Inside native HTTP error for url:https://example.com/api/events/consoleEvents/1%2520hour/AlarmFrames%2520%253E=:1.json?&token=<removed>
Apr 7, 2020 02:20:19 PM DEBUG ***  Inside native HTTP error for url:https://example.com/api/events/consoleEvents/1%2520day/AlarmFrames%2520%253E=:1.json?&token=<removed>
Apr 7, 2020 02:20:19 PM DEBUG ***  Inside native HTTP error for url:https://example.com/api/events/consoleEvents/1%2520week/AlarmFrames%2520%253E=:1.json?&token=<removed>
Apr 7, 2020 02:20:19 PM DEBUG ***  Inside native HTTP error for url:https://example.com/api/events/consoleEvents/1%2520month/AlarmFrames%2520%253E=:1.json?&token=<removed>
Apr 7, 2020 02:20:27 PM INFO Login data not changed, not saving
Apr 7, 2020 02:20:57 PM INFO file location:file:///data/user/0/com.pliablepixels.zmninja_pro/files/zmNinjaLog.txt

pliablepixels · 2020-04-07T22:01:57Z

ref to investigate more silkimen/cordova-plugin-advanced-http#195

pliablepixels · 2020-04-15T14:56:17Z

@fri-K can you please pull my dev branch https://github.com/pliablepixels/zmNinja/tree/dev and test?

fri-K · 2020-04-17T20:15:56Z

I'll bet you're surprised, zmNinja 1.4.3 fixes this bug. Well done, thanks :)

pliablepixels · 2020-04-18T12:09:13Z

sounds good

fri-K mentioned this issue Apr 5, 2020

Fixed double encoded URLs #921

Closed

pliablepixels added the bug label Apr 5, 2020

pliablepixels added the source build label Apr 5, 2020

fri-K added a commit to gospogied/zmNinja that referenced this issue Apr 17, 2020

Merge branch 'master' into ZoneMinder#920-double-encoded-urls

0d35f05

pliablepixels closed this as completed Apr 18, 2020

pliablepixels mentioned this issue Apr 30, 2020

1.4.3 encode URL seems broken at least on iOS (Event/24 hr filters etc) #933

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Double encoded URLs #920

Double encoded URLs #920

fri-K commented Apr 5, 2020

welcome bot commented Apr 5, 2020

pliablepixels commented Apr 5, 2020

fri-K commented Apr 5, 2020

pliablepixels commented Apr 5, 2020

pliablepixels commented Apr 5, 2020

fri-K commented Apr 5, 2020

pliablepixels commented Apr 5, 2020

pliablepixels commented Apr 5, 2020

fri-K commented Apr 5, 2020

pliablepixels commented Apr 5, 2020 •

edited

Loading

pliablepixels commented Apr 7, 2020

fri-K commented Apr 7, 2020

pliablepixels commented Apr 7, 2020

fri-K commented Apr 7, 2020

pliablepixels commented Apr 7, 2020 •

edited

Loading

pliablepixels commented Apr 15, 2020

fri-K commented Apr 17, 2020

pliablepixels commented Apr 18, 2020

Double encoded URLs #920

Double encoded URLs #920

Comments

fri-K commented Apr 5, 2020

welcome bot commented Apr 5, 2020

pliablepixels commented Apr 5, 2020

fri-K commented Apr 5, 2020

pliablepixels commented Apr 5, 2020

pliablepixels commented Apr 5, 2020

fri-K commented Apr 5, 2020

pliablepixels commented Apr 5, 2020

pliablepixels commented Apr 5, 2020

fri-K commented Apr 5, 2020

pliablepixels commented Apr 5, 2020 • edited Loading

pliablepixels commented Apr 7, 2020

fri-K commented Apr 7, 2020

pliablepixels commented Apr 7, 2020

fri-K commented Apr 7, 2020

pliablepixels commented Apr 7, 2020 • edited Loading

pliablepixels commented Apr 15, 2020

fri-K commented Apr 17, 2020

pliablepixels commented Apr 18, 2020

pliablepixels commented Apr 5, 2020 •

edited

Loading

pliablepixels commented Apr 7, 2020 •

edited

Loading