Client error 403

[theilgaard]

Getting this while searching for a song.

Oops: Could not search for chords: 403 Client Error: Forbidden for url: https://www.ultimate-guitar.com/search.php?search_type=title&value=mr%20jones

[kmille]

Which instance do you use? UG started to block IPs. I tried to fix it by using Tor (don't know if/how often external ip s are changed). Ah, I see that now even with Tor it gets blocked.

Commit: f0ab6a3

[4oo4]

Still getting this error on the main instance (freetar.de), but other instances seem to be OK. One solution might be to use AWS API Gateway to proxy requests to UG from a very large pool of IP addresses. API Gateway isn't free but it's extremely cheap.

https://github.com/Ge0rg3/requests-ip-rotator

[kmille]

Still getting this error on the main instance (freetar.de), but other instances seem to be OK. One solution might be to use AWS API Gateway to proxy requests to UG from a very large pool of IP addresses. API Gateway isn't free but it's extremely cheap.

https://github.com/Ge0rg3/requests-ip-rotator

Hey,
that sound's interesting. Also thought about adding a simple caching. The thing with requrests-ip-rotator is:

Please note that these requests can be easily identified and blocked, since they are sent with unique AWS headers (i.e. "X-Amzn-Trace-Id").

Cloudflare will detect and block this (probably?). Also see this issue: Ge0rg3/requests-ip-rotator#51. Maybe some of you has a Cloudflare account and can check that?

Hmm, but this seems to work:

curl 'https://www.ultimate-guitar.com/search.php?search_type=title&value=adele' -H "X-Amzn-Trace-Id: 123" > ~/Downloads/index.html

[kmille]

I tried "some things" and it looks like it's working again. Have not pushed code/released software yet. It's deployed on https://freetar.de. Feel free to test and give feedback!

[4oo4]

I use freetar almost daily and I have not ran into a 403 issue once since whatever changes you made. Before it felt like more than 50% of the time, so whatever you did seems like it's working.