Improve IP Logging for Analytics

[jakkuh]

It'd be awesome if some work could be put into IP fetching, we currently see about a 30% average rate of "unknown" IPs.

I'm not sure how this is even possible as every request has to come from an IP.

Any insight?

[jakkuh]

Here's some additional information regarding how Cloudflare handles HTTP Request Headers in regards to IP addresses.

https://support.cloudflare.com/hc/en-us/articles/200170986-How-does-Cloudflare-handle-HTTP-Request-headers-

There's also a header which supplies the country code of the originating visitor, so maybe if the IP is unavailable for some reason, we can still log the Country via that?

[acelaya]

Accurately detecting the location of 100% of visits is virtually impossible.

Sometimes there are IP addresses that cannot be found in the GeoIp database or the fallback geolocation API, or they can only be partially located (we have the country, but not the city).

Also, there are visitors which do not send any IP address. It is there when real users visit the link from a browser, but if you share a link on Twitter (for example), you will automatically see several visits from their crawlers which cannot be located. This applies to any other kind of web bot.

This kind of visits are currently included with the rest. You can determine how many visits are not coming from real users based on the browser's chart. Those that did not provide a user agent that can be matched to a standard browser, are probably not real users.

Then shlink makes use of this library in order to determine the address https://github.com/akrabat/ip-address-middleware

It checks for the most common and standard headers where IP addresses usually live, but those headers can be configured, so if cloudflare is using some non-standard header, it should be easy to include. I will take a look at the docs.

[acelaya]

I will include this in next release, inspecting requests for the next list of headers.

$headersToInspect = [
    'X-Real-Ip',
    'CF-Connecting-IP',
    'True-Client-IP',
    'Forwarded',
    'X-Forwarded-For',
    'X-Forwarded',
    'X-Cluster-Client-Ip',
    'Client-Ip',
];

That should cover most of the cases.