Gateway: handleSuperfluousNamespace doesn't work with Hostnames

[hacdias]

When the gateway is wrapped with the Hostname Handler, the superfluous namespace redirect doesn't work. This is because HostnameHandler will immediately say that the path is invalid. I suspect this may be happening with other things. It has probably happened before, but we've never noticed.

Solutions:

1. Remove it. Do we really need it?
2. Add it to the hostname handler. Then, we'll be verifying this twice which contains path validation. We already check the URI query twice, but that's cheap: if it doesn't exist, nothing is done. Here we always run the path.IsValid, which costs more.
3. Rework it to make it simpler: just check if URL starts with /ipfs/ipfs or /ipfs/ipns and do the redirect. Do not do path validation. That will happen after redirection anyways. Then add it to the Hostname handler again.

• Add conformance test, hint should say it's quality of life improvement.

[hacdias]

@lidel right now the handler does a 400 Bad Request with an HTML document that does a redirection using http-equiv. What do you think about simplifying this to a Permanent Redirect and streamlining our redirects?

[lidel]

@hacdias that HTML stop-gap was introduced in ipfs/kubo#7930 during NFT bubble, when people started using IPFS but were not familiar with IPFS, there were not man ydocs/resources at the time, so they did integration and addressing poorly, with invalid paths or URIs.

Example: https://opensea.io/assets/ethereum/0x3B3ee1931Dc30C1957379FAc9aba94D1C48a5405/219 which has ipfs://ipfs/Qmcg8f4F9cig2JWXunxJcdBe58Q5myYXPmGfuMn1TVeswD/nft.mp4 🙃

Times changed and most of people know how to use ipfs://, so the http-equiv stop-gap served its purpose, and we could clean this up, but we should keep a redirect to not break things.

Your suggestion to switch to HTTP 301 SGTM 👍