Use HeadObject for lookup

[jamesbornholt]

Our current lookup does two concurrent ListObjects requests. After
thinking about it a bit more carefully, one of them can be replaced with
a cheaper, faster HeadObject request. The "unsuffixed" request we were
doing was purely to discover whether an object of the exact looked-up
name existed, which is what HeadObject does. Switching to HeadObject
reduces the request costs of a lookup.

One disadvantage of HeadObject is when looking up directories. The
unsuffixed ListObjects we're replacing here could discover a common
prefix and return it immediately without waiting for the other request
to complete. But in practice, the two requests were dispatched
concurrently, so the customer still pays for both requests, and the
latency is the minimum latency of two concurrently ListObjects. Now,
the latency for a directory lookup will be the maximum of a concurrent
ListObjects and HeadObject.

An issue in this change is that we expect HeadObject to return 404 when
doing directory lookups, but right now the way our error types are
structured gives us no way to distinguish 404s from other errors. For
now, I'm just swallowing all errors on the HeadObject request, and I'll
follow up with a broader change to fix our error handling story to make
this work.

This is a partial fix for #12, but in future we can do better for
lookups against objects we've seen before by remembering their type.

Signed-off-by: James Bornholt [email protected]

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[jorajeev]

Unclear why you're doing a map_err when you just care about the Ok case. Maybe just warn! with the error code and say we're ignoring the errors for now?

[jamesbornholt]

Ah, this is just leftover from the old code, which mapped the error so it could return it with ?. I’d prefer not to add the warning since it will be very noisy (every directory lookup) and I’m fast-following this change with the one that handles this case correctly, so this change is really a temporary state.