[Test-Proxy] Preserve original format when returning recorded response body content

[mccoyp]

Context

Python's azure-containerregistry library has tests that read a JSON manifest file and send this content to the service. The service creates a digest based on the stream content that was sent; this content is whitespace-dependent. Therefore, the digest we get from e.g. manifest_spaces.json:

{"foo": 2, "bar": "baz"}

will not match the digest we get from e.g. manifest_no_spaces.json:

{"foo":2,"bar":"baz"}

This is because the former digest is calculated with additional spaces in the stream.

In live mode, tests that validate the locally-calculated digest with the service-returned digest succeed when using manifest_spaces.json, since spaces are used in both digest calculations. The problem is that these tests fail in playback -- we calculate the local digest using spaces, but the recorded service response is returned without spaces by the test proxy.

For example, here's a recorded manifest from a response body (source from the assets repository):

"ResponseBody": {
  "config": {
    "digest": "sha256:feb5d9fea6a5e9606aa995e879d862b825965ba48de054caab5ef356dc6b3412",
    "mediaType": "application/vnd.docker.container.image.v1\u002Bjson",
    "size": 1469
  },
  "layers": [
    {
      "digest": "sha256:2db29710123e3e53a794f2694094b9b4338aa9ee5c40b930cb8063a1be392c54",
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "size": 2479
    }
  ],
  "mediaType": "application/vnd.docker.distribution.manifest.v2\u002Bjson",
  "schemaVersion": 2
}

When the test proxy sends the response body content in playback, the recorded JSON is written back without any whitespace -- no spaces between keys and values. Because of this, the digest that's calculated from the response body differs from the one calculated from the local manifest file.

Feature request

If we had a way to request a certain formatting of the response body, for example with a regular expression, we could preserve the actual response body format in playback. As things stand, the response body content in live mode doesn't exactly match the response body content in playback.

@scbedd I could probably help with this, but I would need assistance in figuring out where exactly the response body is fetched from the recording and sent back to the test.

[scbedd]

To properly do this, we need to maintain exactly the body that was sent. This will mean storing the Request/Response body as base64 encoded value.

We can either:

1. Add a new attribute, store the body there. Keep existing RequestBody for visibility only.
2. Replace RequestBody/ResponseBody w/ the base64 value.

In either case, we will need to update at the very least:

• RecordEntry.DeserializeBody()
• RecordEntry.SerializeBody()
• The RecordEntry class itself, to reflect the fact that the body isn't part of the json document, and is instead just a string property (versus emplacing it as an actual additional json object). What this looks like boils down to which path we choose from above.
• Update where we return the selected value. In RecordingHandler.
• Handle temp deserialization for sanitization.
  • The deserialization handling should be the same as the work we do for [Prioritized] Encoded body fragments are decoded appropriately within multipart/mixed payloads #2900

And any attachment points related to these.

[scbedd]

@mccoyp this is now resolved. You will need to re-record the tests, but we now treat these as non-text bodies so they are stored byte-for-byte as they come in.

Resolved in #8430 #8325