S2S call fails when encoding non-UTF-8 data

[oehlschl]

Still trying to get more details on this, but we're seeing PX fail open due to encoding errors when making the S2S call. Example error:

app/vendor/bundle/ruby/2.4.0/gems/activesupport-5.0.7.2/lib/active_support/core_ext/object/json.rb:34:in `encode': "\xE0" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)

I'm guessing there are some ASCII chars in one of the headers or in the cookie. This might be easiest to replicate with tests first.

[oehlschl]

Just replicated by passing a non-ASCII character in any additional header along with the request.

Ex:

curl -H "someheader:œ" ...

This causes the enforcer to fail with the following error:

PerimeterX[pxVerify]
PerimeterXContext[initialize]
PerimeterxCookieValidator:[verify]: no cookie
PerimeterxS2SValidator[verify]
PerimeterxS2SValidator[send_risk_request]: send_risk_request
PerimeterxRiskClient[format_headers]
px_ctx cookie_hmac key = false, value is: 
PxHttpClient[post]: runtime: 0.309
activesupport-5.0.7.2/lib/active_support/core_ext/object/json.rb:34:in `encode': "\xC5" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)
	activesupport-5.0.7.2/lib/active_support/core_ext/object/json.rb:34:in `to_json'
	activesupport-5.0.7.2/lib/active_support/json/encoding.rb:55:in `to_json'
	json-1.8.6/lib/json/common.rb:223:in `generate'
	activesupport-5.0.7.2/lib/active_support/json/encoding.rb:99:in `stringify'
	activesupport-5.0.7.2/lib/active_support/json/encoding.rb:33:in `encode'
	activesupport-5.0.7.2/lib/active_support/json/encoding.rb:20:in `encode'
	activesupport-5.0.7.2/lib/active_support/core_ext/object/json.rb:37:in `to_json'
	perimeterx-ruby-sdk-10c2dbd6e75d/lib/perimeterx/utils/px_http_client.rb:29:in `post'
	perimeterx-ruby-sdk-10c2dbd6e75d/lib/perimeterx/internal/validators/perimeter_x_s2s_validator.rb:73:in `send_risk_request'
	perimeterx-ruby-sdk-10c2dbd6e75d/lib/perimeterx/internal/validators/perimeter_x_s2s_validator.rb:80:in `verify'
	perimeterx-ruby-sdk-10c2dbd6e75d/lib/perimeter_x.rb:119:in `verify'
	perimeterx-ruby-sdk-10c2dbd6e75d/lib/perimeter_x.rb:18:in `px_verify_request'

[oehlschl]

FWIW, this seems like a similar issue: getsentry/sentry-ruby#689

There's no set encoding when reading the data from the Rack request, so it seems like a reasonable solution would be to force the encoding somewhere.