From 94f07d5d658128449ad034b71e1a7881f181ed8b Mon Sep 17 00:00:00 2001 From: Michal Cichra Date: Thu, 9 Mar 2017 13:23:51 +0100 Subject: [PATCH 1/8] [nginx] allow more timers and sockets [nginx] increase number of running timers [nginx] increase default pool size for sockets --- CHANGELOG.md | 1 + apicast/conf/nginx.conf | 2 ++ 2 files changed, 3 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7b8baa7e8..6c5cbef59 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -33,6 +33,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/). - Use per request configuration when cache is disabled [PR #289](https://github.com/3scale/apicast/pull/289) - Automatically expose all environment variables starting with `APICAST_` or `THREESCALE_` to nginx [PR #292](https://github.com/3scale/apicast/pull/292) - Error log to show why downloading configuration failed [PR #306](https://github.com/3scale/apicast/pull/306) +- Increased number of background timers and connections in the cosocket pool [PR #290](https://github.com/3scale/apicast/pull/290) ### Added diff --git a/apicast/conf/nginx.conf b/apicast/conf/nginx.conf index 3f7469f6d..7f60c19db 100644 --- a/apicast/conf/nginx.conf +++ b/apicast/conf/nginx.conf @@ -30,6 +30,8 @@ http { # Enabling the Lua code cache is strongly encouraged for production use # Disabling it should only be done for testing and development purposes lua_code_cache on; + lua_max_running_timers 2048; + lua_socket_pool_size 512; include ../http.d/*.conf; From 4e033f1ec5083b512f8ec9ec99262c21877f82aa Mon Sep 17 00:00:00 2001 From: Michal Cichra Date: Tue, 14 Mar 2017 09:12:17 +0100 Subject: [PATCH 2/8] [luacheck] configure to use ngx stdlib --- .luacheckrc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.luacheckrc b/.luacheckrc index 971ad27d5..6749c8ead 100644 --- a/.luacheckrc +++ b/.luacheckrc @@ -1,5 +1,5 @@ -std = 'min' +std = 'ngx_lua' files["spec"] = {std = "+busted"} -globals = { 'ngx', 'unpack', 'rawlen' } +globals = { 'rawlen' } From 9c1ce6210238896c9fed20cab9db46a060f987f5 Mon Sep 17 00:00:00 2001 From: Michal Cichra Date: Thu, 9 Mar 2017 13:03:38 +0100 Subject: [PATCH 3/8] [management] /status/info endpoint that returns information about timers --- CHANGELOG.md | 2 ++ apicast/src/management.lua | 15 +++++++++++++++ doc/management-api.md | 10 ++++++++++ t/001-management.t | 16 ++++++++++++++++ 4 files changed, 43 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6c5cbef59..eb2753fec 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -42,6 +42,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/). - Ability to lock service to specific configuration version [PR #293](https://github.com/3scale/apicast/pull/292) - Ability to use Redis DB and password via `REDIS_URL` [PR #303](https://github.com/3scale/apicast/pull/303) - Ability to Authenticate against API using RHSSO and OpenID Connect [PR #283](https://github.com/3scale/apicast/pull/283) +- Experimental option for true out of band reporting (`APICAST_REPORTING_WORKERS`) [PR #290](https://github.com/3scale/apicast/pull/290) +- `/status/info` endpoint to the Management API [PR #290](https://github.com/3scale/apicast/pull/290) ### Fixed - `http_ng` client supports auth passsed in the url, and default client options if the request options are missing for methods with body (POST, PUT, etc.) [PR #310](https://github.com/3scale/apicast/pull/310) diff --git a/apicast/src/management.lua b/apicast/src/management.lua index 13ec8c5be..7e2dda499 100644 --- a/apicast/src/management.lua +++ b/apicast/src/management.lua @@ -113,6 +113,20 @@ function _M.disabled() ngx.exit(ngx.HTTP_FORBIDDEN) end +function _M.info() + return json_response({ + timers = { + pending = ngx.timer.pending_count(), + running = ngx.timer.running_count() + }, + worker = { + exiting = ngx.worker.exiting(), + count = ngx.worker.count(), + id = ngx.worker.id() + } + }) +end + local routes = {} function routes.disabled(r) @@ -120,6 +134,7 @@ function routes.disabled(r) end function routes.status(r) + r:get('/status/info', _M.info) r:get('/status/ready', _M.ready) r:get('/status/live', _M.live) end diff --git a/doc/management-api.md b/doc/management-api.md index 1abd6fbaf..b1f8d24e3 100644 --- a/doc/management-api.md +++ b/doc/management-api.md @@ -82,3 +82,13 @@ Available endpoints: ```json { "status": "live", "success": true } ``` + +- `GET /status/info` + Returns internal information about timers and workers. + + ```json + { + "timers": { "pending": 0, "running": 0 }, + "worker": { "count":1, "exiting": false, "id": 0 } + } + ``` diff --git a/t/001-management.t b/t/001-management.t index 7e4e99e6e..7fcaa7fb0 100644 --- a/t/001-management.t +++ b/t/001-management.t @@ -349,3 +349,19 @@ POST /config --- error_code: 406 --- no_error_log [error] + +=== TEST 16: status information +Has information about timers and time. +--- main_config +env APICAST_MANAGEMENT_API=status; +--- http_config + lua_package_path "$TEST_NGINX_LUA_PATH"; +--- config +include $TEST_NGINX_MANAGEMENT_CONFIG; +--- request +GET /status/info +--- response_body +{"timers":{"pending":0,"running":0},"worker":{"count":1,"exiting":false,"id":0}} +--- error_code: 200 +--- no_error_log +[error] From db6cec985ada4b5a80482eb75b700d148000116b Mon Sep 17 00:00:00 2001 From: Michal Cichra Date: Thu, 9 Mar 2017 11:53:30 +0100 Subject: [PATCH 4/8] [proxy] optionally drop synchronous post_action doing synchronous call in the post_action will push the latency to the client on the same connection using timers we can make that fully asynchronous not being tied to the request phase at all setting APICAST_REPORTING_THREADS environment variable to a number greater than 0 will enable reporting in asynchronous timers instead of synchronous call the number defines a maximum number of parallel calls to backend per worker if there is new report coming but no connections is available APIcast will wait for up to 10 seconds before falling back to the synchronous call please note this is **experimental** feature --- CHANGELOG.md | 1 + apicast/conf.d/apicast.conf | 7 +- apicast/conf/nginx.conf | 2 + apicast/src/backend_client.lua | 10 +- apicast/src/proxy.lua | 106 ++++++++++++---- doc/parameters.md | 11 ++ openshift/apicast-template.yml | 6 + t/003-apicast.t | 2 + t/014-apicast-async-reporting.t | 218 ++++++++++++++++++++++++++++++++ 9 files changed, 334 insertions(+), 29 deletions(-) create mode 100644 t/014-apicast-async-reporting.t diff --git a/CHANGELOG.md b/CHANGELOG.md index eb2753fec..dff007120 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -43,6 +43,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/). - Ability to use Redis DB and password via `REDIS_URL` [PR #303](https://github.com/3scale/apicast/pull/303) - Ability to Authenticate against API using RHSSO and OpenID Connect [PR #283](https://github.com/3scale/apicast/pull/283) - Experimental option for true out of band reporting (`APICAST_REPORTING_WORKERS`) [PR #290](https://github.com/3scale/apicast/pull/290) +- Experimental option for true out of band reporting (`APICAST_REPORTING_THREADS`) [PR #290](https://github.com/3scale/apicast/pull/290) - `/status/info` endpoint to the Management API [PR #290](https://github.com/3scale/apicast/pull/290) ### Fixed diff --git a/apicast/conf.d/apicast.conf b/apicast/conf.d/apicast.conf index 9ad9e8274..7c87d582a 100644 --- a/apicast/conf.d/apicast.conf +++ b/apicast/conf.d/apicast.conf @@ -16,6 +16,7 @@ location = /threescale_authrep { local log = ngx.var.arg_log; if log then return '&' .. ngx.unescape_uri(log) end } + set_by_lua_block $backend_endpoint { return ngx.ctx.backend_endpoint } set $path /transactions/authrep.xml?$backend_authentication_type=$backend_authentication_value&service_id=$service_id&$usage&$credentials$log; proxy_pass_request_headers off; proxy_http_version 1.1; @@ -31,7 +32,6 @@ location = /threescale_authrep { } } - location @out_of_band_authrep_action { internal; @@ -43,11 +43,13 @@ location @out_of_band_authrep_action { log_by_lua_block { ngx.var.post_action_impact = ngx.var.request_time - ngx.var.original_request_time - ngx.log(ngx.INFO, '[authrep] ', ngx.var.request_uri, ' ', ngx.var.status) require('module'):log() } } +set $backend_endpoint 'http://127.0.0.1:8081'; + + location / { set $cached_key null; set $credentials null; @@ -62,7 +64,6 @@ location / { set $redirect_url null; set $backend_host 'backend'; - set $backend_endpoint 'http://127.0.0.1:8081'; set $backend_authentication_type null; set $backend_authentication_value null; set $version null; diff --git a/apicast/conf/nginx.conf b/apicast/conf/nginx.conf index 7f60c19db..66d48520e 100644 --- a/apicast/conf/nginx.conf +++ b/apicast/conf/nginx.conf @@ -8,6 +8,8 @@ env OPENSSL_VERIFY; include ../main.d/*.conf; +env APICAST_REPORTING_THREADS; + error_log /dev/null emerg; events { diff --git a/apicast/src/backend_client.lua b/apicast/src/backend_client.lua index cf47825ee..26ca575b9 100644 --- a/apicast/src/backend_client.lua +++ b/apicast/src/backend_client.lua @@ -11,6 +11,8 @@ local setmetatable = setmetatable local concat = table.concat +local insert = table.insert +local len = string.len local http_ng = require('resty.http_ng') local user_agent = require('user_agent') @@ -84,11 +86,15 @@ local function call_backend_transaction(self, path, ...) local args = { self.authentication, ... } + local query = {} for i=1, #args do - args[i] = ngx.encode_args(args[i]) + local arg = ngx.encode_args(args[i]) + if len(arg) > 0 then + insert(query, arg) + end end - local url = resty_url.join(endpoint, '/transactions/', path, '?', concat(args, '&')) + local url = resty_url.join(endpoint, '/transactions/', path, '?', concat(query, '&')) local res = http_client.get(url) diff --git a/apicast/src/proxy.lua b/apicast/src/proxy.lua index 15e9f0f62..c4cf51c6c 100644 --- a/apicast/src/proxy.lua +++ b/apicast/src/proxy.lua @@ -9,6 +9,7 @@ local env = require 'resty.env' local custom_config = env.get('APICAST_CUSTOM_CONFIG') local configuration_store = require 'configuration_store' +local util = require('util') local oauth = require 'oauth' local resty_url = require 'resty.url' @@ -22,13 +23,22 @@ local concat = table.concat local gsub = string.gsub local tonumber = tonumber local setmetatable = setmetatable -local exit = ngx.exit local encode_args = ngx.encode_args local resty_resolver = require 'resty.resolver' +local semaphore = require('ngx.semaphore') +local backend_client = require('backend_client') +local timers = semaphore.new(tonumber(env.get('APICAST_REPORTING_THREADS') or 0)) + local empty = {} local response_codes = env.enabled('APICAST_RESPONSE_CODES') +local using_post_action = response_codes or timers:count() < 1 + +if not using_post_action then + ngx.log(ngx.WARN, 'using experimental asynchronous reporting threads: ', timers:count()) +end + local _M = { } local mt = { @@ -143,7 +153,7 @@ local http = { local backend_upstream = ngx.ctx.backend_upstream local previous_real_url = ngx.var.real_url ngx.log(ngx.DEBUG, '[ctx] copying backend_upstream of size: ', #backend_upstream) - local res = ngx.location.capture(assert(url), { share_all_vars = true, ctx = { backend_upstream = backend_upstream } }) + local res = ngx.location.capture(assert(url), { share_all_vars = true, ctx = { backend_upstream = backend_upstream, backend_endpoint = ngx.var.backend_endpoint } }) local real_url = ngx.var.real_url @@ -191,6 +201,8 @@ function _M:authorize(service, usage, credentials) ngx.log(ngx.DEBUG, 'apicast cache hit key: ', cached_key) ngx.var.cached_key = cached_key else + + self:set_backend_upstream(service) ngx.log(ngx.INFO, 'apicast cache miss key: ', cached_key) local res = http.get(internal_location) @@ -208,6 +220,10 @@ function _M:authorize(service, usage, credentials) -- set cached_key to nil to avoid doing the authrep in post_action ngx.var.cached_key = nil end + + if not using_post_action then + self:post_action(true) + end end function _M:set_service(host) @@ -281,8 +297,6 @@ function _M:call(host) host = host or ngx.var.host local service = ngx.ctx.service or self:set_service(host) - self:set_backend_upstream(service) - if service.backend_version == 'oauth' then local o = oauth.new(self.configuration) local f, params = oauth.call(o, service) @@ -347,22 +361,77 @@ function _M:access(service) return self:authorize(service, usage, credentials) end - -local function response_codes_encoded_data() +local function response_codes_data() local params = {} if not response_codes then - return '' + return params end if response_codes then params["log[code]"] = ngx.var.status end - return ngx.escape_uri(ngx.encode_args(params)) + return params end -function _M:post_action() +local function response_codes_encoded_data() + return ngx.escape_uri(ngx.encode_args(response_codes_data())) +end + +local function handle_post_action_response(cached_key, res) + if res.ok == false or res.status ~= 200 then + local api_keys = ngx.shared.api_keys + + if api_keys then + ngx.log(ngx.NOTICE, 'apicast cache delete key: ', cached_key, ' cause status ', res.status) + api_keys:delete(cached_key) + else + ngx.log(ngx.ALERT, 'apicast cache error missing shared memory zone api_keys') + end + + ngx.log(ngx.ERR, 'http_client error: ', res.error, ' status: ', res.status) + end +end + +local function post_action(_, cached_key, backend, ...) + local res = util.timer('backend post_action', backend.authrep, backend, ...) + + if not using_post_action then + timers:post(1) + end + + handle_post_action_response(cached_key, res) +end + +local function capture_post_action(self, cached_key, service) + self:set_backend_upstream(service) + + local auth_uri = service.backend_version == 'oauth' and 'threescale_oauth_authrep' or 'threescale_authrep' + local res = http.get("/".. auth_uri .."?log=" .. response_codes_encoded_data()) + + handle_post_action_response(cached_key, res) +end + +local function timer_post_action(self, cached_key, service) + local backend = assert(backend_client:new(service), 'missing backend') + + local ok, err = timers:wait(10) + + if ok then + -- TODO: try to do this in different phase and use semaphore to limit number of background threads + -- TODO: Also it is possible to use sets in shared memory to enqueue work + ngx.timer.at(0, post_action, cached_key, backend, ngx.ctx.usage, ngx.ctx.credentials, response_codes_data()) + else + ngx.log(ngx.ERR, 'failed to acquire timer: ', err) + return capture_post_action(self, cached_key, service) + end +end + +function _M:post_action(force) + if not using_post_action and not force then + return nil, 'post action not needed' + end local cached_key = ngx.var.cached_key @@ -371,26 +440,15 @@ function _M:post_action() local service_id = ngx.var.service_id local service = ngx.ctx.service or self.configuration:find_by_id(service_id) - self:set_backend_upstream(service) - - local auth_uri = service.backend_version == 'oauth' and 'threescale_oauth_authrep' or 'threescale_authrep' - local res = http.get("/".. auth_uri .."?log=" .. response_codes_encoded_data()) - if res.status ~= 200 then - local api_keys = ngx.shared.api_keys - - if api_keys then - ngx.log(ngx.NOTICE, 'apicast cache delete key: ', cached_key, ' cause status ', res.status) - api_keys:delete(cached_key) - else - ngx.log(ngx.ALERT, 'apicast cache error missing shared memory zone api_keys') - end + if using_post_action then + capture_post_action(self, cached_key, service) + else + timer_post_action(self, cached_key, service) end else ngx.log(ngx.INFO, '[async] skipping after action, no cached key') end - - exit(ngx.HTTP_OK) end if custom_config then diff --git a/doc/parameters.md b/doc/parameters.md index e8283afa4..3a04fe9a4 100644 --- a/doc/parameters.md +++ b/doc/parameters.md @@ -156,3 +156,14 @@ It requires custom certificate bundle and adding it to trusted certificates. It is recommended to use https://github.com/openresty/lua-nginx-module#lua_ssl_trusted_certificate and point to to certificate bundle generated by [export-builtin-trusted-certs](https://github.com/openresty/openresty-devel-utils/blob/master/export-builtin-trusted-certs). + +### `APICAST_REPORTING_THREADS` + +**Default**: 0 +**Value:**: integer >= 0 + +Value greater than 0 is going to enable out-of-band reporting to backend. +This is new **experimental** feature for increasing performance. Client +Won't see the backend latency and everything will be processed asynchronously. +This value determines how many asynchronous reports can be running simultainesly +before client starts being throttled by adding latency. diff --git a/openshift/apicast-template.yml b/openshift/apicast-template.yml index df9df45c1..63f415c93 100644 --- a/openshift/apicast-template.yml +++ b/openshift/apicast-template.yml @@ -58,6 +58,8 @@ objects: value: "${MANAGEMENT_API}" - name: OPENSSL_VERIFY value: "${OPENSSL_VERIFY}" + - name: APICAST_REPORTING_THREADS + value: "${REPORTING_THREADS}" image: "${IMAGE_NAME}" imagePullPolicy: Always name: "${APICAST_NAME}" @@ -162,3 +164,7 @@ parameters: description: "Turn on/off the OpenSSL peer verification. Can be set to true/false." required: true value: "false" +- name: REPORTING_THREADS + description: "Number of asynchronous reporting threads. Experimental feature." + required: false + value: "0" diff --git a/t/003-apicast.t b/t/003-apicast.t index 9cae03adc..1b3821e76 100644 --- a/t/003-apicast.t +++ b/t/003-apicast.t @@ -469,6 +469,7 @@ env RESOLVER=127.0.0.1:1953; --- http_config include $TEST_NGINX_UPSTREAM_CONFIG; lua_package_path "$TEST_NGINX_LUA_PATH"; + lua_shared_dict api_keys 1m; init_by_lua_block { require('configuration_loader').mock({ services = { @@ -502,6 +503,7 @@ env RESOLVER=127.0.0.1:1953; if ngx.var.host == 'localhost.example.com' then ngx.exit(200) else + ngx.log(ngx.ERR, 'invalid host: ', ngx.var.host) ngx.exit(404) end } diff --git a/t/014-apicast-async-reporting.t b/t/014-apicast-async-reporting.t new file mode 100644 index 000000000..e96a8d741 --- /dev/null +++ b/t/014-apicast-async-reporting.t @@ -0,0 +1,218 @@ +use Test::Nginx::Socket::Lua 'no_plan'; +use Cwd qw(cwd); + +my $pwd = cwd(); +my $apicast = $ENV{TEST_NGINX_APICAST_PATH} || "$pwd/apicast"; + +$ENV{TEST_NGINX_LUA_PATH} = "$apicast/src/?.lua;;"; +$ENV{TEST_NGINX_UPSTREAM_CONFIG} = "$apicast/http.d/upstream.conf"; +$ENV{TEST_NGINX_BACKEND_CONFIG} = "$apicast/conf.d/backend.conf"; +$ENV{TEST_NGINX_APICAST_CONFIG} = "$apicast/conf.d/apicast.conf"; + +$ENV{APICAST_REPORTING_THREADS} = '1'; + +require("t/dns.pl"); + +env_to_nginx('APICAST_REPORTING_THREADS'); + +log_level('debug'); +repeat_each(2); +no_root_location(); +run_tests(); + +__DATA__ + +=== TEST 1: api backend gets the request +It asks backend and then forwards the request to the api. +--- http_config +include $TEST_NGINX_UPSTREAM_CONFIG; +lua_shared_dict api_keys 1m; +lua_package_path "$TEST_NGINX_LUA_PATH"; +init_by_lua_block { + ngx.shared.api_keys:set('42:value:usage%5Bhits%5D=2', 200) + require('configuration_loader').mock({ + services = { + { + id = 42, + backend_version = 1, + backend_authentication_type = 'service_token', + backend_authentication_value = 'token-value', + proxy = { + backend = { endpoint = "http://127.0.0.1:$TEST_NGINX_SERVER_PORT" }, + api_backend = "http://127.0.0.1:$TEST_NGINX_SERVER_PORT/api-backend/", + proxy_rules = { + { pattern = '/', http_method = 'GET', metric_system_name = 'hits', delta = 2 } + } + } + } + } + }) +} +--- config +include $TEST_NGINX_APICAST_CONFIG; + +location /transactions/authrep.xml { + content_by_lua_block { + local expected = "service_id=42&service_token=token-value&usage%5Bhits%5D=2&user_key=value" + local args = ngx.var.args + ngx.log(ngx.INFO, 'backend got ', args, ' expected ', expected) + if args == expected then + ngx.exit(200) + else + ngx.log(ngx.ERR, expected, ' did not match: ', args) + ngx.exit(403) + end + } +} + +location /api-backend/ { + echo 'yay, api backend: $http_host'; +} +--- request +GET /?user_key=value +--- response_body +yay, api backend: 127.0.0.1 +--- error_code: 200 +--- error_log +backend got service_id=42&service_token=token-value&usage%5Bhits%5D=2&user_key=value +--- no_error_log +[error] + +=== TEST 2: https api backend works +with async background reporting + +--- http_config +include $TEST_NGINX_UPSTREAM_CONFIG; +lua_package_path "$TEST_NGINX_LUA_PATH"; +init_by_lua_block { + ngx.shared.api_keys:set('42:foo:usage%5Bhits%5D=1', 200) + require('configuration_loader').mock({ + services = { + { + id = 42, + backend_version = 1, + backend_version = 1, + backend_authentication_type = 'service_token', + backend_authentication_value = 'token-value', + proxy = { + backend = { endpoint = "https://127.0.0.1:1953" }, + api_backend = "http://127.0.0.1:$TEST_NGINX_SERVER_PORT/api/", + proxy_rules = { + { pattern = '/', http_method = 'GET', metric_system_name = 'hits', delta = 1 } + } + } + } + } + }) +} +lua_shared_dict api_keys 1m; +--- config +include $TEST_NGINX_APICAST_CONFIG; +listen 1953 ssl; + +ssl_certificate ../html/server.crt; +ssl_certificate_key ../html/server.key; + +location /api/ { + echo "api response"; +} + +location /transactions/authrep.xml { + content_by_lua_block { + ngx.log(ngx.INFO, 'backend got: ', ngx.var.args) + ngx.exit(200) + } +} +--- user_files +>>> server.crt +-----BEGIN CERTIFICATE----- +MIIB0DCCAXegAwIBAgIJAISY+WDXX2w5MAoGCCqGSM49BAMCMEUxCzAJBgNVBAYT +AkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBXaWRn +aXRzIFB0eSBMdGQwHhcNMTYxMjIzMDg1MDExWhcNMjYxMjIxMDg1MDExWjBFMQsw +CQYDVQQGEwJBVTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50ZXJu +ZXQgV2lkZ2l0cyBQdHkgTHRkMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEhkmo +6Xp/9W9cGaoGFU7TaBFXOUkZxYbGXQfxyZZucIQPt89+4r1cbx0wVEzbYK5wRb7U +iWhvvvYDltIzsD75vqNQME4wHQYDVR0OBBYEFOBBS7ZF8Km2wGuLNoXFAcj0Tz1D +MB8GA1UdIwQYMBaAFOBBS7ZF8Km2wGuLNoXFAcj0Tz1DMAwGA1UdEwQFMAMBAf8w +CgYIKoZIzj0EAwIDRwAwRAIgZ54vooA5Eb91XmhsIBbp12u7cg1qYXNuSh8zih2g +QWUCIGTHhoBXUzsEbVh302fg7bfRKPCi/mcPfpFICwrmoooh +-----END CERTIFICATE----- +>>> server.key +-----BEGIN EC PARAMETERS----- +BggqhkjOPQMBBw== +-----END EC PARAMETERS----- +-----BEGIN EC PRIVATE KEY----- +MHcCAQEEIFCV3VwLEFKz9+yTR5vzonmLPYO/fUvZiMVU1Hb11nN8oAoGCCqGSM49 +AwEHoUQDQgAEhkmo6Xp/9W9cGaoGFU7TaBFXOUkZxYbGXQfxyZZucIQPt89+4r1c +bx0wVEzbYK5wRb7UiWhvvvYDltIzsD75vg== +-----END EC PRIVATE KEY----- +--- request +GET /test?user_key=foo +--- no_error_log +[error] +--- response_body +api response +--- error_code: 200 +--- error_log +backend client uri: https://127.0.0.1:1953/transactions/authrep.xml?service_id=42&service_token=token-value&usage%5Bhits%5D=1&user_key=foo ok: true status: 200 + +=== TEST 3: uses endpoint host as Host header +when connecting to the backend +--- main_config +env RESOLVER=127.0.0.1:1953; +--- http_config +include $TEST_NGINX_UPSTREAM_CONFIG; +lua_package_path "$TEST_NGINX_LUA_PATH"; +lua_shared_dict api_keys 1m; +init_by_lua_block { + ngx.shared.api_keys:set('42:val:usage%5Bhits%5D=2', 200) + require('configuration_loader').mock({ + services = { + { + id = 42, + backend_version = 1, + backend_authentication_type = 'service_token', + backend_authentication_value = 'service-token', + proxy = { + api_backend = "http://127.0.0.1:$TEST_NGINX_SERVER_PORT/api/", + backend = { + endpoint = 'http://localhost.example.com:$TEST_NGINX_SERVER_PORT' + }, + proxy_rules = { + { pattern = '/', http_method = 'GET', metric_system_name = 'hits', delta = 2 } + } + } + }, + } + }) +} +--- config +include $TEST_NGINX_APICAST_CONFIG; + +location /api/ { + echo "all ok"; +} + +location /transactions/authrep.xml { + content_by_lua_block { + if ngx.var.host == 'localhost.example.com' then + ngx.exit(200) + else + ngx.log(ngx.ERR, 'invalid host: ', ngx.var.host) + ngx.exit(404) + end + } +} + +--- request +GET /t?user_key=val +--- response_body +all ok +--- error_code: 200 +--- udp_listen: 1953 +--- udp_reply eval +$::dns->("localhost.example.com", "127.0.0.1") +--- no_error_log +[error] +--- error_log +backend client uri: http://localhost.example.com:1984/transactions/authrep.xml?service_id=42&service_token=service-token&usage%5Bhits%5D=2&user_key=val ok: true status: 200 From 8f9d2a0f44f1e1a343600d15856449e6140c4597 Mon Sep 17 00:00:00 2001 From: Michal Cichra Date: Thu, 30 Mar 2017 11:02:34 +0200 Subject: [PATCH 5/8] [t] update certificate to have proper subject so SSL validation will work $ openssl ecparam -genkey -name prime256v1 -out key.pem $ openssl req -new -sha256 -key key.pem -out csr.csr -subj '/CN=127.0.0.1' $ openssl req -x509 -sha256 -days 3650 -key key.pem -in csr.csr -out certificate.pem --- t/014-apicast-async-reporting.t | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/t/014-apicast-async-reporting.t b/t/014-apicast-async-reporting.t index e96a8d741..1217e2c89 100644 --- a/t/014-apicast-async-reporting.t +++ b/t/014-apicast-async-reporting.t @@ -113,6 +113,8 @@ listen 1953 ssl; ssl_certificate ../html/server.crt; ssl_certificate_key ../html/server.key; +lua_ssl_trusted_certificate ../html/server.crt; + location /api/ { echo "api response"; } @@ -126,25 +128,24 @@ location /transactions/authrep.xml { --- user_files >>> server.crt -----BEGIN CERTIFICATE----- -MIIB0DCCAXegAwIBAgIJAISY+WDXX2w5MAoGCCqGSM49BAMCMEUxCzAJBgNVBAYT -AkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBXaWRn -aXRzIFB0eSBMdGQwHhcNMTYxMjIzMDg1MDExWhcNMjYxMjIxMDg1MDExWjBFMQsw -CQYDVQQGEwJBVTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50ZXJu -ZXQgV2lkZ2l0cyBQdHkgTHRkMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEhkmo -6Xp/9W9cGaoGFU7TaBFXOUkZxYbGXQfxyZZucIQPt89+4r1cbx0wVEzbYK5wRb7U -iWhvvvYDltIzsD75vqNQME4wHQYDVR0OBBYEFOBBS7ZF8Km2wGuLNoXFAcj0Tz1D -MB8GA1UdIwQYMBaAFOBBS7ZF8Km2wGuLNoXFAcj0Tz1DMAwGA1UdEwQFMAMBAf8w -CgYIKoZIzj0EAwIDRwAwRAIgZ54vooA5Eb91XmhsIBbp12u7cg1qYXNuSh8zih2g -QWUCIGTHhoBXUzsEbVh302fg7bfRKPCi/mcPfpFICwrmoooh +MIIBkjCCATmgAwIBAgIJAOSlu+H4y+4uMAkGByqGSM49BAEwFDESMBAGA1UEAxMJ +MTI3LjAuMC4xMB4XDTE3MDMzMDA5MDEyMFoXDTI3MDMyODA5MDEyMFowFDESMBAG +A1UEAxMJMTI3LjAuMC4xMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEBmPcgjjx +OmODfbQGqjkVtbq6qvFC8t0A5FWnL3nRQjI5nB9k7tX7vTx1NzFzq+w3Vf3vX+Fq +sWLyaBIhDSyUHqN1MHMwHQYDVR0OBBYEFAIqtTXT/E0eFC29bQhicIZcM0tlMEQG +A1UdIwQ9MDuAFAIqtTXT/E0eFC29bQhicIZcM0tloRikFjAUMRIwEAYDVQQDEwkx +MjcuMC4wLjGCCQDkpbvh+MvuLjAMBgNVHRMEBTADAQH/MAkGByqGSM49BAEDSAAw +RQIgAWRI+63VAyJyJJFfLGPRNhdasQZXSvICnCm7w6C/RmACIQCjZvsLCah8h2Sa +TyxjtHpkHJAzpVuetPVADc/lNN4l/Q== -----END CERTIFICATE----- >>> server.key -----BEGIN EC PARAMETERS----- BggqhkjOPQMBBw== -----END EC PARAMETERS----- -----BEGIN EC PRIVATE KEY----- -MHcCAQEEIFCV3VwLEFKz9+yTR5vzonmLPYO/fUvZiMVU1Hb11nN8oAoGCCqGSM49 -AwEHoUQDQgAEhkmo6Xp/9W9cGaoGFU7TaBFXOUkZxYbGXQfxyZZucIQPt89+4r1c -bx0wVEzbYK5wRb7UiWhvvvYDltIzsD75vg== +MHcCAQEEIHIYTPgt2XlDTuL6Ly1jIqhlhM3lEspTyVldsaAoaC54oAoGCCqGSM49 +AwEHoUQDQgAEBmPcgjjxOmODfbQGqjkVtbq6qvFC8t0A5FWnL3nRQjI5nB9k7tX7 +vTx1NzFzq+w3Vf3vX+FqsWLyaBIhDSyUHg== -----END EC PRIVATE KEY----- --- request GET /test?user_key=foo From 9ba44dd1af064cab0fdcc7a453e54137c08560b6 Mon Sep 17 00:00:00 2001 From: Michal Cichra Date: Fri, 6 Oct 2017 10:01:52 +0200 Subject: [PATCH 6/8] wrap reading ngx.var in pcall rather than checking context because tests run in the same context --- apicast/src/backend/cache_handler.lua | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/apicast/src/backend/cache_handler.lua b/apicast/src/backend/cache_handler.lua index 6686230f5..b6d776b26 100644 --- a/apicast/src/backend/cache_handler.lua +++ b/apicast/src/backend/cache_handler.lua @@ -1,4 +1,5 @@ local setmetatable = setmetatable +local pcall = pcall local _M = { handlers = setmetatable({}, { __index = { default = 'strict' } }) @@ -23,12 +24,23 @@ function _M.new(handler) return setmetatable({ handler = name }, mt) end +local function cached_key_var() + return ngx.var.cached_key +end + +local function fetch_cached_key() + local ok, stored = pcall(cached_key_var) + + return ok and stored +end + function _M.handlers.strict(cache, cached_key, response, ttl) if response.status == 200 then -- cached_key is set in post_action and it is in in authorize -- so to not write the cache twice lets write it just in authorize - if ngx.get_phase() == 'timer' or ngx.var.cached_key ~= cached_key then - ngx.log(ngx.INFO, 'apicast cache write key: ', cached_key, ', ttl: ', ttl ) + + if fetch_cached_key(cached_key) ~= cached_key then + ngx.log(ngx.INFO, 'apicast cache write key: ', cached_key, ', ttl: ', ttl, ' sub: ') cache:set(cached_key, 200, ttl or 0) end From bac00ab3ce4caa8180a0d3b5ce604f09bfc3f972 Mon Sep 17 00:00:00 2001 From: Michal Cichra Date: Tue, 17 Oct 2017 11:27:24 +0200 Subject: [PATCH 7/8] update changelog --- CHANGELOG.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4e433b51c..9ff410783 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,11 +6,17 @@ and this project adheres to [Semantic Versioning](http://semver.org/). ## [Unreleased] +## Added + +- Experimental option for true out of band reporting (`APICAST_REPORTING_WORKERS`) [PR #290](https://github.com/3scale/apicast/pull/290) +- `/status/info` endpoint to the Management API [PR #290](https://github.com/3scale/apicast/pull/290) + ## Changed - Upgraded to OpenResty 1.11.2.5-1 [PR #428](https://github.com/3scale/apicast/pull/428) - `/oauth/token` endpoint returns an error status code, when the access token couldn't be stored in 3scale backend [PR #436](https://github.com/3scale/apicast/pull/436)] - URI params in POST requests are now taken into account when matching mapping rules [PR #437](https://github.com/3scale/apicast/pull/437) +- Increased number of background timers and connections in the cosocket pool [PR #290](https://github.com/3scale/apicast/pull/290) ### Fixed @@ -123,7 +129,6 @@ and this project adheres to [Semantic Versioning](http://semver.org/). - Use per request configuration when cache is disabled [PR #289](https://github.com/3scale/apicast/pull/289) - Automatically expose all environment variables starting with `APICAST_` or `THREESCALE_` to nginx [PR #292](https://github.com/3scale/apicast/pull/292) - Error log to show why downloading configuration failed [PR #306](https://github.com/3scale/apicast/pull/306) -- Increased number of background timers and connections in the cosocket pool [PR #290](https://github.com/3scale/apicast/pull/290) ### Added @@ -132,9 +137,6 @@ and this project adheres to [Semantic Versioning](http://semver.org/). - Ability to lock service to specific configuration version [PR #293](https://github.com/3scale/apicast/pull/292) - Ability to use Redis DB and password via `REDIS_URL` [PR #303](https://github.com/3scale/apicast/pull/303) - Ability to Authenticate against API using RHSSO and OpenID Connect [PR #283](https://github.com/3scale/apicast/pull/283) -- Experimental option for true out of band reporting (`APICAST_REPORTING_WORKERS`) [PR #290](https://github.com/3scale/apicast/pull/290) -- Experimental option for true out of band reporting (`APICAST_REPORTING_THREADS`) [PR #290](https://github.com/3scale/apicast/pull/290) -- `/status/info` endpoint to the Management API [PR #290](https://github.com/3scale/apicast/pull/290) ### Fixed - `http_ng` client supports auth passsed in the url, and default client options if the request options are missing for methods with body (POST, PUT, etc.) [PR #310](https://github.com/3scale/apicast/pull/310) From 155acf22c92161ebcf45728f2c61d7a1dcf7ddef Mon Sep 17 00:00:00 2001 From: Michal Cichra Date: Tue, 17 Oct 2017 13:37:31 +0200 Subject: [PATCH 8/8] wait for background op to finish --- t/014-apicast-async-reporting.t | 2 ++ 1 file changed, 2 insertions(+) diff --git a/t/014-apicast-async-reporting.t b/t/014-apicast-async-reporting.t index 8979fe090..b9dc4d4ac 100644 --- a/t/014-apicast-async-reporting.t +++ b/t/014-apicast-async-reporting.t @@ -156,6 +156,7 @@ api response --- error_code: 200 --- error_log backend client uri: https://127.0.0.1:1953/transactions/authrep.xml?service_token=token-value&service_id=42&usage%5Bhits%5D=1&user_key=foo ok: true status: 200 +--- wait: 1 === TEST 3: uses endpoint host as Host header when connecting to the backend @@ -217,3 +218,4 @@ $::dns->("localhost.example.com", "127.0.0.1") [error] --- error_log backend client uri: http://localhost.example.com:1984/transactions/authrep.xml?service_token=service-token&service_id=42&usage%5Bhits%5D=2&user_key=val ok: true status: 200 +--- wait: 1