From 3cdb63336ca769883c9589320edf8fe3f21e1b26 Mon Sep 17 00:00:00 2001 From: Mike Patrick Date: Wed, 15 Nov 2023 14:35:14 +0000 Subject: [PATCH 1/3] Enable adding custom LogStasher fields from apps I.e. allow apps to add their own custom fields in addition to those that `GovukJsonLogging` already adds. It looks like `LogStasher.add_custom_fields` can only be called one time, otherwise subsequent calls overwrite previous ones. We found this happening in the wild in the Content Store app, where `govuk_request_id` et al were missing from the Rails framework logs but present whenever the logger had been invoked directly from our own code. I am curious about the possibility of being able to add custom fields to both types of logs, together or separately, but the bug we're currently experiencing is just a conflict between two calls to `LogStasher.add_custom_fields`, so that's what I'm addressing. --- I've manually tested this change locally with Content Store, both with and without its own custom field config. --- CHANGELOG.md | 4 ++++ README.md | 14 ++++++++++++++ lib/govuk_app_config/govuk_json_logging.rb | 20 +++++++++++++++++++- 3 files changed, 37 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 874a820..0b28940 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,7 @@ +# Unreleased + +* Enable adding custom LogStasher fields from apps ([#327](https://github.com/alphagov/govuk_app_config/pull/327)) + # 9.6.0 * Allow YouTube thumbnails from https://i.ytimg.com in the global Content Security Policy ([#328](https://github.com/alphagov/govuk_app_config/pull/328)) diff --git a/README.md b/README.md index 6aaccab..6c4618d 100644 --- a/README.md +++ b/README.md @@ -149,6 +149,20 @@ allow JSON format logs and `Govuk-Request-Id` to be visible. For development logs, in order to see the production style logs, developers should set `GOVUK_RAILS_JSON_LOGGING`in `govuk-docker` -> `docker-compose` files. +### Logger configuration + +To include additional custom fields in your Rails logs, you can declare them +within a `GovukJsonLogging.configure` block in a `config/initializers/` file. + +Example of adding a key/value to log entries based on a request header: + +```ruby +GovukJsonLogging.configure do + add_custom_fields do |fields| + fields[:govuk_custom_field] = request.headers["GOVUK-Custom-Header"] + end +end +``` ## Content Security Policy generation diff --git a/lib/govuk_app_config/govuk_json_logging.rb b/lib/govuk_app_config/govuk_json_logging.rb index c52ecbd..79c35ce 100644 --- a/lib/govuk_app_config/govuk_json_logging.rb +++ b/lib/govuk_app_config/govuk_json_logging.rb @@ -3,7 +3,23 @@ require "action_controller" module GovukJsonLogging - def self.configure + class Configuration + def initialize + @custom_fields_block = proc {} + end + + attr_reader :custom_fields_block + + def add_custom_fields(&block) + @custom_fields_block = block if block_given? + end + end + + def self.configure(&block) + configuration = Configuration.new + + configuration.instance_eval(&block) if block_given? + # We disable buffering, so that logs aren't lost on crash or delayed # indefinitely while troubleshooting. $stdout.sync = true @@ -31,6 +47,8 @@ def self.configure fields[:govuk_request_id] = request.headers["GOVUK-Request-Id"] fields[:varnish_id] = request.headers["X-Varnish"] fields[:govuk_app_config] = GovukAppConfig::VERSION + + instance_exec(fields, &configuration.custom_fields_block) if block_given? end Rails.application.config.logstasher.enabled = true From 9faf031a1077e8d151264be613b09418d24c1486 Mon Sep 17 00:00:00 2001 From: Al Davidson Date: Fri, 24 Nov 2023 14:10:02 +0000 Subject: [PATCH 2/3] Test the adding of custom fields via LogStasher The way LogStasher hooks into Rails makes it difficult to test that configured custom fields actually get added. It was tricky to find an approach to worked, wasn't too awkward and wasn't too tightly coupled to LogStasher. _Notes on some of the esoteric details of these new tests_ `append_info_to_payload`: This is a Rails instrumentation hook (intended for use by third parties). It's run during request handling and LogStasher uses it to "append" its default fields and any custom fields defined by our app. The visible effect in the context of this method and its caller is that those LogStasher fields are set on the `payload` hash that was supplied to the method. So, invoking this method is enough to simulate a controller action running for LogStasher's purposes. And, if the payload hash contains our custom field key (and value, when there is one), we've defined it successfully. `logstasher_add_custom_fields_to_payload`: If a Rails app has defined custom LogStasher fields (with `LogStasher.add_custom_fields`), LogStasher registers a listener with Rails that fires whenever a controller is loaded and that listener adds this method to the loaded controller. Therefore, if a newly-loaded controller has this method, one or more LogStasher custom fields have been defined successfully. --- spec/lib/govuk_json_logging_spec.rb | 65 +++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/spec/lib/govuk_json_logging_spec.rb b/spec/lib/govuk_json_logging_spec.rb index d25d18a..ac8da16 100644 --- a/spec/lib/govuk_json_logging_spec.rb +++ b/spec/lib/govuk_json_logging_spec.rb @@ -3,6 +3,11 @@ require "govuk_app_config/govuk_json_logging" require "rack/test" +class TestController < ActionController::Base + include LogStasher::ActionController::Instrumentation + def index; end +end + RSpec.describe GovukJsonLogging do let(:govuk_headers_class) do Class.new do @@ -78,6 +83,66 @@ def self.headers expect(fake_stdout.read).to match(/test default log entry/) end + context "given a block" do + it "evals the block" do + done = false + expect { + GovukJsonLogging.configure do + done = true + end + }.to change { done }.to(true) + end + + context "and the block configures custom fields" do + describe "any subsequently-created ActionController" do + let(:headers) { { "REMOTE_ADDR" => "10.10.10.10" } } + let(:mock_request) { ActionDispatch::TestRequest.new(Rack::MockRequest.env_for("http://example.com:8080/", headers)) } + let(:mock_response) { ActionDispatch::TestResponse.new } + + before do + GovukJsonLogging.configure do + add_custom_fields do |fields| + fields[:govuk_custom_field] = request.headers["GOVUK-Custom-Header"] + end + end + + @controller = TestController.new + allow(@controller).to receive(:request).and_return(mock_request) + allow(@controller).to receive(:response).and_return(mock_response) + end + + it "has a logstasher_add_custom_fields_to_payload method" do + expect(@controller.methods).to include(:logstasher_add_custom_fields_to_payload) + end + + describe "calling the logstasher_add_custom_fields_to_payload" do + let(:payload) { {} } + + it "executes the block" do + expect(@controller).to receive(:logstasher_add_custom_fields_to_payload) + @controller.send(:append_info_to_payload, payload) + end + + it "adds the custom fields to the payload" do + @controller.send(:append_info_to_payload, payload) + expect(payload.keys).to include(:govuk_custom_field) + end + + context "when the custom field has a value" do + before do + mock_request.headers["GOVUK-Custom-header"] = "My header value" + end + + it "sets the custom field value in the payload" do + @controller.send(:append_info_to_payload, payload) + expect(payload[:govuk_custom_field]).to eq("My header value") + end + end + end + end + end + end + describe "when making requests to the application" do include Rack::Test::Methods From 2305f461f2f4de7654287e09bad0427978d411d6 Mon Sep 17 00:00:00 2001 From: Mike Patrick Date: Mon, 27 Nov 2023 12:00:35 +0000 Subject: [PATCH 3/3] Release version 9.7.0 --- CHANGELOG.md | 2 +- lib/govuk_app_config/version.rb | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0b28940..7b7da96 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,4 +1,4 @@ -# Unreleased +# 9.7.0 * Enable adding custom LogStasher fields from apps ([#327](https://github.com/alphagov/govuk_app_config/pull/327)) diff --git a/lib/govuk_app_config/version.rb b/lib/govuk_app_config/version.rb index 06ab345..c2f94f5 100644 --- a/lib/govuk_app_config/version.rb +++ b/lib/govuk_app_config/version.rb @@ -1,3 +1,3 @@ module GovukAppConfig - VERSION = "9.6.0".freeze + VERSION = "9.7.0".freeze end