Merge pull request #2972 from blockstack/feat/hook-up-estimator

feat: Improved Fee Rate Computation

.github/workflows/bitcoin-tests.yml

@@ -54,6 +54,8 @@ jobs:
           - tests::neon_integrations::filter_low_fee_tx_integration_test
           - tests::neon_integrations::filter_long_runtime_tx_integration_test
           - tests::neon_integrations::mining_transactions_is_fair
+          - tests::neon_integrations::fuzzed_median_fee_rate_estimation_test_window5
+          - tests::neon_integrations::fuzzed_median_fee_rate_estimation_test_window10
           - tests::neon_integrations::use_latest_tip_integration_test
           - tests::epoch_205::test_dynamic_db_method_costs
           - tests::epoch_205::transition_empty_blocks

CHANGELOG.md

@@ -5,7 +5,17 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to the versioning scheme outlined in the [README.md](README.md).
 
-## [Unreleased]
+## [Upcoming]
+
+### Added
+
+- A new fee estimator intended to produce fewer over-estimates, by having less
+  sensitivity to outliers. Its characteristic features are: 1) use a window to
+  forget past estimates instead of exponential averaging, 2) use weighted
+  percentiles, so that bigger transactions influence the estimates more, 3)
+  assess empty space in blocks as having paid the "minimum fee", so that empty
+  space is accounted for, 4) use random "fuzz" so that in busy times we will
+  not have ties.
 
 ### Changed
 

README.md

@@ -361,7 +361,9 @@ Fee and cost estimators can be configure via the config section `[fee_estimation
 ```
 [fee_estimation]
 cost_estimator = naive_pessimistic
-fee_estimator = scalar_fee_rate
+fee_estimator = fuzzed_weighted_median_fee_rate
+fee_rate_fuzzer_fraction = 0.1
+fee_rate_window_size = 5
 cost_metric = proportion_dot_product
 log_error = true
 enabled = true
@@ -378,6 +380,11 @@ are **not** consensus-critical components, but rather can be used by miners to
 rank transactions in the mempool or client to determine appropriate fee rates
 for transactions before broadcasting them.
 
+The `fuzzed_weighted_median_fee_rate` uses a
+median estimate from a window of the fees paid in the last `fee_rate_window_size` blocks.
+Estimates are then randomly "fuzzed" using uniform random fuzz of size up to
+`fee_rate_fuzzer_fraction` of the base estimate.
+
 ## Non-Consensus Breaking Release Process
 
 For non-consensus breaking releases, this project uses the following release process: