Add channel scoring to get_route #1124

jkczyz · 2021-10-14T18:41:56Z

Failed payments may be retried, but calling get_route may return a Route with the same failing path. Add a routing::Score trait used to parameterize get_route, which it calls to determine how much a channel should be penalized in terms of msats willing to pay to avoid the channel.

Also, add a Scorer struct that implements routing::Score with a constant penalty. Subsequent changes will allow for more robust scoring by feeding back payment path success and failure to the scorer via event handling.

codecov · 2021-10-14T19:06:48Z

Codecov Report

Merging #1124 (84c0a48) into main (da498d7) will increase coverage by 0.76%.
The diff coverage is 91.52%.

❗ Current head 84c0a48 differs from pull request most recent head e15a18a. Consider uploading reports for the commit e15a18a to get more accurate results

@@            Coverage Diff             @@
##             main    #1124      +/-   ##
==========================================
+ Coverage   90.60%   91.36%   +0.76%     
==========================================
  Files          66       67       +1     
  Lines       34474    37698    +3224     
==========================================
+ Hits        31235    34443    +3208     
- Misses       3239     3255      +16

Impacted Files	Coverage Δ
lightning/src/routing/scorer.rs	`57.14% <57.14%> (ø)`
lightning/src/routing/router.rs	`95.70% <91.78%> (-0.30%)`	⬇️
lightning-invoice/src/utils.rs	`84.26% <100.00%> (+0.17%)`	⬆️
lightning/src/ln/channelmanager.rs	`85.14% <100.00%> (+0.01%)`	⬆️
lightning/src/ln/functional_test_utils.rs	`97.32% <100.00%> (+2.24%)`	⬆️
lightning/src/ln/functional_tests.rs	`97.93% <100.00%> (+0.53%)`	⬆️
lightning/src/ln/shutdown_tests.rs	`95.87% <100.00%> (+<0.01%)`	⬆️
lightning/src/chain/mod.rs	`50.00% <0.00%> (-8.83%)`	⬇️
lightning-background-processor/src/lib.rs	`94.38% <0.00%> (+0.15%)`	⬆️
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update da498d7...e15a18a. Read the comment docs.

TheBlueMatt

Benchmarks are probably about what I'd expect given the expansion of of in-cmp logic. Something to look into eventually, probably:

Latest git

test ln::channelmanager::bench::bench_sends               ... bench:   9,128,177 ns/iter (+/- 1,349,843)
test routing::network_graph::benches::read_network_graph  ... bench: 1,777,566,078 ns/iter (+/- 23,897,528)
test routing::network_graph::benches::write_network_graph ... bench: 159,355,206 ns/iter (+/- 5,292,557)
test routing::router::benches::generate_mpp_routes        ... bench:  74,852,606 ns/iter (+/- 55,662,508)
test routing::router::benches::generate_routes            ... bench:  72,321,963 ns/iter (+/- 56,925,293)
test bench::bench_sends ... bench: 211,670,987 ns/iter (+/- 188,695,358)

This PR:

test ln::channelmanager::bench::bench_sends               ... bench:   8,284,024 ns/iter (+/- 443,276)
test routing::network_graph::benches::read_network_graph  ... bench: 1,614,888,492 ns/iter (+/- 2,280,811)
test routing::network_graph::benches::write_network_graph ... bench: 143,246,612 ns/iter (+/- 458,059)
test routing::router::benches::generate_mpp_routes        ... bench:  96,742,965 ns/iter (+/- 52,848,627)
test routing::router::benches::generate_routes            ... bench:  96,963,520 ns/iter (+/- 62,975,254)
test bench::bench_sends ... bench: 190,069,734 ns/iter (+/- 168,900,474)

lightning/src/routing/router.rs

lightning/src/routing/mod.rs

TheBlueMatt · 2021-10-14T19:24:23Z

Note for comparison bench for 0.0.101 was (there's obviously a ton of noise in these numbers from github CI).

test ln::channelmanager::bench::bench_sends               ... bench:   7,587,098 ns/iter (+/- 1,026,747)
test routing::network_graph::benches::read_network_graph  ... bench: 2,195,464,293 ns/iter (+/- 116,402,530)
test routing::network_graph::benches::write_network_graph ... bench: 132,630,621 ns/iter (+/- 11,344,574)
test routing::router::benches::generate_mpp_routes        ... bench:  82,229,661 ns/iter (+/- 48,137,908)
test routing::router::benches::generate_routes            ... bench:  83,419,028 ns/iter (+/- 62,531,148)
test bench::bench_sends ... bench: 175,605,757 ns/iter (+/- 148,418,861)

lightning/src/routing/router.rs

jkczyz · 2021-10-14T20:54:21Z

All good feedback! Thanks for divining into benchmarks. All comments addressed.

TheBlueMatt · 2021-10-14T23:51:07Z

Github CI now says (which is, uhmmmm, faster than latest git? yea, no, that's just cause github's CI machines are clearly very heterogeneous....last time I look at that output.........)

test ln::channelmanager::bench::bench_sends               ... bench:   8,363,865 ns/iter (+/- 1,362,103)
test routing::network_graph::benches::read_network_graph  ... bench: 1,626,468,025 ns/iter (+/- 90,985,613)
test routing::network_graph::benches::write_network_graph ... bench: 142,291,900 ns/iter (+/- 6,987,924)
test routing::router::benches::generate_mpp_routes        ... bench:  57,154,598 ns/iter (+/- 32,542,178)
test routing::router::benches::generate_routes            ... bench:  57,344,572 ns/iter (+/- 39,866,175)
test bench::bench_sends ... bench: 197,672,739 ns/iter (+/- 170,050,060)

TheBlueMatt

Needs tests but otherwise good.

TheBlueMatt · 2021-10-15T00:05:52Z

lightning/src/routing/router.rs

@@ -163,12 +164,19 @@ struct RouteGraphNode {
 	/// The effective htlc_minimum_msat at this hop. If a later hop on the path had a higher HTLC
 	/// minimum, we use it, plus the fees required at each earlier hop to meet it.
 	path_htlc_minimum_msat: u64,
+	/// All penalties incurred *after* this hop on the way to the destination, as calculated using
+	/// channel scoring.
+	next_hops_penalty_msat: u64,


Use of next_hops in these confused me somewhat - next_hops_fee_msat (both in a PathBuildingHop and in the add_node macro) refers to only fee after the hop but not including the fee charged at the hop, whereas here we're using it to include the penalty charged at a hop. Maybe path_penalty_msat or so?

Ah, right. I was a little confused by the naming, but now I see the pattern. Should be consistent now.

TheBlueMatt · 2021-10-15T00:08:28Z

lightning/src/ln/functional_tests.rs

@@ -25,6 +25,7 @@ use ln::{chan_utils, onion_utils};
 use ln::chan_utils::HTLC_SUCCESS_TX_WEIGHT;
 use routing::network_graph::{NetworkUpdate, RoutingFees};
 use routing::router::{Route, RouteHop, RouteHint, RouteHintHop, get_route, get_keysend_route};
+use routing::scorer::DefaultScorer;


Can we use either a TestScorer in tests or a DefaultScorer with a score of 0? I find it a bit...awkward? to change all of our routing parameters in all our tests, even if they still pass it kinda makes me nervous.

Sure thing. I removed TestScorer and renamed DefaultScorer to Scorer, using it with 0 in all tests excepted for the newly added one.

TheBlueMatt · 2021-10-15T00:09:40Z

lightning/src/routing/scorer.rs

+impl Default for DefaultScorer {
+	/// Creates a new scorer using 100 msat as the channel penalty.
+	fn default() -> Self {
+		DefaultScorer::new(100)


nit: I think default should be 500, just kinda gut-feel. Lots of "good" nodes charge 1 sat per payment, and we should probably be willing to prefer that over two-three "free" hops IMO. Maybe even 1k. We can tune this later, but 100 just seems very low.

Changed to 500 msats.

jkczyz

I added a test but was getting a little confused with the fee_msat assertions. Let me know if it looks sane. I could add more tests if needed.

jkczyz · 2021-10-15T06:31:34Z

lightning/src/routing/router.rs

@@ -163,12 +164,19 @@ struct RouteGraphNode {
 	/// The effective htlc_minimum_msat at this hop. If a later hop on the path had a higher HTLC
 	/// minimum, we use it, plus the fees required at each earlier hop to meet it.
 	path_htlc_minimum_msat: u64,
+	/// All penalties incurred *after* this hop on the way to the destination, as calculated using
+	/// channel scoring.
+	next_hops_penalty_msat: u64,


Ah, right. I was a little confused by the naming, but now I see the pattern. Should be consistent now.

jkczyz · 2021-10-15T06:32:59Z

lightning/src/ln/functional_tests.rs

@@ -25,6 +25,7 @@ use ln::{chan_utils, onion_utils};
 use ln::chan_utils::HTLC_SUCCESS_TX_WEIGHT;
 use routing::network_graph::{NetworkUpdate, RoutingFees};
 use routing::router::{Route, RouteHop, RouteHint, RouteHintHop, get_route, get_keysend_route};
+use routing::scorer::DefaultScorer;


Sure thing. I removed TestScorer and renamed DefaultScorer to Scorer, using it with 0 in all tests excepted for the newly added one.

jkczyz · 2021-10-15T06:33:20Z

lightning/src/routing/scorer.rs

+impl Default for DefaultScorer {
+	/// Creates a new scorer using 100 msat as the channel penalty.
+	fn default() -> Self {
+		DefaultScorer::new(100)


Changed to 500 msats.

valentinewallace

Nice to get an initial version of this in!

lightning/src/routing/router.rs

lightning/src/routing/mod.rs

lightning/src/routing/router.rs

valentinewallace · 2021-10-15T17:19:30Z

lightning/src/routing/scorer.rs

+//!
+//! # Example
+//!
+//! TODO: Fill in example once interface has been finalized.


I think we're good to fill this in?

Done. Not much to it yet, but could expand if we add more configurations. Also, may want to have an example with InvoicePayer for event handling once that is in. Or at least reference those docs.

valentinewallace

LGTM 🚀

TheBlueMatt · 2021-10-15T20:03:51Z

Can you squash :).

Failed payments may be retried, but calling get_route may return a Route with the same failing path. Add a routing::Score trait used to parameterize get_route, which it calls to determine how much a channel should be penalized in terms of msats willing to pay to avoid the channel. Also, add a Scorer struct that implements routing::Score with a constant constant penalty. Subsequent changes will allow for more robust scoring by feeding back payment path success and failure to the scorer via event handling.

TheBlueMatt

Diff since val's ack was just a squash:

$ git diff-tree -U1 84c0a486e e15a18a50 
$

TheBlueMatt added this to the 0.0.102 milestone Oct 14, 2021

TheBlueMatt reviewed Oct 14, 2021

View reviewed changes

lightning/src/routing/router.rs Outdated Show resolved Hide resolved

lightning/src/routing/router.rs Outdated Show resolved Hide resolved

lightning/src/routing/mod.rs Outdated Show resolved Hide resolved

lightning/src/routing/mod.rs Outdated Show resolved Hide resolved

TheBlueMatt reviewed Oct 14, 2021

View reviewed changes

lightning/src/routing/router.rs Outdated Show resolved Hide resolved

TheBlueMatt reviewed Oct 14, 2021

View reviewed changes

lightning/src/routing/router.rs Outdated Show resolved Hide resolved

lightning/src/routing/router.rs Outdated Show resolved Hide resolved

TheBlueMatt approved these changes Oct 15, 2021

View reviewed changes

jkczyz commented Oct 15, 2021

View reviewed changes

valentinewallace reviewed Oct 15, 2021

View reviewed changes

valentinewallace approved these changes Oct 15, 2021

View reviewed changes

jkczyz force-pushed the 2021-10-default-channel-scoring branch from 84c0a48 to e15a18a Compare October 15, 2021 20:31

TheBlueMatt approved these changes Oct 15, 2021

View reviewed changes

TheBlueMatt merged commit 2398f17 into lightningdevkit:main Oct 16, 2021

jkczyz mentioned this pull request Oct 30, 2021

Add default implementation for tracking metadata #636

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add channel scoring to get_route #1124

Add channel scoring to get_route #1124

jkczyz commented Oct 14, 2021 •

edited

Loading

codecov bot commented Oct 14, 2021 •

edited

Loading

TheBlueMatt left a comment

TheBlueMatt commented Oct 14, 2021

jkczyz commented Oct 14, 2021

TheBlueMatt commented Oct 14, 2021

TheBlueMatt left a comment

TheBlueMatt Oct 15, 2021

jkczyz Oct 15, 2021

TheBlueMatt Oct 15, 2021

jkczyz Oct 15, 2021

TheBlueMatt Oct 15, 2021

jkczyz Oct 15, 2021

jkczyz left a comment

jkczyz Oct 15, 2021

jkczyz Oct 15, 2021

jkczyz Oct 15, 2021

valentinewallace left a comment

valentinewallace Oct 15, 2021

jkczyz Oct 15, 2021

valentinewallace left a comment

TheBlueMatt commented Oct 15, 2021

TheBlueMatt left a comment

Add channel scoring to get_route #1124

Add channel scoring to get_route #1124

Conversation

jkczyz commented Oct 14, 2021 • edited Loading

codecov bot commented Oct 14, 2021 • edited Loading

Codecov Report

TheBlueMatt left a comment

Choose a reason for hiding this comment

TheBlueMatt commented Oct 14, 2021

jkczyz commented Oct 14, 2021

TheBlueMatt commented Oct 14, 2021

TheBlueMatt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkczyz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

valentinewallace left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

valentinewallace left a comment

Choose a reason for hiding this comment

TheBlueMatt commented Oct 15, 2021

TheBlueMatt left a comment

Choose a reason for hiding this comment

jkczyz commented Oct 14, 2021 •

edited

Loading

codecov bot commented Oct 14, 2021 •

edited

Loading