Make ProbailisticScorer half-life more linear
#1752
Milestone
Comments
|
To avoid needing a timer, could a combination of bit shifting and bit setting provide a less aggressive decay? |
|
Probably? I think the ship has move sailed on timers in LDK, though, sadly. |
TheBlueMatt
added a commit
to TheBlueMatt/rust-lightning
that referenced
this issue
Oct 5, 2022
Even at relatively high payment volumes, decaying knowledge of each individual channel every hour causes aggressive retrying of channels as we quickly forget the state of a channel. Even with the historical tracker, this isn't fully remedied, as we'll track the history bounds with the decayed value. Instead, we decay every six hours here, reducing how often we'll retry a channel due to decay. In addition to this, the decay likely needs to be substantially more linear, as tracked in lightningdevkit#1752.
TheBlueMatt
added a commit
to TheBlueMatt/rust-lightning
that referenced
this issue
Oct 5, 2022
Even at relatively high payment volumes, decaying knowledge of each individual channel every hour causes aggressive retrying of channels as we quickly forget the state of a channel. Even with the historical tracker, this isn't fully remedied, as we'll track the history bounds with the decayed value. Instead, we decay every six hours here, reducing how often we'll retry a channel due to decay. In addition to this, the decay likely needs to be substantially more linear, as tracked in lightningdevkit#1752.
TheBlueMatt
added a commit
to TheBlueMatt/rust-lightning
that referenced
this issue
Oct 5, 2022
Even at relatively high payment volumes, decaying knowledge of each individual channel every hour causes aggressive retrying of channels as we quickly forget the state of a channel. Even with the historical tracker, this isn't fully remedied, as we'll track the history bounds with the decayed value. Instead, we decay every six hours here, reducing how often we'll retry a channel due to decay. In addition to this, the decay likely needs to be substantially more linear, as tracked in lightningdevkit#1752.
TheBlueMatt
added a commit
to TheBlueMatt/rust-lightning
that referenced
this issue
Oct 5, 2022
Even at relatively high payment volumes, decaying knowledge of each individual channel every hour causes aggressive retrying of channels as we quickly forget the state of a channel. Even with the historical tracker, this isn't fully remedied, as we'll track the history bounds with the decayed value. Instead, we decay every six hours here, reducing how often we'll retry a channel due to decay. In addition to this, the decay likely needs to be substantially more linear, as tracked in lightningdevkit#1752.
TheBlueMatt
added a commit
to TheBlueMatt/rust-lightning
that referenced
this issue
Oct 6, 2022
Even at relatively high payment volumes, decaying knowledge of each individual channel every hour causes aggressive retrying of channels as we quickly forget the state of a channel. Even with the historical tracker, this isn't fully remedied, as we'll track the history bounds with the decayed value. Instead, we decay every six hours here, reducing how often we'll retry a channel due to decay. In addition to this, the decay likely needs to be substantially more linear, as tracked in lightningdevkit#1752.
TheBlueMatt
added a commit
to TheBlueMatt/rust-lightning
that referenced
this issue
Oct 7, 2022
Even at relatively high payment volumes, decaying knowledge of each individual channel every hour causes aggressive retrying of channels as we quickly forget the state of a channel. Even with the historical tracker, this isn't fully remedied, as we'll track the history bounds with the decayed value. Instead, we decay every six hours here, reducing how often we'll retry a channel due to decay. In addition to this, the decay likely needs to be substantially more linear, as tracked in lightningdevkit#1752.
TheBlueMatt
added a commit
to TheBlueMatt/rust-lightning
that referenced
this issue
Oct 7, 2022
Even at relatively high payment volumes, decaying knowledge of each individual channel every hour causes aggressive retrying of channels as we quickly forget the state of a channel. Even with the historical tracker, this isn't fully remedied, as we'll track the history bounds with the decayed value. Instead, we decay every six hours here, reducing how often we'll retry a channel due to decay. In addition to this, the decay likely needs to be substantially more linear, as tracked in lightningdevkit#1752.
TheBlueMatt
added a commit
to TheBlueMatt/rust-lightning
that referenced
this issue
Oct 11, 2022
Even at relatively high payment volumes, decaying knowledge of each individual channel every hour causes aggressive retrying of channels as we quickly forget the state of a channel. Even with the historical tracker, this isn't fully remedied, as we'll track the history bounds with the decayed value. Instead, we decay every six hours here, reducing how often we'll retry a channel due to decay. In addition to this, the decay likely needs to be substantially more linear, as tracked in lightningdevkit#1752.
TheBlueMatt
added a commit
to TheBlueMatt/rust-lightning
that referenced
this issue
Oct 11, 2022
Even at relatively high payment volumes, decaying knowledge of each individual channel every hour causes aggressive retrying of channels as we quickly forget the state of a channel. Even with the historical tracker, this isn't fully remedied, as we'll track the history bounds with the decayed value. Instead, we decay every six hours here, reducing how often we'll retry a channel due to decay. In addition to this, the decay likely needs to be substantially more linear, as tracked in lightningdevkit#1752.
TheBlueMatt
added a commit
to TheBlueMatt/rust-lightning
that referenced
this issue
Oct 11, 2022
Even at relatively high payment volumes, decaying knowledge of each individual channel every hour causes aggressive retrying of channels as we quickly forget the state of a channel. Even with the historical tracker, this isn't fully remedied, as we'll track the history bounds with the decayed value. Instead, we decay every six hours here, reducing how often we'll retry a channel due to decay. In addition to this, the decay likely needs to be substantially more linear, as tracked in lightningdevkit#1752.
TheBlueMatt
added a commit
to TheBlueMatt/rust-lightning
that referenced
this issue
Oct 11, 2022
Even at relatively high payment volumes, decaying knowledge of each individual channel every hour causes aggressive retrying of channels as we quickly forget the state of a channel. Even with the historical tracker, this isn't fully remedied, as we'll track the history bounds with the decayed value. Instead, we decay every six hours here, reducing how often we'll retry a channel due to decay. In addition to this, the decay likely needs to be substantially more linear, as tracked in lightningdevkit#1752.
|
Thinking about this more, there may be a simple way to introduce quarter-life and eighth-life decays. |
|
Or maybe 3/4 and 7/8 is more accurate way of saying it. 😛 |
|
I'm not sure how worth it it is? I dunno if removing the decay logic from score fetching would be noticeable in the performance (probably not given it's not loading memory), but at this point we just have timers everywhere. I mean annoying "just one more" is maybe nice? But also we already have three (plus persist timers), what's a fourth? |
|
Wanted to avoid the timer approach for now as it seems more involved than what time I can dedicate at the moment. Not to mention all the test changes that would be required. Was hoping there would be some bitwise operations that would be simple to implement. Summing the result of 1 decay and 2 decays gives a rough approximation of a 3/4-life, FWIW. There may be a better approach that is more accurate though... Otherwise, I guess it would be nice if the scorer could be independent of a timer so it could be used out-of-the-box in any typical setup (e.g., remote router) without additional work. |
|
Hmm, right, I suppose we could? If you want to go that direction that'd be fine, I think, at least as a quick-and-dirty thing to land. |
optout21
pushed a commit
to optout21/rust-lightning
that referenced
this issue
Jul 24, 2023
Even at relatively high payment volumes, decaying knowledge of each individual channel every hour causes aggressive retrying of channels as we quickly forget the state of a channel. Even with the historical tracker, this isn't fully remedied, as we'll track the history bounds with the decayed value. Instead, we decay every six hours here, reducing how often we'll retry a channel due to decay. In addition to this, the decay likely needs to be substantially more linear, as tracked in lightningdevkit#1752.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We had an issue where a user's scoring parameters were set to strongly prefer one channel, but in practice it never actually had any liquidity. As a result, they ended up in somewhat of a loop - once every $SCORER_HALF_LIFE, they'd suddenly be retrying the channel again, and would have to retry it again each time a lower-value payment came through, until eventually the liquidity bounds would be very low, and then it would sit still for a $SCORER_HALF_LIFE, before the process started anew.
While we don't want to remove the half-life, the fact that the liquidity bounds jump by a rather large amount at each half-life is a problem, causing a number of retries along the given channel until we finally get the bounds low enough to stop using it. Instead, we really need to find a way to make the bounds updates more linear.
Sadly, currently we apply half-lives while doing routing, which limits us to ensuring we're doing very fast things (eg shifting, not multiplying) but if we instead did scorer update passes on a few-minute timer and updating all the bounds as we go, we could drop the during-scoring updates and also be much more linear (and slower).
The text was updated successfully, but these errors were encountered: