Scheduler, pathmanager, congestion control

[michalmorawski]

Hi,
In previous versions of MPTCP one was allowed to install your own versions of Scheduler, Path Manager, Congestion Control, etc. using sysctl.
I cannot find such a feature in the new (build-in-kernel 5.15 - ubuntu 2022.04) version of mptcp. Neither sysctl nor 'ip mptcp' has such options. Would you like to advise me on how to modify the default mptcp behaviour (i.e., change the Path Manager, Scheduler, and congestion controller)?

[matttbe]

Hi @michalmorawski

This MPTCP Upstream implementation is a re-work of the previous one (MPTCP Out of tree / fork) to fit upstream requirements. Some choices have been made and the design needed to be different. So it is normal to have differences.

Regarding the extensions, the global idea is to move the customisation part to userspace. In other words, we try to avoid having multiple path-manager and packet schedulers with many different options in kernelspace because it is complex to maintain, especially when there are multiple stable releases in place. It is also quicker to develop them in userspace when the API with the kernel is in place.

Yet it is already possible to adapt MPTCP behaviours to your needs:

• there are 2 path-managers:
  • the "in-kernel" one, controllable via ip mptcp (or other tools like mptcpd)
  • the "userspace" one -- very similar to the "Netlink" path manager from MPTCP out of tree -- controllable via mptcpd plugins for example (or other apps implementing the Netlink API)
  • there are also sysctl: https://docs.kernel.org/networking/mptcp-sysctl.html
    • net.mptcp.allow_join_initial_addr_port to control the C bit flag
    • net.mptcp.pm_type to change the PM
    • net.mptcp.add_addr_timeout for the add-addr, mainly to accelerate tests
• there are 2 packet schedulers:
  • the "in-kernel" one having a generic behaviour, implementing a simplified version of the Blest algorithm. It is sending over the different subflows except the backup ones if there are working non backup ones
  • the "eBPF" one -- currently in development, see BPF: packet scheduler #75 -- to allow other implementations outside the kernel
  • regarding the sysctl:
    • only net.mptcp.stale_loss_cnt on how quickly a subflow is considered "stalled"
• there are no MPTCP specific congestion control (yet):
  • mainly because they are only needed in very specific cases: there are often linked to interesting mathematical problems, hence studied in academic research papers but it doesn't seem to be that important in real use cases.
  • yet, if someone needs them and wants to implement new ones, it is still possible to create TCP congestion control looking at MPTCP info (or do that with eBPF).

If something is missing, it is probably because nobody needed it before (or reported it or better: implemented it).
Of course, any contributions are welcome!

[matttbe]

Hi @michalmorawski ,

Does my previous reply answer your questions? Can we close this ticket?

[matttbe]

Hi @michalmorawski

I suggest to close this ticket.

Feel free to re-open it if my previous reply was not clear enough.

[Suvam77]

Hi @matttbe

Can you please clarify a bit more about the two path manager types you mentioned above, i.e., 'in-kernel' and 'userspace'?
And can you please describe the specific use of mptcpd and how it is different from ip mptcp in terms of operation?

[matttbe]

Hi @Suvam77

Can you please clarify a bit more about the two path manager types you mentioned above, i.e., 'in-kernel' and 'userspace'?

When you use the 'in-kernel' path manager, the kernel will do some actions depending on the endpoints you added per net namespace: ip mptcp endpoint add ADDRESS (...). The same action will be taken for all MPTCP connections from the same net namespace, e.g. always announce the same IP or always create the same subflows.

With the userspace one, the kernel doesn't do any actions but a userspace application like mptcpd can ask the kernel to create new subflows per connection. It is then possible to do different actions per connections to cover more specific use cases.

And can you please describe the specific use of mptcpd and how it is different from ip mptcp in terms of operation?

ip mptcp will allow you to control the endpoints used by the in-kernel path-manager: https://man7.org/linux/man-pages/man8/ip-mptcp.8.html
The idea is to control the endpoints manually or via a script.

Network Manager 1.40.0 (and higher) can also manage the endpoints but here in a more "automatic way".

mptcpd is a bit particular because it can be used with both the "in-kernel" path-manager and the "userspace" one. Indeed, depending on the plugin, it can control the endpoints for the "in-kernel" path manager but plugins can also be written to use the "userspace" path manager and have a finer control from the userspace, e.g. reacting when a new MPTCP connection is being created.

[michalmorawski]

Yes, of course. Sorry for my silence. I am waiting for Scheduler example written as BPF. Eagerly 😊. I have seen, it is already on the bakclog table. Michał. From: Matthieu ***@***.***> Sent: piątek, 16 września 2022 18:33 To: ***@***.***> Cc: Michał Morawski ***@***.***>; ***@***.***> Subject: Re: [multipath-tcp/mptcp_net-next] Scheduler, pathmanager, congestion control (Issue #300) Hi @michalmorawski<https://github.com/michalmorawski> , Does my previous reply answer your questions? Can we close this ticket? — Reply to this email directly, view it on GitHub<#300 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AGK2KGLGJTVBNV66OXMZVJDV6SOOPANCNFSM6AAAAAAQH4DQRY>. You are receiving this because you were mentioned.Message ID: ***@***.***>

…

________________________________ [Politechnika Łódzka / Lodz University of Technology] [Facebook] <https://www.facebook.com/Politechnika.Lodzka/> [Twitter] <https://twitter.com/p_lodz_pl> [Youtube] <https://www.youtube.com/channel/UCbBnxpxg1DbSrIBwvebJghg> [Linkedin] <https://www.linkedin.com/school/politechnika-łódzka/>

________________________________ Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata. Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez pomyłkę, prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie. This email contains information intended solely for the use of the individual to whom it is addressed. If you are not the intended recipient or if you have received this message in error, please notify the sender and delete it from your system.