IPv6 support master plan

[yarikoptic]

I think that #39 grew out of the proportions to be any longer efficient. So I have decided to organize a new ticket to discuss the high-level development pieces which could largely be taken in steps. Let's continue discussion here at that level, and whenever we finalize the possible plan, detail specific steps in separate issues.
So, logical workflow to move off the dead point could IMHO could be be

1. Provide basic support for IPv6 addresses in Fail2Ban, at the beginning assuming very conservative prefix (/128). For that -- review/adopt previous work (https://github.com/grooverdan/fail2ban/tree/ipv6, https://github.com/grooverdan/fail2ban/tree/ipv6_2 etc ), without touching actual action definitions, incorporating prefix within the ban ticket (for possible future treatment in 2.) and possibly adopting/using https://github.com/google/ipaddr-py .
2. Investigate more and decide on how to deal with variable across use-cases prefixes. Suggested approaches were to

   • variable bantime (@sebres IPv6 Support #39 (comment)) depending on the number of failed attempts within the subnet

   • automatic shrinking of the prefix (@rotanid IPv6 Support #39 (comment)) given growing number of attacks

   • adhere to some very conservative default while maintaining a custom list of network prefixes for known providers/addresses

     Also @neirbowj information in IPv6 Support #39 (comment) .

3. Look into establishing definitions for support of both ipv4 and ipv6 within action files. See my summary of considered approaches in IPv6 Support #39 (comment)

Although every next step depends on the previous one, I think they have enough of independence to continue discussion in separate issues. ATM
I just want to hear from other @fail2ban/contributors and the community if this sounds reasonable.

[brianjmurrell]

I think this reflects the separation of issues that I was previously advocating. Basic, equivalent-to-current-IPv4 support for IPv6. Land it, release it.

Then, and only then, start to look at general wider prefix handling generic enough to be applicable to both IPv4 and IPv6. Land it, release it.

And then go from there.

[yarikoptic]

I am not sure we will land/release 1 alone since, as was stated, it could immediately be exploited by an attacker to cause resources exhaustion/DoS. May be only if treatment of IPv6 addresses would be made optional with a big fat warning on possible ramifications.

[mrc0mmand]

Maybe something like gradual banning of larger subnets would help? Eg. if f2b bans 25 IPs from /121 subnet, it will ban entire /120 and remove all previous bans for /121. If it later bans another 50 from /120, it will again remove all bans for /120 and ban entire /119. This could be still exploited in some way, but it wouldn't be so easy/fast. (Also I'm not sure if it's technically correct/possible, it's just a thought).

[sebres]

@yarikoptic I can make a IPv6 version as described in your plan, but:

• I will make it iptables/debian only, need help from another contributers for other systems;
• unfortunately I have no interest to make it, as long as Ban time incr #716 and my feature branch based on it + Collection of all kinds of changes, fixes, refactorings [Was: F2B performance branch] #1116 are not completed resp. both are not mainline (not master), because I see too many conflicts hereafter;
• I see the subnet support as a necessary part of the IPv6, also because of memory exhaustion, increase factor for failure count or time of ban should be applied for subnet's (single IP would be a subset of it), and many other things, why it is really necessary.

[rotanid]

the plan sounds good - and i would prefer the optional "1" with default to off and big warning being released. there are use cases, where this is better than f2b doing nothing for IPv6 at all.
@mrc0mmand please read the ticket! as far as i understand it, "automatic shrinking of the prefix" does exactly what you are suggesting.

[mrc0mmand]

@rotanid I honestly don't know how I could miss that, but now I see it. So sorry and thanks - it's exactly what I mean.

Also, even with 'automatic prefix shrinking' there would be a huge amount of IP addresses, which would be kind of uncomfortable or even slow if we store them in iptables. Wouldn't be eg. ipset better 'default' solution for that? I know even ipset has its limits, but it would be much faster (and probably convenient) than storing that load of IPv6 addresses in iptables.

[mfechner]

Could it be possible to implement at first only a block for the specific IPv6 IP?
I get tons of logs in my server that fail2ban cannot block the address, because it is an IPv6 address.
You can add the /64 or /56 block feature later.

[Xoib]

@idefix6 It has already been discussed in #39. Btw, this new topic is to discuss the proper way to implement this, if you need a workaround there are plenty in #39.

[buanzo]

Currently we have one table per jail with all banned IPs. We could apply a
hierarchical approach, based on ipv6 grouping, to ease the load? Sorry if
this has been discussed already.

[yarikoptic]

Currently we have one table per jail with all banned IPs. We could apply a
hierarchical approach, based on ipv6 grouping, to ease the load? Sorry if
this has been discussed already.

I don't think it was discussed, but I am also not yet familiar with this
so can't stay how feasible it will be.

[Elompenta]

Do you have any new information to support IPv6 banning?

[sebres]

Short - no, we have not. But I work on the solution, unfortunately fewer as secondary priority (there much to do).
@fail2ban/developers any thoughts?

[f0o]

👍 to IPv6 Support :)

[yarikoptic]

+1
....
Did it help? Not really... We know that it is a needed feature, and this issue is not to seek feedback on its desirability, but about the plan. Provide feedback accordingly. Or better -- submit PRs addressing specific parts of the plan for review so feature actually gets implemented. Thanks

edit 1: all non-constructive/0-bit-of-relevant-information comments get removed from this thread

[grmrgecko]

So, as I understand it:

1. Get fail2ban to parse IPv6 addresses from logs. I don't think they are logged differently from IPv4.
2. At first, have it detect only the /128 prefix for attacks and later have features where it'll see that multiple IPv6 addresses from a similar prefix is attacking and block that prefix.
   From my research, ISPs normally give a /64 prefix to customers, but would be good to be able to grow the prefix depending on the range of attacks.
3. At first, do not modify actions for IPv6, but later maybe have the option to have actionban6 and actionunban6 to specify what to do specifically for IPv6 addresses.
   As far as I know, UFW will support banning IPv6 in the same way you ban IPv4, so an actionban6 type thing is not needed for it, but I can see where you may need something for say example ip6tables.
   I think having an actionban6 type variable would be easy to implement.

I believe that getting the basic support can be done rather quickly, but as I have not reviewed the code in full I could be mistaken.

I believe IPv6 support is important as many hosting companies provide IPv6 (such as my own). I would be happy to help implement support for IPv6 once this semester in school is over.

To cut down on spam, I think people should thumb up the post using githubs "Add your reaction" button instead of writing a post saying +1 or something. I for one want IPv6 support, so I have thumbed up the post.

Thank you for your work on this project fail2ban team.

[yarikoptic]

There is a perspective PR #1374 so everyone interested is welcome to review/try

[sebres]

We have introduced a new branch with IPv6 support: 0.10, and the new PR #1410
The decision is at least firstly - 0.9 will never get IPv6-support.

Everyone interested to test 0.10 is welcome to review resp. try this new vesion.

[mjholtkamp]

Maybe I'm joining a bit late in the discussion, but I would also like to have IPv6 support in fail2ban and I have a potential contribution.

I tried to find a solution for the second item mentioned in the beginning of this issue: "Investigate more and decide on how to deal with variable across use-cases prefixes". I chose to implement "automatic shrinking of the prefix" and I think I have a nice working library:

https://github.com/mjholtkamp/py-iptree

Are people interested in using this library? I do not know if it's a good match, since iptree uses the ipaddress package (included in py3.5, separately installed for py2.7), instead of relying on the socket interface that fail2ban currently uses.

iptree is tested under both python 2.7 and 3.5.

A nice bonus is that the automatic shrinking also works for IPv4 addresses, if you choose to use that.

[sebres]

Thx, I will take a look into your library...

I would also like to have IPv6 support in fail2ban

Well, we have it in 0.10 (even pre-released and yet without subnets, but ... :))

[mjholtkamp]

Of course I meant to have IPv6 support with subnet support ;)

Did you get a chance to take a look into the library yet? If you have suggestions on changes, I'm open to it and I wouldn't mind putting some effort into integrating the library in fail2ban (I think there might be some changes internally), but I want to know if it's desired. If you already now that it will never be merged I think it will be a waste of effort :)

[basd82]

When will support for ipv6 be implemented ?

[sebres]

@basd82 It is already implemented in 0.10.x (ca. one and a half years).

[basd82]

@sebres is there debian/ubuntu repo for that version ?

[costela]

@basd82 both debian testing and ubuntu bionic have it:

$ rmadison -u debian fail2ban
(...)
fail2ban   | 0.10.2-1       | testing            | source, all
fail2ban   | 0.10.2-1       | unstable           | source, all

and

$ rmadison -u ubuntu fail2ban
(...)
 fail2ban | 0.10.2-1                     | bionic/universe           | source, all

Also, a github issue isn't the best place to ask these questions.

[szepeviktor]

What parts of IPv6 support are missing? As this issues is open.

[sebres]

As this issues is open.

• subnet
• all banning actions must provide IPv6 capability

[yarikoptic]

@basd82 backports of 0.10.2-1 are available from "secret" debian-devel of neurodebian: http://neuro.debian.net/debian-devel/pool/main/f/fail2ban/ I was not yet brave enough to push them into the main repo of neurodebian... but soon I guess ;)

[f0o]

@sebres How can I, as a banning action provider, detect if I should ban a IPv4 or IPv6 without having to run the provided IP through regex?
Can I provide two separate ban/unban actions for v4 vs v6 targets?

[sebres]

@f0o this is very easy: we've a conditional section that allows to differentiate between IP families.
For example take a look into any action supporting IPv6, e. g.

fail2ban/config/action.d/iptables-common.conf

Line 79 in b5dd5ad

[Init?family=inet6]

Short self explaining example:

[Definition]
...
actionban = printf "%%b\n" "ban <_ip_type> <_ip_var>:<port>"
actionunban = printf "%%b\n" "unban <_ip_type> <_ip_var>:<port>"

[Init]
_ip_var = <ip>
_ip_type = IPv4

[Init?family=inet6]
_ip_var = [<ip>]
_ip_type = IPv6

Of course it is not necessary to use some parameters extra (like in the example above), you can simply overwrite actionban, actionunban, etc. in section [Init?family=inet6].
So yes, you can provide two separate ban/unban actions for v4 vs v6 targets, but IMHO it is better to use parameters, at least saves time (and code) and more dynamical, e. g. if action will be extended then you must not adjust entries in both sections (if the same syntax).

Additionally, you've a new run-time tag <family> which contains the family of the IP (ban/unban only). So you can use it like this (or similar):

actionban = if [ '<family>' = 'inet4' ]; then ...; fi;

[omega-software]

Hi,

I think the ideal solution would be to ban the smallest range whose disappearance would make the rate of bad traffic go down below the acceptable threshold (i.e. what used to be represented by maxretry for a single IPv4 address).

So for example, if we're receiving a brute-force attack from a specific /64 range, but 99,999% of it comes from a specific /128 address within that range, we would only block the /128 one, and not the entire /64 range unless it surpasses the threshold in its own merit. But before considering /64 we would have to check if /127 might be enough to mitigate the problem, then /126, etc. If none of the smaller prefixes are enough, then it means the whole /64 is either evil or incompetent and should be banned entirely.

Obviously, the threshold cannot be the same for all prefixes, otherwise we would ban entire ISPs just because they have some bad apples, or even the whole internet (prefix /0).

[sPOiDar]

I think the ideal solution would be to ban the smallest range whose disappearance would make the rate of bad traffic go down below the acceptable threshold (i.e. what used to be represented by maxretry for a single IPv4 address).

The smallest recommended routable prefix is a /64, so just banning that immediately makes sense to me. If some ISP has chosen to do something stupid by routing a smaller prefix, oh well.

Cloud providers seem to be the only culprits who commonly provide less than a /64 prefix, but the smaller cloud providers are where a large portion of malicious traffic originates anyway, often from multiple boxes, so again, not overly concerning to me.

The proposal to check existing individually blocked IPs and calculate minimal subnets could very quickly become hugely expensive in terms of memory and processing power - optimal storage and lookup of the massive quantities of individual numbers in IPv6 ranges is still an evolving area of research, and when your potential set is the entire IPv6 space, you have a very serious problem.

[jorhett]

I agree with the /64 approach. But just to be safe, perhaps an option to ban only the offender or the entire /64 depending on user preference.

I see no value provided (and high risk) in Fail2ban trying to guess subnet sizes.

Not sure it's easy to do, but banning only the offender and then switch to banning the entire /64 on a 2nd or 3rd failure from that same /64 would be nice. Some sort of logic in recidive?

[omega-software]

@sPOiDar Yes, I agree with both things you said. Straight banning of /64 is probably reasonable, but still not perfect, as in some cases /64 could be too much or too few I was only thinking loud on what the ideal would be.

I'm not sure the cost would be prohibitive, as making a decision for a particular prefix length doesn't require you to traverse the entire subtree over again (we only have two bits and need to know how bad is each of them, which was already -mostly- calculated when traversing shorter prefixes). It might be possible to write an algorithm below O(2^n).

[sPOiDar]

as in some cases /64 could be too much or too few I was only thinking loud on what the ideal would be.

There's no algo I can think of off the top of my head that can sanely determine what the upper bounds should be for the "too few" case here without relying on external services to try and determine allocated ranges, and as for "too many", my take on that is above.

[szepeviktor]

@Slamdunk You can add IPv6-specific sections, for example

[Init]

ss_ip = <ip>

[Init?family=inet6]

ss_ip = [<ip>]

[Slamdunk]

@szepeviktor thank you, missed that of the first read 😅