[anti-abuse-wg] Fail2ban usage, was Draft Minutes - AA-WG @ RIPE80

Hi Alessandro,

    Hi Jordi and all,

    TL;DR:  Fail2ban can deal with missing or non-responding abuse teams
             automatically, without the need to load RIPE with extra costs.

[Jordi] Yes and not!

If you mean reporting to existing and *working* abuse-c, yes, but if the abuse-c doesn't work, doesn't exists, bounces, or returns an email to fill-in a form (a non-standard form), you're lost and have no other way to "monitor" the fail2ban bounces and fill the form manually.

LACNIC, as APNIC, should not be any more a problem, soon, as they both got this policy accepted by the community. In APNIC is already implemented since a year ago. LACNIC is still in implementation phase.



    In the draft minutes I read:

         Jordi said he thinks it will work because smaller providers use more and
         more Open Source tools and it's very common to use Fail2ban. He uses it
         himself, and it takes a couple of hours to implement that. So, he
         disagreed, but pointed out there there are lots of different opinions on
         the matter.


    I can confirm that abuse reporting by email works.  When I started reporting I 
    noticed some ISPs were receiving lots or reports each day.  In some cases, the 
    frequency suddenly dropped.  Most likely, that's the result of the ISP starting 
    to work on my reports and clean up.

    Based on such evidence, I recently changed my reporting script.  Now, I don't 
    use Fail2ban; I use ipqbdb, which works in a similar way.  It features an 
    abuserdap utility which looks up abuse addresses.  It takes as argument an 
    exclusion-file, which I manually fill with the addresses that seem to be 
    permanently bouncing.  Currently, the utility returns no address if either no 
    address is found in RDAP, or all the addresses found there are also found in 
    the exclusion file.  (See bash snippet below).

    Like Fail2ban, ipqbdb bans addresses for a limited time.  Wrong passwords 
    deserve a particularly short time period, because they can be given by legit 
    users.  However, users coming from IP addresses not supported by a responding 
    abuse team can be safely banned for a longer period.  I do one month.


    On Tue 07/Jul/2020 10:33:58 +0200 PP wrote:
    > The complaint to RIPE mechanism should only be an escalation mechanism when the 
    > ISP does not respond.


    Besides costs, that would make RIPE behave different than other LIRs.  I log 
    how many RDAP lookup fail.  Most of them are in LACNIC and APNIC.  Figures are 
    as follows:

    Total RDAP lookups           99,   3.03% of which failed
    Total RDAP lookups          107,   5.61% of which failed
    Total RDAP lookups          102,   3.92% of which failed
    Total RDAP lookups          140,  17.14% of which failed
    Total RDAP lookups          125,   6.40% of which failed
    Total RDAP lookups          115,   8.70% of which failed
    Total RDAP lookups          127,   7.09% of which failed
    Total RDAP lookups          113,   4.42% of which failed
    Total RDAP lookups          415,  21.93% of which failed
    Total RDAP lookups         1542,  39.49% of which failed
    Total RDAP lookups         1996,  49.10% of which failed
    Total RDAP lookups         1297,  55.05% of which failed
    Total RDAP lookups          242,  31.40% of which failed
    Total RDAP lookups          125,  40.80% of which failed
    Total RDAP lookups          149,  43.62% of which failed
    Total RDAP lookups           89,  30.34% of which failed
    Total RDAP lookups           55,  18.18% of which failed
    Total RDAP lookups           53,  18.87% of which failed
    Total RDAP lookups           61,   9.84% of which failed
    Total RDAP lookups           64,  25.00% of which failed
    Total RDAP lookups         1259,  49.80% of which failed
    Total RDAP lookups         1725,  60.46% of which failed
    Total RDAP lookups         1746,  64.83% of which failed
    Total RDAP lookups          643,  62.99% of which failed
    Total RDAP lookups           73,   5.48% of which failed
    Total RDAP lookups          148,   8.11% of which failed
    Total RDAP lookups          163,  11.04% of which failed
    Total RDAP lookups          155,  21.94% of which failed

    The relevant snippet of code is below:

             let rdap_lookup++
             readarray -t <<< "$(abuserdap -x $XCLUDE -vs $rdap_url 2>> $RDAP_LOG)"
             rcpt=${MAPFILE[0]}
             if test -z "$rcpt"; then
                 let rdap_failed++
                 # since Tue 19 May 2020, ban for 1 month.  Don't use -l here!!
                 ibd-ban -i $key -c 0 -t 2592000 -r "IP without abuse team"
             fi
             lastline="Recipient found in ${MAPFILE[1]}"

         # [...]

         if [ "$rdap_lookup" -gt 0 ]; then
             printf 'Total RDAP lookups     %8d, %6.2f%% of which failed\n' \
                 "$rdap_lookup" "$(echo "100*$rdap_failed/$rdap_lookup"| bc -l)"
         fi


    Best
    Ale
    -- 

































**********************************************
IPv4 is over
Are you ready for the new Internet ?
http://www.theipv6company.com
The IPv6 Company

This electronic message contains information which may be privileged or confidential. The information is intended to be for the exclusive use of the individual(s) named above and further non-explicilty authorized disclosure, copying, distribution or use of the contents of this information, even if partially, including attached files, is strictly prohibited and will be considered a criminal offense. If you are not the intended recipient be aware that any disclosure, copying, distribution or use of the contents of this information, even if partially, including attached files, is strictly prohibited, will be considered a criminal offense, so you must reply to the original sender to inform about this communication and delete it.