[anti-abuse-wg] Blocklists on RIPEstat (Was: UCEPROTECT DNSBL possibly abusive practice and RIPEStat Blacklist entries widget)

Dear colleagues,

In preparation for the upcoming AA-WG session, I want to provide an 
update on the status of the revamp of the blocklist feature on RIPEstat.

In 2021 and since the last status update we collected and reviewed a set 
of blocklists that we would see ideal to be included. We did this 
together with the help of our panel of domain experts.

Following the review, we performed a technical feasibility analysis, 
which filtered the blocklists related to supported resource type (IP and 
domain) and access method (DNS). For each of the remaining blocklists, 
we started to contact the respective operator and asked for permission 
to use the blocklist on RIPEstat.

The technical implementation of the required backend started with a 
delay but we are glad to report that we finished the development last 
month and released the feature a week ago.

Currently, the best way to use the new feature is via this RIPEstat widget:

https://stat.ripe.net/widget/dns-blocklists

Some details on the current solution:
- The data is fetched in realtime unless cached (with a maximum cache 
time of 2 hours)
- The supported resource type is a single IP address (IPv4 & Ipv6)
- To prevent abuse a daily rate limit of 100 uncached queries per user 
is in place

What we want to do next:
- Extend the number of blocklists
- Add support for domain queries
- Include the feature in the new UI (UI2020)
- Add a similar API for whitelists

As we are still reaching out to some blocklist operators it is unclear 
how many blocklists will be included but we expect the number to be 
significantly higher than the old blocklist API call. Soon we will also 
review whether the new blocklist feature is mature enough to replace the 
old one.

All details on the project planning are available here: 
https://ripestat.featureupvote.com/suggestions/196464/project-blocklists-2021

Last but not least I want to thank the participants of the panel for 
their invaluable input and support!

I hope you find the new feature useful and I am looking forward to your 
feedback.

All the best,
Christian

On 08/07/2021 10:21, Christian Teuschel wrote:
> Dear colleagues,
> 
> I would like to share a short update regarding the Blocklist project for
> RIPEstat. As you might have noticed, we changed the name to “blocklist”
> earlier this year in March.
> 
> This month, we started evaluating new blocklists with the support of a
> panel of seven domain experts. Once the evaluation phase has been
> completed, we will share a fresh update.
> 
> You can follow more detailed design decisions on our public feature
> tracker:
> https://ripestat.featureupvote.com/suggestions/196464/project-blocklists-2021
> 
> To make sure you get all the latest news and updates on Twitter, follow
> #RIPEstat
> 
> Best regards,
> Christian
> 
> On 19/03/2021 14:24, Christian Teuschel wrote:
>> Dear colleagues,
>>
>> Thank you for your suggestions and input. Here is an update on what we
>> plan to do next:
>>
>> We will continue looking at blocklist candidates and prepare an
>> analysis/implementation plan. This will be done with a small group of
>> experts who volunteered to help with the vetting process (contact me
>> off-list if you would like to join this group).
>>
>> This work will start in Q3. Once implemented, the changes will be
>> available via the new RIPEstat UI. We might also add filters that allow
>> you to decide which sources to use - with a smaller set of blocklists
>> selected by default.
>>
>> Another outcome is that we will also change the name from "blacklist" to
>> "blocklist". This will have an impact on API endpoints, widgets and
>> documenation. Work on this will start next week.
>>
>> We will update this working group once our analysis is ready. In the
>> meantime, if you are interested in RIPEstat, we also post developments
>> on Twitter under #RIPEstat.
>>
>> Best regards,
>>
>> Christian
>>
>> On 12/03/2021 09:51, Christian Teuschel wrote:
>>> Dear colleagues,
>>>
>>> I can see a few suggestions for additional blocklists to include. It
>>> would be helpful if we could get any others by 19 March.
>>>
>>> We will then get started on an analysis that we will share with the
>>> community a little later in the year.
>>>
>>> Kind regards
>>> Christian
>>>
>>> On 09/03/2021 10:37, Christian Teuschel wrote:
>>>> Dear colleagues,
>>>>
>>>> Thinking about a course of action - it looks there is an agreement to
>>>> have more RBLs on RIPEstat. It would be good to have a list of
>>>> candidates that the community feels would be useful. Once we have this
>>>> list, we can perform a feasibility analysis and present this to the
>>>> community.  We can then take it from there.
>>>>
>>>> Let me know if this approach works for you.
>>>>
>>>> Best regards,
>>>> Christian
>>>>
>>>> On 04/03/2021 17:16, Christian Teuschel wrote:
>>>>> Hi Elvis and Suresh, dear colleagues,
>>>>>
>>>>> Putting exact numbers on how many operators are using UCEProtect is
>>>>> difficult, but through feedback from users, network operators and
>>>>> members we understand that it is in use and that the provisioning of
>>>>> this RBL on RIPEstat has value.
>>>>>
>>>>> If I am reading the feedback in this discussion correctly, the sentiment
>>>>> is leaning towards adding more RBLs instead of less and if that is the
>>>>> case we are going to look into how and when we can achieve this. Please
>>>>> let me know if that is aligned with your requirements/expectations.
>>>>>
>>>>> Best regards,
>>>>> Christian
>>>>>
>>>>> On 04/03/2021 09:54, Elvis Daniel Velea wrote:
>>>>>> Hi Christian,
>>>>>>
>>>>>> while it may be useful to have their data source, it only shows the RIPE
>>>>>> NCC favors one or two operators and I think that is damaging to the
>>>>>> whole idea of being impartial.
>>>>>>
>>>>>> You either include a good list of blacklist operators and their data or
>>>>>> none. Including only a couple will lead to the impression that only
>>>>>> those are important enough to be considered by the RIPE NCC.
>>>>>>
>>>>>> my 2 cents,
>>>>>> Elvis
>>>>>>
>>>>>> On 3/3/21 8:27 AM, Christian Teuschel wrote:
>>>>>>> Dear colleagues,
>>>>>>>
>>>>>>> RIPEstat is a neutral source of information and we aim to provide users
>>>>>>> with access to as many data sources as possible to provide insights.
>>>>>>>
>>>>>>> UCEProtect was added as a data source prior to 2010 and is still used by
>>>>>>> several network operators to filter traffic into their networks.
>>>>>>> Including it as a data source in RIPEstat allows users to see whether
>>>>>>> resources are included in their lists.
>>>>>>>
>>>>>>> RIPE NCC does not pay for, support or endorse their practices, although
>>>>>>> we understand that continuing to include UCEProtect as a data source
>>>>>>> could be misunderstood as such. We also do not use their lists to filter
>>>>>>> traffic on our services.
>>>>>>>
>>>>>>> Our goal remains to provide the best visibility and tools for network
>>>>>>> operators to diagnose their networks. We have also heard your feedback
>>>>>>> regarding including more RBLs. It is something that we have considered
>>>>>>> in the past, and we are open to revisiting this.
>>>>>>>
>>>>>>> RIPEstat is driven by the community. We would like to hear from you
>>>>>>> about whether including UCEProtect as a data source is useful.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Christian
>>>>>>>
>>>>>>> On 02/03/2021 00:08, Kristijonas Lukas Bukauskas via anti-abuse-wg wrote:
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> I noticed that RIPE NCC uses uceprotect-level1, uceprotect-level2 and
>>>>>>>> uceprotect-level3 in RIPEStat Anti Abuse Blacklist Entries widget.
>>>>>>>>
>>>>>>>> There have been controversial positions about this blacklist recently:
>>>>>>>>
>>>>>>>> 1)
>>>>>>>> https://success.trendmicro.com/solution/000236583-Emails-being-rejected-by-RBL-UCEPROTECL-in-Hosted-Email-Security-and-Email-Security
>>>>>>>>
>>>>>>>> <https://success.trendmicro.com/solution/000236583-Emails-being-rejected-by-RBL-UCEPROTECL-in-Hosted-Email-Security-and-Email-Security>
>>>>>>>>
>>>>>>>> 2) https://blog.sucuri.net/2021/02/uceprotect-when-rbls-go-bad.html
>>>>>>>> <https://blog.sucuri.net/2021/02/uceprotect-when-rbls-go-bad.html>
>>>>>>>>   
>>>>>>>> UCEPROTECT blacklists the whole range of IP addresses, including the
>>>>>>>> full IP range of some autonomous systems:
>>>>>>>>    UCEPROTECT states, '/Who is responsible for this listing? YOU ARE NOT!
>>>>>>>> Your IP was NOT directly involved in abuse but has a bad neighborhood.
>>>>>>>> Other customers within this range did not care about their security and
>>>>>>>> got hacked, started spamming, or were even attacking others, while your
>>>>>>>> provider has possibly not even noticed that there is a serious problem.
>>>>>>>> We are sorry for you, but you have chosen a provider not acting fast
>>>>>>>> enough on abusers'/) [http://www.uceprotect.net/en/rblcheck.php
>>>>>>>> <http://www.uceprotect.net/en/rblcheck.php>].
>>>>>>>>    It asks for a fee if some individual IP address wants to be
>>>>>>>> whitelisted
>>>>>>>> (http://www.whitelisted.org/ <http://www.whitelisted.org/>),
>>>>>>>>    It abuses people who decide to challenge their blacklist by publishing
>>>>>>>> conversations in their so-called /Cart00ney/
>>>>>>>> (http://www.uceprotect.net/en/index.php?m=8&s=0
>>>>>>>> <http://www.uceprotect.net/en/index.php?m=8&s=0>;
>>>>>>>> http://www.uceprotect.org/cart00neys/index.html
>>>>>>>> <http://www.uceprotect.org/cart00neys/index.html>).
>>>>>>>>    And the other type of threatening: http://www.uceprotect.org/
>>>>>>>> <http://www.uceprotect.org/>
>>>>>>>>    Does RIPE NCC have any position on this specific blacklist?
>>>>>>>>
>>>>>>>> Thank you!
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
> 

-- 
Christian Teuschel
@christian_toysh
RIPE NCC