[atlas] Proposal: Measure well-known CDNs,[CDN-HTTP]

Hello,


On Thu, 29 Dec 2022 at 15:15, Stephane Bortzmeyer <bortzmeyer at nic.fr> wrote:
>
> On Tue, Dec 20, 2022 at 05:48:08PM +0100,
>  Lukas Tribus <lukas at ltri.eu> wrote
>  a message of 60 lines which said:
>
> > - where have those security concerns been previously discussed?
>
> Several times on this list. This is a recurring discussion, for many
> years.
>
> > Are you suggesting that people deploy ATLAS probes in security
> > sensitive inside parts of corporate networks?
>
> I believe that the concerns were more about the security of the server
> than the security of the probe. Nobody wants Atlas to be used as a
> botnet against unsuspecting HTTP servers.

I was specifically addressing the decision to make this an "opt-in"
option for the probe owner. I fully agree with the proposal to limit
the request to HEAD and GET with limited response size. With the
proposed limitations, the fact that measurements are public, atlas
credits have non-zero costs, I don't have huge concerns about Denial
of service/load concerns of the measurement destination (HTTP)
servers. However this is orthogonal to the concerns about load/traffic
of the probe itself, which is why I believe we have a proposal of an
opt-in option per probe.


> > And those security concerns affect only GENERIC-HTTP not other
> > currently available measurements like DNS?
>
> For a typical DNS server, the "cost" does not depend on the request
> (at least for authoritative DNS servers). On the contrary, for HTTP,
> the cost can vary immensely from a static favicon.ico to a request
> involving many SQL statements.

Additional limitations may be "global destination based rate limiting"
or robots.txt parsing (User-Agent/request URI disallowed from
robots.txt -> stop there), although I believe that with the proposed
limitations botnets are already an order of magnitude more efficient
for an attacker.

However this is unrelated to whether or not HTTP measurements are
opt-in for probe owners or not, which is what my entire point was
based on.



thanks,
lukas



>
> > we are talking about small HTTP HEAD and GET requests here.
>
> The GET can be small but incurring a huge cost for the server.