Short URL Providers RBL List

[Marc Perkel]

I don't know if it will be useful but I made a short URL provider list
that is DNS readable.

I got the list here:

http://longurl.org/services

It's a host name RBL and you can read it as follows:

dig tinyurl.com.shorturl.junkemailfilter.com

Let me know if you find a use for it.

[John Rudd]

There's lots of valid uses of those services, to just having a
blacklist for the providers doesn't seem, to me, to be useful.  I
think what would be more useful is a way for SA to expand the
short/tiny/etc. URLs in a way that lets SA check against the expanded
URL.



On Thu, Nov 5, 2009 at 07:01, Marc Perkel <marc@...> wrote:

> I don't know if it will be useful but I made a short URL provider list that
> is DNS readable.
>
> I got the list here:
>
> http://longurl.org/services
>
> It's a host name RBL and you can read it as follows:
>
> dig tinyurl.com.shorturl.junkemailfilter.com
>
> Let me know if you find a use for it.
>
>
>
>

[Mike Cardwell-16]

Marc Perkel wrote:

> I don't know if it will be useful but I made a short URL provider list
> that is DNS readable.
>
> I got the list here:
>
> http://longurl.org/services
>
> It's a host name RBL and you can read it as follows:
>
> dig tinyurl.com.shorturl.junkemailfilter.com
>
> Let me know if you find a use for it.

Been done. See http://rhs.mailpolice.com/#rhsredir

--
Mike Cardwell - IT Consultant and LAMP developer
Cardwell IT Ltd. (UK Reg'd Company #06920226) http://cardwellit.com/
Technical Blog: https://secure.grepular.com/blog/

[Marc Perkel]

Obviously you wouldn't block on this by itself. It is a piece of information that combined with other information might be useful in detecting spam.

John Rudd wrote:

There's lots of valid uses of those services, to just having a
blacklist for the providers doesn't seem, to me, to be useful.  I
think what would be more useful is a way for SA to expand the
short/tiny/etc. URLs in a way that lets SA check against the expanded
URL.



On Thu, Nov 5, 2009 at 07:01, Marc Perkel marc@... wrote:
  

I don't know if it will be useful but I made a short URL provider list that
is DNS readable.

I got the list here:

http://longurl.org/services

It's a host name RBL and you can read it as follows:

dig tinyurl.com.shorturl.junkemailfilter.com

Let me know if you find a use for it.




    

[Marc Perkel]

It's really not a black list. It's a lookup list. I'm just using RBLDNSD
to provide the list.

John Rudd wrote:

> There's lots of valid uses of those services, to just having a
> blacklist for the providers doesn't seem, to me, to be useful.  I
> think what would be more useful is a way for SA to expand the
> short/tiny/etc. URLs in a way that lets SA check against the expanded
> URL.
>
>
>
> On Thu, Nov 5, 2009 at 07:01, Marc Perkel <marc@...> wrote:
>  
>> I don't know if it will be useful but I made a short URL provider list that
>> is DNS readable.
>>
>> I got the list here:
>>
>> http://longurl.org/services
>>
>> It's a host name RBL and you can read it as follows:
>>
>> dig tinyurl.com.shorturl.junkemailfilter.com
>>
>> Let me know if you find a use for it.
>>
>>
>>
>>
>>    

[John Rudd]

On Thu, Nov 5, 2009 at 08:21, Marc Perkel <marc@...> wrote:
> Obviously you wouldn't block on this by itself. It is a piece of information
> that combined with other information might be useful in detecting spam.

and:

> It's really not a black list. It's a lookup list. I'm just using RBLDNSD to provide the list.

Neither of which changes the point of what I'm saying.

The point isn't: "should you block or not".  I never said anything
about that, one way nor the other.  As you have just stated,
blacklists get used for things other than just blocking.  My point has
nothing to do with blocking.

The point is: the URL shortening service isn't the interesting part of
the equation.  The expanded URL is.  So, telling people "this is a URL
shortening/compacting service" is really of minimal value, IMO.
Unless you then use that as a decision to try to expand the URL or not
("if it's in Marc's RBL, then try to expand it; if not, then don't").
And, again, that latter part is the main value: expanding the URL.

> John Rudd wrote:
>
> There's lots of valid uses of those services, to just having a
> blacklist for the providers doesn't seem, to me, to be useful.  I
> think what would be more useful is a way for SA to expand the
> short/tiny/etc. URLs in a way that lets SA check against the expanded
> URL.
> On Thu, Nov 5, 2009 at 07:01, Marc Perkel <marc@...> wrote:
>
>
> I don't know if it will be useful but I made a short URL provider list that
> is DNS readable.
>
> I got the list here:
>
> http://longurl.org/services
>
> It's a host name RBL and you can read it as follows:
>
> dig tinyurl.com.shorturl.junkemailfilter.com
>
> Let me know if you find a use for it.
>
>
>
>
>
>
>

[Jonas Eckerman]

Marc Perkel wrote:

> I don't know if it will be useful but I made a short URL provider list
> that is DNS readable.

Thanks. That could be usuable in my URLRedirect plugin. A current list
of URL redirectors is the main thing missing from that plugin.

It would be even better it included info about wether a URL shortener
uses HTTP redirects (wich is what my plugin checks). My plugin won't
check for frames, meta refresh or other "redirection" variants that
requires content to be fetched.

> Let me know if you find a use for it.

I've got a use for it. Now I just need to implement it as well. I'll
post here when/if I implement it in URLRedirect.pm.

For more info on URLRedirect.pm check at
http://whatever.frukt.org/spamassassin.text.shtml

Regards
/Jonas
--
Jonas Eckerman
Fruktträdet & Förbundet Sveriges Dövblinda
http://www.fsdb.org/
http://www.frukt.org/
http://whatever.frukt.org/

[Jonas Eckerman]

John Rudd wrote:

> The point is: the URL shortening service isn't the interesting part of
> the equation.  The expanded URL is.

If the service uses HTTP redirects it can be checked pretty cheep, wich
is what my URLRedirct plugin does. It adds the redirected-to URL to a
messages metadata so that other checks (URIBL for example) sees it.

Using a DNS based well kept after list of redirectors could make that
plugin much more useful than it is currently.

Regards
/Jonas
--
Jonas Eckerman
Fruktträdet & Förbundet Sveriges Dövblinda
http://www.fsdb.org/
http://www.frukt.org/
http://whatever.frukt.org/

[RW-15]

On Thu, 05 Nov 2009 20:05:25 +0100
Jonas Eckerman <jonas_lists@...> wrote:

> Marc Perkel wrote:
>
> > I don't know if it will be useful but I made a short URL provider
> > list that is DNS readable.
>
> Thanks. That could be usuable in my URLRedirect plugin. A current
> list of URL redirectors is the main thing missing from that plugin.
>
> It would be even better it included info about wether a URL shortener
> uses HTTP redirects (wich is what my plugin checks).

One other thing is that sometimes the links have already been cancelled
for abuse, and the redirection goes to a page saying that. Such pages
aren't going to be in any URIBL list, but obviously they are very
strong spam indicators.  Ideally there would be a regex to match those
links on each redirection service.

[Marc Perkel]

RW wrote:

On Thu, 05 Nov 2009 20:05:25 +0100
Jonas Eckerman jonas_lists@... wrote:

  

Marc Perkel wrote:

    

I don't know if it will be useful but I made a short URL provider
list that is DNS readable.
      

Thanks. That could be usuable in my URLRedirect plugin. A current
list of URL redirectors is the main thing missing from that plugin.

It would be even better it included info about wether a URL shortener 
uses HTTP redirects (wich is what my plugin checks). 
    

One other thing is that sometimes the links have already been cancelled
for abuse, and the redirection goes to a page saying that. Such pages
aren't going to be in any URIBL list, but obviously they are very
strong spam indicators.  Ideally there would be a regex to match those
links on each redirection service.

  

I love that idea. I thought this might go somewhere!

[Jonas Eckerman]

RW wrote:
> On Thu, 05 Nov 2009 20:05:25 +0100

>> Thanks. That could be usuable in my URLRedirect plugin. A current
>> list of URL redirectors is the main thing missing from that plugin.
>>
>> It would be even better it included info about wether a URL shortener
>> uses HTTP redirects (wich is what my plugin checks).

> One other thing is that sometimes the links have already been cancelled
> for abuse, and the redirection goes to a page saying that. Such pages
> aren't going to be in any URIBL list, but obviously they are very
> strong spam indicators.  Ideally there would be a regex to match those
> links on each redirection service.

Since my plugin adds redirect targets to the message metadata that check
could be done with a normal URL rule.

(It might be useful if the plugin flags redirections that redirects to
the same domain.)

/Jonas
--
Jonas Eckerman
Fruktträdet & Förbundet Sveriges Dövblinda
http://www.fsdb.org/
http://www.frukt.org/
http://whatever.frukt.org/

[Jonas Eckerman]

Mike Cardwell wrote:

>> I don't know if it will be useful but I made a short URL provider list
>> that is DNS readable.

> Been done. See http://rhs.mailpolice.com/#rhsredir

Thet don't seem to be the same thing. Quote from the page you linked to:
---8<---
This includes any website which provides an open mechanism to redirect a
web browser to another website, ie, by adding a
url=http://anotherwebsite in the URL.
---8<---

I would not consider open redirector of that type to be the same thing
as a URL shortener service (especially not a well run service).

/Jonas
--
Jonas Eckerman
Fruktträdet & Förbundet Sveriges Dövblinda
http://www.fsdb.org/
http://www.frukt.org/
http://whatever.frukt.org/