Wednesday, July 10, 2013

Transparent Squid3 with DansGuardian Content Filtering

In many organizations today use varies kinds of open-source web controlling systems, this tutorial is focused on Squid and DansGuardian based  web content caching and filtering system installation and configuration on an Ubuntu base.

Squid is  a web caching proxy which supports HTTP,HTTPS,FTP, etc. It is licensed under GNU GPL and it also has an extensive access control system.  On the other hand DansGuardian is an open source web content filter which can filter any type of content based on pages, URLs, phrases even pictures. For more details can found on  Squid and DansGuardian.

On my example I’m going to show how to integrate a squid/dansguardian system as a transparent proxy system. As shown in the proxy is between the firewall and the local LAN.
Transparent Proxy


I have used certain IP ranges about they can be changed according to your requirement.
Once the setup is done then we have to install the Ubuntu latest server edition to the server, Make sure you use a server with two Gigabit Ethernet ports and it is better to have at least 16GB of RAM and 2GHz processor and a 1T HDD.

Then after installing the Ubuntu latest server edition, we have to install all required packages, this can be done by using apt-get but it should be updated first so as sudo:

sudo apt-get update
sudo apt-get install squid3 clamav-freshclam iptables dansguardian

After installing we have to edit the squid configuration file,
sudo vi /etc/squid3/squid.conf

On that make the changes as required…
....
http_port 3128 transparent
....
acl intranet src 172.16.0.0/16
acl localnet src 127.0.0.1/255.255.255.255
....
http_access allow intranet
http_access allow localnet
....
always_direct allow all
....
cache_dir ufs /var/spool/squid3 716800 256 256
....

You may have to change the intranet IP ranges and the cache directory sizes, also please refer the squid documentation.
After editing the squid.conf we need to restart the squid service.

sudo /etc/init.d/squid3 restart

After we do these we might need to log squid activities, but for squid3 to log we have to link its log paths to squid2.0 paths because most of the loggers and Ubuntu repository is configured as Squid2.

ln -s /var/log/squid3 /var/log/squid

After that before configuring the Dansguardian we have to download a blacklists from a site such as http://urlblacklist.com/?sec=download

Then,
tar -xzf bigblacklist.tar.gz
sudo mv blacklists /etc/dansguardian/blacklists
sudo chown -R root:root /etc/dansguardian/blacklists

When configuring you would likely want to customize some filtering options. In particular, you may want to comment (#) or uncomment options in these files:
  • /etc/dansguardian/lists/bannedextensionlist
  • /etc/dansguardian/lists/bannedsitelist
  • /etc/dansguardian/lists/bannedurllist
Then edit the conf file
sudo vi /etc/dansguardian/dansguardian.conf

...
#UNCONFIGURED - Please remove this line after configuration
...
filterip = 127.0.0.1
...
daemonuser = 'proxy'
...
daemongroup = 'proxy'
...
accessdeniedaddress = 'http://localhost/cgi-bin/dansguardian.pl'
...

Now configure the logging,
sudo chown -R proxy:proxy /var/log/dansguardian
sudo /etc/init.d/dansguardian start

Finally we have to configure iptables to redirect internet traffic
sudo iptables -t nat -A OUTPUT -p tcp --dport 80 -m owner --uid-owner proxy -j ACCEPT
sudo iptables -t nat -A OUTPUT -p tcp --dport 3128 -m owner --uid-owner proxy -j ACCEPT
sudo iptables -t nat -A OUTPUT -p tcp --dport 80 -j REDIRECT --to-ports 8080
sudo iptables -t nat -A OUTPUT -p tcp --dport 3128 -j REDIRECT --to-ports 8080
sudo iptables-save > iptables
sudo mv iptables /etc/dansguardian/iptables
/sbin/iptables-restore < /etc/dansguardian/iptables
sudo chmod 750 /etc/rc2.d/S60iptables


Also we have to add the following to the end of /etc/dansguardian/lists/exceptionextensionlist: (this will allow Ubuntu to download updates)

# Archives & packages
.bz2
.gz
.tbz2
.tar
.deb
.gpg


This is how I configured my network to several thousands of clients and hope this will help someone, enjoy...!!!

7 comments:

  1. ලෝකයේ දුටු නුදුටු දේ දකින්න, ලෝකයේ සිමා මායිම් හබා යන්න, අලුත් විදියට ලෝකය දකින්න වෙර දරණ, ඔබ වෙනුවෙන්ම නිර්මාණය වූ ලංකාවේ හොදම තාක්ෂණික ෆෝරමය, ℯkathuwa.org. එන්න අප හා එක්වන්න දැනුමෙන් සන්නද්ද වන්න. ඔබ තවත් මේ ලෝකයේ මං මුලාවුවෙකු නොවේ!

    ReplyDelete
  2. Squidblacklist.org is the worlds leading publisher of native acl blacklists tailored specifically for Squid proxy, and alternative formats for all major third party plugins as well as many other filtering platforms. Including SquidGuard, DansGuardian, and ufDBGuard, as well as pfSense and more.

    There is room for better blacklists, we intend to fill that gap.


    It would be our pleasure to serve you.

    Signed,

    Benjamin E. Nichols
    http://www.squidblacklist.org

    ReplyDelete
  3. There is room for better blacklists, we intend to fill that gap.

    We are the worlds leading publisher of Squid 'Native ACL' formatted blacklists, that allow for web filtering directly with Squid proxy. Of course we also offer alternative formats for the most widely used third party plugins, such as DansGuardian and Squidguard. And while our blacklists are subscription based, they are as a result of our efforts, of a much higher degree of quality than the free alternatives.

    We hope to serve you,

    --
    Signed,

    Benjamin E. Nichols
    http://www.squidblacklist.org

    ReplyDelete
  4. hello,
    Can you help me please i want to bloc the access to facebook (while
    using https and https) . I'm trying to do this with squid like a
    transparent proxy

    acl CONNECT method CONNECT
    acl FACEBOOK dstdomain .facebook.com
    acl FACEBOOK dstdomain .orkut.com
    #insert your own rule
    http_access deny FACEBOOK
    http_access allow all
    .............
    And finally i should redirect the traffic to squid with iptables but
    when i do it with this rule it seems to bloc all the internet traffic
    not only facebook

    iptables -t nat -A PREROUTING -s 192.168.20.0/24 -p tcp -j REDIRECT
    --to-port 3130

    I do really need help .

    ReplyDelete
  5. hello,
    Can you help me please i want to bloc the access to facebook (while
    using https and https) . I'm trying to do this with squid like a
    transparent proxy

    acl CONNECT method CONNECT
    acl FACEBOOK dstdomain .facebook.com
    acl FACEBOOK dstdomain .orkut.com
    #insert your own rule
    http_access deny FACEBOOK
    http_access allow all
    .............
    And finally i should redirect the traffic to squid with iptables but
    when i do it with this rule it seems to bloc all the internet traffic
    not only facebook

    iptables -t nat -A PREROUTING -s 192.168.20.0/24 -p tcp -j REDIRECT
    --to-port 3130

    I do really need help .

    ReplyDelete
    Replies
    1. Dear Lahmar,
      I have the same issue with facebook and other web sites which uses https to transfer their traffic. As its encrypted U cannot block facebook by its dstdomain. Anyway if you can find ip addresses of fb cdn then you may. But I am not recommending it. There are ways you can decrypt https traffic by a CA crt and filter. These are done in many UTM solutions. But I also failed with the previous method

      Delete