Multiple Servers for Optimum Filtration

This guide with cover a deployment concept of using multiple server working in different roles to cover a variety of infrastructure and BYOD requirements to provide holistic filtration for a network. This requires a minimum of two ClearOS server but can be done with one so long as your default firewall/gateway has similar features to perform the role.

Transparent and Non-Transparent filtration

ClearOS provides a multitude of configuration option. Each with their advantages and disadvantage. To get the most out of both options, we will deploy both to work in conjunction with each other. This combines their strengths and removes their weaknesses.

Transparent filtration

Transparent proxies can filter non-SSL traffic but cannot filter SSL traffic without violating sound security principles. The only way to filter SSL without manipulating the workstation is block HTTPS. Moreover, they cannot negotiate with the web browser for authentication without employing a captive portal. They are limited to identifying the user by IP only as well. Because they can only identify by IP, they can only categorize traffic by the IP source.

However, they are straight-forward and fairly turnkey. They are great for Bring Your Own Device (BYOD) situations.

Non-Transparent filtration

Non-transparent filtration is great for identifying users, working with authentication models in the browser, filtering HTTPS and HTTP sites based on URLs. Reporting is more precise and because you can identify the user, you can classify them differently, even on the same machine.

However, they are lousy at ease of use. Non-transparent filtration required configuration of the browser. This can be mitigated by technologies like WPAD but there may exist issues with automatic detection.

Best of Both

Using both you can ensure filtration of all the traffic that leave your network.

Transparent Box

Your transparent box needs to pass port 80 traffic outbound and redirect it to your transparent proxy server. This happens automatically on ClearOS when you select transparent mode. Additionally, you will block all other traffic going out except for traffic that you approve. This will block traffic that is not filtered. The obvious exception to this is to allow the second box to navigate ports out.

BYOD device, devices not configured and services which don't support proxy settings will be able to surf the internet without settings. They won't be able to navigate https sites and you are limited to categorizing everyone the same way or by IP groups.

Non-Transparent Box

For the non-transparent box we can do the full treatment of user authentication, filtration policy groups, filtration of SSL traffic and others. To make this easy, we will set up WPAD on the network and a PAC file.


This section will help you seamlessly deploy the solution in such a way to maximize your filtration and minimize the downtime and downside risks in deployment. The will ease your environment into a filtered mode and allow your users to adjust to the changes. Feel free to bypass any steps as appropriate to your environment.

  • 1) Set up ClearOS as your gateway. ClearOS is a valuable tool in content filtration even without the content filter services. Out of the box, and running as only a firewall, tools on ClearOS can help you analyze user traffic over periods of time or in realtime.
  • 2) With ClearOS in place as the gateway, engage the transparent proxy. This will allow your traffic to process through the proxy. This can save you costs on bandwidth and speed up your connection if properly configured. You can also see more detail about user traffic now using the proxy report. Or you can monitor user connections to the internet live.
  • 3) Turn on Filtration on the gateway. With filtration on, you will start to automatically block sites that are generally bad on the internet. If the filter is blocking too much at first, you can ease up on the policy or you can even put the filter into an allow all mode that will still log everything and tell you what it would have done but still permit the traffic. The reports from this give you a great view of what is happening on the http traffic but will not yet cover https.
  • 4) With the gateway filtration in place, you can implement other things like filtration groups by IP address exempted users and machines versus filtered one. You can even filter viruses and malware now. This is the way that your filtration will look like for BYOD devices.
  • 5) On a separate ClearOS server (or in a virtual machine) we will run our non-transparent proxy and filtration. For users to use this proxy they will need settings in their browser. At this point you are only testing and you can manually put those settings in your test workstation to try it out. Validate that your authentication works (whether using the directory that is builtin or the AD Connector) if you are using authentication. Validate that your policies work (if using authentication and policies). And validate that reporting is working.
  • 6) Optional and Recommended - Autodetection of Proxy: To set up auto-detection of the proxy server, we recommend a blanket approach. Do all the things (if available). Set up a website on premise that can serve up your wpad.dat file. Follow the WPAD guide to help you maximize the WPAD configuration. If you have Active Directory, set up a Group Policy Object (GPO) that will direct your browsers to this file. To minimize downtime, customize your file to only set your workstation through the proxy and send others DIRECT. Doing this will allow you to ease in the users at a future time or little by little. You can also load balance your users through a plurality of proxies if you need higher resiliency or if your user count is high.
  • 7) Once your proxy is in place and working well with your test environment, begin migrating your devices to use the proxy by having the proxy server service groups of users either through modifications of your WPAD file, changes in GPO, or manual changes of groups of computers. This will allow you to place on users and back them off in case there is an issue or a misconfiguration.
  • 8) Once all your users are in place and you have a good percentage of the BYOD crowd either auto-detecting the proxy or satisfied with just HTTP, you can pull the plug on the firewall for non-filtered traffic. Now all web traffic is either HTTP through the gateway or HTTP/S though the non-transparent proxy.

What is nice about this method for migration is that it allows for an easy transition from a wide open network to a completely monitored and filtered one. If done properly with good testing and with patience, your users not even notice. search?q=clearos%2C%20clearos%20content%2C%20kb%2C%20bestpractices%2C%20maintainer_dloper%2C%20maintainerreview_x%2C%20keywordfix&btnI=lucky

content/en_us/kb_bestpractices_multiple_servers_for_optimum_filtration.txt · Last modified: 2014/12/23 23:05 by dloper