Domain categorisation should we well known within the Blue Team, but may be lesser known within the Red Team. It can be a useful defense mechanism used by next generation proxies and web based security gateways when used correctly, but it can also be used by Red Teams or attackers to falsely categorise a website in order to bypass the security polices on these next generation devices.
The question is:
Can you trust domain categorisation, and how can you manipulate it?
What is Domain Categorisation
An online definition describes domain/website categorisation as:
A website category lookup is a web service used to determine the categories of a domain name. Domains are classified using a combination of machine learning (ML) and human inspection. Web crawlers collect and organise website content, which ML algorithms then examine and categorise. Human experts verify the results.
Why Domain Categorisation is useful
Domain categorisation can often prove as an effective defense against many Red Teams. Red Teams may be unexperienced and not know about domain categorisation or as new domains are always uncategorised these Teams are merely lucky when tackling corporate proxies. However, more mature environments are highly likely to restrict the categories that can be accessed to only those that are most trusted: typical sectors are Finance, Government, and Utilities. If a Red Teams phishing or C2 domain is blocked due to categorisation by the proxy it often means the end of that campaign unless by chance it lands on a user outside of the coporate network controls or with a less restrictive policy applied, eg. a mobile provider or 3rd party WiFi ISP.
We previously mentioned Domain Categorisation during our Darkside of Red Teaming presentation, the slides can be found here
Domain Categorisation Links
Below are a number of useful links for checking/submiting domain categorisation:
- Bluecoat/Symantec - https://sitereview.bluecoat.com/sitereview.jsp
- McAfee - https://www.trustedsource.org
- Palo Alto Wildfire - https://urlfiltering.paloaltonetworks.com
- Websense - https://csi.forcepoint.com & https://www.websense.com/content/SiteLookup.aspx (needs registration)
- Fortiguard - http://www.fortiguard.com/iprep
- IBM X-force - https://exchange.xforce.ibmcloud.com
- F-Secure SENSE - https://www.f-secure.com/en/web/labs_global/submit-a-sample
- Checkpoint - https://www.checkpoint.com/urlcat/main.htm (needs registration)
- Squid - https://www.urlfilterdb.com/suggestentries/add_url.html
Domain Categorisation and Malware
Recently, Netscylla has observed some strange categorisations where sites that were previously marked as malware/phishing are suddenly re-categorised as one of the following:
- social media / media website
- finance website
- e-commerce / business website
- gambling & adult content
This is an obvious blatent attempt of organised phishing/malware groups likely abusing the above links to re-categorise their domains as an attempt to bypass security gateways.
Therefore, the security staff at Netscylla have thought its useful to highlight the topic of Domain Categorisation, to help security teams battle the never ending phishing and malware attempts that have risen in recent months. It is always useful to have the above links handy: and to check and re-submit a categorisation reports in addition to submitting reports on the Netcraft reporting tool to aid in the take-down of these dodgy domains/websites for the benefit of all: