URL filtering is a feature of most Next-Generation Firewalls (NGFW) and some Web Proxies. It compares web traffic against a URL filtering database to block employees from reaching malicious phishing sites and potentially other undesirable Internet locations such as gambling sites, adult sites, etc. URL filtering works well, but the approach has a major flaw when it comes to protecting employees against today’s fast-moving phishing attacks and newer zero-hour threats.

URL Filtering Flaw

URL filtering databases contain information on known threats as well as domain categorizations to govern employee web traffic. This filtering can indeed protect users from known malicious sites. However, with today’s phishing sites typically only lasting a few minutes to hours, URL databases must be continually updated with real-time phishing threat feeds in order to protect users from newer emerging threats. Thus, the freshness and accuracy of real-time threat feed data is key to its effectiveness. Even if URL filtering databases are updated every five minutes, it can still take several hours to several days to implement the threat feed data—which is common with human-vetted threat intelligence. By that time, it’s already out of date and won’t protect users from newer, live, zero-hour threats. This flaw leaves organizations and their employees exposed to a wide variety of fast-moving phishing and social engineering threats that involve URLs.

What to Do

To protect users from new phishing sites, many organizations augment the filtering databases provided by their NGFW or Web Proxy vendor with threat feeds from third-party specialist vendors. This is feasible since most NGFWs can ingest third-party threat feeds, also known as blocklists. To optimize a system’s effectiveness at blocking newly discovered threats, IT security staff need to do three things:

• Update URL filtering databases as frequently as possible, ideally in real-time or every five minutes.
• Employ high levels of automation to continually access and quickly operationalize real-time phishing threat feed data. Even a one-hour delay from first ingestion to filtering operationalization leaves organizations exposed to a large number of newly emerging threats.
• Employ smart policies about blocking individual URLs vs. entire domains. This is important because an increasing number of phishing pages are hosted on compromised websites or legitimate infrastructure, where blocking entire domains may disrupt business operations and/or user productivity.

What to Look for in Phishing URL Threat Intelligence Feeds

The breadth, freshness, and accuracy of phishing threat feed data are all important factors for making URL filtering effective, especially against newly discovered zero-hour threats. So, when evaluating third-party phishing threat feeds or blocklists, be sure to consider the following:

• Human vs. Automated Threat Detection: Is the threat detection that’s powering the threat feed fully automated or are humans involved? Human-vetted threat intelligence incurs delays that make it less effective for blocking newer, short-lived threats. Fully automated detection provides faster, real-time threat data.
• Accuracy: How many false positives are in the feed? Ideally, you want zero false positives to avoid filtering or blocking traffic to legitimate sites. However, many threat feeds have a relatively high false positive rate, over 10%-20% in some feeds. Carefully evaluate each vendor’s methods for detecting phishing URLs and explicitly test for false positives.
• Live vs. Dead URLs: How often are URLs re-checked, and if no longer active, removed from the feed? To keep the focus on blocking live threats, and to keep frequent blocklist updates at a manageable size, look for phishing threat feeds powered with automated URL re-checking and retirement.
• Refresh Frequency: How often is the threat feed refreshed? Consider threat feeds that are continually refreshed in real-time or every few minutes. Threat feeds that are not refreshed more frequently than once an hour means they will not keep up with many newly discovered threats.
• Breadth: How many new phishing URLs are added per day? And what types of phishing threats are covered? Many phishing threat feeds only offer a few hundred to a couple thousand new phishing URLs per day, and most are focused only on fake log-in pages for credential stealing. But there are tens of thousands of new phishing pages going live each day. Look for phishing threat feeds with larger numbers of new daily detections and that cover all major kinds of phishing threats, including rogue software, phishing exploits, social engineering scams, and command and control servers (C2s) used for phishing callbacks and data exfiltration.
• Machine Readable: Is the threat feed available as machine readable threat intelligence (MRTI)? Since both automation and speed are critical to making URL filtering effective for blocking new zero-hour threats, look for threat feeds that are machine readable and support the data formats you need for your security infrastructure, such as NGFW, Threat Intelligence Platform (TIP),or Security Orchestration, Automation and Response (SOAR) platform.
• Trials: Can you get a free trial of the threat feed? There is no better proof of the breadth, freshness, and accuracy of a threat feed than to get a free trial and run your own tests. Make sure your vendor can offer you a trial for in-house testing and evaluation.

SlashNext Phishing Threat Intelligence Feeds

SlashNext Real-Time Phishing Threat Intelligence feeds are powered by SlashNext award-winning SEER® detection technology. By using browsers in a purpose-built cloud, SlashNext is able see through the growing variety of URL obfuscation techniques and evasion tactics, including shortened links, URL redirects, multi-stage attacks, phishing pages hosted on compromised websites, and those hosted on legitimate cloud infrastructure. SlashNext dynamically inspects page contents and server behavior in real-time to accurately detect phishing sites that evade URL filtering and domain reputation analysis tools with near zero false positives. With more powerful real-time phishing site analysis, plus a widely sourced network of suspicious URLs to evaluate, SlashNext detects previously unknown phishing URLs that may avoid detection by other systems or never be found in any other commercial or free phishing URL threat feeds or databases.

Want more? You can try SlashNext Real-Time Phishing Threat Intelligence free for 15 days.