3 Key Problems Security Professionals Have With Traditional Penetration Testing

By: Rena Chua, Bug Bounty Advisor, HackerOne 

Penetration tests are a fundamental part of any security apparatus, but they’re traditionally seen as a one-and-done annual exercise. You hire a consultant, they run a pen test, you get a report. However, is that enough? With the rise of digital transformation, many companies are finding traditional penetration testing increasingly ineffective.

Penetration testing is a good baseline for evaluating system vulnerabilities and an industry best practice that supports routine security hygiene. Many companies also use pentests to pass vendor assessments and meet compliance standards like HITRUST, SOC 2 and ISO 27001. However, traditional pentests aren’t enough to demonstrate security effectiveness. 

Forrester Consulting conducted a “Total Economic Impact (TEI) Study“ on HackerOne Challenge, a time-bound hacker-powered security program. Based on customer interviews, the study identifies 3 key problems with traditional pentesting solutions and evaluates the benefits of time-bound testing using ethical hackers. 

1. Traditional pentesting missed critical vulnerabilities

Traditional pentesting was missing critical vulnerabilities and sometimes focused on irrelevant vulnerabilities that would not occur in a real attack, e.g., needing physical access to a machine. This left systems vulnerable and risked breaches that could result in large remediations costs, lost customers and revenue, and reputational damage. 

One participant of the study said: “[Pentesting] used to be a frustrating process. What they were finding wasn’t relevant. For example, they said the password was being exposed in the computer’s memory. Why does it matter? If you broke in and got physical access to the computer, you could put in a key-logger. They weren’t finding practical exploits.” 

2. Expanding attack surfaces leave security teams stretched thin

Fuelled by the COVID-19 pandemic, companies are rushing to meet remote work requirements and customer demands for digital services. As a result, attack surfaces have dramatically expanded, leaving security teams stretched thin and not staffed to cope.  

To illustrate this point, Marten Mickos, CEO of HackerOne recently shared: “Businesses realised that they have been too slow with their digital transformation and cloud migration. HackerOne research revealed digital initiatives had accelerated as a result of COVID-19 for 37% of security leaders in Singapore. Nearly 40% were forced to go through it before they were ready. The strain this puts on security teams is immense.” 

3. Creating an in-house bug bounty program would have been too labour-intensive

Several companies looked at creating their own bug bounty programs but concluded that building out the systems and processes internally would have been too costly and time prohibitive. One participant shared: “My predecessor had been thinking about using bug bounties. The prospect of doing it all on our own was daunting, especially when companies like HackerOne offer this service.” 

An Alternative to Traditional Pentesting: A Time-Bound Bounty Challenge

Just as bug bounty programs have proven the value of using a large and diverse set of hackers to identify security vulnerabilities on a 24/7 basis, those same hackers can bring their speed and expertise to bear over a set period of time in a time-bound bounty challenge — HackerOne Challenge.

A time-bound bounty program is repeatable as desired and combines structured testing with unstructured ethical hacking, targets designated systems and applications for vulnerabilities. Several of our customers have switched from traditional penetration testing to time-bound bug bounty challenges and one of the common pieces of feedback we get from customers is that they are getting much better results than traditional pen-testing and at a more cost effective price.

The Forrester TEI Study identified the following key results among customers using HackerOne Challenge: 

HackerOne found more vulnerabilities and provided better remediation recommendations

The most important benefit was finding more vulnerabilities, both in terms of numbers and criticality, in order to remediate them and create better system security. HackerOne found vulnerabilities across internal- and external-facing systems, websites, mobile applications, and internet-of-things (IoT) devices. 

Ethical hackers’ creativity was a major reason more vulnerabilities were uncovered. One participant stated: “We had some vulnerabilities found in our first HackerOne Challenge. We learned a lot from the triaging and created a remediation cycle for high and critical vulnerabilities.” Another participant explained that their “pen tests used to be limited by the skill level of the assigned team. Sometimes they get very settled in how they approached a problem. When you bring in the crowd with HackerOne, you have different perspectives and better results.” 

Companies can deliver code faster

Customers eliminated vulnerabilities in development and delivered secure applications to their production environment. HackerOne leverages familiar industry standards — (ISO/IEC 15408) and CVE (Common Vulnerabilities and Exposures) — and applies them with the angle of identifying software vulnerabilities at every phase of the software development life cycle. 

By applying security analysis and monitoring at every step, customers were able to ship secure code faster, as one participant indicated “We would need a lot more time to complete on our own what we do with HackerOne.” 

The effort to manage pentesting decreased

HackerOne manages the vulnerability testing process to reduce customers’ effort and provide results faster. To illustrate, one participant said: “Traditional pen tests can be very expensive and take months. You can get past a lot of that with HackerOne.”

In summary, the security landscape is ever evolving and organisations will have to step up their game. 

Traditional pentesting is no longer sufficient in today’s modern digital world. Focused, time-constrained security testing using the creativity of hackers (HackerOne Challenge) helps organisations find more relevant vulnerabilities and ship secure code faster. With hacker-powered security, organisations pay only for found and validated vulnerabilities, and hackers bring nearly unlimited diversity of skills, approaches, and experience. In other words, organisations get an army of hackers eager to uncover and report bugs of all types.