Penetration Testing FAQ

Buying security assessment services can be confusing. There is little in the way of standardisation within the industry, and the costs, quality and value offered varies considerably. The following represents the most common questions we are asked:

Vulnerability assessment, penetration testing or ethical hacking - What's the difference?

In general, the above terms are used interchangeably within the industry, although it is always a good idea to clarify the supplier's perception of the term used to ensure you are comparing like for like offerings. Although they are generally interchangeable, there are semantically some differences between the vulnerability assessment and penetration testing/ethical hacking: Vulnerability assessments tend to be performed using automated scanning tools. These tools when used in isolation however have a number of limitations, not the least of which is the inability to exploit the potential vulnerability, to confirm its presence and demonstrate the real-world risk associated with its exploitation.

Penetration testing and ethical hacking will normally provide a number of important additions: Firstly, a range of tools and technologies will be used. Secondly, potential vulnerabilities will normally be exploited to confirm their existence, and simulate a real attacker more closely. Not all issues can be exploited (for example, some require very specific scenarios or actions by third parties to be exploitable) but the vulnerabilities’ existence will be proved/disproved as far as reasonably possible.

Ask any potential suppliers the following questions:

  1. Will you be manually enumerating public information sources for information useful to a potential attacker and using this information during testing?
  2. Will you be using a variety of tools and technologies to test systems?
  3. Will you be manually verifying the output of any automated scanning solutions to confirm the presence of potential vulnerabilities?
  4. Will you be performing manual analysis on web applications within the project scope?
  5. Will you be exploiting vulnerabilities as far as reasonably possible to simulate a real-world attack?
  6. Will you be including screenshots, supporting evidence etc, of compromised systems in the report?

If the answer to any of these questions is a No, you are probably buying what would be considered by most UK providers as a vulnerability scan and not a penetration test (unless this is due to a project requirement).

Where will the test be performed from?

This depends on the test scope. If the scope of systems is entirely external, Internet visible systems then the test will normally take place from the provider's testing facility remotely. Tests involving internal, non-Internet visible systems will normally be tested at the client’s premises as required. Testers will need the ability to connect their testing platforms (normally laptops) to the target system/network, a physical space to work and a steady supply of coffee as a minimum! Internet access can also be useful. Consider the potential impact of a third-party system connecting to the test environment before commencement of the test as this will likely contravene your security policy and may need specific authorisation.

How long will the test take?

This is an impossible question to answer, although accurate estimates can normally be made for a specific project based on previous experience. Testers may quickly find a number of serious, exploitable issues or may spend a considerable amount of time attempting to exploit an obscure anomaly in a particular web page. What's really important is that the client is kept abreast of the test progress and immediately alerted of any high-risk issues, problems or other potential issues that could affect project delivery.

Is there potential to negatively impact the availability of systems or data?

In short, yes! Anyone that can guarantee you otherwise should be questioned. Consider what penetration testing really is: It is placing systems into situations and scenarios that the developer did not envisage. Vulnerabilities arise because of unexpected or unconsidered input. Whenever you have a system outside an anticipated situation, you have the potential for unexpected repercussions.

That said, it is very unusual to negatively impact systems in any noticeable way. Experienced testers will know the common issues and have mitigating strategies. The most common issues are caused by:

  1. Systems with insufficient spare capacity. Testing will exert an additional load on systems. This is perfectly normal and carefully managed as part of the test. However, if the system has no spare capacity or is prone to "falling over" then testing will exaggerate the issue. Memory leaks are a common example of this.
  2. Insufficient bandwidth. If your WAN/Internet links are heavily utilised with little spare capacity, this once again will exaggerate the issue.
  3. Some legacy systems can respond badly to testing and need a reboot to continue normal operation. These include outdated UNIX platforms, older turnkey appliances, and telephone systems. If you have anything of concern, flag it up as soon as possible in order to discuss risk mitigation strategies.

Any kind of permanent damage or damage requiring a restore is extremely unlikely. Also, don't forget, a real attacker sitting halfway around the world won't ask before investigating your network.

What will I see in my system logs?

Firewall logs will likely show a large number of connection requests on odd port numbers, this will be part of the initial port scanning. Web servers and application logs will show large numbers of 404 file not found errors, associated with a large number of strange looking URLs. It’s quite common for even a basic URL scan to generate in excess of 20,000 requests, so don't be too alarmed. Intrusion Detection/Prevention systems should provide a number of alerts on various attack attempts being attempted. The tester's IP addresses should be supplied to the client upon request in order to differentiate between the legitimate test and other potential attacks.

Should I disable my IDS/IPS system for the penetration test?

This is a question we are increasingly asked, and to be honest, there is no right or wrong answer. Testing under certain standards (PCI DSS for example) mandates that Intrusion Prevention systems be disabled for the duration of the test for the IP addresses in use by the penetration testers. Outside of these mandates there are two schools of thought:

  1. The IDS/IPS is a legitimate layer in the purchasing organisation's defence, and so should be present and operational during the test as this is the most realistic scenario.
  2. That the IDS/IPS provides an additional layer of protection, but should not be relied upon solely, and that disabling the IDS/IPS for the testing team's IP addresses will allow a greater number of underlying vulnerabilities to be identified.

Sec-Tec are happy to discuss the pros and cons of each perspective.

How much information should I provide to the provider?

Once again, the only answer can be whatever you feel comfortable with. Clients tend to have an opinion on this. Some customers furnish the testing team with comprehensive network diagrams, passwords to simulate "normal users" (to see if the privileges associated with the account can be escalated) and any other information. Some prefer to provide only the minimal information and rely on the penetration testers to discover the information for themselves. The vast majority of clients, however, are somewhere between these two extremes.

For more information :

Sec-Tec Penetration Testing Services Page