Penetration Testing: What is it?

This blog will discuss the meaning of Penetration Testing. A requirement that applies to all systems categorized as HIGH within the U.S. Government Sector and all 3rd Party Service Cloud Providers systems categorized as MOD and HIGH who must comply with FedRAMP. It addresses the requirements noted under NIST's 800-53 Control CA-8. See

Penetration Testing includes a combination of unique processes and methodologies that are followed by different teams [Blue & Red Teams]. In fact, Penetration testing is split within the two (2) components of Cybersecurity: (I) Defensive [Blue Team]; and (ii) Offensive [Red Team]. In this blog we will provide a general overview of the various options and how they differ from the general Security Assessment that falls within the Defense In-depth Component under Information Assurance.

What is Penetration Testing?

It is security testing in which assessors mimic real-world attacks to identify methods for circumventing the security features of an application, system, or network. It often involves launching real attacks on real systems and data that use tools and techniques commonly used by attackers. Most penetration tests involve looking for combinations of vulnerabilities on one or more systems that can be used to gain more access than could be achieved through a single vulnerability.

Who is required to perform a Penetration Test?

Any organization, that is concerned about cyber attacks should perform at least one Penetration test on systems deemed to be mission critical to identify deficiencies on their IT Security Program. But we understand that budget is restricted for many, therefore, this applies to:

  • Organizations that wish to comply with FedRAMP requirements in order to sell services to the government.
  • Organizations that fall under the Government sector, especially government contractors who process government data within their network.

The organizations noted above must performed a penetration test annually for each individual system accredited under FISMA or FedRAMP. The requirements are as follow:

  • FISMA Accredited Systems- applicable to systems categorized as HIGH and required annually. In regards to MODERATE systems, the test isn't always mandated and testing can be ad-hoc in accordance with organizational policies.
  • FedRAMP Accredited Systems- applicable to each individual systems that is categorized as MODERATE or HIGH. The test must be performed annually in order to remain accredited.

Keep in mind, that the requirements noted above have an impact on the budget, as it doesn't address the cost of implementing all other controls.

Why would a Penetration Test be useful?

In addition to identifying vulnerabilities that may be linked to a specific vulnerability, the test can help identify the following:

  •  How well the system tolerates real world-style attack patterns.
  • The likely level of sophistication an attacker needs to successfully compromise the system.
  • Additional countermeasures that could mitigate threats against the system.
  • Defenders’ ability to detect attacks and respond appropriately.

Penetration testing can be invaluable, but it is labor-intensive and requires great expertise to minimize the risk to targeted systems. Systems may be damaged or otherwise rendered inoperable during the course of penetration testing, even though the organization benefits in knowing how a system could be rendered inoperable by an intruder. Although experienced penetration testers can mitigate this risk, it can never be fully eliminated. Penetration testing should be performed only after careful consideration, notification, and planning.

What techniques fall under the Offensive Component of Cybersecurity?

The techniques under the Offensive Component incorporate internal and external testing. Each one requires a different level of knowledge on the system. The techniques noted below must be performed by a 3rd Party Service Provider not associated with the organization. They are as follows:

Technique Description
Overt aka White Hat Testing This methodology involves performing external and/or internal testing with the knowledge and consent of the organization’s IT staff, enabling comprehensive evaluation of the network or system security posture. Because the IT staff is fully aware of and involved in the testing, it may be able to provide guidance to limit the testing’s impact. Testing may also provide a training opportunity, with staff observing the activities and methods used by assessors to evaluate and potentially circumvent implemented security measures.
Covert aka Black Hat Testing This methodology takes an adversarial approach by performing testing without the knowledge of the organization’s IT staff but with the full knowledge and permission of upper management. Some organizations designate a trusted third party to ensure that the target organization does not initiate response measures associated with the attack without first verifying that an attack is indeed underway (e.g., that the activity being detected does not originate from a test). In such situations, the trusted third party provides an agent for the assessors, the management, the IT staff, and the security staff that mediates activities and facilitates communications.

This type of test is useful for testing technical security controls, IT staff response to perceived security incidents, and staff knowledge and implementation of the organization’s security policy. Covert testing may be conducted with or without warning.

The purpose of covert testing is to examine the damage or impact an adversary can cause—it does not focus on identifying vulnerabilities. This type of testing does not test every security control, identify each vulnerability, or assess all systems within an organization. Covert testing examines the organization from an adversarial perspective, and normally identifies and exploits the most rudimentary vulnerabilities to gain network access.

If an organization’s goal is to mirror a specific adversary, this type of testing requires special considerations—such as acquiring and modeling threat data. The resulting scenarios provide an overall strategic view of the potential methods of exploit, risk, and impact of an intrusion. Covert testing usually has defined boundaries, such as stopping testing when a certain level of access is achieved or a certain type of damage is achievable as a next step in testing. Having such boundaries prevents damage while still showing that the damage could occur.

What techniques fall under the Defensive Component of Cybersecurity?

The Defensive Component Team that is typically engaged on Penetration Testing is known as the Blue Team. The Blue Team is composed by assessors who may also be part of the Defense In-depth component, but that are trained by the organization to exploit vulnerabilities internally and sometimes externally on systems deemed to be mission critical. The process is very similar to the Overt aka White Hat process noted under the Offensive Component, but a little less in-depth.

How do the above techniques differ from the Defense In-depth Assessment Process?

The assessors under the Defense In-depth components focus mainly on ensuring that the systems under development can transition into the Accredited List of systems by ensuring the vulnerabilities found comply with the organization's Risk Appetite Level and that such risk is maintained throughout the Systems Life Cycle as part of Continuous Monitoring. Their focus is to find vulnerabilities and/or deficiencies of controls impacting the operation of such under the organization, not to go into in-depth testing of system to determine if a specific vulnerability can indeed be exploited. The techniques leveraged, while technical are meant to simply validate compliance against organizational policies and procedures.

The main focus... to identify vulnerabilities and keep track of mitigation by retesting the controls deemed to be deficient to ensure they were fixed to comply with policy requirements. On the other hand, the focus of the components under Cybersecurity is to exploit the organization's system for a specific vulnerability and/or mimic an external attack to identify deficiencies associated with failures in the organization's IT Security Program.

What's the cost?

The cost can be high, as it is per system. Overt testing is less expensive, carries less risk than covert testing, and is more frequently used—but covert testing provides a better indication of the everyday security of the target organization because system administrators will not have heightened awareness.

There's some flexibility on the Overt test when it comes to the teams requirement, as the organization could train a set of individuals internally to perform the test from within as long as they are NOT involved on the Engineering and/or development of the systems being tested.

The staggering cost can be as high as 50K per week with a minimum of two weeks engagement. Certainly, there's much to consider when engaging a 3rd Party Service Provider who claims to do penetration testing. It is not a simple scan. There's much more in-depth requirements that must be in place for it to be valid.

How can CyberAdeptness help?

We don't really focus on Offensive Security and our focus on Defensive is minimal. From the Defensive Component, we can make recommendations on the approach and the tools to use based on organizational specific needs impacting what is deemed "MISSION CRITICAL". Something that we also help identify as part of the Enterprise Level Risk Assessment.

Our main focus lies on the implementation of a Risk Management Framework and/or improvement of an existing framework and the controls applicability from a Defense In-depth point of view, so that the test performed by Defensive and Offensive security teams either fails and/or is restricted within a specific layer in the network and easily detected before damage can be done.

However, if your organization wishes to find someone to perform the test, we highly express caution as to who you use to perform testing. There's many companies that claim to offer Penetration Testing but they don't really comply with the requirements nor do they perform a true test. A vulnerability scan is not a penetration test. To be on the safe side, make sure the provider meets or exceeds the following:

  • At least one in the team has Technical hands-on Expertise with a minimum of ten (10) years hacking environments.
  • Technical personnel must have python or other programming knowledge in order to develop scripts that are uniquely designed for the organization. [If OVERT]
  • Provides a list of the documentation that will be provided prior to and after the engagement.
  • Provides a clear list of tools to be used and that address the technology being assessed. [If OVERT].
  • Clearly denotes the risk associated with each technique used and how the organization must prepare in the event the system tested is damaged or taken down.

Certainly there's more items to keep in mind, but the above is the minimum the organization should require. If the organization needs help recruiting a team to perform this task, we will gladly provide guidance to help in the decision making process. We have the knowledge and a more in-depth list of requirements, but we simply don't offer it as an actual service.

Interested on learning more? Contact US via our site form at 

NOTE: Due to security concerns, we limit our web content to bare bones. Please note that you will receive a response within forty-eight (48) hours or less.

By Karen Baez | on Friday, November 9 2018 19:47


1. Shana

Saturday, April 20 2019 | 01:15

Hi there! I know this is kinda off topic nevertheless I'd figured I'd
ask. Would you be interested in exchanging links or maybe guest authoring a blog post
or vice-versa? My website covers a lot of the same topics as yours and I think we could greatly benefit from each
other. If you happen to be interested feel free to shoot me an email.
I look forward to hearing from you! Superb blog by the way!

Add a comment

HTML code is displayed as text and web addresses are automatically converted.

They posted on the same topic

Trackback URL :

This post's comments feed