Threat Modelling Evaluation
After we have fixed the threats and assessed the risks around them, we will need to determine whether our efforts are working. It is essential to test the applied fixes to gain confidence in the measures taken against previously identified threats. This will be the objective of the threat modeling validation process. Validation will require testing. Testing refers to a critical function performed by “quality assurance” testers: creating and managing tests. Testing will require the ability to understand issues found when looking for denial of service threats, manage repudiation, look for structured query language (SQL) injection, use fuzzing, and create or manage tests that are not based on threat modeling. The threat modeling validation process will be carried out in three phases: validation of threat model, validation of the quality of threats and mitigations, and validation of the information captured.
Objective of Validation
The validation process aims to establish the effectiveness of mitigations. In other words, we intend to validate that the threats are addressed. This is a crucial step in the threat modeling process.
Validation of the threat model
The first process in evaluating the cyber threat model’s effectiveness will involve validating the whole threat model. First, we will determine whether the diagram matches the final code. By doing so, we will establish whether the threat model’s processes are well-placed to mitigate the threats. Secondly, we will establish whether the threats are enumerated. The minimum requirement for this process will be the determination of STRIDE for each element that touches a stride boundary.
We will also determine whether the testers or quality assurance (QA) have reviewed the threat model. At some point, it may be necessary to deliver or deploy the software; thus, quality assurance will be crucial. Like everything else in software development or the deployment of systems, threat models can be subjected to a quality assurance process. It’s important to “close the loop” and ensure that threats have been appropriately handled. The quality assurance approach often reveals problems with threat models. As we get close to finishing, we will want to confirm whether the threat model works. We will use a checklist to ensure that all items in the threat list are addressed, the model is close to reality, and that all bugs are closed. Checking the model’s conformance to reality will indicate whether the list of threats found is relevant to what we build. We will also check whether the threat model mitigates all threats and whether all mitigations are done properly. All these items will be checked before the final security review.
Validating the quality of threats and mitigations
The second stage of the threat model validation process is the validation of the quality of threats and mitigations. We will determine whether the identified threats describe the attack, the context of the attack, and its impact. In this way, we will ensure that the threat model addresses all possible vulnerabilities.
We will also validate the mitigations. Particularly, we will determine whether the mitigations are associated with specific threats, whether they are described, and whether they file bugs. For instance, fuzzing is not mitigation. Instead, it is a test tactic. We will also test mitigations by further causing bugs to trigger and creating tests to see whether the mitigations are working. These tests will either be automatic or manual. Since the level of expertise needed to code automated tests is high, we will mainly opt for manual tests. However, since the preferred bugs do not regress, automation is the best option. We will include variations of the attack to make the testing more effective. In developing tests for threats, we will consider how mitigations can be attacked. Testing will call for expertise in shell coding or cryptography. Shell coding is the creation of user interfaces for accessing an operating system’s services. Therefore, testers might require skills in creating command-line or graphical user interfaces. Cryptographers are skilled in using codes to protect communications and information. Their expertise will be needed to protect information adequately during testing.
As we get closer to completion, we will review mitigation test bugs to ensure that they have been closed. This is where tags such as the tmtest will be helpful. Where some bugs will not have been closed, we will triage them like other bugs, with appropriate attention to their gravity. We will fix the bugs, or if necessary, move them to a later revision stage.
Validation of Information Captured
Software dependencies are relationships among code that exist as a result of external influences on the software development process. Since they affect the development of systems, we will also validate them. Basically, we will determine other code being used, the security functions embedded in the code, and assumptions made during the development of the threat model. For instance, we may have assumed that GenRandom will give us crypto-strong randomness. This is a valid assumption. On the other hand, we may have assumed that protection from malformed messages will be obtained from linear predictive coding, which is not true. These examples show the importance of validating our assumptions to ensure the threat model’s effectiveness.
Outputs of the Validation Process
The main output of the validation process is the effectiveness of the threat model. After evaluation, we will establish what threats the threat model works against, the model’s areas of weakness that need improvement, and the impact of threats on the model.
References