Testing Methodology
1. Understand the Security Requirements
The security requirements of a product drive how resistant a product should be against attacks. Before we evaluate the security of a product, we aim to have a good understanding of its security requirements. In case the security requirements have not yet been specified, we work with the customer to define the desired security level for the product.
2. Identify the Assets
What are the valuable assets for this product in relation to the security requirements? In this activity, we determine what the assets are (e.g. authentication key, PIN, transaction counter, firmware code), and where they are located in the product. Often, we define so-called primary and secondary assets. A primary asset is critical to the security, whereas a secondary asset can often be used as a stepping stone towards a primary asset.
3. Understand the Design
In this activity, we analyse the available design documentation, manuals and the information available on the internet on the product and its components. The more detailed the design documentation, the more effective we are in finding potential weaknesses. In a “black box” evaluation we work with very little information on the product so that we will have to use reverse engineering techniques, whereas in a “white box” evaluation, we are provided with detailed design information and source code of the product.
4. Threat Analysis
Now that we understand the product and its security requirements, we identify the threats that are relevant to this product. Different attack paths towards the assets are analysed and rated in relative terms of difficulty and cost for an attacker.
5. Select Tests
Selecting the appropriate tests is critical in each security evaluation project because one can never execute all possible attack scenarios. The experience of our people ensures that the best possible selection is made based on the Threat Analysis. Further, the security requirements for the product and the available budget for the project influence the width and depth of the tests.
6. Implement and Execute Tests
The implementation of tests can be straightforward when we use a tool that has the tests already implemented, such as our Java Card security test tool. In other cases, it requires significant manual work, for instance when we test a proprietary application protocol. After implementation, we execute the tests and analyse the behaviour of the product. Unexpected behaviour is followed by manual testing to gain a good understanding of the issue.
7. Impact Analysis
This final step forms the basis of our reporting and is critical to our customers. Having identified several technical anomalies, we analyse the impact of these issues on the product. It is our experience that what may first seem a minor technical issue can have a large impact on a product, as well as the other way around. For each issue, we also work out a recommendation how to address the security weakness and improve the security profile of the product.
Several Inspector hardware components are now also available with SDK. Riscure...