Higher Expectations for Security Assessments

We've all heard the saying that you can't test security into software, so if we must do these tests, why not demand more value from them.  If someone performed a security assessment on my software, I would expect them to contribute knowledge in a meaningful way to help me build better software in the future.  I don't think a report with best practices meets that expectation.

Existing Knowledge Base and Patterns
Before we get much further, let's talk about the knowledge and resources I would start with.  I know this may not be common in many organizations, but I would expect to have a specific set of secure architecture, configuration, and coding patterns that satisfy security requirements for my application, organization, data protection needs, and the threat actors targeting my application AND these patterns are tailored for the language and frameworks I'm using.  These patterns may not be very complete or robust, but they are at least a starting point to be added upon in the future.

For example, if I'm using ASP.NET MVC 3, I will have concrete examples of how to switch out the default HTML encoder within the Web.config file with the Microsoft Web Protection Library (AntiXSS Library) and then have example code that implements HTML or JavaScript encoding within my view such that cross-site scripting attacks are rendered inert (you could also mention input validation, but I skipped it in this example).  I would also have examples for configuring the application to require SSL/TLS, secure and HTTPOnly flags on cookies, and other similar security related configuration settings.  There are many more possibilities that should be included within this library of patterns. Each would also include one or more validation patterns that ensure a security control is present and working properly such as:

• Unit tests
• Automated tests using Capybara, Watir, or other similar frameworks
• QA test cases
• Automated static analysis rules
• Dynamic testing test cases
• Many more...

Security Assessment Expectations

After I receive an assessment report, what do I expect the assessor to provide to me (in addition to a report discussing the issue, risk rating, best practice, etc).  First, I would expect a completed test case of some kind to reproduce the issue.  This may require collaboration between the developers, QA testers, and the security tester, but I should receive automated (preferable) or manual test cases (detailed instructions for QA to follow for example, probably created by QA while walking through reproduction with the assessor) that I can run at any time to validate whether the vulnerability has been fixed successfully.  These test cases may reflect the capabilities of the organization and the validation patterns that already exist. 

Second, I would expect the assessor to help evaluate and update existing secure coding, configuration, architecture, and validation patterns to ensure a solution exists for future development.

Third, I would expect the assessor to provide a mini training session (may only be 10 minutes using a web based conferencing solution), explaining the vulnerability, and then support the development lead or architect in training the developers and QA testers in applying the new patterns.

Finally, I would expect the assessor to help link specific weaknesses in my application to techniques commonly used by the threat actors targeting my organization and application.