Security Testing Roles - Expanding on Integrating Security Testing into the QA Process

If you haven't read my previous post: Integrating Security into the QA Process and/or listened to the podcast from Rafal Los: http://podcast.wh1t3rabbit.net/down-the-rabbithole-episode-3-qa-and-security-can-we-make-it-work-, do that first.  The content below breaks down my thoughts on the type of security testing that can be integrated into each role (developer, QA, Security). This covers security testing, and does not include any other activities like threat modeling, architecture review, design review, following secure coding standards during development, etc.

Development Team:

• Use an automated static code analysis tool  (for example: Ounce, Fortify, or Veracode)
• Perform peer code reviews
• Construct and run unit/functional/automated tests for previously identified security issues using libraries like Watir/WatiN/Watij/Capybara/etc

QA Team:

• Test security controls that can be broken down into positive, concrete tests with a clear start and end (may require step by step directions and training) -- possible focus mainly on controls that ALL application should have in place
• Construct (or have developers construct) and run unit/functional/automated tests for previously identified security issues using libraries like Watir/WatiN/Watij/Capybara/etc (listed for both QA and Dev.)
• Perform testing for business logic or domain specific vulnerabilities (a list of these business logic rules should be specified in the application's requirements)

Security Team (possibly a team within QA!?!):

• Manually Perform negative testing and using penetration testing experience to identify issues
• Perform automated static code analysis (for example: Ounce, Fortify, or Veracode)
• Perform manual code review
• Perform automated security scanning (for example: AppScan, Web Inspect, or Burp Suite)

I also want to emphasize that identifying a vulnerability through testing should ALWAYS result in the organization creating a new or correlating to an existing application security requirement (like a functional or business requirement) and an associated secure coding standard (specific code/configuration example and discussion showing how to accomplish the application security requirement in a particular development language, framework, and/or library).  The goal is to proactively avoid vulnerabilities in the future, decreasing costs and improving security for all projects within the organization rather than for one specific application.  Feeding solutions back into the requirements specification phase of software development will help accomplish this.

Integrating Security into the QA Process

I just listened to the latest podcast from Rafal Los's site "Down the Rabbit Hole":
http://podcast.wh1t3rabbit.net/down-the-rabbithole-episode-3-qa-and-security-can-we-make-it-work-

On the show, they discuss integrating security testing into the QA process, which isn't terribly new, but what I got really excited about was the discussion about *how* to integrate it.  Security testing is difficult for QA teams. Security testing requires specific skill sets and knowledge.  QA teams don't know how to run automated security testing tools, have trouble interpreting the assessment results, and usually focus on testing defined requirements rather than having an open-ended mandate to look for security defects.

On the podcast, they recommend breaking security testing up and distributing it across the organization as much as possible.  Security testing should be a normal part of developing and testing software.  In regards to QA testing, define specific security tests, in a similar manner to the way functional or business requirements are tested, and provide those to the QA team.  Step by step directions, training on a particular tool (like the web developer toolbar plugin for Firefox, not necessarily a static code analysis tool), or custom scripts may be required and included in these test definitions (use templates if possible).  Link all of these security defect QA tests to functional or business requirements (this may require the organization to define new ones).  Then, add these requirements back into the software development process, so they are included during the planning, design, or coding phases.  Finally, provide secure coding standards that show step by step directions or specific code examples for accomplishing the functional or business requirements in the team's language, framework, library, or other software component.

Security B-Sides Kansas City Presentation

Security B-Sides Kansas City is happening Wednesday, October 26, 2011.  I will be presenting there at 10AM.  The topic is using Watir-WebDriver (a browser automation framework/driver) and Ruby to perform web application security unit testing.  While doing the research, I also got a chance to use another framework, Capybara, so I included examples from it as well.  The presentation will mostly consist of demos.  I will discuss what Watir-WebDriver is, alternate frameworks and languages to use, and how to apply them to web application security unit testing.  Then, I will walk through specific examples/demos showing how to use the frameworks to exploit/unit test vulnerabilities in the OWASP Broken Web Applications Project/VMware image.  The unit testing framework I chose is RSpec. I have demos ready for the following issues:

• SQL Injection
• error message based
• matching contents of a union select over the application usernames/passwords
• Cross-site Scripting
• Reflected - URL Based
• Reflected - Using a custom POST request
• Stored
• Autocomplete
• Session Fixation
• Open Redirect
• Enumerating authorization/access controls
• Information Disclosure through HTTP headers

After the presentation, I can provide anyone that asks with completed unit tests for all of the vulnerabilities listed above for both Ruby/Watir-WebDriver and Ruby/Capybara, using RSpec as the unit testing framework.  To run these demos at home, simply download and run the OWASP Broken Web Application VMware image using VMware Player.