Wednesday, January 20, 2010

How Often Should I Reassess My Web Applications?

There are a couple approaches for determining when an application should be undergo a security assessment. First, organizations often require new tests after a fixed period of time. This period of time may vary based on the risk level the organization has attributed to each application or application type. It's common for organizations to conduct security assessments of high and medium risk applications every six months to one year. For low risk applications, the time period is often a year to two years. The risk level of an application is often determined based on the type of data and functionality within the application. For example, an Internet facing application that handles credit card transactions would be considered high risk; while an application that simply provides product information and is not subject to regulatory, compliance, or legal requirements may be low risk. Periodic assessments usually supplement the next two approaches.

The second approach is associated with major changes implemented in the application. There are a variety of changes that should trigger a new application assessment. Any changes to a security mechanism should undergo validation. Security mechanisms usually include things like authentication processes, authorization controls, session management features, and data validation and encoding components (think cross-site scripting, SQL injection, etc.). Changes that add or modify a feature in the application should trigger a retest based on the risk level of the application and the sensitivity of data or functionality that the change effects. So, a new feature that adds an “about” tab to a website probably doesn’t need to undergo rigorous security testing; however, a new feature that collects users’ social security numbers absolutely should undergo testing (including evaluation whether collection of this PII complies with the organization’s policies, that there is a need to know for this sensitive data, that the sensitive data is adequately protected at rest and in transit, and that only authorized, authenticated parties can reach this data). Organization’s typically set up criteria and standards around PII, credit card information, and other sensitive data types that automatically trigger a reassessment when an application change related to those data types occurs.

The third approach is to use application security testing as a security gate within an organization’s development process. In this case, specific application types or risk levels would require an assessment before they can be deployed to production. This is usually one of the final steps in a secure software development process and acts as a sanity check or a process improvement opportunity rather than a catch-all for security issues. These types of tests are often highly targeted and would not encompass an assessment of the whole application; instead the security assessment would focus on high risk, new, or updated components introduced in the application.

Previously, I wrote an ISSA article, Titled “Web Application Security Portfolios”, which covered some of this detail. An expanded version of this article can be found in my post here. The article discusses ideas around managing portfolios for each application within an organization, identifying data types and compliance requirements, and tracking security activities.


dre said...

This seems to presuppose that assessments are what is necessary in the first place.

Instead, custom application lifecycle management solutions should include software security practices and tie them directly to a framework-specific OWASP ESAPI (or a platform that resembles "an ESAPI"). The ESAPI should include validation to a certain OWASP ASVS level (1-4) and type (A or B in the case of levels 1-2).

I believe strongly in avoiding hamster-wheels-of-pain for information security management, even if it's an unknown, murky water such as application security at stake. Attempting to "re-assess" applications or sets of applications (based on risk posture) at any interval (i.e. it doesn't matter if it's weekly, yearly, or every 5 years!) it a complete waste of time.

Penetration-testing activities may still be useful every so often, but this should not be mandated by any regulations or standards. Note that this means that I do indeed dislike how PCI DSS "prescribes" what you refer to as `[requiring] new tests after a fixed period of time ... [and] common for organizations to conduct security assessments of high and medium risk applications every six months to one year'. Most (if not all) organizations should seek alternate or compensating controls to satisfactorily meet Requirements 5, 6, and 11.

It would be better if PCI DSS and many other regulations/standards were re-written to denote that penetration-testing and vulnerability testing (whether manual or automated) should be replaced with an ESAPI-equivalent that has been verified to a level/type agreeable to what PCI DSS intends to protect. Perhaps ASVS L2B or L3 would be closest to appropriate?

I sincerely dislike how immature our industry has been for several years around these matters. I politely invite you to change what it is that you do, and what your customers do, in order to extend budget, save money, and implore better practices.

We (as an industry) no longer need penetration-testing or vulnerability testing in the manners that you describe. I would compare your strategy and tactics to the use of medicine before bacteria was discovered and a paradigm shift occurred where medical "doctors" were actually able to save lives instead of quite the opposite.

Nick Coblentz said...


I agree in some respects with you... The so called "Security Whack-A-Mole" approach is a pretty inefficient way to reduce risk in application assets. It also does not have a lasting impact on the way future applications are developed within an organization. This is where security training, secure coding standards, and training and accountability against those coding standards help.

This is also why many of my blog posts discuss secure software development processes, application security program management, secure coding techniques (see my struts 2 blog posts), and frameworks like the Software Assurance Maturity Model.

Unfortunately, as you hinted, organizations only seem to be interested in penetration tests, vulnerability assessments, code reviews, and etc. I have encountered very few organizations that truly have a secure software development process other than... yeah we do assessments before we deploy to production.

This is also evident in how few organizations seek out strategic consulting services centered around developing a long term application security program. I and other members of the AT&T Consulting team offer comprehensive services around defining and implementing secure software development processes and application security programs. I believe this is the future of our industry and that we are slowly but surely moving in this direction. Give it a five more years, and I think you will like how the situation evolves.

With that said, assessments do have their place. Assessments are VERY useful for organizations that have a mature process and need to identify gaps in their security policies and secure coding standards. If an organization abides by all their rules and standards and still yields a vulnerable application, a security assessment is a great way to point it out and learn which controls and practices need to be added.

Additionally, Security assessments, penetration testing, and code reviews are often the "Gateway Drug" to application security. Most organizations start with this activity and over time they realize more is necessary. They begin reaching out to OWASP, WASC, and consulting organizations, asking questions like "What next?"

I believe assessments are still very necessary to organizations; the part that I believe you are trying to point out is that they aren't always leveraged correctly by development groups, nor do these groups take the proper lessons away from these engagements.

Nick Coblentz said...

There are also several situations where applications assessments are very well suited. Some of those situations include:

1. Organization A acquires Organization B along with thousands of web application assets. Organization A has an application security program, but Organization B does not. Organization A needs to determine their risk posture and the level of effort required to remediate these newly acquired applications.

2. An organization needs to satisfy regulatory, compliance, or legal requirements (Yes, I know everyone will get bent out of shape about this one... but the current state of these requirements mandate one or a combination of assessments, penetration testing, and code reviews)

3. An Organization wants to provide assurance to customers that purchase developed software.

4. A Customer wants to ensure the third party software development shop provides an application with an appropriate risk-level. Possibly as part of the contract agreement. (See the OWASP Secure Software Contract Annex project)

I'm sure there' more...

As you can see, there are many strategic reasons for carrying out assessments in addition to just checking to see if were "secure".