Web Application Vulnerability Unit Testing Cheat Sheet (Capybara and Watir-WebDriver)

For an example template for Watir-WebDriver/RSpec or Capybara Test Case, start here:

• http://nickcoblentz.blogspot.com/2011/11/using-watir-webdriver-or-capybara-for.html

For common questions with Watir-WebDriver or Capybara, look here next:

• http://watirwebdriver.com/web-elements/ (and the "advanced interactions" items)
• https://github.com/jnicklas/capybara

Finally, refer to the cheat sheet below for issues I came across while trying to write test case for web application vulnerabilities:

Detect and Close JavaScript Alert Boxes:
Watir-WebDriver
begin
browser.driver.switch_to.alert.accept #raises an exception if no alert is present
puts 'alert box found'
rescue
puts 'alert box not found'
end

Capybara:
page.driver.browser.switch_to.alert.accept

Perform Arbitrary HTTP POST Requests:
Watir-WebDriver


Send Keyboard Commands:
Watir-WebDriver
browser.text_field(:name,/login/i).send_keys(:arrow_down)

Capybara:
page.find_field('login').native.send_keys(:arrow_down)

Search For Text/HTML Within a Nested Frame:
Watir-WebDriver


Returning Content From Javascript:
Watir-WebDriver
browser.execute_script(‘return “asdf”’) #=> returns “asdf” to ruby

Include JQuery: Watir-WebDriver browser.execute_script(%q| var el = document.createElement("script"); el.setAttribute("src","http://code.jquery.com/jquery-1.6.4.min.js"); document.body.appendChild(el);|)

Return HTTP Headers:
Watir-WebDriver

Using Watir-WebDriver or Capybara For Web Application Vulnerability Unit Testing

Back in October, I gave a Security B-Sides presentation filled with demos showing how to construct and execute unit tests for web application security vulnerabilities.  The goal was to:

1. Allow QA teams or developers to execute unit tests to demonstrate that a web application vulnerability remains fixed 1 day, 1 week, 1 month, or even 1 year from the date it was remediated (For example, security unit tests run as part of a continuous integration process).
2. Provide a mechanism for security teams to demonstrate a vulnerability instance to web application stakeholders.  One that can be run by the stakeholders themselves, as many times as needed, with little or no knowledge of security testing techniques.
3. Allow security testers to write testing tools or scripts that interact directly with the browser, eliminating many false positives occurring due to the inability to execute JavaScript or other similar browser dependent components.

The presentation slides and unit test scripts are available below.  Before trying out the unit tests, make sure you have the OWASP Broken Web Application Virtual Machine downloaded and running.  Next, install Ruby, Watir-WebDriver, RSpec, escape_utils, and Capybara.

To install everything on Windows, here's what I did:
1. Install Ruby (1.9.x) (http://rubyinstaller.org/downloads/)
2. Install Watir (http://watir.com/installation/#win)
Get an admin command prompt
gem update --system
gem install watir
gem install watir-webdriver

3. Install RSpec and escape_utils gem install rspec
gem install escape_utils

4. Install Capybara
gem install capybara

To run the test cases, use the following commands:
rspec -f d "OWASP Broken WebApps RSpec.rb"
rspec -f d "OWASP Broken WebApps Capybara.rb"

Presentation Materials:

• Using Watir & Ruby for Web Application Vulnerability Unit Testing
• OWASP Broken WebApps Capybara.rb
• OWASP Broken WebApps RSpec.rb

Potential BlackBerry Playbook Application Permissions Vulnerability - Need Help Confirming

I'm posting some details about a potential BlackBerry Playbook application permissions vulnerability with the idea that others out there may help confirm and test the issue's existence.  I can only take the exploit so far, and I need help from those with a BlackBerry AppWorld Vendor Account and a Playbook device (I have neither of these at the moment).

A completed exploit/application is available here: http://dl.dropbox.com/u/1132296/PermissionsTestModified.bar

Vulnerability Discussion

What I *think* I have found is a vulnerability that will allow an attacker to develop a BB Playbook application (think malware) with specific device permissions without those permissions being reported by App World.  What I have discovered is when you compile a WebWorks SDK application, the SDK uses a config file containing permissions as an input, and compiles those permissions into a flash file. The flash file manages, fullfills, and grants access to device APIs like accessing the camera, accelerometers, or sensitive user/device information.   I suspect that AppWorld determines the permissions for the application via that config file OR possibly a method call like getPermissions() in the flash file, but I have no way to confirm this.  I can alter the config file so it shows zero permissions, but then deploy an application with full permissions on the BB Playbook simulator successfully.  Now I need to confirm that AppWorld reports that no permissions are granted to the application despite the fact that these privileged APIs are being called successfully.  If it is successful, It will tell me that AppWorld relies on the config file to notify users of application permissions (which means I can get arbitrary permissions without the user ever knowing).

If that first attempt is unsuccessful, then I have already successfully disassembled, altered, and reassembled parts of the flash file and reincluded it within the application, so its just a matter of discovering how to obfuscate those permissions from AppWorld.  Most, if not all, the permissions enforcement occurs in the compiled flash file rather than by the underlying operating system itself.

The specific help I need is in publishing the application to AppWorld, observing the permissions reported for the application via AppWorld, and observing the permissions reported for the application via the Permissions Module on the Playbook control panel, and then running the application on a Playbook itself (rather than the simulator, which works just fine, but doesn't report permissions at all).  I don't own a BB Playbook and I don't currently have a Vendor account, so it is difficult to confirm whether this issue really exists.

If you have a BB AppWorld Vendor Account and a BB Playbook that you can confirm my results on, please read the details below and publish your results.

Instructions


First, install all the prerequisites needed to compile and deploy a BB WebWorks SDK application.  For a walk through, take a look at this guidance: http://us.blackberry.com/developers/tablet/webworks.jsp

Next, we will create a very simple WebWorks App that uses a few privileged API calls (specifically the Blackberry.app API: http://www.blackberry.com/developers/docs/webworks/api/blackberry.app.html).  We will use the example provided in the API documentation to create an index.html page:



Then, add the JQuery libraries referenced by the index.html page to the /js folder.

Now, we will create two configuration files, one containing the proper permissions and one without any permissions.  config.xml should read:


config2.xml:

Compile the application into a .bar file as you would any other WebWorks application. First create a zip archive at the root level of the application, then run the following command:

"c:\Program Files (x86)\Research In Motion\BlackBerry WebWorks SDK for TabletOS 2.1.0.6\bbwp\bbwp.exe" PermissionsTest.zip -d

In the /bin directory, a PermissionsTest.bar file will be created. Unzip this file (its just a zip file containing the compiled flash file and html, JavaScript, and XML files. Open the /bin/PermissionsTest/META-INF/MANIFEST.MF file. There will be two lines we will modify. They are (the SHA-512-Digest value will differ):

Archive-Asset-Name: air/config.xml Archive-Asset-SHA-512-Digest: 9x6Jp-WRWEV14A0DGdO7rIIXxEB7V-Ya6Ke1pRnM9oeckJB6GzS9EqzoDosXyaUEEaLDebxE6o36UalIqtv2gQ

Archive-Asset-Name: air/config2.xml Archive-Asset-SHA-512-Digest: Y5hA0NxFXOCHpfy5utM-9oMWG5elciLxKNWl0AcU4azyXWDBOrq6v4tw9cU0coG3jXzWqg4Od3OtZsEcqxNLwA

Delete the lines for config.xml, then rename config2.xml to config.xml

Next, go to the /bin/PermissionsTest/air/ directory and delete "config.xml", and rename "config2.xml" to "config.xml"

Go back to the /bin/PermissionsTest directory and make a new zip archive.  Change the file extension from ".zip" to ".bar".

Deploy the application to the simulator (or AppWorld):


"c:\Program Files (x86)\Research In Motion\BlackBerry WebWorks SDK for TabletOS 2.1.0.6\bbwp\blackberry-tablet-sdk\bin\blackberry-deploy.bat" -installApp -device <device ip here> -package bin\PermissionsTestModified.bar -password <simulator device password>

When you open the application, and click "Populate - APP" it should retrieve the author and title of the application (which require privileged API access).

So Why does this work?


Permissions seem to be enforced by the Adobe AIR application executable rather than the protected operating system itself.  if you decompile the swf file generated by the compilation process, you will see the following interesting code fragments:

public static const values:Object={"configXML":"config.xml", "version":"1.0.0", "content":"index.html", "author":"Nick Coblentz", "description":"This application tests whether privileged APIs can be called without AppWorld reporting those permissions to the user.", "name":"Permissions Test", "foregroundSource":"index.html", "hasMultiAccess":true, "onFirstLaunch":false, "onRemotePageLoad":false, "onLocalPageLoad":false, "debugEnabled":true, "accessList":new Array(new webworks.access.Access(webworks.config.ConfigConstants.WIDGET_LOCAL_DOMAIN, true, new Array(new webworks.access.Feature("blackberry.app", true, "", null)))), "widgetExtensions...lots more...

and the isFeatureAllowed() method in webworks.config.ConfigData which is used by webworks.FunctionBroker to determine whether to service JavaScript JSON requests for privileged API access.

Request For Help


If anyone chooses to lend a hand, I would welcome help further understanding the inner workings of the generated flash file, validation that the vulnerability exists, and validation of the permissions reported by AppWorld and the Playbook device itself.  Feel free to add comments on the blog or email me directly.