Web Application Security Testing

Editor's Notes

1. The risk that a web application might incur in a security incident such a major data breach depends on several risk factors such as the exposure into the public internet, the likelihood of being the target for a potential attacker as well as the knowledge, tools and techniques available to the attacker to break into the application. In order to mitigate such risks, sites that are potentially at risk to become a target for attacks are vulnerability assessed and security tested. These traditional security tests typically include source code analysis to find vulnerabilities in source code and penetration testing to find vulnerabilities in the application and can be done both manually of with the help of tools.   There are certainly benefits on performing these security tests: besides compliance with applicable standards (e.g. PCI-DSS explicitly calls for these tests in section 6.6), they provide a level of assurance that no vulnerabilities exist that might cause harm to the application and as consequence a business impact to the organization and the customers using it.   But even if security testing satisfies a compliance requirement and provides for an increased level of software security assurance, it is logical to ask whether we can consider an application secure because we found, for example, no high and medium risk vulnerabilities with these security tests.   For example, it is logical to think that you can test only for the threats you know of. Moreover, the argument that a secure application “ equals ” no vulnerabilities found with tests is difficult to make if you consider this data: According to MITRE all vulnerabilities that application security tool vendor claim to find cover only 45% of the known vulnerability types (over 600 in CWE) According to a University Of Michigan study 76% of financial sites still have at least one critical design flaw that can be exploited to cause financial fraud, identify theft, reputation-brand damage, denial of service to customers Two of the last reported data breaches (datalossdb.org), 130 million credit card numbers @ Heartland Payment Systems (reported Jan 2009) and 4.2 million credit card and ATM records @ Hannaford Bros (reported March 2008) involve the exploit of application vulnerabilities that were previously tested with vulnerability assessments in compliance with PCI-DSS requirement 6.5 testing for minimum (OWASP T10) vulnerabilities and 6.6 testing of such vulnerabilities with either manual, automated, source code or vulnerability assessment From the perspective of security testing web applications, it is the author of this presentation opinion that these data advocate the need to a new approach toward security testing: a risk based, threat driven approach. Hence, the aim of this presentation is to provide the basics of the risk based testing methodology for testing web applications that includes: Cybercrime intelligence Attack tree analysis, Attack vector analysis Use and misuse cases Data Flow Analysis (DFD)-secure architecture analysis.   Each of these threat analysis techniques can help security testers in deriving tests for testing web applications countermeasures against most likely attacks scenarios such as: Cybercrime intelligence data (e.g. IC3 and public sources) can help security testers to simulate real attack scenarios to test the application using the same attack vectors and by assessing for similar vulnerabilities used by cybercriminals. Threat tree analysis for cybercrime attacks help security testers simulate the attacks against different targets by adopting the same attacker goals and by adopting the same means (attacker methods, tools) to achieve such goals Use and misuse cases help security testers to derive test cases to validate that the application functionality cannot be abused such as by bypassing security controls or by exploiting security flaws that might lead to business impacts such as information disclosure, fraud, denial of service. Attack vector analysis provides testers with the attack vectors for testing countermeasures such as filtering of encoded cross-site scripting and SQL injection commands. DFD-secure architecture analysis provide testers information about the application scope for risk based testing such as the location of the application entry points that need to be tested with the previously determined attack vector, the access levels that are required to access them and the countermeasures that need to be tested for mitigation of identified threats.   Graphical examples of these threat analysis techniques will be provided and it will be explained how these threat analysis techniques can help testers to derive tests cases for testing web application countermeasures. The presentation will also put risk-based security tests in the context of traditional security and quality testing as well as of the testing environment and tools can be leveraged by quality and functional tests. The roadmap for adopting risk based security by info security managers, testers and developers will also provided. This presentation will include references to the security testing methodology published in the OWASP testing guide. Marco Morana Bio Marco Morana serves as one of the leaders of OWASP (Open Web Application Security Project) organization where he is actively involved in evangelize on web application security through presentations at local chapter meetings in USA as well as internationally. Marco has recently been awarded a contract from Wiley Publishing to co-author a book on Application Threat Modeling. Besides being the OWASP Cincinnati chapter lead, Marco is also active contributor to OWASP projects such as the application threat modeling methodology for secure coding guideline and the security testing guide (ver. 2 and 3). Besides contributing to OWASP, Marco works as Technology Information Security Officer for a large financial organization in North America with responsibilities in the definition of the organization web application security standards, management of application security assessments during the SDLC, threat-fraud analysis and training of software developers, project managers and architects on different topics related to application security. In the past, Marco served as senior security consultant and independent consultant where his responsibilities included providing software security services for several clients in the financial and banking, telecommunications and commercial sector industry. Besides security consulting, Marco had a career as technologist in the security industry where he contributed to the design business critical security products currently being used by several FORTUNE 500 companies as well by the US Government. Marco work on software security is referred in the 2007 State Of the Art report by the Information Assurance Technology Analysis Center (IATAC). Marco received the NASA’s Space Act Award in 1999 for the patenting the S/MIME SEP (Secure Email Plug-in) application. Marco research work on application and software security is widely published on several magazines such as In-secure magazine, Secure Enterprise, ISSA Journal and the C/C++ Users journal. Marco’s ideas and strategies for writing secure software are posted on his blog: http://securesoftware.blogspot.com . Public OWASP bio: http://www.owasp.org/index.php/Marco_Morana        
2. http://www.ddj.com/184405196
3. Functional Security Requirements From the perspective of functional security requirements, the applicable standards, policies and regulations drive both the need of a type of security control as well as the control functionality. These requirements are also referred to as “positive requirements”, since they state the expected functionality that can be validated through security tests. Examples of positive requirements are: “the application will lockout the user after six failed logon attempts” or “passwords need to be six min characters, alphanumeric”. The validation of positive requirements consists of asserting the expected functionality and, as such, can be tested by re-creating the testing conditions, and by running the test according to predefined inputs and by asserting the expected outcome as a fail/pass condition. In order to validate security requirements with security tests, security requirements need to be function driven and highlight the expected functionality (the what) and implicitly the implementation (the how). Examples of high-level security design requirements for authentication can be: Protect user credentials and shared secrets in transit and in storage Mask any confidential data in display (e.g., passwords, accounts) Lock the user account after a certain number of failed login attempts Do not show specific validation errors to the user as a result of failed logon Only allow passwords that are alphanumeric, include special characters and six characters minimum length, to limit the attack surface Allow for password change functionality only to authenticated users by validating the old password, the new password, and the user answer to the challenge question, to prevent brute forcing of a password via password change. The password reset form should validate the user’s username and the user’s registered email before sending the temporary password to the user via email. The temporary password issued should be a one time password. A link to the password reset web page will be sent to the user. The password reset web page should validate the user temporary password, the new password, as well as the user answer to the challenge question. Risk Driven Security Requirements Security tests need also to be risk driven, that is they need to validate the application for unexpected behavior. These are also called “negative requirements”, since they specify what the application should not do. Examples of "should not do" (negative) requirements are: The application should not allow for the data to be altered or destroyed The application should not be compromised or misused for unauthorized financial transactions by a malicious user. Negative requirements are more difficult to test, because there is no expected behavior to look for. This might require a threat analyst to come up with unforeseeable input conditions, causes, and effects. This is where security testing needs to be driven by risk analysis and threat modeling. The key is to document the threat scenarios and the functionality of the countermeasure as a factor to mitigate a threat. For example, in the case of authentication controls, the following security requirements can be documented from the threats and countermeasure perspective: Encrypt authentication data in storage and transit to mitigate risk of information disclosure and authentication protocol attacks Encrypt passwords using non reversible encryption such as using a digest (e.g., HASH) and a seed to prevent dictionary attacks Lock out accounts after reaching a logon failure threshold and enforce password complexity to mitigate risk of brute force password attacks Display generic error messages upon validation of credentials to mitigate risk of account harvesting/enumeration Mutually authenticate client and server to prevent non-repudiation and Man In the Middle (MiTM) attacks Threat modeling artifacts such as threat trees and attack libraries can be useful to derive the negative test scenarios. A threat tree will assume a root attack (e.g., attacker might be able to read other users' messages) and identify different exploits of security controls (e.g., data validation fails because of a SQL injection vulnerability) and necessary countermeasures (e.g., implement data validation and parametrized queries) that could be validated to be effective in mitigating such attacks.
4. PCI DSS brings precision, but with FUD: Greater detail surrounding technical/ non-tech security requirements Cardholder data such as PAN need to be masked in display and need to be protected in storage and transit with encryption, the same for card holder data and expiration data on the card Track 2 data that include CVV2, PINs cannot be stored even if encrypted
5. To derive security requirements from use and misuse case [20] , it is important to define the functional scenarios and the negative scenarios, and put these in graphical form. In the case of derivation of security requirements for authentication, for example, the following step-by-step methodology can be followed. Step 1: Describe the Functional Scenario: User authenticates by supplying username and password. The application grants access to users based upon authentication of user credentials by the application and provides specific errors to the user when validation fails. Step 2: Describe the Negative Scenario: Attacker breaks the authentication through a brute force/dictionary attack of passwords and account harvesting vulnerabilities in the application. The validation errors provide specific information to an attacker to guess which accounts are actually valid, registered accounts (usernames). The attacker, then, will try to brute force the password for such a valid account. A brute force attack to four minimum length all digit passwords can succeed with a limited number of attempts (i.e., 10^4). Step 3: Describe Functional and Negative Scenarios With Use and Misuse Case: The graphical example in Figure below depicts the derivation of security requirements via use and misuse cases. The functional scenario consists of the user actions (entering username and password) and the application actions (authenticating the user and providing an error message if validation fails). The misuse case consists of the attacker actions, i.e., trying to break authentication by brute forcing the password via a dictionary attack and by guessing the valid usernames from error messages. By graphically representing the threats to the user actions (misuses), it is possible to derive the countermeasures as the application actions that mitigate such threats. Step 4: Elicit The Security Requirements. In this case, the following security requirements for authentication are derived: 1) Passwords need to be alphanumeric, lower and upper case and minimum of seven character length 2) Accounts need to lockout after five unsuccessful login attempt 3) Logon error messages need to be generic
6. The Guide Contents A series of articles on the most common web application security problems Some process information, but not much… The world desperately needs a body of knowledge on application security. One important piece of this body of knowledge is about application security testing.
7. Reporting Requirements The security posture of an application can be characterized from the perspective of the effect, such as number of vulnerabilities and the risk rating of the vulnerabilities, as well as from the perspective of the cause (i.e., origin) such as coding errors, architectural flaws, and configuration issues. Vulnerabilities can be classified according to different criteria. This can be a statistical categorization, such as the OWASP Top 10 and WASC (Web Application Security Statistics) project, or related to defensive controls as in the case of WASF (Web Application Security Framework) categorization. When reporting security test data, the best practice is to include the following information, besides the categorization of each vulnerability by type: The security threat that the issue is exposed to The root cause of security issues (e.g., security bugs, security flaw) The testing technique used to find it The remediation of the vulnerability (e.g., the countermeasure) The risk rating of the vulnerability (High, Medium, Low) By describing what the security threat is, it will be possible to understand if and why the mitigation control is ineffective in mitigating the threat. Reporting the root cause of the issue can help pinpoint what needs to be fixed: in the case of a white box testing, for example, the software security root cause of the vulnerability will be the offending source code. Once issues are reported, it is also important to provide guidance to the software developer on how to re-test and find the vulnerability. This might involve using a white box testing technique (e.g., security code review with a static code analyzer) to find if the code is vulnerable. If a vulnerability can be found via a black box technique (penetration test), the test report also needs to provide information on how to validate the exposure of the vulnerability to the front end (e.g., client). The information about how to fix the vulnerability should be detailed enough for a developer to implement a fix. It should provide secure coding examples, configuration changes, and provide adequate references. Finally the risk rating helps to prioritize the remediation effort. Typically, assigning a risk rating to the vulnerability involves a risk analysis based upon factors such as impact and exposure.
8. Security Test Data Analysis and Reporting Goals for Security Test Metrics and Measurements The definition of the goals for the security testing metrics and measurements is a pre-requisite for using security testing data for risk analysis and management processes. For example, a measurement such as the total number of vulnerabilities found with security tests might quantify the security posture of the application. These measurements also help to identify security objectives for software security testing: for example, reducing the number of vulnerabilities to an acceptable number (minimum) before the application is deployed into production. Another manageable goal could be to compare the application security posture against a baseline to assess improvements in application security processes. For example, the security metrics baseline might consist of an application that was tested only with penetration tests. The security data obtained from an application that was also security tested during coding should show an improvement (e.g., fewer number of vulnerabilities) when compared with the baseline. In traditional software testing, the number of software defects, such as the bugs found in an application, could provide a measure of software quality. Similarly, security testing can provide a measure of software security. From the defect management and reporting perspective, software quality and security testing can use similar categorizations for root causes and defect remediation efforts. From the root cause perspective, a security defect can be due to an error in design (e.g., security flaws) or due to an error in coding (e.g., security bug). From the perspective of the effort required to fix a defect, both security and quality defects can be measured in terms of developer hours to implement the fix, the tools and resources required to fix, and, finally, the cost to implement the fix. A characteristic of security test data, compared to quality data, is the categorization in terms of the threat, the exposure of the vulnerability, and the potential impact posed by the vulnerability to determine the risk. Testing applications for security consists of managing technical risks to make sure that the application countermeasures meet acceptable levels. For this reason, security testing data needs to support the security risk strategy at critical checkpoints during the SDLC. For example, vulnerabilities found in source code with source code analysis represent an initial measure of risk. Such measure of risk (e.g., high, medium, low) for the vulnerability can be calculated by determining the exposure and likelihood factors and, further, by validating such vulnerability with penetration tests. The risk metrics associated to vulnerabilities found with security tests empower business management to make risk management decisions, such as to decide whether risks can be accepted, mitigated, or transferred at different levels within the organization (e.g., business as well as technical). When evaluating the security posture of an application, it is important to take into consideration certain factors, such as the size of the application being developed. Application size has been statistically proven to be related to the number of issues found in the application with tests. One measure of application size is the number of line of code (LOC) of the application. Typically, software quality defects range from about 7 to 10 defects per thousand lines of new and changed code [21]. Since testing can reduce the overall number by about 25% with one test alone, it is logical for larger size applications to be tested more and more often than smaller size applications. When security testing is done in several phases of the SDLC, the test data could prove the capability of the security tests in detecting vulnerabilities as soon as they are introduced, and prove the effectiveness of removing them by implementing countermeasures at different checkpoints of the SDLC. A measurement of this type is also defined as “containment metrics” and provides a measure of the ability of a security assessment performed at each phase of the development process to maintain security within each phase. These containment metrics are also a critical factor in lowering the cost of fixing the vulnerabilities, since it is less expensive to deal with the vulnerabilities when they are found (in the same phase of the SDLC), rather then fixing them later in another phase. Security test metrics can support security risk, cost, and defect management analysis when it is associated with tangible and timed goals such as: Reducing the overall number of vulnerabilities by 30% Security issues are expected to be fixed by a certain deadline (e.g., before beta release) Security test data can be absolute, such as the number of vulnerabilities detected during manual code review, as well as comparative, such as the number of vulnerabilities detected in code reviews vs. penetration tests. To answer questions about the quality of the security process, it is important to determine a baseline for what could be considered acceptable and good. Security test data can also support specific objectives of the security analysis such as compliance with security regulations and information security standards, management of security processes, the identification of security root causes and process improvements, and security costs vs. benefits analysis. When security test data is reported it has to provide metrics to support the analysis. The scope of the analysis is the interpretation of test data to find clues about the security of the software being produced as well the effectiveness of the process. Some examples of clues supported by security test data can be: Are vulnerabilities reduced to an acceptable level for release? How does the security quality of this product compare with similar software products? Are all security test requirements being met? What are the major root causes of security issues? How numerous are security flaws compared to security bugs? Which security activity is most effective in finding vulnerabilities? Which team is more productive in fixing security defects and vulnerabilities? Which percentage of overall vulnerabilities are high risk? Which tools are most effective in detecting security vulnerabilities? Which kind of security tests are most effective in finding vulnerabilities (e.g., white box vs. black box) tests? How many security issues are found during secure code reviews? How many security issues are found during secure design reviews? In order to make a sound judgment using the testing data, it is important to have a good understanding of the testing process as well as the testing tools. A tool taxonomy should be adopted to decide which security tools should be used. Security tools can be qualified as being good at finding common known vulnerabilities targeting different artifacts. The issue is that the unknown security issues are not tested: the fact that you come out clean it does not mean that your software or application is good. Some studies [22] have demonstrated that, at best, tools can find 45% of overall vulnerabilities. Even the most sophisticated automation tools are not a match for an experienced security tester: just relying on successful test results from automation tools will give security practitioners a false sense of security. Typically, the more experienced the security testers are with the security testing methodology and testing tools, the better the results of the security test and analysis will be. It is important that managers making an investment in security testing tools also consider an investment in hiring skilled human resources as well as security test training.
9. According to MITRE all vulnerabilities that application security tool vendor claim to find cover only 45% of the known vulnerability types (over 600 in CWE)
10. According to a University Of Michigan study 76% of financial sites still have at least one critical design flaw that can be exploited to cause financial fraud, identify theft, reputation-brand damage, denial of service to customers
11. In same cases the cyber attack vectors are reported step by step Use "xp_cmdshell“ to download hacker tools to the compromised MSSQL server. Obtain valid Windows credentials by using fgdump Install network "sniffers" to identify card data and systems involved in processing credit card transactions. Install backdoors that "beacon" periodically to their command and control servers Target databases, Hardware Security Modules (HSMs), and processing applications in an effort to obtain credit card data or brute-force ATM PINs. Use WinRAR to compress the information they pilfer from the compromised networks.
12. Most of cybercrime attacks target both the browser and the web application. You are as secure as the weakest link and the weakest link is always the human element, so phishing and social engineering is the easier way to get CC data directly from a user. Other attacks use drive by download to install malware to perform MITM, clickjacking or man in the browser attacks exploit browser vulnerabilities and in the exeuctable content (browser plugins, adobe and macromedia flash, activex controls) From the web application pespective the attacks can exploit SQL injection vulnerabilities to upload sniffers, get the data by altering the query, attack the weak encryption and attack session such as using session fixaction, hijacking the session in transit or being cached logged
13. It is important to be context aware of the application while you are testing
14. Security in depth tests scope: entry points, access levels and countermeasures Spoofing Identity 􀂃 Attempt to force the application to use no authentication; is there an option to allows this, which a non-administrator can use? 􀂃 Can you view a valid user’s user s credentials on the wire or in persistent storage? 􀂃 Can “security tokens” (e.g. a cookie) be replayed to bypass an authentication stage? 􀂃 Tampering with the data 􀂃 Is it possible to tamper with than rehash the data? 􀂃 Create invalid hashes and digital signatures to verify they are checked correctly. 􀂃 Repudiation 􀂃 Do conditions exist that prevent logging or auditing? 􀂃 Is it possible to create requests that create incorrect data in an event log? Information Disclosure 􀂃 Attempt to access data that can be accessed only by more privileged users. 􀂃 Make the application fail in a way that discloses useful information to an attacker (for example, error messages) 􀂃 Kill the process and then perform disk scavenging, looking for sensitive data written to disk. 􀂃 Denial of Service (Dos) 􀂃 Flood a process with so much data it stops responding to valid requests. 􀂃 Does malformed data crash the process? 􀂃 Elevation of Privilege 􀂃 Can you execute data as code 􀂃 Can an elevated process be forced to load a command shell, which in turn will execute with elevated privileges?