|Register for our Free Newsletters
|Other Carouselweb publications
Code review and WAP firewalls for PCI Compliance
: 29 January, 2009 (Special Report) |
Ulf Mattsson, Chief Technology Office at Protegrity provides a detailed review of attaining PCI compliance using Web Application Firewalls and Code Reviews
Organisations handling credit cards feel pressure building as the deadline for PCI Requirement 6.6 compliance has passed and well documented breaches have heightened the public and regulatory agencies' concerns about how well companies are securing consumer-specific information. Despite some initial advances, sensitive information is still frequently stolen. Internal threat an issue, magnified by extended partnerships which ultimately lead to more tasks will be performed outside company facilities. In increasingly complex technical and business environments, no one security approach can deal with all the new and innovative intrusions. But the lack of a security silver bullet doesn't mean data security is impossible. It simply means that businesses have to take a multi-pronged approach to data security.
This article is based on a project case study in protecting an enterprise application environment, including web-oriented applications. The article is PCI 6.6-oriented and compares the use of Web Application Firewalls (WAF) or code reviews for web-facing applications. It also addresses code scanning that is not web related. Extending the code reviews into the non-web applications, we also briefly discuss other types of protections. Other articles already discussed how to protect from SQL Injection into the database, or internal threats, including a DBA that impersonates a user. The section 'Protecting the Data Flow' includes a few pointers to resources discussing protection of the enterprise data flow. The code review section is longer since this is an evolving area from a PCI perspective focusing on WAF and complementary code scanning. This article will compare WAF and web-based code reviews, and point to resources discussing the whole data flow, which then involves much more than C/C++ code scanning. The part concerning code analysis is not web-oriented, but it's about C/C++/Java source code scanning, though it has some general parts.
The case study from company ABC recommended using both WAF and code reviews. Internal and external users are accessing sensitive client and credit card data via web based applications and other types of applications. ABC is considering Web applications as #1 focus of an attack. ABC reviewed recent research that shows that the majority of cyber attacks are performed at the Web Application level. ABC considers that their e-business Websites are at immediate risk of being hacked. ABC's primary issues are PCI compliance, and a concern about the escalating threat against financial data from organized crime and insiders. Time is a critical factor in selecting solutions to prevent breaches. ABC is a security aware organization that will need both short term and long term solutions to this problem. The case study from company ABC will analyze and identify an approach to develop and maintain secure systems and applications, including selecting suitable static-analysis code scanning tools for application development. ABC positioned different approaches to prevent data theft (and the attack-paths to the data - different types of apps, databases â€¦) including WAF, data (protection (encryption, hashing, tokenizing) and C++ code scanning. The solution for ABC is based on the conclusion that every layer of defense is critical. A holistic and layered approach can provide the best level data security and the sooner sensitive data gets encrypted, the more secure the environment. ABC is planning an enterprise data protection approach and protects data across the information life cycle. ABC acknowledges that secure development will take a long time to implement, partly based on expensive and time-consuming manual code reviews. The short term solution is based on protecting the external web-facing applications with a WAF combined with data encryption in files and databases. This will give ABC a quick and cost effective data security implementation that will meet PCI requirements in this area. ABC is complementing this with a medium term solution including code reviews and scanning of internal code non-web applications. ABC also identified a long term project that will include penetration testing and scanning and review of the web application code base.
PCI Requirement 6 - Developing and maintaining secure applications
Payment Card Industry (PCI) Data Security Standard (DSS) Requirement 6, Develop and maintain secure systems and applications. PCI 6.6 itself has two halves, "code review" (the act of finding/fixing vulnerabilities) and "application firewalls" (device designed to thwart website attacks) that merchants may choose between.
Requirement 6 is about "developing and maintaining secure applications and systems." Requirement 6.1 requires that vendor-supplied security patches be applied within one month of release. Securing and fixing custom application code is not as easy as downloading a patch from your favorite software vendor. Web application vulnerabilities must be identified, fixes developed, tested, and deployed. In short, you're on your own for the entire process. Setting aside the fact that these two options should not be perceived as competitive, rather complementary, the Council is giving merchants the choice acknowledging budget constraints.
PCI Requirement 6.6 mandates the following:
PCI DSS version 1.1 Requirement 6.6: Ensure that web-facing applications are protected against known attacks by applying either of the following methods. Having all custom application code reviewed for common vulnerabilities by an organisation that specialises in application security. Installing an application layer firewall in front of web facing applications.
Testing Procedure: For web-based applications
PCI DSS version 1.1 Requirement 6.6 Testing Procedure: For web-based applications, ensure that one of the following methods are in place as follows. Verify that custom application code is periodically reviewed by an organization that specializes in application security; that all coding vulnerabilities were corrected; and that the application was re-evaluated after the corrections. Verify that an application-layer firewall is in place in front of web-facing applications to detect and prevent web-based attacks. The confusion stems from the interpretation of the requirement. First, let's clear up some high-level misconceptions. Requirement 6.6 is not just for "level ones." It does not specify service providers or merchants. It does not specify either source code reviews or web-application firewalls.
Complying with Requirement 6.6
Requirement 6.6 is about protecting web applications, plain and simple. Given our modern threat landscape, it is no wonder that PCI Requirement 11.3.2 dictates "application penetration tests" be performed after every "significant change." Meaningful web application security management requires frequent assessments as code and threats evolve continually. Requirement 6.6 is about developing a repeatable methodology that connects the "Find" (the vulnerability detection) process to the "Fix" process for the systematic, efficient elimination of vulnerabilities from web applications. Additional practical PCI guidance can be found at .
What does PCI 6.6 mean
The ultimate goal is to ensure secure web applications. For applications developed or customized in-house, the following process must be continually performed: Identify vulnerabilities (find), correct them (fix), and test to confirm that the correction is effective (prove). Find, fix, prove, find, fix, prove.
PCI quarterly network scans - too little too late
The quarterly network scans will find some SQL injection and catch missing basic input validation, but generally they cannot find application-level vulnerabilities. But Web applications need to be checked on a continuous basis, these quarterly network scans should not be relied on to tell you if your Web apps are vulnerable.
Vulnerabilities must be detected, communicated, and corrected
A long term goal of Requirement 6.6 is the establishment of a web application vulnerability lifecycle - leading to the effective elimination of risk. Vulnerabilities must be detected, communicated, and corrected. This can be done through various measures including, black box testing (run-time assessment), white box testing (source code review), binary analysis, static analysis, remediation by developers or web application firewalls.
Runtime Assessments, Source Code Reviews, Binary and Static Analysis to find vulnerabilities in web applications
There is a misconception that all detection techniques try to achieve the same end goal and compete for the same budgetary dollars. The fact of the matter is that each testing ideology brings different benefits to the table at different prices, almost all of which are complementary and help paint a complete picture of application weaknesses. While Vulnerability Scanners are required for PCI DSS section 11.3 and can be used for section 6.6, WAF helps organizations meet 8 of the 12 PCI DSS requirements. That's eight PCI DSS requirements that WAF helps meet versus just two that vulnerability scanners can help meet.
Requirement 6.6 Option 1 - Application Code Reviews
The application code review option does not necessarily require a manual review of source code. Keeping in mind that the objective of Requirement 6.6 is to prevent exploitation of common vulnerabilities (such as those listed in Requirement 6.5), several possible solutions may be considered. They are dynamic and pro-active, requiring the specific initiation of a manual or automated process. Properly implemented, one or more of these four alternatives could meet the intent of Option 1 and provide the minimum level of protection against common web application threats:
1. Manual review of application source code
2. Proper use of automated application source code analyzer (scanning) tools
3. Manual web application security vulnerability assessment
4. Proper use of automated web application security vulnerability assessment (scanning) tools.
Requirement 6.6 Option 2 - Application Firewalls
PCI Requirement 6.6 can be quickly met through installing a web application firewall.
In the context of requirement 6.6, an "application firewall" is a web application firewall (WAF), which is a security policy enforcement point positioned between a web application and the client end point. This functionality can be implemented in software or hardware, running in an appliance device, or in a typical server running a common operating system.
PCI Auditors seek evidence of due care.
The PCI Council has not asserted itself as an authority on application security; it leaves the verification of compliance to approved auditors. What the PCI Auditors seek is evidence of due care. Automated tools alone only cover roughly half of the web Application Security Consortium's Threat Classifications. If an application is worth protecting, test it thoroughly with both automated and human means. Web applications are continually changing, as is the threat landscape. Test the application in production as frequently as is meaningful, for example, with each code change. Vulnerabilities identified become a known liability and must be managed. Vulnerabilities must be communicated clearly and effectively to groups tasked with remediation. Testing custom application code must be done methodically, and retesting must follow the same processes where possible. Patch development, validation of remediation, and corrections will be simplified if you follow a consistent methodology.
Web Application Attacks
Buffer overflows, SQL injection and Cross Site Scripting
Buffer overflows and SQL injection are not new, but attackers still manage to make effective use of them to gain access and administrative privileges to databases. Intrusion prevention systems are of use in dealing with buffer overflows. SQL injection is a popular method of attack, since modern databases utilise SQL - Structured Query Language - to enable users to access and manipulate data stored in a database. The basic procedure for a SQL injection exploit is to provide a valid request at the beginning followed by a single quote and a ";" with an additional request appended which contains the actual command the attacker hopes to implement. By piggybacking the "bad" code onto the good code it is possible to trick an incorrectly configured database into carrying out unauthorised executions. Cross site scripting occurs when a web application gathers malicious data from a user. The data is usually gathered in the form of a hyperlink which contains malicious content. The user will most likely click on this link from another website, instant message, or simply reading a web board or email message. Usually the attacker will encode the malicious portion of the link to the site in HEX (or other encoding methods) so the request is less suspicious looking to the user when clicked. After the data is collected by the web application, it creates an output page for the user containing the malicious data that was originally, in a manner to make it appear as valid content from the website.
Finding vulnerabilities in applications
Finding vulnerabilities in web-facing applications
Regardless of your classification as a Merchant or Service Provider, if you have a web-facing application, it must be assessed. This will be far more exhaustive than a network vulnerability scan, and will require authentication to access the majority of application functionality. This testing requires human expertise to exercise the application, validate findings, and test for logical vulnerabilities and other threats a testing tool cannot identify.
Vulnerabilities in custom application code
Vulnerabilities in custom application code can be found in a variety of ways. The Web Application Security Consortium has classified 24 different types of attacks targeting web applications. Half of those threats (13 technical vulnerability classes) can be identified at some level of effectiveness through automated means, including run time code testing as well as source code analysis. As with any detection technology, there is a certain signal-to-noise ratio; human validation is required to separate true vulnerabilities from false findings. There are many variables in application security testing, so your mileage will vary. There are 24 threat classifications, with two current appendices (HTTP Response Splitting and Cross Site Request Forgery), which have not yet been formally ratified into the WASC Threat Classification document.
PCI Requirements 11.3.2 and 6.6 require this. For context, reread PCI requirement 6.1. Proving you have installed a patch to commercial applications and operating systems is easy. Proving you have corrected a weakness in custom application code is a little more complicated. This is where having a consistent testing and reporting methodology will come in handy. If you own the web application code -- fix it. If you do not own the code, or have valid business case or cost restrictions that are impediments to fixing the raw code, correct the vulnerability through other methods (e.g., a web application firewall).
Ensure the mitigation correct the vulnerability in practice and in writing
After significant investment in managing the web application vulnerability lifecycle, an auditor (SOX, PCI, or any other auditor) needs documentation to prove the fix worked. Ensure the mitigation applied does in fact correct the vulnerability in practice and in writing. The PCI 6.6 compliance process of "Find, Fix, Prove" can be simplified further. If the "Find" process is done with sufficient precision and creates proper documentation, the "Find" process can be done in a continual or ongoing manner -- and will in turn document proof of the "Fix" actions as they occur. Auditors like to see trends, especially when they involve continual detection and removal of vulnerabilities -- this makes proving due care very easy. With a clear understanding of PCI Requirement 6.6, compliance is not only achievable, but can provide great value to web application owners and users. This requirement creates a need for visibility into the lifecycle for vulnerability detection and correction, and will serve to mature web application security. Applying metrics to the efficiency of detection, the cost of producing vulnerable code, and the associated costs of correction will only serve to advance the goal of total web application security.
Different types of Firewalls
Traditional Network Firewalls
A WAF is different from other firewall approaches. Traditional firewalls which perform packet filtering only cannot monitor and block by user, which is required for compliance. Also, without a white list security model, this type of solution cannot protect against parameter tampering, session hijacking and cookie poisoning attacks, among others. The bottom line is that network firewalls do not understand enough information about the application and it's state over time to provide adequate application security functionality.
1st Generation Web Application Firewalls
Reverse proxy only Web application firewalls introduce latency, because they terminate traffic and require changes to the network, DNS and the application itself. They may even break applications in the event of large traffic loads.
Application Delivery Solutions with Application Security Add-ons
Layer 7 content switches and first generation Web app firewalls share something in common: generally they both mandate deploying reverse proxies to modify and manage traffic. As a consequence, many application delivery vendors acquired Web app security technology and integrated it into their content switches. However, these joint solutions have retained all of the challenges of legacy Web app firewalls. For example, they often rely on manually defined white lists to validate Web requests. They protect session IDs by signing cookies and obfuscating URLsâ€”intrusive measures that often have unexpected consequences. Combining Web application security and delivery also introduced many new challenges. The extensive regular expressions and content parsing in Web security significantly degrades the performance of application delivery products, upwards to 50%. And lastly, most application delivery vendors do not specialise in Web security, so they do not regularly research new application threats or automatically update security policies.
Holistic Security - Protecting the Enterprise Data Flow
Scanners identify vulnerabilities. If those vulnerabilities are not fixed, but still known, management is accountable. We know that it often takes months to fix vulnerabilities in the application. WAF provides a unique solution: it prevents the vulnerability from being exploited, allowing time to fix the code - thus eliminating the accountability issue. The easiest way to protect against these sorts of exploits
Data at Rest Encryption - configuration files, log files, web pages and data
Web Server Applications are typically accessing configuration files, log files, web pages and data. It is recommended to secure these files by encrypting each type of files using different encryption keys. With a mature encryption solution, all files are encrypted yet access can be granted to any of these types based on the user's role.
Complementing application scanning tools
Vulnerability scanning tools cannot verify cryptographic storage at all. Code scanning tools can detect use of known cryptographic APIs, but cannot detect if it is being used properly or if the encryption is performed in an external component. Like scanning, testing cannot verify cryptographic storage. Also do not use weak algorithms, such as MD5/SHA1. Favor safer alternatives, such as SHA-256 or better. Code review is the best way to verify that an application encrypts sensitive data and has properly implemented the mechanism and key management. Database scanning, i e scanning for database vulnerabilities like default/weak passwords, improper configuration, patch level, etc. since application security is very dependent on database authentication (based on the transparency requirement), for instance a weak password would be critical. This is related to PCI 2.1, 2.2, 6.1, 6.2 and 8.5.
Protecting the Data Flow
Limit the exposure of sensitive data bytes inside applications and the LAN
Many applications have no need to view all bytes in every data field that is passed through. One approach to protect this information in application memory and in transit is to use masking or partially encrypt sensitive fields to hide the not needed bytes from exposure. This can be enabled by using some mode of AES encryption algorithm that is providing full or partial format preserving encryption or preservation of length or data type. This allows arbitrary fields to be encrypted to a given target format. This alleviates the need to change the database, and minimizes the application end point changes to a minimum. Some of these types of AES encryption modes may not be approved for use when permanently storing PCI data.
Validate the encryption mode with a certified PCI assessor
It is always a good practice to check if the AES encryption mode is approved for the type of use that you are planning. You may check with a certified PCI assessor. In Special Publication 800-38A, five confidentiality modes are specified for use with any approved block cipher, such as the AES algorithm. The modes in SP 800-38A are updated versions of the ECB, CBC, CFB, and OFB modes that are specified in FIPS Pub. 81; in addition, SP 800-38A specifies the CTR mode.
Source-level analysis is clearly still required
An application firewall is an excellent solution for protecting against knowable front-end attacks, and can be the only solution for applications where deeper analysis is not possible or permitted. Having said that, source-level analysis is clearly still required, because a majority of customer credit information exposures occur because of issues with access control, authorization, and data storage/transmission. These problems are, and will continue to be, outside the capability of a firewalling technology to distinguish.
Vulnerability Scanners and Application Code Reviews
Manual code reviews have negative aspects like time, cost, skills required etc. but may detect bugs that are hard to find in other ways. Tool-based code reviews may detect a different set of bugs. There are a few different code review alternatives discussed above.
Performance and frequency
Scanning web sites in production can disrupt website performance. Applications, especially Web applications, change frequently, so the target of vulnerability scanning and code review is a moving target, and new vulnerabilities can be introduced at any time. In many cases the application can change before a review cycle has been completed
Attacks change frequently
Attacks, especially Web attacks, also change frequently. A few years ago, no vulnerability scan or code review would have found response splitting problematic. Then a paper describing response splitting attack techniques required developers to send the same code back to review.
Source code may not be available
For many applications the source code is not readily available or understood - and, in some cases, cannot easily be changed by the organization using the Web application. This could be either because the application is a third-party application or because the original developers of a legacy application are no longer available to explain what they did.
Code reviews are often not security oriented
One of the problems with manual code reviews; they are more often done for functionality purposes. It is expensive and time-consuming process to go through manual code reviews like the OWASP-based source-code review. On the code review side, just about all forms of testing options are still on the table. Black and white box, with or without automated scanning assistance, and that kind of flexibility is a good thing. The catch is the person/firm doing the testing "must have the proper skills and experience to understand the source code and/or web application, know how to evaluate each for vulnerabilities, and understand the findings." This goes for tool use as well. That's going to be the little bit fuzzy part since our industry is new and doesn't really have formalized certification or education processes. So it'll be up to the merchant to prove the case to their auditor or bank.
A massive legacy code base
We not only develop code at a staggering pace, we have a massive legacy code base. While many leading organizations follow secure software development lifecycles, and many more will be adopting at least some level of code scanning over the next few years thanks to PCI 6.6, it's naive to think that even the majority of software will go through secure development any time soon. On top of that, we are constantly discovering new vulnerability classes that affect every bit of code written in the past. And, truth be told, no tool will ever catch everything, and even well-educated people still make mistakes.
Senior skills needed
Manual code reviews and manual assessments of scan results are only as good as the reviewer. Skill sets vary widelyâ€¦and can be very expensive. Manual code fixes are only as good as the developer. Skill sets vary widelyâ€¦and can be very expensive. Often, manual code fixing introduces new vulnerabilities.
Application vulnerabilities can be a significant class of vulnerabilities and no scanner or WAF can identify. Application vulnerabilities can be introduced by bad design or programming. The best way to find those vulnerabilities is by a penetration test. Penetration tests should be performed by a security expert and can be better than code review in finding problems from the overall system view.
Comparing WAF with Scanners and Code Review
Web applications and WAF
Companies need to do security code reviews, specifically designed for web applications coding errors and web application vulnerabilities. Then the feedback from the review process - which requires automated tools to integrate into the Web application development design templates and scripts and tools. Web applications are a special breed of living code -- always online, always accessible, always being modified, and always subject to attack. Diligent web application security demands frequent assessment/attack research and findings targeting specific web applications are posted daily.
WAF - immediate protection and without changing the application
A WAF can be deployed to provide immediate protection and without changing the application. Vulnerability scanners and application code review are both still require developers to manually fix code - this takes time and isn't always possible. WAF's Dynamic Profiling technology automatically profiles applications and user behavior, automatically provides accurate protection for web applications and cardholder data, and automatically adjusts as applications and user behavior change to provide continuous protection of web applications and cardholder data, and can be usd to provide valuable information to developers to improve the application under normal cycles.
A WAF is a complement to the development processes
A WAF is useful, and complementary to building security into the development processes. The WAF is probably the best choice in many situations. The WAF is the best first step as it can provide an immediate solution for immediate threats. It can also provide new immediate solutions as other methods uncover issues over time, or as new attack methods evolve over time. even if the customer is aware of web-threats and develops his web-application with security in mind, the customer is aware of the PRESENT threats, Not about FUTURE threats. So actually - today's secured applications will not necessarily stay secured tomorrow. There is a great opportunity for a feedback loop in both directions from WAF to code review and/or pen testing and/or scanning solutions.
Deploy a WAF and build a plan for a long term code review
A WAF can help to balance different options. One issue is that PCI puts two very different techniques against each other. Organisations are going to choose only one technique to achieve compliance, where, in reality, they should be using both. Looking past that, the wording works very well for web application firewalls, in the sense that most organizations are likely to choose to deploy a WAF rather than go through a very long and very expensive process of code review.
WAF is an aid to Web Application Developers
WAF provides critical information on usage patterns and changes in usage patterns that can GUIDE code review teams and point out problems so they can fix any underlying logical issues in the application.
After WAF is deployed, code review and code fixing can proceed at a controlled pace
WAF secures web applications and cardholder data without incurring the time and cost to bring 3rd party consultants or maintaining a separate dedicated group to review code. After WAF is deployed, code review and code fixing projects can proceed at a controlled pace, reducing risk of errors and reducing the extra costs of emergency-mode development. The basic premise is that we need to assume that any browser that connects to our applications is completely compromised. Attacks like cross-site request forgery are just too difficult for the average browser/application to defend against. A big part of the problem is that the browser is a multi-session tool. Unlike most of our client/server desktop applications, the browser needs the ability to mix content from multiple sources, often in a single window. It's how the whole darn Internet works. Some organizations don't understand what an application firewall does or how does, to use it, and may use a network scanner as a substitute for an app firewall.
Selecting a WAF solution
WAF selection criteria
The clarification provides more depth on what is required of a solution in order to meet Option 2 for Section 6.6. Several vendors views this clarification as a positive step for the industry as there have been frequent misleading claims by solutions attempting to claim application security functionality where none in fact exists. The new guidance provides a step in the right direction in defining the specific functionality that Web application security comprises.
An important part of the guidance stresses the need for a solution to provide specific application security functionality, saying: "Increasingly, WAF technology is integrated into solutions that include other functions such as packet filtering, proxying, SSL termination, load balancing, object caching, etc. These devices are variously marketed as "firewalls," "application gateways," "application delivery system," "secure proxy," or some other description. It is important to fully understand the data-inspection capabilities of such a product to determine whether the product could satisfy the intent of Requirement 6.6."
Only a WAF in blocking mode to satisfy PCI 6.6 requirements
Be aware that simply buying expensive WAF hardware does not meet this requirement. Configuring that application-layer firewall to fix known vulnerabilities is required, and entails the risk of mis-configuration, and potentially blocking legitimate traffic to your website -- but you must configure the WAF in blocking mode to satisfy PCI 6.6 requirements that the vulnerability has been corrected.
PCI require a sophisticated Blocking WAF
And the list they provided is quite detailed and extensive requiring a sophisticated product, no marginal network security device with a few webappsec checks is going to cut it here. Of course the catch here is the device must be configured to "block" the attacks, not just alert on them. That's going to be the most challenging part in my estimation as this is not a trivial process. An issue that's not been brought to the front is what happens from a PCI perspective if an organization chooses code review (or if the clarification allows for pen test / scanning in the future) and that review turns up an issue requiring a long fix cycle.
WAF - Critical Requirements
A 'sophisticated WAF' should search for REQUEST vulnerabilities and should look for REPLY vulnerabilities (look for forbidden patterns...). These capabilities are very different from IDS/IPS and network sniffers.
Soft appliance, a hardware appliance or any combination WAF should be able to be deployed as software, a soft appliance, a hardware appliance or any combination of the three. This will enable the WAF to be a completely "green" solution, coupled with deployment flexibility, make it an ideal choice for shared hosting and virtual server environments. A WAF should also be able to operate as an in-line gateway or out-of-band monitor.
Latency issues with traditional application firewalls
Most application firewalls, whether they are implemented as separate reverse-proxy server machines, co-located with the application on the same host machine, or co-located with network firewall machines, generally operate in real-time, intrusively in-line with the applications they protect. This introduces latency while the application firewall examines the traffic, logs the activity, alerts IT Operations and/or network firewalls to suspected attacks and passes traffic on to the application. Additional latency is introduced when HTTPS traffic is examined. For instance, secure socket layer ('SSL') protocols used in HTTPS are terminated and decrypted prior to examination; in some implementations, traffic is additionally encrypted again before passing traffic on to the Web, application, and/or database servers for final HTTPS termination. Application firewalls are not configured to take advantage of security events or behavioral anomalies detected elsewhere in the environment in the same approximate timeframe, although correlation with those events is a typical practice when auditing the forensics of events via log files, long after the events have occurred.
Web application firewalls combined with an escalation system
Automated, synchronized threat monitoring and response between the application level and database level provides a highly effective protection against both external and internal attacks. An escalation system  can solve most of the latency issues with traditional application firewalls by dynamically switch Web application firewalls between different protection modes is described below.
The Future of WAF Technology
Neither approach can solve the web application security problem
It's increasingly clear that no matter how good we are at secure programming and no matter how effective our code scanning and vulnerability analysis tools are, neither approach can "solve" our web application security problem.
Need to change how we view WAFs
I don't think stand-alone external WAFs will ever be effective enough to provide us the security we need for web applications. Rather, we need to change how we view WAFs. They can no longer be merely external boxes protecting against generic vulnerabilities; they need tighter integration into our applications.
Web application firewalls, applications, databases and file systems combined with an escalation system
Think of it as a combination of a web application firewall, an agent on the application server watching activity (what a user clicks on, where data goes) and a database agent or passive monitor watching all SQL activity
A Multi-layer security advisory framework
A Multi-layer Security Advisory System provides a framework to effectively deal with threats of some classes of attacks. The warning system has 5 risk-of-attack-levels (Threat Levels) which when triggered, initiate specific actions by local servers within the same policy domain. Information about data security events is collected from sensors at different system layers (web, application, database and file system). The Threat Level is propagated to systems that are connected within a data flow. The Threat Level will also adjust for time of day, day of week, and other factors that are relevant.
A Score-card to keep track of usage abnormalities
A score-card is maintained for each subject (user or service account/proxy-user, ip-address, application, process â€¦) and object (database column, file â€¦) with a history of processing sensitive data. The score-card summarizes current and historical information about data access patterns for each entity (subjects and users). The score-card also includes a 'finger-print' that reflects historical deviation from acceptable access patterns at the level of s/i/u/d (select/insert/update/delete) operations. A high score-card value will initiate more extensive analysis before releasing data to the subject. The dynamic and automatic altering of the protection policy between multiple system layers includes modifying the protection policy of data at one or several of the system layers. The modification is performed based on a result of the prevention analysis. The score-card can also keep track of when a remote system need to reconnect to the central system to renew or recharge it's capability to encrypt and decrypt data. The policy may allow the local system to only operate stand alone for a certain time or processing a fixed number of crypto operations between each host connection and central password renewal. This behavior will act like a rechargeable key box and can automatically shut down the local access to sensitive data in case the local system is stolen, cloned or compromised in some other way.
Selecting a code review approach
Web development and code management
Security Development Lifecycles
Microsoft's Trustworthy Computing Security
Related to this is Microsoft's Trustworthy Computing Security Development Lifecycle (SDL) initiative. SDL describes requirements for different phases in development, with the main goal to reduce the number of vulnerabilities in software products. It has been proposed that Yyy should follow SDL rules by then end of 2008. For the Implementation Phase it's said that:
Apply coding and testing standards
Coding standards help developers avoid introducing flaws that can lead to security vulnerabilities. Testing standards and best practices help to ensure that testing focuses on detecting potential security vulnerabilities rather than concentrating only on correct operation of software functions and features. 'Fuzzing' supplies structured but invalid inputs to software application programming interfaces (APIs) and network interfaces so as to maximize the likelihood of detecting errors that may lead to software vulnerabilities.
Apply static-analysis code scanning tools and code reviews
Tools can detect some kinds of coding flaws that result in vulnerabilities, including buffer overruns, integer overruns, and uninitialized variables. Microsoft has made a major investment in the development of such tools (the two that have been in longest use are known as PREfix and PREfast) and continually enhances those tools as new kinds of coding flaws and software vulnerabilities are discovered.
Code reviews supplement automated tools and tests by applying the efforts of trained developers to examine source code and detect and remove potential security vulnerabilities. They are a crucial step in the process of removing security vulnerabilities from software during the development process.
Separate code reviews as a way to enhance security
Both PCI DSS and SDL mention separate code reviews as a way to enhance security. In addition SDL mentions the use of static-analysis code scanning tools. Such tools often assists during code reviews, but may also be applied during normal development.
Static/dynamic analysis and other definitions
There are two principal types of program analysis. Static, or compile-time, analysis is aimed to investigate a program's run-time properties without actually executing it. Normally this is performed by source code inspection, but binaries may also be used. Dynamic, or run-time, analysis is performed when observing the program at execution. Testing, debugging and performance monitoring are examples of dynamic analysis.
Example of a very simple static analysis
An example of a very simple static analysis would be searching for specific words like strcpy using a file-search utility like grep. The goal would be to identify where unsafe functions are used.
A search for strcpy will however also list values like strcpy_s. If strcpy is part of a comment, this will also be presented as a valid occurrence. Such output is called a false positive, i e something reported as a vulnerability though not. False positives are a big issue in static analysis since they give the user more data to evaluate than necessary. Each program construction reported as a vulnerability must be considered and reviewed.
Suppose strcpy is renamed to something else, for instance with a macro like '#define mycopy strcpy'.
In this case a word search for strcpy won't list any occurrence of mycopy, though strcpy really is used. This is called a false negative, i e a real vulnerability that hasn't been presented.
An ideal analysis tool
An ideal analysis tool would have no false negatives and no false positives, only true vulnerabilities would be presented. That is however not realistic. Instead they are often somewhat related; a lower false positive rate means a higher false negative rate, and vice versa.
Free open source static-analysis tools
There are different free open source static-analysis tools, as desribed further below. These are only marginally better than the simple word-search as above. They search for specific unsafe calls like strcpy as listed in an internal database, and when found they present the position and a general comment about the problem. This handling gives a lot of false positives, but also a lot of false negatives since they only look for some calls. Such simple tools are of limited value.
More advanced tools
More advanced tools try to interpret the context of the word, based on the programming language used. This is called semantic analysis. The better this analysis is, the fewer false positives there will be. The free open source tools do perform some semantic analysis, meaning at least some false positives will be skipped.
In addition to look for certain language-specific words, advanced tools also look at the general program context. An intra-procedural analysis only looks at what happens within a specific function. This may be inaccurate, for instance when external entities like globals are used. An inter-procedural analysis tries to consider all parameters of the function, and the interaction of functions. This is much more complicated than intra-procedural analysis, considering different possible parameter values and paths for execution. Related to this is flow-sensitive and path-sensitive analysis, which try to consider the program flow and the different paths possible.
Inter-procedural analysis may in some cases not be impossible
Even if supported by the tool, inter-procedural analysis may in some cases not be possible. If there are third-party libraries for which source code isn't available, or there are yet unimplemented functions, the tools can't inspect what happens inside these calls. This may result in false negatives produced.
Tools often try to simplify analysis, to get better performance
Tools often try to simplify analysis, to get better performance. Doing such as inter-procedural and flow-sensitive analysis could consume considerable resources. A tool may for instance only consider min/max values when handling integer input. Such simplifications are also a source of false negatives.
In general, a tool will never be totally accurate, but the better it performs different types of advanced analysis, the more vulnerabilities it will find.
Using static-analysis tools during development
There is a wide range of static-analysis tools, from simple syntax checkers to advanced tools performing semantic, inter-procedural and flow-sensitive analysis. The advanced tools are also getting more advanced for each version, applying new types of analysis and vulnerability detection rules.
Examples of what tools may look at are:
Â· resource leaks
Â· references to NULL pointers
Â· use of uninitialized data
Â· buffer array overruns
Â· unsafe use of signed values
Â· use of resources that have been freed
Â· concurrency handling
Without doubt a tool capable of such analysis would be valuable during development. A tool may however be used in different configurations. The questions are when, where and by whom should the tool be applied? There are different options:
1) When the code is written by the developer
First option would be to run the tool when the code is being written. In this case it's the developer that runs the tool, in the local IDE used. Later versions of commercial tools also support a mixed handling where there are local instances, but still some central repository for handling overall metrics, for instance to see if coding skill is evolving over time.
There are both advantages and disadvantages with the local approach:
Â· it's easier and faster to handle a bug if caught directly when the code is written; the programmers know their own code best, and the code is in current focus
Â· handling a bug locally means it's not propagated to the central repository, thereby affecting other developers
Â· running a tool and interpreting the output will educate the developers in secure coding. Tools have contextual help that explains a given vulnerability and how to handle it
Â· tools are often licensed per user, one tool instance per developer could mean a large total cost for the tool
Â· running a tool too early could mean an unnecessarily high number of false positives. Tools are less precise when the code is in an initial phase, and inter-procedural analysis doesn't really apply when many code pieces are missing (later versions of commercial tools however claim to be better in this aspect)
Â· each developer must be trained in using the tool. Interpreting tool output is for senior developers, with appropriate security knowledge. Marking a valid bug as a false positive could mean the weakness is lost
Â· each developer workstation needs a local installation of additional software
2) At build time, when scheduled builds are performed
A second option would be use a tool integrated in a central build process. The scan is performed when the total application is built. This is an option often used with commercial tool offerings.
Â· a central configuration means a group of senior developers may evaluate tool output before it's distributed to responsible developer, the analysis could be transparent to developers.
Â· providing error reports to responsible developers means education in secure coding still applies
Â· server installations minimizes the number of software deployments necessary
Â· reports are easily generated concerning overall secure coding metrics, and may be accessed by everyone
Â· tool is executed a bit later in the development process, not until the code has been checked into the build system. This will reduce false positives caused by premature code
Â· tool license cost may still be based on number of users, or it may be some general server license. The licensing cost could still be high
Â· bugs are not handled directly, but if the build is performed often the code is still current and not that hard to modify
Â· errors are propagated to the central repository, thereby affecting other developers until corrected
Â· is using a specific group of reviewers, they may become a constrained resource. They will however likely become tool experts after some time, speeding up the process
3) At certain code review checkpoints, by a security oriented code reviewer
A third option would be to use the tool as an assistant during code reviews, to be performed at certain project milestones like before product release.
Â· tool license cost should be smaller, since only a few security oriented users will have the tool. License could however also be based on code size and other aspects.
Â· tool is executed late in the development process, which will reduce false positives caused by premature code
Â· senior security oriented developers are evaluating output before it's distributed to responsible developer, the analysis could be transparent to developers
Â· distributing error reports to the developer in charge means education in secure coding will still apply. Errors have been filtered by the code reviewer, though
Â· reports may be generated concerning overall secure coding metrics
Â· bugs are not handled directly, but rather late in the development process. Fixing an error will take longer time and be more costly, code may not even be current for developer when bug is presented
Â· errors are propagated to the central repository, thereby affecting other developers until corrected
Â· the security reviewer may not know the code, which could slow down the interpretation of the tool output
Â· the group of reviewers will likely become a constrained resource, possibly handling different projects. They will however become tool experts after some time, speeding up the process
All these three cases could apply for ABC development
All these three cases could apply for ABC development. The first case seems like an attractive way to go; the errors are handled directly by the developer and won't affect others, the developers will become more skilled as time goes by. But based on the cost involved to pursue such a configuration, it's absolutely necessary to first verify how good the tool is in the ABC environment. A similar high cost would also be true for the second configuration.
A specific platform library, where system calls are hidden
ABC has a specific platform library, where system calls are hidden. There are different types of wrapper functions and classes used throughout the code. Vital parts like encryption and SSL are implemented in external libraries. All of this means inter-procedural analysis will be important for a ABC analysis tool. Until a tool is tested in the ABC environment, it's not possible to say how good it will be, and it shouldn't be used on a large scale.
Since there are much code already developed in ABC
Since there are much code already developed in ABC, and all this code is about to have a code review, the third option could be a good starter. This will give an indication of general code quality, and should minimize the initial cost for the tool. If the tool used is found to be very helpful, and the tool budget allows, the tool may later be propagated into the whole developer community either as option one or two.
ABC is a highly security oriented organization
Another consideration is that ABC is a highly security oriented organization. A higher degree of false positives could be accepted for a ABC scan, since this normally also means a higher percentage of errors identified. But this also means there will be more time spent with the tool output, each error reported must be evaluated. This is a big issue in a time-constrained environment, which is a reality for many developers. If using the first configuration, an option would be to restrict the vulnerability rule set for local development, and then have a more thorough rule set for a central build or security review.
Tool selection criteria
A static analysis tool would certainly be valuable during development, no matter where it's applied in the development chain. But all tools will of course not be equally good. There are different things to consider when selecting a static analysis tool, especially for a commercial tool. Some considerations will depend on how the tool is going to be used.
1) Multiple language support
The tool must support the programming languages used for development. This is a requirement no matter where the tool is applied.
The main languages used for ABC development are C/C++ and Java; support for these is a basic tool requirement. But ABC also has some code built with for instance C#, PL/SQL and T-SQL. Support for these additional languages would be a further advantage, though not a requirement.
2) Capability in finding vulnerabilities
The principal task for the tool is to find code vulnerabilities. Strong capability in this area is another major requirement. This is a requirement no matter where the tool is applied.
This ability is two-fold; there should be a high rate of true errors identified compared to the total number of errors, but there should also be a low rate of false positives. These are often a bit contradictory; a lower false positive rate often means a higher rate of missed errors.
Being a security oriented organization, a higher degree of false positives could be accepted for a ABC scan if this means a lower false negative rate. The target for a security oriented organization must be to have the smallest amount of bugs possible, even if this means time for analysis will be extended.
3) Integration into development environment
If the tool is to be used as part of normal development operations, it's important that the tool integrates smoothly into the development environment used, for instance Visual Studio. If necessary to run a separate tool, it will likely be less often used than if closely integrated, and additional time must be spent on handling two different output lists.
If used in a central build environment, the tool must of course integrate into what's used there. Since ABC is a multi-platform product, there must be support for at least one UNIX version and Windows.
4) Management of true errors and false positives
Tools normally provide some explanation why an error has been reported. The more specific this explanation is, the easier and faster it will be to evaluate if the reported item really is an error, or if it's a false positive. Good explanations are also important for educational purposes.
When an error has been fixed it shouldn't be listed any more. It should also be easy to mark en error as a false positive when this has been decided, and this mark should be consistent (saved in-between invocations). Otherwise it will be necessary to mark it as a false positive for each execution, and much time will be spent on the same errors.
Related to this is the possibility to have different types of reports generated, for instance providing trends in number of errors reported. This may be useful for education and management.
5) Management of rule set
Tools are often based on some rule set, where different errors are grouped. There will always be false positives produced. It's important that it's possible to tweak the rule set, to adjust the amount of errors and/or false positives reported. A recommendation is to start with a small subset of rules in the beginning, to not get overwhelmed by tool output, and then step-by-step extend the rule set used. In the end a security oriented organization like ABC must be using an extensive list of rules.
Related to this is the complexity to add internal rules, to extend the default rule set. This is for advances users, but may be necessary in some situations, like when using external libraries or to catch certain error types. Writing an extension could mean writing a separate C/C++ library, or using some internal tool language format.
Assuming the tool budget isn't unlimited, price may be an important parameter for a tool. If using one copy of the tool per developer, cost may easily be very high since the tools are often licensed per user.
License cost may often be selected either as an annual fee, or a perpetual one with some maintenance cost.
Considered paths to go
Concerning tool selection, there are three paths to go from here:
* Use free tools only
* Select a commercial tool based on trials with all four possible vendors
* Select a single commercial tool, and either buy a single license or perform trial
1) Use free tools only
Using a free tool for C/C++ like Flawfinder doesn't seem to be an option. Especially since ABC has a platform library, which is the only place for unsafe calls as for instance listed in Flawfinder. Free tools could possibly be used as education sources, learning the unsafe functions if not known already. The GPL license type must also be considered.
Using Microsoft's PREfast should be added to normal development process. All existing ABC C/C++ code should be scanned with PREfast, and before new code is being checked in, it should have been scanned with PREfast (since compilation time will be longer when using PREfast, it probably shouldn't always be used). Code scanning with PREfast will however be restricted to the Windows environment, some UNIX specific parts in ABC won't be handled.
Looking at Java, the FindBugs tool should be a good choice. It has a LGPL license, and is even used in the Fortify commercial tool. All existing ABC Java code should be scanned with FindBugs, and before new Java code is being checked in, it should have been scanned with FindBugs.
2) Select a commercial tool based on trials with all four possible vendors
Using a commercial tool is recommended. A commercial tool will be more advanced concerning inter-procedural analysis than PREfast, and is expected to find more vulnerabilities. The C/C++ code is likely where most bugs will be found in ABC, being less secure programming languages than for instance Java.
The choice of a commercial tool is however not that clear. Based on the public tests available, there doesn't seem to be any major differences concerning bug-finding capability. Different ranking is rather based on tool management.
A general recommendation is to test the tool in the own environment, and most vendors support trials. An environmental test is maybe even more important for ABC, with its platform library and different types of wrapper functions/classes. Strong ability in inter-procedural analysis is important.
3) Select a single commercial tool, and either buy a single license or perform trial
If decided not worth to spend too much time on tool evaluation, one commercial tool vendor could be chosen. A feeling is that all products will provide good assistance. A tool may certainly save time, since more bugs can be handled in an early phase. When evaluating the tool cost, a consideration must be how much time a tool will save, and the earnings from that.
'â€¦An industry rule of thumb is that a bug which costs $1 to fix on the programmer's desktop costs $100 to fix once it is incorporated into a build, and thousands of dollars if it is identified only after the software has been deployed in the field...'
The decision falls back to how many unknown bugs the tool will find, and how early these bugs are identified thanks to the tool.
The case study from company ABC recommended using both WAF and coding reviews. ABC is planning an Enterprise Data Protection approach and protect data across the Information Life Cycle. The primary issues are PCI compliance, and a concern about the escalating threat against financial data from organized crime and insider threats. Time is a critical factor in selecting solutions to prevent breaches. WAF and data encryption will give ABC a quick and solid data security solution to start with. ABC is complementing this with a long term approach including code reviews and scanning. ABC positioned different approaches to prevent Data Theft (and the attack-paths to the data - different types of apps, databases â€¦) including WAF, data (protection (encryption, hashing, tokenizing), C++ code scans. Implementation of Secure Development may take time and ABC are looking at PCI 6.6 as a part of their short term year's budget and install appliances. The appliance is easier to implement and will cost less, based on research done by ABC. WAF is the most effective mechanism to immediately address security issues since the security rule set can be adjusted to stop new attack types without the time required to change the application code. WAF can protect custom applications, 3rd party applications, and legacy applications - even in cases where the organization does not control the source code (as for SAP, Oracle, PeopleSoft web applications and portals) and where the people who understand the application are no longer accessible. The solution for ABC is based on the conclusion that every layer of defence is critical. A holistic and layered approach can provide the best level data security and the sooner sensitive data gets encrypted, the more secure the environment. Early data encryption will protect sensitive data at rest and while it's moving between the applications and databases and between different applications and data stores. An effective code-scanning tool would definitely be useful in ABC development. Being a security oriented organization, it's very important to minimize the number of bugs. The use of code scanning tools is also mandated by Microsoft's SDL. No matter what tool used, this should be accompanied with code reviews, appropriate testing including such as fuzzy testing, code standards that are followed, and proper education. No matter what tool configuration selected, manual code reviews, education, coding standards and proper testing must also be applied.