Static analysis works for mission-critical systems, why not yours?

1© 2017 Rogue Wave Software, Inc. All Rights Reserved. 1
Confronting the mission-
critical software testing
challenge
Episode 2:
Static analysis works for mission-
critical systems, why not yours?
Walter Capitani
Product manager, Klocwork

Presenter
Walter Capitani
Product manager, Klocwork
Rogue Wave Software
walter.capitani@roguewave.com
Twitter: @walter_capitani

1. How do you select a static code analysis
tool?
2. What kind of defects/issues are you looking
for?
3. How/when/where should you deploy static
code analysis?
4. Common myths and barriers to adoption
5. Q&A
Agenda

Poll #1
What is the primary method you use to test code?
• Code reviews
• Unit tests
• Manual tests at build time
• Automated tests at build time
• Automated testing using CI tools

How do you select a static
code analysis tool?

Decision metrics for static code
analysis
• What kind of defects are you looking for?
Security
issues
Memory
leaks
Application
crashes
Other
defects
Improve
quality
Enforce
compliance
Improve
security
Other
• What are you trying to accomplish?

What kind of defects/issues
are you looking for?

• Find common issues in code
• Not easy to spot with the human eye
– Not generally found by code review
– Many are traditionally found with dynamic testing after a failure has
occurred in testing or the field
What kind of defects are we looking
for?
Buffer overflows
Security exploit
or program
crashes
Null pointer
dereferences
Your program
crashes
Memory leaks
Processor runs
out of memory
and locks up
Uninitialized data
usage
Data injection
Platform/OS
specifics
Privilege
escalation, etc.
Concurrency
Deadlock
Suspicious
coding practices
Variable
assignments,
function calls

What is static code analysis?
Performs one or
more processes
Syntax Analysis
Data Flow Analysis
Symbolic Logic
Analysis
Requires
source code
The most accurate
tools must be able to
compile the code
No changes to your
existing build flow
Different types
of analysis
Intra-procedural
(simplest analysis)
Inter-procedural
Inter-file

Syntax Analysis
• Creates a lossless transformation of the source code
• Generates the ‘Abstract Syntax Tree’
• Can be used to find Coding Style Issues and Simple Defects
– Simple security defects (e.g. use of banned encryption API)
– Simple coding style issues (e.g. no dynamic memory allocation)
This function
allocates
memory
Name = “malloc”,
Source Code Abstract Syntax Tree

Syntax Analysis -example
if(i = j) j++;
if(i == j) j++;
Defect: Assignment
operator used in
conditional statement
Assignment operator
replaced with intended
comparison operator
Vulnerable Code
Fixed Code

This seems to work well, but…
• These defects are contained in a single program
statement
• They are not dependent on values from external
functions
• Syntax Analysis can only find a limited set of defects
To find more interesting defects
you need to perform
more sophisticated analysis

Data Flow Analysis
• Monitoring of the lifecycle
of data objects:
– Creation
– Assignment
– Usage
– Deletion
• Must be monitored across all
paths in the Control Flow Graph
– Function calls
– Compilation units
• Can find program crashes across functions and files

Data Flow Analysis - example
• This function a()will cause the program to crash at line 3
• This function g() will cause the program to crash if position is outside
the valid range – how do we know if this will happen?
1 void a(){
2 int buffer[32]; // valid range of 0..31
1 buffer[35] = 5; // buffer access outside valid range (35)
4 return;
5 }
1 void g(int position, int value){
3 buffer[position] = value;
4 return;
5 }
3 buffer[35] = 5; // buffer access outside valid range (35)
Defect: Array bounds
violation

• Data Flow Analysis tracks what potential values are actually used when
function f() calls function g()
4 return;
5 }
1 void f(){
2 g(10,55); // calls function g with position=10, value=55
3 return;
4 }
No defect: values within
valid range
Vulnerable Code

1 void h(){
3 return;
1 }
function h() calls function g()
4 return;
5 }
3 buffer[position] = value; // buffer access outside valid range (35)
Defect: Array bounds
violation (program crash)
Vulnerable Code

1 void h(){
3 return;
1 }
function h() calls function g()
3 if (position < 0 || position >31 0) // Check position is valid
4 return;
6 return;
7 }
No defect
Fixed Code
3 if (position < 0 || position > 31) // Check position is valid
4 return;

This also seems to work well, but…
• Data Flow Analysis alone can only understand actual
numeric values (or ranges of values)
• What if there are no numeric values at all? How do we
determine valid data flow paths?
To find more interesting defects
you need to augment data flow analysis
with Symbolic Logic

Symbolic Logic
• Define functional behavior between symbols
• Don’t necessarily know what the values will be at runtime
• Used to infer software behavior
1 void f(int i, int j){
3 i = j;
4
5 /* set the value of k */
6 if (i == j)
7 k = get_tainted_data(); // Since i equals j, k is tainted
8 else
9 k = 0;
10
11 /* read the value of k */
12 if (i != j) // Since i = j, k will not be used
13 buffer[k] = 0;
14 return;
15 }
3 i = j;
12 if (i != j) // Since i == j, k will not be used

Symbolic Logic
• Symbolic logic determines that since i = j, there is no use of tainted
data at line 13
• Otherwise a tool must “guess” at the defect
3 i = j;
4
6 if (i == j)
8 else
9 k = 0;
10
13 buffer[k] = 0;
14 return;
15 }
3 i = j;
12 if (i != j) // Since i == j, k will not be used

Symbolic Logic
• If we change line 12, then a defect appears!
3 i = j;
4
6 if (i == j)
8 else
9 k = 0;
10
13 buffer[k] = 0;
14 return;
15 }
12 if (i == j) // Since i == j, k will be used
Defect: Unvalidated input in
array index (program crash)
Vulnerable Code

How/when/where should you
deploy static code analysis?

What are we trying to accomplish?
• This will guide what kind of implementations our static code analysis tools
should support
Improve
quality
Enforce
compliance
Improve
security

Frequency of analysis
Once per release
Every check-in
Continuous integration
Nightly/Weekly
Developer desktop
Can be used to ensure that no
issues are introduced with any
check-in
Good for security compliance,
minimizes backlog of work to
do in release phase
Detect issues as they are typed
Most efficient method to save
developers’ time
Typically used for compliance
purposes, limited value to
improving code

Development Cycle
Edit Save Compile Test Check In Build
Analyze
& Fix
• Late stage “rework” reduces tool adoption
• Timelines compromised
• Issues are more expensive to fix
Traditional analysis done after compile/build

 Eliminates new defects from being checked back into the team level build
 No extra work for developers
 In-context checking and fixes
 Continuity of development flow
Edit Save
Analyze
& Fix
Compile Test Check In Build
Development Cycle
Best practice: Analysis earlier in the cycle

 Improves coding practices
 Alerts the developer immediately when they enter a defect
 Provides entire path from “source to sink” of how the issue occurs
 Provide help on how to remedy
 Provides links to the specific coding standards that may be violated
 Allows you to edit and customize that advice with simple HTML editing.
 The key is that not only do we help the developer, by telling them immediately it
is an excellent “teachable moment.”
 Finally, since the developer makes the fix immediately, your code base is never
impacted.
Edit Save
Analyze
& Fix
Compile Test Check In Build
Development Cycle
Desktop analysis advantages

Common myths and barriers to
adoption

Poll #2
What are the barriers to adoption in your
organization?
• Cost of the SCA tool
• Lack of perceived value to developers
• To complex to integrate
• Too many false positives
• No barrier – we use an SCA tool already

#1: The truth about false positives
• All automated safety systems have false positives
– That what static code analysis is:
an automated safety system for your software developers
• Safety systems in automobiles:
– Blind Spot Detection systems
– Back up sensors
• All of these systems will generate erroneous warnings sometimes
• As long as the Signal-to-Noise ratio is reasonable, and we can tune the
system to generate the results we are looking for, these systems add value

Common sources of false positives
• Enabling the wrong defect checkers
• “Developer false positives”
• Third party libraries

How to manage false positives
• Refer to our decision criteria
– Look at defects that serve your purpose:
• Improve Security  Enable security defects
• Improve Quality  Enable memory leak defects
• Prioritize what defects specific developers should look at:
– Advanced static code analysis tools provide different ways for
developers to view defects
• Tune the static code analysis tool to reduce or eliminate defects from third
party libraries

#2: Static code analysis is for junior
developers
• If that were true, it would mean software written by senior
developers would be bug-free
• Sophisticated tools with data flow analysis find issues that
may even get past senior developers, particularly in large
complex code bases (what about 3rd party and legacy code)
• Static code analysis tools evolve over time to find new
security defects – even senior developers will need training
to spot these new threats…

#3: Should test and QA find bugs?
• 80% of defects are introduced in development
• Each defect found in test costs 50x to fix
• Test and QA can spend more time trying to make the
product better, rather than reporting issues that could
have been found at the development stage

Summary

Summary
• Select a tool that supports the corresponding depth
of analysis
Decide what kind of defects you need to find
• Select a tool that supports the appropriate workflow
Decide what you are trying to accomplish
Proper configuration and tuning of the SCA tool helps with
developer adoption
1
2
3

Q&A

Follow up
Free e-book:
Building better code with static code
analysis
www.roguewave.com/programs/building-better-code-with-sca
Learn more about Klocwork static code analysis:
Kate Andreeva
Inside Sales Account Executive
kate.andreeva@roguewave.com

Stay tuned
Confronting the mission-critical software testing
challenge
Feb. 22: What if you could eliminate the hidden costs of development?
Combat different types of development inefficiency by examining error-prone tasks, waiting for
resources, “bug fix crowdsourcing,” and more to learn what the industry is doing about them and
what you can do to get ahead.
Available on-demand www.roguewave.com/sca
Episode 1: How to achieve security, reliability, and productivity in less time
Episode 2: Static analysis works for mission-critical systems, why not yours? (Soon!)

Static analysis works for mission-critical systems, why not yours?

Related slideshows

More Related Content

Static analysis works for mission-critical systems, why not yours?