It’s the never-ending dilemma; the ‘Coke or Pepsi’ debate of the software and security world, and there’s still no definitive answer.
As the application security market grows, so too does the variety of tools available to organizations seeking to secure their applications. And with both open source and commercial tools popping up and solid options on either side, the decision isn’t made any easier to the question emerging in organizations around the world: When it comes to selecting tools for source code analysis, should we choose open source or commercial?
A few months ago, we released The Ultimate List of Open Source Static Code Analysis (SCA) Tools and heard that many found it useful when deciding between the options for open source SCA platforms.
Now we’re taking a look at the pros and cons of open source static code analysis tools versus commercial source code analysis tools. Each has its’ own benefits and downsides, but there are worthy tools with both commercial and open source licenses. What it comes down to are the needs of the organization looking to secure the apps they are building and releasing.
Open Source Pros
Free of upfront costs
The obvious plus to using an open source tool of any kind is that it will usually lack any sort of a price tag. (Note: We’re talking about open source tools that, to quote Richard Stallman, are “free as in free speech, not as in free beer” – but they also will only rarely have an upfront price to pay for the tool). The free nature of the tools make open source an easy choice for those without any budget or with very little budget which would be better spent on customization and add-ons for a tool with no upfront costs. There are other financial and resource considerations to take into account as you implement the tool into your specific environment that we’ll delve into in a bit, but the free upfront cost definitely makes open source the best option for the cash-strapped.
More eyes on the code
One of open sources primary advantages is the fact that anyone can help fix found issues, and in fact many of the open source static code analysis tools, similar to what we wrote about a few months ago are continuously tested and improved by healthy communities. The vitality of an open source tool depends on the constant work of volunteers using the code and either fixing issues or reporting them, and there are plenty of tools that have just that. With communities of developers and InfoSec people behind some of the open source SCA tools, new issues are bound to be discovered, and can be quickly repaired most of the time before any serious damage can take place. On the other hand, however, static code analysis tools don’t exactly have Linux or Mozilla size communities behind them, so it pays to be aware of how active the community is.
Some organizations choose open source tools because of the ability to alter the tool to their specific needs since they have the source code. Once you obtain the open source tool code, you’re free to modify it for whatever you need, if you indeed know how. For those technical enough to know how to modify the tool to best suit your needs, the flexibility that comes with the ability to customize the tool to your specific workflow or for your organization’s business logic properties makes open source a great option. For those just starting out with security tools and not as familiar with the nuances of security queries and mitigations, complicated open source tools may not be the best way to go.
Smart way to ease into Static Code Analysis tools
For those who have never used an automated static analysis tool, open source offers a cost-free (upfront, at least) alternative to get you and your teams feet wet with a new type of platform. It’s a great way to test out tools on your code, see which work best and find how you’d integrate it into your current processes. If you find the tool you chose works well with your development team and their flow, it might be the tool for you. If you find that even after configuring it to your precise needs it still misfires – produces too many false positives, lacks the strength in reporting or integrations with your existing processes, etc. – than it may be time to look into other open source or commercial options.
Open Source Cons
Limited support and liability
Open source projects are normally passion projects taken on by one person or a small group, and the support comes with the ebbs and flows of attentiveness that passion projects usually take on. This means there can be long periods of time in between version updates.
Another possible issue is the style of updates released by the open source community: These public updates don’t share the same responsibility and accountability as their commercial counterparts. The flexibility that open source tools enjoy with public contributions can also make these tools less reliable in some ways, especially if your organization requires a dependable warranty.
More eyes don’t necessarily make all bugs shallow
Unfortunately for many, Linus’s Law has been proven wrong, with a few major dramas caused just last year. Heartbleed and Shellshock dazed the world as two globally implemented open source components, OpenSSL and Bash, were found to have glaring security issues. Jim Zemlin, executive director of the Linux Foundation, stated at the time that in each case, “no one was looking at the code.” And while the issues that would be found in a static code analysis tool wouldn’t have nearly the same kind of impact as Shellshock or Heartbleed, it’s still a factor to consider.
Cost in resources and staff time
Another factor to take into consideration when considering commercial versus open source static code analysis tools is to look at the resources you’ll need to put in to implementing the tool.
‘There is no such thing as a free lunch’ isn’t neglected with open source. While open source static code analysis tools have a much lower starting price, it’s important to consider the costs surrounding the tool, namely:
- Cost of paying your team or a third party to customize the tool to your needs
- Cost of possible gaps in documentation and lack of warranty in most cases
- Cost of paying for support, updating, or additional charges which aren’t upfront
- Cost of risks involved in open source software, primarily issues that may arise with special customizations
Harder to Scale – Especially coding in different languages
In an enterprise organization, it’s rare that all projects and applications will be coded in the same language.
The majority of the popular open source static code analysis tools are built to scan and analyze one specific language. When you’re only working on a few, similar applications, such tools might work. But what happens when you find you need to develop an iOS app when you’ve been developing web applications? If your tool isn’t built for Swift or Objective-C, you’ll need to start the search over again for a compatible tool.
For this reason, if your organization is planning on scaling in the near future, it’s important to keep this in consideration during your search.
Any reputable business offering commercial static code analysis tools will have a strong support team available for customer issues. Along with this is the more personal relationship you’ll have with a company, usually in the form of a dedicated technical support and account manager. The more individual service allows for a more robust relationship with the tool and offers more individualized help with set-up, customization, and questions you and your team may have while using the tool. You’ll know during your trial if the commercial tool you’re trying out is going to be there when you need them, especially if you’re paying for the service.
Main focus on their product
The companies behind these tools are supported by the licenses sold – so there’s that whole added business incentive to offer competitive, well-crafted tools. The research and development departments behind the majority of commercial tools are expansive, and the teams are constantly working to improve the product. While comparable to the communities behind open source tools, commercial source code analysis tools are built for the customer – and the future customer. Teams behind commercial products are always working to provide a more productive, feature-packed and innovative tool for their customers.
In addition to the amount of money invested by commercial tool builders in R&D, there’s usually more of an effort to proactively receive feedback from customers and implementing updates to satisfy their needs directly.
With the additional resources available to commercial tool builders, the results are usually packaged in a prettier, more UI-enhanced way, making the process of reporting to different security stakeholders a more streamlined process. In addition, data on trends over time are most likely to be available in commercial tools, making them a natural choice for those needing strong data on code quality and persistent issues in their code depositories over long periods of time .
More complete vulnerability coverage
Another plus for most commercial tools is the wider coverage offered for paid licenses, especially when it comes to vulnerabilities. As evident in the list of open source static code analysis tools we posted, most open source tools are built to detect a specific, limited number of security vulnerabilities and many are built for one or a specific set of languages. The majority of commercial tools have at least one product dedicated to serve the enterprise, including support for a wider variety of languages and a wider set of out-of-the-box vulnerability coverage.
Long term viability with road-map
As with the company having a main focus on the product providing their financial lifeline, commercial tool builders have more long-term viability. While there’s always a chance that the business behind a proprietary tool will shut down, there’s a greater chance that the one or two people behind the project will abandon it or lose interest. In these cases, it’s difficult to find another developer to step into their shoes – it’s more likely that you’ll need to find another tool. At least with commercial tools, the available road-map will offer a good idea of where the company is headed in the future both near and far, offering you and your security stakeholders more ease of mind in your decision.
Without a substantial budget allocated for securing your applications, the cost of a commercial source code analysis tool could be prohibitive. While there are always costs to take into consideration no matter which tool you’re choosing, as discussed earlier, commercial tools come with upfront costs that will automatically deter some in their static code analysis tool search. And while it’s critical to calculate the approximate ROI of any platform used by your organization, in some instances commercial costs can automatically be foregone by security stakeholders.
Too big companies lose their passion and support quality
One of the biggest downfalls of companies offering proprietary source code analysis tools is when the organization gets too big or loses focus on the tool in favor of other company offerings. Similar situations occur with open source tools, where the developer abandons one project for bigger and better ones, but in both situations it’s something to consider when selecting your tool. How much research and development is being put into the tool now, and what’s in the roadmap for its’ future? These are questions to ask yourself when making a decision.
Only the vendor can fix issues in the code
With open source tools, code vulnerabilities can be fixed almost as soon as they’re found. When you’re relying on a vendor, you’re relying on their expertise for finding issues and also their turnaround times for fixing any problems they find. If you’re choosing a commercial tool, it’s important to know their response rates and how satisfied current customers are with the vendor. Fortunately, with the wider appearance of agile and continuous development teams in both the commercial and open source worlds, updates are being released at a faster rate than ever before – great news for everyone.
The Bottom Line:
The success you’ll have when using either type of tool is only as high as how much time and resources you have to put into the tool to set it up to your exact needs, therefore doing the best job it possibly can. A tool is only as successful as it’s embedded into the Software Development Life Cycle (SDLC), so no matter which choice you end up making, it’s vital to remember: any fool with a tool – is still a fool.
The worst thing to do with either type of tool would be to loosely implement a static code analysis tool (free or of cost) and not fully integrate it into your SDLC. If you choose to go that route, not only will you be wasting the time, money and resources it took to do the tool search, evaluate it for your needs and make the decision, you’ll continue to allow insecure code slip through in your applications.
When it comes to making a decision between open source or commercial source code analysis tools, the choice should boil down to this: What are your requirements? Here are some questions to ask yourself when considering which path to go down:
When looking at tools:
- What’s your budget? Do you know if your upper-management has a preference for open source or commercial?
- How frequently are updates and fixes released for the tool?
- What are your needs in relation to the tools license?
- How large and dedicated to continued updates is the tool’s community or the company behind it?
- What kind of support can you expect from the community or company behind the tool?
- How much training and resources will the tool require to get set up and familiarized in necessary areas?
- What do you know about the tools’ false positive and false negative issues?
- Does the tool come with its own defect-tracking and reporting system?
- Will you be able to integrate the tool with your existing set, if applicable?
For a more thorough assessment worksheet, head to the Web Application Security Consortium, which offers this downloadable and printable evaluation criteria sheet to work off of.
Make sure to read our AppSec How-To: Choosing Your SAST Tool for a dive into best practices when selecting your AppSec solution!