Software Assurance

Making sure software apps function as intended and are free from known vulnerabilities and implementation flaws

Recently, I was asked to examine an acquired Software as a service (SaaS) app to see if it needed updating because the acquirer felt it was based on some older technology and needed to understand that more fully. This was for a very mature system with little or no new feature development planned. My task was to make whatever recommendations were necessary without planning new product features.

There's quite a bit to unpack there. So lets try to get to the bottom of it and come up with a plan.

TL:DR — Some steps I took, and all the gory details of how I thought it through using the principles of "Software Assurance" to provide a foundation for recommendations to bring an acquired SaaS app to a state where security and service can be assured by the acquiring organisation. Hope its useful. If you need help with this kind of work do get in touch! If you'd like me to come and talk about this at your meetup also contact me!

Check platform dependencies, licensing, documentation, safety, look for code smells and quality security issues update all and republish

To save you time looking for a simple checklist based approach to this, let me say that if you find one it will likely be too simplistic for your particular situation. Every software product is created and brought to life uniquely and once a software product or business is acquired much of the context around that can be lost. You will need to unpick the software platform, understand it, then fold it back up and stitch it in to the acquiring organisations way of doing things. There is no one size fits all approach to software assurance. That said, there is a wealth of best practice you can lean on in deciding where to focus and to inform your plan.

Some of this best practice information is to be found at SAFECode, a global industry forum sponsored by major software industry luminaries where business leaders and technical experts come together to exchange insights and ideas on creating, improving, and promoting scalable and effective software security programs. They talk in detail about something called "Software Assurance"

Defining what is meant by software assurance

SAFECode define what software assurance means and make a whole series of recommendations for best practice.

This is exactly what I was being asked to do. Software assurance is a good term and worth adopting as a shorthand for the challenges, tools and competences which need to be managed in order to bring about confidence in a software platform. Importantly, SAFECode reinforce that there is no simple approach to the issue, but they do provide a set of processes on which build such confidence.

This is exactly what I was having some trouble trying to articulate. A heroic effort or checklist wasn't going to be enough to provide ongoing confidence and really would just be setting up some future failure event. It has to be about a commitment to an ongoing process. The nice thing about SAFECodes definition is that it provided a set of foundations to use as the headings by which to go ahead and examine the SaaS app in question. The SAFECode information and papers are well structured and good relevant reading that can be put into action on projects of any size!

The elephant in the room with a software acquisition — a loss of decision accumulation and a lack of software assurance

Apps that are acquired by a business as a product that is in production are quite different from new projects and should be evaluated against various factors that can impact the success of the acquisition. One key problem with acquisition is that you may not have the whole context of the app. Don't take it from me, take it from Sarah Drasner, Director of Engineering, Web Infrastructure at Google and author of Engineering Management for the Rest of Us which if you are reading this article you probably like! It is a thoughtful book by a reflective practitioner in the field.

The greatest example of all time of a Software as a Service (SaaS) app acquisition leading to loss of decision accumulation — Twitter under new management

Sarah beautifully crystalised the issues involved in acquiring an app without all the context, what she calls "decision accumulation", using the greatest example of all time where a new team has taken over a mature gigantic SaaS app in service — Twitter's recent acquisition and evisceration of its staff.

Sarah said (on Twitter, indenting and list format mine for readability):-

Sarah went on to add:

Why do we all agree on this?

• Legacy and heavily used systems have decision accumulation over the years: some decisions were made because of the technology available at the time, some due to a feature need, some due to dependency management, some were made to make the system more secure or to test it.
• When the staff were fired/walked out the door, they took with them all of this context.
• The existing code can tell you what it does and how, but it can’t tell you why.
• Good devs are curious: they explore context before they get to work.
• None of that happened, under the assumption that “I’m so smart, I can figure everything out on my own, quickly.” You see this a lot with junior devs or folks who haven't had to maintain large systems over time.
• Let’s say a massive rewrite was even feasible for them, which I have high doubts of. This doesn’t take into account the opportunity cost of the time spent doing this.
• Do they have SLOs in place? How will they maintain those while rebuilding?
• Even if all of those were easy, which they aren’t, and even if they had a perfect new system all set up, what about the cost and risk of a massive migration? Migration costs are huge, and ensuring stability in that process even more so.
• How are they testing any of that new infrastructure, and given that this is probably a multi-year project, what’s the plan for shipping new features in the interim? Who is going to set up security and trust reviews for the changes they are making (critical in social media)?
• If you have a lot of hubris with no curiosity, you might not understand: how to build upon lessons learned from the old system, the opportunity cost of trying something new, the complexity of a massive rewrite, or why the system became complex for the current features
• A dev who can build something alone quickly may not have the skillset to evaluate these tradeoffs. Deciding that if you are smart and can’t figure it out, it must be broken is irresponsible.

This is why a curious mindset is one of the greatest strengths an engineer can possess."

So how can you provide software assurance for an acquired app or software solution?

So what do you do about the Elephant in the room, this loss of context, of decision accumulation? You might be tempted into thinking that because your app runs that is good enough. Accepting the product into service in the acquiring organisation requires more. You need to be able to ensure that it conforms to the acquiring organisations standards for service quality, and any relevant regulations that might apply. In order to do that you need to get to know whats going on underneath the surface.

In my experience, there is nothing that gets out of date quicker than software that is not maintained. It may have been in decline or subject to decisions taken in haste for short term reasons.

Approach it like a car with no service history

Assume it to be like a car you bought at auction and don't know the history of. You will need to check it over carefully and make sure everything is working the way it needs to, nothing has been bodged or painted over for sale and it isn't dangerous to drive!

When you buy a used car, you would normally check everything works according to the original specification in the manual, and that any worn out parts like tyres or brakes are replaced, then ensure any maintenance or safety advisories that need to be completed are done. After doing this your car should be capable of performing safely exactly as it was designed to do. Software assurance is a practice that is similar and follows the same kind of process with the objective of the technology performing safely exactly as it was designed to do.

It is possible you will need to recommend that you pause the software service to remediate very serious issues, but to do this you risk alienating customers. It is even possible you might have to consider cancelling the product altogether. Hopefully, you won't need to do recommend such drastic action.

Just like servicing your car and making sure it has good tyres before adding a fancy expensive exhaust, you should take the steps toward software assurance to ensure your SaaS app has an acceptable level of service before you start to add new features according to your product roadmap. 

Recommended steps towards software assurance

Step 1 — Review the background and context

The software you are reviewing isn't likely to be as gigantic as Twitter but all these considerations form part of the process of review and comprehension of what it is. The first step is to make a list of the things you know about the SaaS app. Find out as much as you can about it.

• Is it in production?
• Does it have paying customers?
• Are any of the existing team available now?
• What service level objectives or guarantees or targets does it have and are these contractual?
• What languages are used of the app and the platform?
• What functionality outside of the core features could be consolidated?
• Does it support payments, invoices?
• Is there an API?
• Does it use third party API's to connect to other services?
• Where is It is hosted?
• Do you have access to the source code and a development instance or staging instance?

Step 2 — What documentation is there?

Production SaaS apps soon depart from any originally documented minimum viable product (MVP). Documentation may be written as comments in code, documents, wikis, support tickets or just readme.txt files in the code.

Documentation is unlikely to be as clear as it was at the outset. Updates and new features can be quite fluid because they need to reflect sometimes conflicting demands of the market, defects in the system, platform updates, regulatory changes and competitive or sales requirements. In a perfect world a product manager would weigh all of these inputs in order to deliver the right blend of features at the predicted cost and with the right level of quality to meet the requirements of the business and produce new documentation or change requests. Often details are far more opaque and may require trawling through lists of support tickets and email or other documents.

Sometimes there is no documentation at all, and sometimes the software itself does not represent the behaviour that has been documented.

Step 3 — Conduct a review of the software platform and environment

You are going to need to determine the state of the software and its platform in terms of the code base, your future ability to maintain it, and your ability to operate it. A good way to start to answer these questions is to ask yourself, "Is it legacy code", followed by, "How reliable is it?", "How maintainable is it?", "How will it be deployed?", and "How secure is it?".

Is the software Legacy Code?

A good definition of when software source code becomes legacy code can be found in Elliote Harolds presentation about testing legacy code. Elliote says that code can be considered to be legacy code if the following indicators are present:

• It pre-dates the current team,
• is built with technology which is unsupported now,
• it does not have an automated test suite,
• there is no secure code deployment process with governance

Testing Legacy Code

"Learn strategies for retrofitting test frameworks onto existing code, and developing a test suite for code that never had one before."

Testing Legacy Code at Devoxx Belgium 2016 - Elliotte Harold

Making a determination that your software is legacy code is a good yardstick but is only part of what needs to be reviewed. If you do have legacy code without an automated test suite all is not lost. You can use the strategies described by Elliote Harold to contain the scope of your work and provide useful data.

Is the software reliable, maintainable and secure?

In addition to test results you need to find out how reliable, maintainable and secure your SaaS app is. This can be be achieved by using a code inspection tool to do a one-off baseline scan of your entire source code repository.

Use the data you find to review of the software platform and environment. At this point a checklist might prove helpful. Here's an example using the legacy code and code inspection results as headings for review.

Figure 1. A simple software review checklist

Software review checklist issues explained

a. Does the software and its platform pre-date your current team?

You may not have access to the original development team. If you have, then that will help navigate the structures of the code and will help explain some of the decision accumulation that made the solution what it is, but it may not help you with your objective of providing software assurance. In my experience your access to a former team, if any, is likely to diminish quickly over time as they either get new roles in an acquirer or elsewhere. You should act quickly to get what you need to know about the software. You might consider short consulting engagements to get the knowledge transfer you need. This can sometimes work well, and sometimes not so much!

b. Is the software built with technology which is unsupported now?

A well ordered SaaS application written, for example, in HTML, CSS and PHP, will naturally rely upon many dependencies in order to function. These are not inherently bad. think of it as standing on the shoulders of others, and not having to continually re-invent the wheel. Dependencies include hardware platform specifics, third party software libraries, as well as any third party systems or services perhaps accessed via an API. It is important to consider all the dependencies found, as they can affect the cost and complexity of integrating a SaaS app into your existing infrastructure. There is no way to avoid looking at the code to see what third party technologies are lurking deep in the directory structure. You will need to check for any dependencies that are out of date, obsolete, or defunct, and for dependencies outwith the directory tree, for example configuration files in specific locations.

Software which is unsupported can cause serious issues such as:

• security vulnerabilities,
• the inability to access newer more effective functionality in revisions of the dependency concerned,
• lack of support for changes to web standards, technologies and web browsers,
• pre-dating regulatory changes,
• other concerns in terms of warranties, licensing, fitness for purpose etc.

You should create a table or spreadsheet listing all the dependency issues you encounter and their status. Here is a fictitious example:

c. Does the software and its platform have an automated test suite?

With a legacy code platform there may be no reliable testing framework in place and you will need to be able to provide assurance both that changes don’t have unintended consequences to the existing SaaS app and that new development is properly tested.

Inspection and testing are a benefit, not an overhead to development because they make it easier to find bugs and other design defects. Tests make it easier to add features or make small changes with assurance. Tests make it easier to refactor source code over time. Tests give confidence that the software is working as designed.

Is there at least a smoke test?

Smoke testing

"Smoke testing is preliminary testing to reveal simple failures severe enough to, for example, reject a prospective software release."

Smoke testing (software) - Wikipedia

Difference between testing and existing platform reports

You probably have logs and reports from the SaaS app in production. These aren't tests. Platform reports are after the fact information in response to activities, backups, transactions etc. Tests are before the fact indications of quality and capability before deployment into production.

d. Is there a secure code deployment process with governance?

Software projects need to exhibit best practice governance over source code in order to provide the foundation for software assurance. It is a basic fundamental, but in an acquired technology you may find that there are missing governance steps, or that things are just assumed to work because they were always done by 'Dave', or that the process is otherwise subject to short cuts.

There are a variety of software and platform solutions available to ensure secure deployment of software. There are too many and projects vary too much to make a specific recommendation aside from to say choose wisely from a vendor you trust!

That said, some tools that provide extensive governance over software deployment that I have extensively used include, GitHub, a collaborative version control system owned by Microsoft, Assembla a web-based version control and source code management software as a service provider for enterprises and AWS CodeDeploy, an automated code deployment system designed to maintain application uptime owned by Amazon. There are many others. Your choice will be influenced by many factors including the technology stack your organisation has standardised on.

e. Is the software reliable?

Reliability is about bugs. The total number of bugs, their severity and the amount of effort required to remediate them. An inspection report from a code scanning tool can help by telling you how many bugs you have at a point in time.

Code scanning tools are reliable ways to check for coding errors in languages that they support. They can save hours of time on bugs from simple typos, or syntax errors to more complex logic problems.

Removing demonstrably wrong code increases the reliability of your code.

f. Does the software pass security review?

Security review is about finding all the vulnerabilities in software source code that could be exploited by a hacker or other bad actor.

These security issues can be caused by vulnerabilities or by code weaknesses. Vulnerabilities are things like out of date security algorithms. The total number of known vulnerabilities should be zero, but if it is not then you need to know the amount of effort required to remediate all known vulnerabilities. Code weaknesses are things like insecure password variables in source code. Obvious failings such as hardcoded credentials in code have no place in professionally managed software. These impact security just as much as known vulnerabilities and need to be eradicated.

g. Is the source code maintainable?

Maintainability is a measure of technical debt, For example lack of test coverage or code smells particular to the language concerned that indicates a deeper potential problem. Code with code smells often works, but might be hard to understand due to odd naming of functions or variables, dupication of near identical code, too many loops or branches and the like.

Code smells

"A code smell is any characteristic in the source code of a program that possibly indicates a deeper problem. Determining what is and is not a code smell is subjective, and varies by language, developer, and development methodology."

Code smell (computer programming characteristic) - Wikipedia

Code inspection tools

Continuous code inspection provides metrics which can assure confidence in developed software. Automatic reviews of source code with static analysis of code to determine code quality, detect bugs, code smells, security issues and duplicated code should be performed in addition to Smoke tests and Unit tests.

SonarQube, is an example of an automated code inspection tool. It is open-sourced by SonarSource, and offers reports on duplicated code, coding standards, unit tests, code coverage, code complexity, comments, bugs, and security recommendations for around thirty software languages.

SonarQube consolidates these in a dashboard under the headings of Reliability, Security, Security Review and Maintainability. SonarQube can record metrics history and provides evolution graphs. SonarQube also provides fully automated analysis and integration with most build systems.

You can drastically improve software assurance by making automated code inspection part of your process.

As well as the community edition, there are commercial services offerings from the creators, SonarSource - who have large numbers of top software organisations as users.

SonarQube

"SonarQube provides the capability to not only show health of an application but also to highlight issues newly introduced. With a Quality Gate in place, you can Clean As You Code and therefore improve code quality systematically."

Sonarqube is licensed under the GNU Lesser General Public License v3.0 - GitHub

Is the source code ‘DRY'

"Don’t repeat yourself" (DRY) is a common principle of software development aimed at reducing repetition of software patterns to avoid redundancy, reduce technical debt, and improve software maintainability. For example, a given subsystem (for example ‘a payment’) must have a single, unambiguous, authoritative representation within the whole system in order for it to be DRY.

Indicators of potential headaches include duplicated code, duplicated dependencies directories. Fixing these things removes code, removes ambiguity and increases testability.

h. What is the process for development and deployment lifecycle management, (Devops)?

DevOps is a software development methodology that combines software development (Dev) and IT operations (Ops) to "shorten the systems development life cycle" by providing continuous delivery while " ensuring high quality". It aims to bridge the often found gap between development and operations in organisations, and to improve collaboration and communication between teams with different and sometimees competing priorities.

DevOps

"a set of practices intended to reduce the time between committing a change to a system and the change being placed into normal production, while ensuring high quality""

DevOps(set of software practices) - Wikipedia

You'llneed to familiarise yourself with the mechanism for committing changes to the source code and placing them into production. Hopefully it isn't just copying a file to a location on the server using FTP!

i. What are the possible legal and reputational damage risks with the software as is?

The security of an acquired software based product should not be a tick box exercise. The new owner will be warranting and providing services based on software that they have not written but are maintaining and developing further. A security issue could cause severe repercussions for example from a data breach or loss of revenue from the service. You should review the software with standards set by the acquiring organisation for services in production for example any quality certifications or other standards adhered to.

Step 4 — Make recommendations for delivering software assurance for the SaaS app

Bringing the software up to date so that it can be accepted into service

Here's my list of recommendations:

1. Go or no go decision point
2. Address immediate critical bugs and vulnerabilities in legacy code by refactoring for these things alone,
3. Adopt a systemic approach to bringing the platform up to date by incorporating ‘devops’,
4. Appoint a product owner and assign or contract development team resources,
5. Implement secure source code control and deployment management,
6. Improve test coverage with a smoke test and then unit testing of all refactored work,
7. Automate code inspection, with a metrics goal of zero issues over the project period
8. Refactor for remaining reliability issues and maintainability.

1 — Go or no go decision point

You may be find yourself in the uncomfortable positioning of dealing with issues so intractable that the only sensible course of action to you should be the cancellation of future work and termination of the platform. There are a whole series of organisational and management issues that may then arise as consequences but from a technical perspective at least the recommendation could be as simple as "This SaaS app needs to go into the end of life stage of the product cycle". Ask yourself if the software can be remediated? Is there sufficient resource in your organisation to do it? Will it be cost effective to do so? Make the right recommendation and be prepared to justify it.

This is easy to do if the recommendation has a foundation in facts. Decent management understand the difference between facts and pushing an agenda. Be aware that there may well be business imperatives you have no visibility of that override your recommendation. You will have shone enough light on the issues at hand to inform the ultimate decision makers. Afterwards if your recommendation is not accepted and further issues emerge and there is some kind of collective amnesia about the reasons identified you will be able to refer people back to your recommendation.

I've had to recommend cancelling a running product in a major telecommunications company in partnership with a major vendor before and it was a very unpopular decision with the team and their management and the vendor at first. It was the right one, because it was burning cash at an alarming rate, had unrealistic revenue forecasts based on a price point the market was not prepared to pay for the service and in any case it did not meet the needs of the customer segment it was aimed at. There was also a gigantic flaw in the design which resulted in a single point of failure in a so called high availability design. Eventually, resistance was overcome, partly I think because someone (me) dared to say the unsayable, that the product needed to go and that needed to start with a timetable for end of life management. Making that decision cost me some quite a bit of short term personal capital but saved a great deal of money in the longer term and avoided the wasted opportunity cost continuing to expend resources on what was an obsolete technology. It is perhaps, the product decision of which I am most proud, although it didn't seem like it at the time.

Commercial and Licencing issues

Licensing is an important factor to consider when making your go or no go decision. You should carefully review the terms of any licences for any software used including any restrictions on use, and warrants given to particular customers and ensure that these licences are appropriate. This should not be ignored or marked as too complicated.

2 — Address immediate critical bugs and vulnerabilities in legacy code by refactoring for these things alone

it is important to set the expectation that this work does not all need to be done "at once", but that the critical items must be addressed first and then over time the other issues can be addressed by further refactoring and improvement. This is a marathon not a sprint! Set out your planned deliverables for each work package in a spring or group of sprints aligned around a theme. You will need a smoke test before you can test the results of updating all your third party dependencies. The smoke test is not the goal, the theme of is about providing software assurance by having the latest supported and relevant third party dependencies! Themes for work are important framing for your management who will not necessarily understand the nuances of all the tasks but will understand the overall theme that you aim to fix all known security vulnerabilities in your third party dependencies.

Assuming you decided to go on, you will need to examine customer contracts or end-user licence agreements for any service level commitments and possible financial risk of downtime or a long period of maintenance.

Don't attempt to rebuild from scratch

If you decide on a rebuild from scratch you will be taking on long term overhead costs and if you keep the current system running while you rebuild you may still have to make emergency changes or even add new features and have to face issues around testing and software assurance anyway. You will either extend the timetable or add additional cost.

You will also be taking the risk that a new set of bugs and issues won't end up being significant issues previously encountered and resolved or just in different problems in different areas than the current code you have. Your team will inevitably become one further step removed from the decision accumulation in the original software. I have experienced engineering managers new to a company demanding a complete rewrite to a particular favourite language or framework on the flimsy justification that they have been successful with it in previous work and therefore it is right this time. This is a rookie error, but a surprisingly common one.

Keep the lights on

Remember the comments about the recent acquisition of Twitter above. Remember that a migration to a completely new code base is likely to present its own set of risks and huge costs. Look what happened to Netscape and they planned their rewrite meticulously and deliberately!

Wait, what happened to Netscape?

"The decision was made to start from scratch using the standards-compliant rendering engine... This decision meant that Netscape's next major version was severely delayed...The browser struggled to make an impact on a disappointed community."

Netscape 6 - Wikipedia

Really, don't rebuild from scratch unless theres no alternative

The only exception for taking a product and starting from scratch would be if the framework or language used has become completely obsolete and unsupported. For example recently the mobile app development framework known as Appcelerator Titanium was discontinued by its acquirer which meant that apps based on it would become obsolete despite a valiant effort to make all private source code of the Titanium SDK public in the open source Titanium SDK GitHub repository. Appcelerator apps were written in Javascript and their own specialist MVP framework called Alloy. Dart and Flutter were just emerging from Google as replacement technologies and with Firebase as the replacement to the Appcelerator Mobile Backend as a service (MBaaS) and this was my recommendation for a particular clients apps. See: The Appcelerator offering has been discontinued (Axway)

It took a significant development cycle of around six months of elapsed time to rewrite these apps. This had an objective of feature parity with what we had before and one additional major technical change in functionality but it was well worth it. The apps are now using a supported, well resourced open source framework for building beautiful, natively compiled, multi-platform applications from a single codebase. It is quicker now to add features, the support for new versions of operating systems that apps need to run on is much improved, the development tooling is far better, the community developed packages are excellent at helping avoid re-work of commonly needed mobile app functionality. Flutter is now mainstream but this was a very big decision to recommend back in 2018!

3 — Adopt devops practices

DevOps emphasises automation and monitoring at all steps of software creation, from planning, development, building, testing, publishing to production, operating and monitoring in a cycle that repeats and is always running. This is why it lends itself to the use of Continuous integration and continuous delivery (CI/CD) tools. It also encourages the use of agile methodologies to improve the flow of work, and to deliver value to customers faster.

DevOps best practices include:

• Continuous integration and continuous delivery (CI/CD),
• Automation of infrastructure management and configuration,
• Monitoring and logging,
• Microservices architecture,
• Agile methodologies for software creation,
• a cultural shift towards collaboration and communication between development and operations teams.

DevOps adoption is a vital part of software assurance for a SaaS app and its platform.

4 — Appoint a product owner and assign or contract development resources

You'll need someone in the weeds of the requirements to manage the development resources you put in place and drive theme based sprints towards delivering software assurance. This needs to be measured. You have the baseline, that is to say, the number of issues and the detailed analysis of the source code. You can set a goal of a specific level of software assurance for a given amount of time and effort. This is detailed work and you can read much more about difference between a product owner and a product manager over at Product Focus.

Product Owner

Development resources

In terms of development, you may be able to use existing resources, contract a third party company, or work with independent subcontractors. All of these approaches can work well provided you have the required budget for the task, a clear set of deliverables and set up your devops environment for the collaboration and communication required for success.

5 — Implement secure source Code control and code deployment management

Secure source code control and code deployment management are critical tools for oversight and provide clear benefits, including:

• Version control: — Secure source code control systems like Git allow developers to track changes to their code over time, making it easier to identify and fix bugs.
• Backup: — Source code control systems also act as a backup for code, allowing developers to restore previous versions in case of data loss or corruption.
• Collaboration: — Developers can work on the same codebase simultaneously without overwriting each other's changes.
• Deployment management: — Code deployment management systems automate the process of deploying code to different environments, such as development, staging, and production. This allows for a more efficient and consistent process for releasing new code.
• Security: — Protection against unauthorised access to code and unauthorised changes.
• Automation: — By automating the build, testing, and deployment processes, developers can focus on writing code and not on the operational aspects of the software development process. 
• Repeatability: — Each time you use automated tooling you are sure that the configuration settings and inputs to build, test and deployment are performed in exactly the same way. 
• Speed: — Automating the deployment process allows for faster delivery of software changes to end-users.
• Auditing: — An audit trail, allowing developers and system administrators to trace the deployment of specific code changes and configurations.
• Rollback: — The ability to rollback to a previous version of the code in case of issues or errors.
• Environment separation: — Code deployment management systems allow to separate different environments (development, test, production, etc) and deploy code accordingly, so it can be tested before it goes to production.

Automated code deployment

Automated code deployment is crucial to providing efficient repeatability for frequent tasks. It reduces the prospect of human error in deployment to almost zero because deployment is automated as part of the check in and merge process. It is another must! I have used GitHub Actions to build and test, Assembla integrated with GoCD continuous delivery of software and recently AWS CodeDeploy, which is particularly relevant if your platform is on AWS. There are so many tools to choose from that there is a curated list of build automation software and a comparison of continuous integration software at Wikipedia. You can start small but do start. Automation saves so much error prone repetitive rework each time you deploy a build to dev, test or production that investing in automation here is a no brainer.

6 — Improve test coverage with a smoke test and then unit testing of all refactored work,

Smoke Test

Because we are talking about a mature system in production, the first step to achieving software assurance must be to develop a smoke test for the whole system. You will dramatically improve software assurance by writing an overall test for the main part of the system. A smoke test should look for obvious answers and obvious bugs. It should be runnable in minutes so that every single change in development can be tested to see if the change breaks anything. Otherwise how will you know if a change you made in one part of the system had an unexpected consequence in another part?

Unit tests

Unit tests should be added for all new or refactored functionality. These detailed tests should be part of a secure software deployment process and include the obvious path and corner cases as needed. An example testing framework is PHPUnit - a programmer oriented testing framework for PHP. Remember though, what Elliote Harold says in his YouTube presentation (paraphrased), which is that 100% test coverage is not possible for legacy code until it is completely refactored, but more tests are better than fewer tests and some tests will be better than none at all.

Automated tests

Tests must be completely automated. Failures must be clear and obvious. Any proposed changes must have unit tests as part of the development process. Eventually, after a long haul, comprehensive test coverage will be achieved.

6 — Automate code inspection, with a metrics goal of zero issues over the project period

Adopt a methodical approach to code inspection and statistics. SonarQube call this "Clean as you code" in a nod to McDonalds workplace mantra "clean as you go". Make the reporting in your chosen tool align with your product goals and use it to focus on the issues at hand. SonarQube organises this well in a dashboard focused on Reliability, Security, Security Review and Maintainability. Make it part of your check in and build system to run via your automated code inspection tool and produce reporting information. Review the reporting information as part of your development process. Block deployments that fail code inspection against whatever level of quality makes sense for your project.

7 — Refactor the software code.

Firstly, build your smoke test. Once you have it, you can start to address the critical security issues and security review of your software code.

Your first work will be to remove identified security issues and vulnerabilities and redeploy your code, testing and iterating until it passes your smoke test. You should not undertake this refactoring until you have some automated testing in place, otherwise you'll get in to an unknown state, and run the risk of quickly encountering widespread issues with no baseline of what the expected outputs should be.

Once testable you can mitigate known security vulnerabilities by reviewing the latest supported and relevant third party dependencies and updating, sometimes deprecating them where required. Remove all known security issues from your source code by adopting best practices for credentials management, protocols used and so on. This could touch quite a lot of your source code and require quite a lot of change and needs a good methodical approach to be successful.

Now you can go on to tackle reliability issues and maintainability. This includes evaluating the app's performance, reliability, and usability and design. It is also important to consider the app's scalability, as this can impact its long-term viability.

Design

A mature app needs to be evaluated to ensure compatibility with current technology including web browsers and operating systems and for mobile app review guideline compatibility if it is destined to for delivery as a mobile app in the App Store, Google Play, or another destination app store.

New functionality

Now you've created a revision of the acquired software capable of being accepted into service by the acquiring organisation you can start to think about new functionality. You are going to want to develop a product roadmap taking into account resources, available budget, a list of known requirements, and your overall product strategy.

But that is for another day.

See also:

• Assessing the Security of Acquired Software: One size does not fit all!
• Fundamental Practices for Secure Software Development
• SAFECode and the Cloud Security Alliance (CSA) Release Guidance for the Secure Development of Cloud Applications
• Testing Legacy Code - Elliote Harold (YouTube)
• Things You Should Never Do, Part I (rewrite the code from scratch) - Joel Spolsky
• The evolution of PHP - PHP Annotated (Youtube)