If a project runs over time and over budget, then commitment to software testing is the first thing to go by the board. That’s according to the chief executive of one vendor who doesn’t want to be named, as he deals largely with government departments.
“There is a distinct lack of interest in testing in the public sector,” he claims. “There’s far more interest in the private sector.
“It is evident those organisations that have moved to Agile development factor testing into each iteration. Therefore, if things get tight [in time], they can be rapidly addressed, compared to trying to shave three or four months off at the end of a big project [when testing is cut].
It used to be quite common in government projects for the budget to be broken down into percentages for individual components such as testing. But methodologies have changed to a large degree. There’s Agile and there’s iterative, which encompasses such technologies as .NET and Java.
“Testing is an area of IT that has never been seen as sexy,” the chief executive says. “It is about a group of people trying to break something and, as such, is almost considered as a negative force in systems analysis.
“It has become the poor cousin. But without it, all the good things will be undone if the reputation of what has been built suffers from poor quality control.”
He claims that in the public sector there are not always the people available who have the wherewithal to drive testing. Budgets don’t necessarily cater for large numbers of people who may not be employed 100% of the time after projects finish.
He wonders whether offering testing as a shared service might be a way around this.
When several government departments were asked what percentage of project budgets they generally allocated to testing, their answers varied, considerably in some cases. Some commented that it wasnt possible to provide a precise percentage set aside for software testing, due to the number of variables that must be considered when comparing IT projects.
“Aside from the size and nature of projects, there are also different types of testing, such as unit testing within the software development phase and then regression, integration, user acceptance and performance testing,” a representative said.
“Taking that into account, the ministry will generally dedicate between 15% and 20% of a software project’s budget to testing. There may, on occasion, be individual projects that fall outside that range due to their specific characteristics,” a spokesperson said.
Another government organisation said that in 2010-2011 it spent between 7.5% and 8.5% of IT project budgets on testing.
A spokesman says the testing costs include employee, contractor and overtime effort. Total project costs included all capital and operating costs incurred by the project.
Other IT professionals at government entities said that keeping track of the testing effort as a percentage of overall development spend is an important metric, because it provides an indicator on effectiveness and efficiency in the application lifecycle management.
“Key areas of this cover project leadership, development effort, business analysis, testing (unit, release, integrations, UAT) and architecture/design,” they pointed out.
“Testing will vary, depending on the complexity and risk of the project. As an overall percentage, the range for test expenditure tends to be between 24% and 34%, based on an application lifecycle management approach. This includes costs arising from project management, business analysis, architecture/design, testing, development and other related project costs.”
However, generally speaking testing costs would not be expected to fall below 10% for a major project IT experts believe, excluding third-party test costs as part of the development cycle. “If these costs were included, we would estimate an average total test cost of 15% to 19%.”
