Effective Engineering 
                Consulting Services
                                           "Helping Engineering Excel!"

Our Premise
The Problem
Our Promise
Our Approach
Our Deliverables
Case Study
Our Results
Our People
Contact Us


Case Study:  Software Company in Networking Market  [Company name withheld upon request]


This client develops complex, highly distributed, server/client/web-based software used to manage complex data networks and network-based applications for enterprise and service-provider customers.  Their software enables customers to identify and correct problems before they seriously impact their highly critical network operations.  On-time delivery of product releases and high product quality are essential.

“Tom Dennis was instrumental in assessing the engineering organization, and in recommending and implementing changes associated with the enforcement of our development methodology, the consolidation of QA into a separate and co-equal organization within engineering, and the consolidation of all Release Engineering activities into a single group.  He also recommended effective changes in the way these organizations operated.  The net result of the implementation of his recommendations was on-time delivery of product releases, high quality products with minimal customer reported issues, and motivated employees.  His efforts helped to significantly reduce development and product support costs, and contributed to increased product revenues.” 
  – Chris Crowell, Vice President of Engineering

Case Study:

This $60M software company in the networking market derived about equal revenues from sales to new and existing customers, and from support and maintenance contracts from existing customers.  The biggest engineering-related issues facing this company were delays in the delivery of new software releases (both major and minor releases), and quality issues reported by customers.

Problems Identified
This 250-person engineering organization was lacking the enforcement of discipline of a solid development methodology.  Further, it had the critical functions of test- and release- related activities spread out among five departments.  This decentralized approach gave no one the overall responsibility for these two critical functions, and this led to lack of ownership and finger pointing within the organization.  That development delays and quality issues were present was not surprising.

Actions Taken:
1) Methodology Enforcement: Forced adherence to the formal development methodology was mandated.  This meant that design specs were to be written, reviewed, and signed off on before coding began.  It meant that independent code reviews were to be completed before code submission to the source code control system.  It meant that software development tools, such as tools to identify and correct runtime errors, to identify and correct memory access and memory leak errors, to analyze code coverage (whether or not code has been executed and tested), to analyze software performance and eliminate performance bottlenecks, etc., were run on all code before code submission to the source code control system.  It meant that thorough unit testing was to be completed before code submission to the source code control system.  It meant that independent code reviews on all code changes to correct bugs were to be completed before submission to the source code control system.  It meant that all other elements of the software development methodology were to be complied with.  Metrics of compliance with this were to be maintained and reviewed.

2) Test Organization: Five disparate test groups spread among the development groups (integration test, core functionality test, applications test, device management test, and performance test) were consolidated into one department of four groups (core functionality test, applications test, device management test, and performance test), independent of the development groups, each with a manager, and all reporting into one director who was at a peer level with the development directors.  This consolidated test organization was given the charter, for each new release, of concentrating initially on working very closely with development on the integration testing of portions of new releases, and when ready, moving to an independent software quality assurance (SQA) activity of the complete release where tests were to be conducted more independently of development interaction.  This department was to perform integration testing, functional testing, system testing, white-box testing, black-box testing, negative testing, performance testing, and regression testing (to ensure bugs have been properly fixed).  They were to be the internal proxy for customers regarding the quality of the releases they were testing.  It was the test department’s responsibility to make the determination of whether or not a release was ready to go to customers.

3) Release Engineering: All functions related to release of the product were consolidated into one group called Release Engineering, under a strong manager.  This included responsibilities for release coordination, release project management, source code control system and tools, build engineering and build tools, bug tracking and correction system and tools, daily bug tracking and correction assignment meetings, weekly project review meetings, sustaining engineering coordination, and regular interaction with all functional areas in the company that had an affect on or were affected by the release of new products (included development, test, usability engineering, performance engineering, technical documentation, customer support, product management, and marketing).  The Release Engineering group had the responsibility for all releases of product to the field, both new releases and sustaining engineering releases (patches and maintenance updates).  This group worked with all of the other groups in determining the schedules for all releases, and for tracking those schedules to ensure on-time delivery per committed schedules.  This group also instituted significant release process changes, including to the bug review process, to the weekly project reviews, etc.

1) Methodology Enforcement – The result of enforcing adherence to the development methodology was higher quality initial code, reduced testing time by the test group due to fewer bugs being found, delivery of quality code in shorter timeframes, and fewer customer problems being reported.  The overall economics of this is summarized below.

2) Test Organization – By combining the test groups under one department at a peer level with the development departments, the testers were given a stronger sense of purpose and co-equal status with development.  It also gave the testers greater freedom and testing variety to reduce what often had become tiresome and demoralizing repetitious drudgework.  It enabled the elimination of duplicative testing (and unneeded people who were doing duplicative testing), the identification of marginal performers (managers, project leaders, testers), and enabled the groups to pinpoint key quality problems to root causes in development so that quality could be designed in rather than tested in.  This resulted in significantly improved quality.  The overall economics of this is summarized below.  

3) Release Engineering – By concentrating all release-related activities under one group, significant improvements in communications and coordination among groups resulted.  It not only gave those in Release Engineering the focus to concentrate on the specific tasks necessary to get good quality product delivered on time, but also provided the other groups (development, test, documentation, product management, etc.) a common point of interaction to work with.  Projects moved from missing delivery dates by 3-6 months on 12-18 month projects to delivering on time. 

Overall Economics
The net result of the changes implemented resulted in about $4M in development cost savings, and in at least $5M in additional revenues.  This is explained below. 

(1) Development Costs Savings of $4M: This was determined as follows: (a) People: The organization as a whole was able to complete a project with 10 fewer people by enforcing the process and therefore not needing to do things over, by eliminating redundant efforts, and by keeping everyone fully informed along the way.  With an average loaded salary of $150K/person/year, this provided $1.5M in development costs savings. (b) Time: The organization was able to deliver higher quality product on time, saving 3-6 months.  Assuming a 4-month time savings on a 1-year long, 50-person project, the net savings was $2.5M in development costs.  Thus the net development cost savings was $4M.

(2) Revenue Improvement of $5M: Whenever a project is late, revenue that would have been made from the time the project should have been delivered until it is actually delivered is lost forever.  For this $60M company, $30M was from sales and $30M was from maintenance.  In this company, one-half of the revenues came from new releases, or $15M/year from new releases.  If a release is delayed by 4 months (one-third of a year), then $5M in revenue is lost.  By delivering on time, that $5M can be reclaimed.  In fact, the revenue gains will actually be greater, since sales will not be reduced due to late delivery, and since the higher quality of the product will also improve sales.



Society of Professional Consultants


Copyright © 2002-2013 Effective Engineering Consulting Services, All Rights Reserved