In the earlier days of web-based software development, along with most of our peers, we usually worked to a traditional ‘waterfall’ software development process and fixed-price project delivery. In waterfall methodology, you moved from one stage of the process onwards completing each one fully before starting the next. So you undertook the system design fully before starting development, completed development in full before testing, and delivered the near-complete system to the client at the end of the testing phase. The idea being that you can control the scope and thus the costs by not letting it change at all along the way, or at least following a very strict process if you did.
This was theoretically sound, however, in practice it never really worked. And it is probably the reason behind much of the bad reputation large software projects have today. In reality, you could not design a system in enough detail before starting development to be sure you had it completely right first time. What’s more, you couldn’t be sure it was really what the client wanted before they could see it working in detailed actuality on a computer screen and not just on paper.
And, if you did try to get the system design and specification totally complete before undertaking the development, you regularly found yourself ‘painting the Forth Bridge’ as once you had completed the specification with the client, so much time had passed that the business itself had changed and you had to go back to the beginning and rewrite all the business processes again. We found ourselves on several occasions having spent a year writing a system specification with a client only to find at the end of it that it was no longer relevant.
This clearly was not practical for either the client or the supplier, and over the last decade or so ‘Agile’ development methodologies have gained prominence and are now pretty well universally accepted as the only way to go if you want any kind of custom-developed software.
Given that even industry-specific business software, written specifically for a particular business sector or type, typically still requires a lot of customisation and integration to work for the particular client, then it is clear that custom software development will remain central to successful business software systems for the foreseeable future.
Pure agile development was perhaps a swing too far the other way for commercial considerations; though done correctly, I am sure, experts would be confident to deliver on time and in budget using proper agile methodologies. My personal view is that agile works very well for internal development teams where contracts and budgets are not so strictly delineated at the outset of the project, and, because an external consultancy is not being used, the cost-per-day is usually much lower; so you have a lot more time potentially for the same budget. Though the down side of that can be that the urgency factor required to achieve the time and cost targets can potentially be missing, so you don’t save money in the end anyway.
With a fixed-price project you often found yourself in a potential confrontation scenario, with the client demanding as much functionality as possible for the budget agreed, and you the supplier trying to stick to what you understood the scope that was agreed in the first place to stay in budget! As someone once said, “in a fixed-price project it is the clients’ intention to get as much work done as possible, and the suppliers intention to do as little as possible for the same budget, and they will fight over that dichotomy to the end!”
And as a final note, with agile development, in reality it was very hard to work as a remote supplier on a contract and get the job completed; not usually because of any failing on our side, but because it was so hard to get enough of the client staff’s time and input needed to finalise the system design and specification as the development moved forward, and to get the consensus within the business, within the sprint times needed to ensure the development process was not delayed or slowed up.
‘Flexible’ process
Out of these experiences we started to evolve our own methodology as a practical blend of waterfall and agile development that somewhere along the line I labelled as ‘flexible development’, which seems to have stuck.
Our process is typically three-phased, after each one the project can either progress or not:
An initial outline scope and high-level budget for a project, which will usually only take a few days of consultant time to complete. If that outline is acceptable and indicates the project can be delivered in the budget and timeframes needed, we then undertake a more detailed system design phase, working together with the client, which, depending on the size of the project, will take from one or two to several weeks to complete.
At the end of that, we can provide a reasonably detailed system design and scope of work, and a budget estimate with a % accuracy estimate; e.g. this project will cost in the region of £50,000 with an accuracy estimate at this stage (depending on the level of detail of the scope and design) of, for example 20%. So, in this example we are saying it will cost between £40-60k and will take x weeks or months to complete. Which is dependent on the size of the development team that is feasible to assign to the project; that is, it is usually not the case that the more people you put on the development the faster it will be done, there is usually an ideal number to ensure the most productive progress is achieved.
And then, if that budget and time estimate is now still within the original expectations, or is otherwise acceptable, we can move into development. This still does not provide for a fixed-price, but it does give the budget parameters, and a clear change control process for the client and developers to work within to stay in budget. i.e. if it turns out the scope is incomplete in practical terms during the process of development, then the budget can be increased, or the scope can be decreased. Also it is clear that the budget has to remain estimated, none of us (neither client nor supplier) can guarantee to have covered every aspect of the requirement in this type of scoping, and it is understood that we are going to uncover things we did not think of as we progress. It does sound pretty simple really, and it is in practice!
One aspect of this which has proven controversial, but nevertheless completely practical, is our development contract term that states, “…should any differences in interpretation occur resulting in any disagreement on the scope of work and what it means, then the interpretation of Ballard Chalmers is accepted as being the interpretation and understanding that has been used in preparing any time and cost estimates for the project, however much these may differ from the client’s interpretation of the same documentation, discussion, or other information. If any differences occur then these would be reviewed and agreed and the design and time estimates adjusted as required.”
On the face of it — it sounds like we have carte blanche to decide whatever we like about what the scope meant; however, in practicality it makes complete sense, we can only estimate a job on what we understand it to mean, and not what the other person understood, it’s only fair.
What we are actually guaranteeing is our expertise, honesty, integrity, and partnership to work together to deliver what is needed and wanted to the best of our abilities, and with the client fulfilling all of their obligations to do the same. Working together on this basis we will all do our best to get through the process of developing the custom business software needed, or integrating the systems, whatever the adventure we have undertaken together may be.
Software development
The development phase is itself semi-agile (flexible!). If we haven’t done them already in the design phase, we will develop wire frames (outline mock ups of user screens), user interface designs, and outline the business processes to provide the next level of design detail and a proof of concept, to confirm we are on the right track.
With that signed off we can start development in earnest. Depending on the size of the project we will deliver several releases of the system to the client, in agile ‘sprint’ fashion, and review and take feedback as we progress. Out of that usually comes Change Requests, which are estimated for time and cost and then either approved and added to the project or rejected and omitted. These can include functionality requests to be added by the client, or functionality the developers have seen is required but not included, or where they have found the original assumptions were incorrect and additional time (and cost) will be required to meet the objectives.
In this process our guiding concept is ‘Prediction’ — knowing and alerting what is expected in the future in both time and cost terms. What we don’t want is any last-minute surprises for the client, or ourselves. A part of that is our unique weekly reporting system we have developed which includes an ‘uplift factor’ which factors in the current progress of the project and extrapolates that out through the rest of the project time and budget. So if we are currently running at 5% under the time and budget, that 5% will be applied through the remainder of the project estimates and the 5% reduction applied to the final cost. With that you can always track your final expected end cost, even without a fixed-price.
We have an online project management system that we share with clients for project reporting, and also use Microsoft Excel and Project heavily to track progress and ensure all parties remain informed and comfortable with the project as it moves through development.
Once the main development is completed we have normal testing and user acceptance testing phases, and closely involve the client in this process. You can often find our clients in our own office working together with our project team on testing and acceptance to ensure a top quality end result that meets what they expected and want to achieve.
And, by the way, this process above extends to our own offshore team which we have nurtured and evolved over a decade to the stage where we can totally confidently deliver enterprise software development with the benefit of offshore rates for the core system development, supported by our UK program and project management team and engineers. We have delivered several very large projects, which would not have been considered at all by our clients if it were not for the economy of offshore development that we offer, so this is a very important part of our service delivery for software development services.
Of course once we have completed a system development we provide ongoing Support for that system; in fact our longest serving client has been with Geoff Ballard for over 20 years and we still provide development, consultancy, training, support and other services to that company. We also provide cloud hosting where required, working with suitable partners, whether ‘public cloud’ deployments such as with Microsoft’s Azure platform, or ‘private cloud’ deployments at for example Rackspace, or the clients’ designated hosting provider, or indeed at their own in-house data centres.
In closing
Our aim is to provide what is best for our client for the ongoing secure and robust live phase of the system that we have developed, and to ensure that all the effort that was put into the development of the system is now rewarded with the business-use that it was intended for, and – most importantly – the resultant return on investment!
Our expertise in this area has grown to include complete custom-developed systems built on .NET and SQL Server, designed and developed from scratch, typically as full business-process systems used to manage all or part of a business or activity; SharePoint Server for enterprise collaboration, document management and business process; BizTalk Server for developing enterprise integration layers, typically to integrate cloud-systems with on premise back office systems; and of course being deeply expert in SQL Server which underpins almost every Microsoft-technology platform based system that is deployed.
We see custom business software development as a Partnership between our client and ourselves and we expect our client to buy in to our flexible development process at the outset of the project. Of course, where it is demanded we can and do provide fixed-price project work, but the costs, to cover the contingencies and warranties and so on, are in reality always higher than with flexible development.
Once a client has experienced this with Ballard Chalmers, they are convinced I am sure! And we become their long-term partner and trusted expert, ready and available whenever we are needed. That is certainly our aim anyway, and one thing is certain — we will always try our best to do so.
By Andrew Chalmers, Managing Director
