One way of looking at the art and science of software engineering is that it is a process of mapping human desires and wishes – the insides of peoples’ heads – onto a computer system. This is not a particularly novel idea, and it’s one that you are probably familiar with, but it’s an important one. Engagement with a client can be boiled down to a conversation wherein we discover the client’s needs and wishes, and then present an instantiation of our interpretation of what they have expressed. There is an awful lot of chance for error in this. Mapping the contents of their heads to vibrations in the air and symbols on paper or a screen is a lossy process. Our interpretation of what we hear or read is a lossy process. Implementing the ideas, dreams and wishes into an information system is a lossy process. It’s a wonder software ever gets built at all.
Fortunately, as a profession we’ve built up a suite of habits and conventions that can help us to bring clarity to the conversation and to the design and implementation of a solution. In broad terms, we are all used to starting the conversation with the first question: “What” The trouble is, that the follow-up conversations are often not… optimal. Surely you’ve heard conversations like this:
John: So Ms Client, what do you want to do?
Meixiu Client: Well, we would like a web application where the customer can login and…
Paul: Ah! Well, I think we can mount it on AWS, and use Ruby-on-Rails with a HTML5 front end and websockets, and use OAUTH 2.0 for authentication…
George: Yes, and put the API gateway in front of a Kafka cluster that talks to a cluster of microservices deployed in Docker on ECS…
Richard: And I’d recommend that we use Cassandra for transactional time series data and a Mongo database for customer records…
Meixiu Client: …
We have a bad habit of leaping very quickly from the first question to the last question, skipping over most of the conversation to rush toward the solution. It’s not a surprise, we are trained to come up with solutions, and this industry attracts people whose heads are shaped to seek solutions. The trouble is we’ve not finished articulating what the problem really is. Error is inevitable. “How?” is a good question, and a conversation that proceeds like the one above would almost definitely see us build an incredibly solid and robust solution, but it needs to be the last question, not the first.
So then. What are the four questions?
We’ve seen the first question already: “What?”. lot of our initial conversations must be about “What?”. The client is very likely not to have a well articulated and consistent vision of what they want or believe they need, and it is crucial that our understanding of “what” aligned with their understanding of “what”
Meixiu Client: We want a web application where the customer can login and download a CSV of their transactions.
John: Great! Exciting stuff! Have you thought about whether they should be able to select a range of transactions by date?
Meixiu Client: No we hadn’t! That’s a great idea, we should allow them to pick a date range as well!
“What?” gives us the opportunity to help the client articulate their requirements precisely, and indeed gives us an opportunity to upgrade and modernise their expectations and desires. We should expect to spend a lot of time focussing on “What?”, but that does not imply that we are spending a lot of time writing down requirements and specifications. “What?” does not preclude an agile process where the behaviour of the solution evolves as it is being built. Instead it gives a target to launch the development process towards. There’s no point in building a motor boat if the client actually needs an air conditioner.
The second question is highly underrated, but terrifically powerful: “Why?”
Paul: So Ms Client, what can we do for you next?
Meixiu Client: Well, I think we need to get some AI into this system
George: Why?
Meixiu Client: …
If the client cannot tell you why they need something, run away. If they can only tell you that they want something, but not why, run away. Better yet, don’t run away. Stay at the table and see if you can help them figure out why. This may be one of the most important things we can do with our clients – help them to make sure they really do want whatever they have asked for.
At a coarse level, the main reasons a business would want to embark on an expensive engineering exercise would be to either increase revenue or reduce costs. Those are the two pillars that the capitalist system stands on. Growth and profit. Of course, a client may have entirely altruistic reasons for doing something, desiring to provide some public good or cultural improvement: it would be fantastic to be able to work with folk like that. But we need to be clear on why the engineered solution should exist. “Why?” defines purpose.
We should expect that “why?” will lead back to restating “What?”, and that you will spend quite a bit of effort looping back and forth between “what?” and “why?” the purpose and intent are well articulated and well understood.
The third question is the one most often overlooked: “When?”. This is a simple question that should have a simple answer, but provides some very useful trip wires. If the client says ‘next week’, we need to be able to politely disabuse them of the notion that it might be possible. A desired short delivery time is a useful place to begin negotiation and education, and is the driver for trimming the scope of the solution and the resources needed to build the solution. “Next week” might usefully be negotiated up to “two weeks”, and a solution cobbled together with WordPress and MySql on a simple web hosting provider.
Don’t fall into the bad engineers’ habit of ascribing time pressure just to the client’s dreams: we are very prone to the desire to want to build a mansion when the client needs a hut, often because we have not fully understood that the real need is for a hut. Scope creep can come from either party.
The other worrying response to “When?” are very long delivery times. Sometimes this will be unavoidable, some businesses have very specific times that they can deploy software, or specific legislative or regulatory constraints that impose delay. Other times the client may suggest time frames on the order of years: this is a potential problem as over the span of years the needs and desires of the client are likely to change, and the optimal technological solution will definitely change. The risk of the drift away from the original purpose and intent is that the implementation will be endlessly reworked without ever quite being finished or deployed.
The final question is the one that we love to get our teeth into: “How?”. It’s also the least important!
There’s never just one optimal technological solution, which makes this question the exciting one for us: it’s where we get to be creative and imaginative, and to play with all those neat new toys we have been reading about. Ultimately this is mainly a question for us, not for the client. Ms Meixiu Client really doesn’t care whether we use Java or Scala, Docker or EC2 instances. They probably care about the operating and development costs, and might have concerns about ongoing maintenance in the future, but the “how?” of technology is largely irrelevant to the behaviour of the system, and whether the solution is boosting revenue and shaving costs.
“How?” is not purely an engineering question though it needs to be informed by questions of available skills and resources, and is probably the source of refinements of the intent and purpose. For example a proposed “how” may be the first time that the client expresses a strong and meaningful desire to host the service themselves, rather than a cloud hosted solution. It’s also probably the first time that you get to grips with secondary requirements such as system availability and response times – which might take us all the way back to asking “What? and “Why?” again.
Four questions. Let’s look at them again.
- “What?” provides the intent, an expression of need and desire.
- “Why?” provides the purpose, the raison d’être for the engineering project to proceed at all.
- “When?” shapes how the project will proceed from concept to delivery.
- “How?” defines the delivered outcome of the project.
Finally, hold in your mind that this is not a simple linear progression. The answers from any of these questions should cause us to loop back to re-examine previous answers. It’s a conversation, not an algorithm.