Permanent prototype
Each project follows more or less one of the following models, sometimes several of them combined:
Design first,
Iterative,
Permanent prototype.
The first two ones are well-known.
The first one is the classical Waterfall model. Prior to development, the project is designed in every possible detail, so that we have both an overall view and a detailed view of the work to do. Suited for environment where formalism is required, it is also often used by mistake in projects where nobody knows quite well what the project is about.
The second one is the also classical Agile. Projects are constructed through frequent (at most a few weeks on a project with the overall duration of at least a few months) iterations, allowing some nice things like retrospection and adjustment of the process.
That's the boring stuff. The interesting stuff is the last model: permanent prototype. This one is practically not covered by the literature, while it helps describing many of the projects built by IT industry. Often, projects don't have prior design, and they are not iterative in any way, so the other models can't be applied. The lifespan of those projects can be described like this:
There is an idea. It's not described, nor understood by its owner; nevertheless, it can serve as a starting point for a project.
Some people work on the task of describing the project. Since they don't have a good overview of the project and since nobody knows what it would look like, this description is not extremely useful or exact, but still can serve to explain the concept.
The team is created and starts programming immediately. A set of elements (see below) force the quality to be particularly low, and this quality level is accepted by the team who believes that it will be improved later in the lifecycle.
The project is constantly modified, increasing the mess in the absence of Agile.
It is finally cancelled, or maybe released in the wild while being broken.
If released, the project is then maintained, year after year, by programmers who scream and cry while looking at the source. At least, it's a good source for TheDailyWTF stories.
There are several factors that make low quality acceptable at step 3. Time pressure is coupled with the logic “We can make it better later” and “Simply make this work”. The wrong logic I was talking about recently that standards can be applied to an existent project is here too. Finally, the illusion of organization has its influence too: since it is believed that there is someone, somewhere, who have thought about the project and knows what it will look like in every of its aspects makes it difficult to refuse to go non-Agile.
Such project is nothing other than a prototype: we should release it fast to see some stuff working, and we don't care about quality. In a case of a prototype, everything is clear: the only thing that matters is functionality; quality doesn't and shouldn't matter, because it's not the point of a prototype. The point is that a short prototype (or a bunch of prototypes) should be followed by a real project, with no code reuse. Simply tweaking the source of a prototype doesn't magically make it a product.
What's wrong with projects which can be modeled as permanent prototype is the misunderstanding, the lack of communication between the stakeholders and the team. It should be clear to the stakeholders that what is actually built right now is only a prototype, not a final product. Being a prototype, it doesn't create technical debt; in other words, being low quality is not a flaw, but a benefit: instead of putting accent on pointless—in this context—elements such as readability of code, the team can put all the effort in going from nothing to wow in a tiny amount of time. Where misunderstanding becomes clear is at the moment where stakeholders tell that the prototype which was built will now be used in production, as well as maintained. At this very point, technical debt grows from zero to extreme—something which is often omitted by stakeholders, and not explained to them by IT people.
To escape permanent prototype model, there are two solutions:
Go Agile. If the project is not clear from the beginning, you have no excuse: avoiding Agile would lead systematically to project failures. If the project is clear and well documented and is an excellent candidate for Waterfall, do the project right from start. Don't wait the next month or year to start using standards and care about security. Do it right now.
Do real prototyping and be transparent with stakeholders. Explain what prototyping is about. Make sure everybody understands. Be clear about the consequences, including the fact that all the code will be thrown. Another important aspect is to set up a mechanism which ensures that the code will indeed never be seen in production. One of such mechanisms is to write prototype in a language which is not the one chosen for the final product.
Let's take an example of a project which has two parts: one is pretty clear, and another one, the essential one, is highly unpredictable.
Start by creating a prototype for the unpredictable part. While not knowing on which platform the final product will be hosted, let's pick C# for the prototype. This will guarantee that we'll throw all the code when moving to a programming language which has enterprise-level support on Linux servers.
The prototype turns well, but there are also a few points which are not clear in the second part of the project. Let's create a few prototypes for those dark spots too.
After writing the prototypes for this second part, it appears that we got the first part wrong. Let's redo the original prototype until it matches our new vision. Note that the stakeholders are constantly involved and giving their feedback, which helps in getting prototypes right. The stakeholders know what prototype is, so they provide feedback only on the overall concept, not the different bugs they may encounter.
Now that we have a clear overall vision of the project, we can start the project itself. We won't look at source code of the prototypes, but we definitively will reuse the concepts and ideas they demonstrate.
The project itself is being built and uses short iterations with constant feedback from the stakeholders.
At some point, stakeholders notice that there is a huge problem, unforeseen until now. It's now time to create an additional prototype. It allows going very fast, while keeping the code base clean and avoiding technical debt growth.
The prototype appears to represent successfully the new vision of stakeholders. The code base is progressively and carefully refactored to fit the new vision. The stakeholders are constantly involved and have a clear view of current progress.
While doing refactoring, businesspeople decide to hold the project, because IT personnel is needed for a more important one. It's not a problem, since the project is up and running and the last version created a hour ago through continuous delivery works correctly.
Six months later, the project is resumed. Since the technical debt is low, the new team can start working ASAP on the project, without fearing to break stuff.