There is more to the creation of a successful application (or system) than just learning the language and mastering the techniques. Today's commercial software projects are among the most complex engineering undertakings of humankind. Programming is essentially the art of dealing with complexity. There were many attempts to apply traditional engineering methods to control software's complexity. Modularization, software reuse, software IC's, etc. Let's face it--in general they don't work. They may be very helpful in providing low level building blocks and libraries, but they can hardly be used as guiding principles in the design and implementation of complex software projects.
The simple reason is that there is very little repetition in a piece of software. Try to visually compare a printout of a program with, say, a picture of a microprocessor wafer. You'll see a lot of repetitive patterns in the layout of the microprocessor. Piece-wise it resembles some kind of a high-tech crystal. A condensed view of a program, on the other hand, would look more like a high-tech fractal. You'd see a lot of self-similarities--large-scale patterns will resemble small-scale patterns. But you'd find very few exact matches or repetitions. Each little piece appears to be individually handcrafted. Repetitions in a program are not only unnecessary but they contribute to a maintenance nightmare. If you modify, or bug-fix, one piece of code, your are supposed to find all the copies of this piece and apply identical modifications to them as well.
This abhorrence of repetition is reflected in the production process of software. The proportion of research, design and manufacturing in the software industry is different than in other industries. Manufacturing, for instance, plays only a marginal role. Strictly speaking, electronic channels of distribution could make the manufacturing phase totally irrelevant. R & D plays a vital role, more so than in many other industries. But what really sets software development apart from others is the amount of design that goes into the product. Programming is designing. Designing, building prototypes, testing--over and over again. Software industry is the ultimate "design industry."
In the third part of the book I will attempt to describe the large-scale aspects of software development. I will concentrate on the dynamics of a software project, both from the point of view of management and planning as well as development strategies and tactics. I will describe the dynamics of a project from its conception to shipment. I will talk about documentation, the design process and the development process. I will not, however, try to come up with ready-made recipes because they won't work--for exactly the reasons described above.
There is a popular unflattering stereotype of a programmer as a socially challenged nerd. Somebody who would work alone at night, subsist on Twinkies, avoid direct eye contact and care very little about personal hygiene. I've known programmers like that, and I'm sure there are still some around. However most of the specimens of this old culture are becoming extinct, and for a good reason. Progress in hardware and software makes it impossible to produce any reasonably useful and reliable program while working in isolation. Teamwork is the essential part of software development.
Dividing the work and coordinating the development effort of a team is always a big challenge. In traditional industries members of the team know (at least in theory) what they are doing. They learned the routine. They are performing a synchronized dance and they know the steps and hear the music. In the software industry every team member improvises the steps as he or goes and, at the same time, composes the music for the rest of the team.
I will advocate a change of emphasis in software development. Instead of the old axiom Programs are written for computers. I will turn the logic upside down and claim that Programs are written for programmers. This statement is in fact the premise of the whole book. You can't develop industrial strength software if you don't treat your code as a publication for other programmers to read, understand and modify. You don't want your 'code' to be an exercise in cryptography.
The computer is the ultimate proofing tool for your software. The compiler is your spell-checker. By running your program you attempt to test the correctness of your publication. But it's only another human being--a fellow programmer--that can understand the meaning of your program. And it is crucial that he do it with minimum effort, because without understanding, it is impossible to maintain your software.