Black Boxes

Every programmer should know the famous pattern “low coupling, high cohesion” by heart. It's one of the most important patterns because they apply to every single piece of code, even within other software patterns. In a more generic language, the darker the box, the better.

Black boxes are valuable tools for teaching. Surround an electric grinder with a big black box with two openings and a button and ask the student to tell you what do the box does (to make it easier for them label the input and output holes). They will press the button and hear the noise, but nothing will come out. then they'll probably try putting something in the input hole and after a few experiments with the box they will figure it out what the box is for.

The same concept is also very important for software. When you write a hardware driver you never know what kind of program will use it. For instance, USB drivers cannot assume anything about anything as virtually any device can be connected to it and any software can interact with it. Libraries in a higher level (like mathematical) must also not assume much because virtually every program will end up using it.

This concept repeats itself all the way up the line, from kernel modules, basic libraries, specific libraries, user programs, users, groups, institutes. Whenever one user's cycle is tied up with the other's the probability of having major consequences for small problems increases.

Low coupling also have a good deal in redundancy. From traditional software engineering we know that whenever two programs use the same method you should extract it from both and build one generic that doesn't belong to any of them, but to a generic library that both use, and this concept must be extended to all levels of software development.

Like libraries, production procedures (user's actions) should never rely on intermediate results of the other user's procedure.

It's very tempting to couple two production schedules when you know you can handle and a lot effort is done in bioinformatics institutes to talk between groups to increase this coupling, it's even seen as productive work. But coupling is evil, it reduces cohesion of schedules which is one of the first software quality measurements to be discarded. Increasing schedule coupling will eventually increase software coupling as more and more will depend on other's procedures and the same happens with cohesion, more and more the same tools will do different things for different people at different times.

Software doesn't care if it's cohere or low-coupled, as long as its syntax is right, it runs. Those concepts must come down from the user level (programmers) but programmers quite often are following development schedules (most of the time too tight) and trying to cope with legacy code at the same time. Thus, these values must come down from the very top: teams' and institutes' policies.

Teams should prepare and define interfaces of their own software with the outside world instead of hacking exceptions to their own functionality just because there is a similar problem in a different team. If there is a common problems to be solved in different teams the solution (ie. library) should be external to both teams. Of course teams can talk to each other, solve problems together, share libraries and algorithms but everything that is shared should be outside any team.

Once the common problem is identified, solved and a new shared library is developed, who will maintain that? Either team will have no time to maintain the software for the other team, especially during tight deadlines time. A new team must be created: the platform team.

The platform team is not only responsible for maintaining the shared libraries, it's responsible for identifying common problems on all teams, help to solve them in a generic way and maintain the libraries. It's therefore this team's responsibility to assure software quality in a broader sense, making sure that all quality rules are being followed by all teams, that they're working together but not mixed up with high-coupling and low-cohesion, that they have proper test cases and debug environments, that they hack less than they refactor etc. It's also that team's role to search for international standards, new techniques, algorithms and third-party software to leverage the quality of local software.

What's more important, however, is the role of this generic team within the institute. The institute should define general rules about development and software quality, merging those policies with current good practices the teams already do and trying to reuse any standard or policy already in place. It's a task for all teams to outline all policies and to define what's important to them and what's not and it should be task of this new separated team to group, filter and serve as a technical base to define the general policies for the institute.

Of course, this new team would not have the power to impose every change in every project, as many of them can be running successfully already without problems but a healthy institute (in terms of software quality) helps this team to enforce the general policies to every new or problematic project. It's also this team's responsibility to help all other teams when creating new projects or refactoring problematic ones to assure all steps will be taken to conform with the standards and policies.