How now wit! Whither wander you?

by Mathijs van Zutphen, Philosopher, educator, artist and creator of VISH, www.vish.nl. He participates as a philosopher in the Knowledge Stream Life Sciences of the Summit for the Future.

The challenges of software engineering

On november 30th the Club of Amsterdam organized its 'Future of Software Architecture' conference. The meeting was hosted by Syntens, a government sponsored 'Innovationnetwork' for entrepreneurs, who have their office in the center of Amsterdam in a modern highrise overlooking the water. Cargo ships lazily coasted by, as the winter sun gave way to the long winter evening. The meeting proved to be what is becoming a trademark of the Club of Amsterdam's events. New insights arose out of a series of constructive confrontations between radically opposing viewpoints. The big surprise was that a seemingly straightforward technical subject, in appearance almost dull, generated such different, almost diametrically opposed, perspectives. The contrast inspired a frantic discussion about complexity, history, values, and the role of technology in today's and tomorrows world.

Maarten Boasson, professor at the University of Amsterdam and chair of the Industrial Complex Computer Systems Department, started of the evening in minor mode. The title of his presentation "Do we want to keep software Architecture alive" says it all; software architecture is a terminal patient. Boasson emphasised that in his view software architecture concerns itself with the technical design of systems. This is a relevant remark, since the term software architecture has acquired such diverse meanings over the years. From a purely technical term it has evolved into an all inclusive concept, containing everything from generic properties of computer systems, to users, to a wider context of stakeholders, 'software architecture' can describe all possible relations and structures that exist between these elements. In fact this conceptual inflation of the term 'Software Architecture' will be reflected by the diverse opinions that will be exchanged during the evening's discussion.

The succes of the digital revolution, or perhaps we should say semi-conductor revolution, has left no aspect of society untouched. The explosion of computing power, expressed by Moore's law, has generated a myriad of complex systems that pervade all corners of reality. The discipline of software engineering seems to be split between the drive to design robust functioning systems which exhibit predictable properities and the ever quickening development of business needs, discovering new market opportunities which require ever newer applications. In this dynamic we are constructing our tower of Babel on an imperfect foundation of frantically, and shabbily integrated legacy systems and components. In short: software architecture is a mess. There is no comprehensive theory, and practical (i.e. business) considerations drive most of software development, leaving little room for reflection on design and the pursuit of quality. Education and research are key to creating quality systems, but these considerations are completely marginalized by forces driving the development of technical systems. Software is unstable and unreliable, and we're not just talking about PC operating systems. Onboard computers in airliners also do fail for no other reason than bugs in the programming; which is a daunting realization. (For more on this: http://catless.ncl.ac.uk/Risks).

Unfortunatele we cannot step back and start all over again from scratch. Boasson's presentation amounts to a passionate plea to start focusing on quality and appreciate the crucial impact of great minds. Really good software, if it exists at all, is without exception the product of a few brilliant programmers, whose skills cannot be matched by a mulitude of mediocre developers. The road ahead is unclear, but if we want improvement, we should learn from our mistakes and develop general principles for understanding how architecture is related to specific applications and not merely adopt the fashion of the moment. Whether this will actually happen, or even has a chance of happening, is uncertain. The final conclusion is depressing, looking back on three decades of systems engineering, software architecture, as a way to mindfully design the best possible systems, is more or less a failure.

A greater contrast with the next presentation can hardly be imagined. Maarten Visser, a self-styled Performance, Productivity and Integration evangelist takes us into the promises of Web 2.0 and the emerging reality of an ever more complex array of communication and information processing applications. His focus is on communities which organize themselves around online applications, and he explains how these community needs will become important drivers for the development of next generation information and communication tools. The massive amount of information available is so overwhelmingly large and complex that we will need new layers of interpretation between the information and our needs. Such 'semantic' web technologies are to help us find what we need in the exploding informational complexity that is the internet. Visser bounces with energy as he performs his fast-paced sermon with an exuberance and optimism that indeed resembles divine inspiration. A numbing list of emergent technologies are presented: blogs, XML and RSS (Really simple syndication) feeds are already superseded by SSE (simple sharing extensions), RDF (Resource Description Framework) and an new ambitiously named standard called WOL (Web ontology language). Things don't appear to become less complex folks! Everything is to become connected to everybody everywhere, with machines and applications taking care of the semantics of it all…

Niek Jetten's subsequent presentation brings home the point already bemoaned by Boasson: software engineering in general, and software architecture specifically, are driven by business interests, and as we all know: business has a dynamic all of its own. In effect we learn why the interests of business (low-cost, high-pressure, short time-to-market, scalability) are so diametrically opposed to the interest of software architects attempting to build robust quality system with predictable properties, requiring research, reflection, and ample time.

Given the contrasting views in the presentations the discussion promised to be lively, and so it was. In spite of Boasson's critical assessment of the quality of most software, the general focus seems to be on new developments, and since these developments have an impact beyond the technology itself the discussion quickly expanded into non-technical areas. Software architects value robust systems, web communities seek intangible values in their interaction with each other, business is interested in financial value and healthy bottom lines, these represent very different value domains.

Even though the high expectations of Web 2.0 may be exagerated (it wouldn't be the first time the internet will have inspired fantasies) the internet will increase its influence, as certain as its internal chaos will increase as well. As the internet is sucking in information and communication functions that are traditionally provided by other media like radio, television and telephones, it will become the dominant knowledge and and communication system of the global village, and its architecture will be a crucial asset (or liability) in our brave new world. It seems clear that software engineering will increasingly be driven by what users, and communities of users, want and need. Isn't that a good thing? Perhaps, but the internet already is such an intense cocktail of interrelated technologies, components, systems and protocols. How are we to guarantee its stability and sustainability? How are we to rely on a system containing so many possible glitches?

A real solution to the problems outlined by Boasson does not present itself. So what if we just reinterprete the problem? Could software engineering become somewhat akin to a behavioural science; empirically studying autonomous organic systems? So that we recognize the inherent instability of systems and admit that the level of complexity is beyond the reach of our understanding? Properties of a system then become emergent rather than predicted by design. Boasson is visibly appalled by this suggesting while Visser is eagerly nodding, having found a metaphor that strikes a chord with his visions.

Did we discuss the future of software architecture? Not everyone agreed. To some the discussion lacked technical content, to others the entire session had too limited a focus on technology and insufficiently addressed the humanity of it all. Well, we did end up discussing quite a range of topics. What do value systems, human needs, communities and social developments have to do with software architecture? Well, as with many things, context determines everything, whether we like it or not, and systems are an integral part of their supersystem. As such the "Future of Software Architecture" exemplified the Club of Amsterdam's for diversity, creativity.