Design for safety
The recent testimony of Facebook whistle-blower Frances Haugen to politicians on both sides of the Atlantic focused heavily on her former employer’s preference for the pursue profit over the safety of its users, strongly echoing the beginnings of the regulation of “big tobacco”.
The slogan that emerged from these meetings was the apparently clear and simple request for Facebook to “design for safety”.
However, ‘design’ and ‘safety’ are loaded terms and need careful expansion, because this phrase applies to many business sectors, especially physical ones such as transport and medicine, as well as the more data-intensive ones of finance and social media.
Of the two terms, ‘design’ is probably used incorrectly more often, because it is often confused with ‘styling’: design is all about the efficient use of materials, structures, processes and manufacturing techniques to a achieve a functional goal, whereas styling is a more abstract analysis and refactoring of a design to achieve a visual aesthetic. Apple products have gained popularity because of their styling which even led it to try to patent some aspects of it, such as rounded corners on display windows, which have now been introduced in Windows 11.
Clearly when assessing the design of an algorithm in the world of Facebook, aesthetics have very little importance other than perhaps the readability of the source code. The HTML displayed to solicit and display user interactions should largely be ignored by regulators, but doubtless user experience designers will want a seat at the review table.
‘Safety’ poses the problem of being a too abstract a term for the problem domains in which it is applied. Ms Haugen’s testimony and the subsequent media analysis focused largely on the mental harm that Facebook causes to its users, sadly leading to many cases of low self-esteem, self-harm and ultimately suicide.
Clearly in other domains, safety has a much more understandable direct physical connotation, ‘safety belt’ perhaps being the most obvious phrase.
Safety also connotes a much broader ecosystem and operational resilience scope, such as the safety of the world’s financial system, which has been heavily shored up by regulation and capital adequacy requirements (in truth, similar to automotive airbags!) since the crash of 2008.
Last and by no means least, we need to consider the notion of ‘proxy harm’ as embodied by COP26 Climate Conference, which at the time of writing is trying to address the long-term safety of the human race from the threat of global warming.
Having unpacked these two all-encompassing terms, the obvious next question is: “How do we design for safety?”. Obviously there are many domain-specific techniques that have little of no value in other business sectors because they are focused on specific physical and chemical interactions, but there are definitely many opportunities to standardise how designs are reviewed and contractual relationships are governed.
Until now, legislators and regulators have resorted to the technique of ‘principles-based’ wording, which tries to create durable governance and control by focusing on abstract requirements for behaviour that then require careful and costly interpretation and constant review/testing.
The danger of this approach is that it can easily fall into the trap that leads to the confusion mentioned early between design and styling. There is a natural bias to the stylistic or opinion-based approach because it has clear visual properties and broader comprehension, whereas formally assessing designs via testing, simulation and complex calculations takes time and money.
Eventually the stylistic approach however is doomed to fail. Whistleblowing disclosures such as the Panama and Pandora Papers have shown how many unique and complex, interlinked personal and corporate relationships have been created to avoid tax and hide sources of income.
Big data has been a simplistic first attempt to swing the pendulum back towards a more systematic analysis of design and techniques such as blockchain to define and maintain long-term stateful relationships between transactions and entities. However the energy costs of using these approaches are now themselves ‘unsafe’ because of the amount of energy, and hence carbon emissions, produced.
Somehow we are going to have to transition from big data to some form of smart data to achieve a way of evaluating ‘design for safety’ activities. This may well mean a refocusing of the technology lens on semantic mechanisms, with all the inherent dangers of misusing terms such as glossaries, taxonomies and the dreaded term ‘ontology’.
‘Design for safety’ has become a catch-all phrase often used alongside the term ‘cultural change’ for describing how a current business modus operandi or political situation has become unacceptable and needs action to either repair it or move it to different state and supporting processes. Yorkshire County Cricket Club has found itself in the eye of a media storm primarily because of an outdated ‘culture’ and selection ‘tradition’ which had served it well for 150 years.
Having standardised the phrase, we now need to standardise how a design is analysed and what comparative measures can be applied to enable rigorous assessment.
Clearly, many job roles will be affected by more automated and systematic assessment techniques, especially those in positions that tend to be little more than temporal/structural buffers (also known as committees) rather than any particular analysis skill. Many safety functions rely solely on multiple pairs of eyes in the hope that any defects can be spotted on a second or third pass.
‘Design for safety’ is an important operational construct across both commercial and political spheres of influence, but it is in grave danger of becoming little more than a simplistic soundbite or social media slogan.