Here’s a text version of the talk I did on January 28, 2022 to the Infoscope Research Lab at Ryerson University in Toronto, Canada.
Entrelazamientos más Ubicuos
Many thanks to the folks at Infoscape Research Lab for inviting me to talk about my book.
It’s difficult to talk about anything produced in the last couple of years without reference to the pandemic.
It’s been a strange gap between publication in Summer 2020, and now.
The pandemic appears to have messed with my sense of time.
The publication seems, at once, to be like it was only yesterday, and yet, also, a lifetime ago.
I really can’t settle on one or the other.
My somewhat blurred focus today will eventually be on the final and probably most speculative chapter.
In a recent Spanish translation of this text by Ana Fabbri, we decided to change the title from Deeper to More Ubiquitous Entanglements. Thus, the new Spanish title!
In very general terms, the chapter looks at the apparent changing user experience of digital culture.
The discussion looks beyond the social media paradigm to explore so-called immersive, ubiquitous media futures.
Along these lines, it chimes fairly-well with Facebook’s proposed move to a new business model: The Metaverse… Well, maybe not…?
Following Greenfield, I position ubiquitous computing as a moment when the consensual hallucinations of virtual reality tumble out into the everyday world… in effect, turning VR inside out.
Just to note, much of my talk will be quite theoretically heavy…
So, to provide some more concrete context, I will begin by saying that the kind of media virality I have been writing about for too many years now…
… That is, the idea that digital networks function as complex vectors for contagion…
… can only be fully grasped, if we resolve a series of often-forced distinctions, apparent in digital culture.
Indeed, inspired by a methodology developed by Caillois in the 1930s, A Sleepwalker’s Guide begins by attempting to resolve a prevailing distinction.
That is… the user experience of social media, has been frequently defined in opposition to sleep.
Crary, e.g., contends that sleep marks the last line of defence against the onslaught of periods of protracted digital wakefulness.
In digital culture, power is always on, or permanently on charge…
… Digital labour and leisure time are never fixed. Platforms decide when work starts and ends.
Sleeping isn’t easy, with all these attentional demands, excesses of light, infinite linking and smart thinking.
Contrary, then, to the prevalent figure of the zombie smartphone user…
… sleep-texting their way to social media oblivion… this digital dystopia requires vigorous cognitive engagements.
The concept of A Sleepwalker’s Guide is, nevertheless, more nuanced than this…
It is definitely not like the zombies on the cover… which I must point out is a design chosen by the publisher…
The artist, Mikey Georgeson’s more revealing illustrations and fiction machines, portray a sleepwalker that does not easily fit into tidy distinctions.
Things are a lot messier!
The Guide positions the user experience as intuitively felt; occurring in neurologically designed comfort zones…
But, also, provoking concrete actions and habits, and, importantly, a collective dynamic of propensity, compulsion, impulse.
To put this another way, sleepwalking is a blurry line of indistinction…
It is where the non-phenomenological world of somnolent experience slips into ambulant impetus.
As Gabriel Tarde once described the porous somnambulists of his 19th Century crowd theory… “There are insensible degrees of separation between the nonconscious imitation of others and the feelings of volition and activations of mechanical habit…”
For sure, what I have called Media Somnambulism is an update on Tarde’s initial concept of a collective dynamic.
As follows, user experiences are neither asleep nor wide awake…
… Or, indeed, in-between…
They are, at once, somnambulistic.
Each chapter in the Guide therefore deploys a conceptual sleepwalker intended to overcome a series of further distinctions.
The distinctions in need of resolving, throughout the book, include… …those made between facts, feelings and fakes …
For example, in the war on fake news, we see a clear distinction made between…
Positivistic fact-checkers
Post-truth advocates of facts that somehow disappear down a virtual rabbit hole.
This distinction presents a conspicuous deadlock between, what Connolly observes as, miserly positivists, like Daniel Dennett, and the overgenerosity of an outmoded post modernism…
To resolve this theoretical impasse, the Guide turns to A.N. Whitehead’s speculative philosophy, and a concept he introduces called aesthetic facts.
In short, aesthetic facts upset the distinction made between fact and feeling.
… Latter chapters address the many immunological distinctions apparent in social media security discourses and platform design.
Drawing on examples of contagious race hate and violence, the discussion challenges some of the failed conventional immunological distinctions made between, e.g., Self/Non-self and Delete/Ignore
The distinction that needs resolving in chapter five is a big one! The one between mind and matter. But, before I come to that…
It is important to note that this book is all about user experience. There are several points to make about this…
Firstly, user experience has become increasingly automated by algorithmic infrastructures… characterised by ‘more-than-human’ capacities.
It is these infrastructures that provoke and algorithmically churn contagion, without seeming to care about the political or cultural consequences.
As follows, the social media business model of virality/growth and the spreading of race hate, and violence have become disastrous counterparts.
Secondly, social media contagion shifts the focus from individual user experiences to an all-encompassing collective experience…
Indeed, Media Somnambulism points to a hypnotic collective dynamic, that is, itself, ‘more-than-human’.
As such, the concept of the sleepwalker is closely aligned to Clough’s work on the technological nonconscious… latterly expanded on by Thrift, Hayles and Grusin.
Thirdly, evidently, the widespread distribution of computing into everyday environments, will ignite further anxieties concerning surveillance, privacy and autonomy.
But the expansion of data power also occurs alongside an industrial-scale commodification of the user experience…
Indeed, this expansion belongs to a much broader manifestation of Experience Capitalism…
… traced back to Toffler’s profound notion of experience industries in the 1970s. As Toffler puts it:
‘a revolutionary expansion of certain industries whose sole output consists not of manufactured goods, nor ordinary services, but pre-programmed ‘experiences’.
It is, Toffler further contends, ‘the experience industry of the future and the great psychological corporations, or psychcorps … that ‘will dominate.’
Similarly, in the 1980s, we find the development of ‘an aesthetic, experiential view’ of consumption.
This is marketing focused on the symbolic, but also hedonic pursuits of fantasy, feelings, and fun.
By the 1990s, the experience economy begins to take hold… owing more to the aesthetic of Disneyland, than a Fordist factory.
Now there is a burgeoning UX industry… … a convergence of interaction design, coding, data science, online marketing and behavioural psychology, consistent with Toffler’s psychcorps.
To summarize, then, Experience Capitalism, is a kind of expropriation of experience.
It parasites collective experiences, mass attention and emotional engagement…
… so as to steer users toward all-important purchase intent.
The viral design and business model of social media (virality/growth) are apposite here, since the aim is to cultivate, capture, intensify, and spread the sharing of felt experiences.
Borrowing from Langlois and Elmer’s work on corporate social media, we might say that Experience Capitalism…
Patterns experiences – through likes, shares, posts etc.
Works on relational interactions and the capacity of machines to learn from impersonal social context rather than individual subjective experience.
An example of this latter focus on the cultivation of relational interaction is Facebook’s most lucrative ad revenue stream: its so-called Lookalike Audiences.
Finally, my approach is further couched in Whitehead’s philosophy of experience… and its attempt to reverse a bifurcation that distinguishes between an animated mind, on one hand, and dead matter, on the other.
Indeed, drawing on Andrew Murphie’s Whitehead inspired take on computational media… far from being dead matter, media is considered well and truly alive!
So, in what ways are media considered to be alive… And what are the implications of these new ubiquitous vitalities?
Chapter five makes the point that sleepwalkers will have to contend with the disappearance of computing power into the environments they inhabit.
Experiences will be captured through everyday interactions with cars, fitness gadgets, training shoes, watches, kettles, mirrors, speakers, furniture, pavements, and streetlamps.
Visual conventions of graphical user interfaces (GUIs) will also disappear.
Encounters with computing will be increasingly non-visual… and thus hidden from users.
Events will be triggered by non-task interactions, fleeting moments of unknowable, yet locatable contact, and new kinds of accidentally engendered glitch.
As several media theorists have already argued, encounters with these hidden events of digital culture will escape user cognition and the autonomy of human intension.
Historically grasped as an augmentation of human cognition, computing power is now generating experiences that occur outside of human memory, perception, and attention.
To some extent, this includes a move away from purely cognitive to affective computing…with the potential to become more sensitive to feely and emotional interactions with bodies and faces.
The capture of visceral data will produce new patterns; generating inferred future performances.
All this will happen without the need for direct human input, but, nevertheless, it will steer human actions to new places.
Notable media theorists have again expressed concern over an all-pervasive operational level of computing power… blurring the divide between human subjects and technological objects.
As Mark Hansen argues:
“Digital networks possess the capacity to gather and to exploit all kinds of data without us having any knowledge, and, to a great extent, any possibility for knowledge, of such activity.”
Nevertheless, the entanglement of passive humans and active machines is very complex.
Computers are imitation machines after all!
Media Alive is animated by machines that mimic human interaction.
Media Alive will not necessarily become a superior, autonomous post-human intelligence.
On the contrary, some machine learning technologies are already propagating mimicked human error, stupidity, bigotry, and racism.
Lastly, Media Alive seems to have converged with Nature Alive.
Computing power has seeped into nonhuman worlds.
This is not all bad news… On one hand, sensors capture entangled experiences relating to destructive impacts of the Anthropocene, on e.g. animal life and rotting landscapes.
However, on the other hand, widespread consumption of digital media increases the toxic assemblages of material extraction, human conflict, and the proliferation of obsolete, non-recyclable junk; re-entering and contaminating the Earth.
It would appear that Media Alive plays a considerable role in Nature Deceased.
So, how has the study of HCI responded to Media Alive? Well, arguably, it seems to be trapped in a bifurcated model of interaction.
Or what has been termed, in HCI circles, as the Phenomenal Matrix.
How did we get here?
The theoretical frame of HCI has shifted through three overlapping paradigms
linked to ergonomic couplings of bodies and machines
limited to the cognitions of a mind-machine coupling
based on embodied interaction
How we ‘understand the world, ourselves, and interaction’, in the latter, is derived ‘from our location in a physical and social world as embodied actors’.
These new contexts are crucially linked to technological change.
Firstly, GUIs introduced a representational turn in interaction theory.
This mapped onto 2nd paradigm methods, like cognitive-task-analysis and mental modelling.
Secondly, the rise of social technology prompted researchers towards analysing distributed cognition.
Subsequently, as computing power enters tangible, physical environments, researchers begin to see the limits of purely cognitive approaches.
This trend toward tangible technology, prompts a new HCI frame…
…. A frame focused on embodiment, significantly grasped through phenomenology.
The Matrix owes an initial debt to Husserl’s move from experience grasped through the realm of abstract ideas (idealism) to experience grasped through encounters with concrete phenomena.
However, notably, more attention is given to Heidegger and Merleau-Ponty, than Husserl.
Along these lines, embodied interaction escapes Husserl’s ‘mentalistic model of experience in the head’.
Interaction is now out there, ‘in the world; a world that is ‘already organized in terms of meaning and purpose’.
This is an ontology, arrived at through interaction.
It provides a ‘property of being manifest in the world’ in which interactions take place.
To explain the experience of interacting, embodied interaction initially draws on Heidegger’s distinction between ready-to-hand and present-at-hand.
A distinction that switches between automatic unconscious interaction and mindful attention.
On one hand, then, when a device is ready-to-hand, a user acts through a tool.
Interaction is defined as an unconscious extension of the body.
On the other hand, though, when a tool is present-at-hand, it becomes an object of conscious attention.
Like this, users act on the tool; mindful of it as an object of activity.
Similarly, the Matrix borrows from Merleau-Ponty’s distinction between the ‘physical embodiment of a human subject, with legs and arms’ and a ‘cultural world’ in which subjects extract meaning from objects.
The Matrix is indeed based on these environmental exchanges, between embodied subjects and their mindful interactions with objects.
So, although the Matrix escapes Husserl’s mental prison to grasp how experience emerges from fleshy human interactions with the world…
… subjective human perception remains stubbornly central to its ontology.
The Matrix always begins with the human who has the experience.
In other words, where the action is, can be grasped ontologically, as it is sensed (in the head, in the hand or through some other bodily interaction) by the human.
So, if Whitehead does HCI, what kind of radical departure from the Matrix would be on offer?
In Whiteheadian terms, the Matrix not only traps experience in a bifurcated relation between mind and matter.
It also constrains the terms of reference applied to experience to a subject-predicated-object relation.
Which is to say, it is always the subject (the user) who experiences the object
In chapter three, I develop a phenomenal syntax of experience…
Taken as a verb, phenomenal experience predicates the subject as the possessor of her own experiences… generally from a first-person perspective.
Taken as a noun, phenomenal experience happens to, or objectifies, a person… Experiences can alienate us, or they can be learnt from. But they are always experienced personally.
By extension, the antonym of experience, inexperience, presents a person who is found to be without the object of experience.
As a recent expression of digital culture, the user experience provides an extended hypernym of phenomenal experience. Experience not only happens to… but is also performed by the user.
Again, in chapter three, I make the point that there is a long tradition of freeing experience from this subject-predicate-object relation.
For example, even R.D. Laing, a phenomenal psychiatrist, argues, in The Politics of Experience, that the ‘relation of experience to behaviour is not that of inner to outer’.
Experience, Laing contends, is not ‘inside’ the head; experience is ‘out there in the room’.
More powerfully, Laing stresses the political importance of collective experience.
Some of the concerns he expressed, significantly pre-empt, current anxieties about social media contagions of conformity.
Laing’s experience thesis profoundly contends that by inducing similar collective experiences, it is possible to more effectively steer a population towards more aligned and imitated actions.
Decades before social media fuelled contagions of racist populisms, Laing understood the power of inducing the feelings of a population…
… encouraging them to share the same feelings; to ‘want the same thing; hate the same things; feel the same threat’ – would ensure that ‘behaviour is captive’.
This is political control realized in the contemporary stirrings of user experiences, so that they can become entrained…
Once complete, then, ‘you have acquired your consumers & cannonfodder’.
In many ways, Whitehead’s aim is to similarly disentangle experience from this syntax.
Since, to begin with, he sees subjective phenomenal experience as decidedly unreliable.
These are aspects of Whitehead’s philosophy, which are, as Stengers contends, divergent from phenomenology.
Indeed, Whitehead’s Process and Reality…introduces its own sleepwalker!
“We sleep; we are half-awake; we are aware of our perceptions, but are devoid of generalities in thought; we are vividly absorbed within a small region of abstract thought while oblivious to the world around us. We are attending to our emotions – some torrent of passion – to them and to nothing else; we are morbidly discursive in the width of our attention; and finally we sink back into temporary obliviousness, sleeping or stunned.”
There is no doubting the elegance of this quote…
But is this not the same switching between Heidegger’s two states of experience – embodied in the Matrix?
Well, as Stengers further notes, at its most exceptional; it’s most plastic, the human brain is a mere foothold in the experience of reality.
It is certainly not a phenomenal cognitive command post!
In sharper contrast to the Matrix, a Whiteheadian HCI does not couch experience in terms of human consciousness at all…
This is because Whiteheadian experience begins with an ostensibly uncanny, yet profound, proposition…
Worldly experience, and the cosmos it floats in, heralds the arrival of human subjective experience… It is not human consciousness that draws attention to experience.
It is, on the contrary, experience that draws attention to an anomalous human perception of worldly experience!
In short, then, Whitehead offers a constraining philosophical point of departure…
Since it is not phenomenal human consciousness that sheds light on experience.
Quite the reverse, it is experience in the actual world that draws attention to the aberration that is human consciousness.
Like this, Whiteheadian event theory, confronts the limits of the Matrix’s focus on perception constrained to the here and now.
This ‘small region of abstract thought’ can never absorb the temporal thickness of the events of nature alive.
Of course, HCI researchers will question the value of an approach that side-lines human cognition.
In chapter two, I draw attention to similar concerns expressed by Kate Hayles, who questions Whiteheadian new materialist’s side-lining of higher levels of cognition.
Hayles calls this a performative contradiction:
‘Only beings with higher consciousness can read and understand their arguments, yet few, if any, new materialists acknowledge the functions that cognition enables for living and non-living entities.’ Hayles continues…
‘Reading new materialism, one looks in vain for recognition of cognitive processes, although they must necessarily have been involved for these discourses to exist at all.’
It’s a valid argument. But one drawn from a biased cognitive theoretical frame…
Indeed, human perception is not entirely discounted from Whiteheadian HCI.
Firstly, the decentring of the cognitive subject should not become confused with the disappearance of thought.
Although cognition is regarded as something that arrives late, higher consciousness is not altogether discounted.
There are moments of self-enjoyment – of ‘one arising out of the composition of the many’.
But self-enjoyment must not be misinterpreted as the point wherein mind bifurcates from matter.
Mind and matter need to be taken together…
Secondly, yes, high-theory analysis must clearly be filtered through higher cognition, but that should not prevent speculation about pre-cognitive experience.
Nonconscious habits, for example, which are often misleadingly ‘designated as belonging to the mind’, develop through complex processes, enabling decisions to occur prior to cognition.
————
Significantly, then, while discarding the phenomenal syntax is vital to grasping Media Alive.
Getting rid of it is not equal to rejecting human perception!
Perception and event must not be prized apart in any way, shape or form.
But, perception alone does not produce the reality of experience.
Perception does not decide if things are more or less real!
Which is to say, in a Whiteheadian frame, embodied interaction only goes as far as declaring mere instants of percipient, and sometimes false, events in experience.
What the Whiteheadian sleepwalker profoundly tells us is that it is, inversely, the event of experience that produces perception.
To start to conclude then… perhaps those who lament the loss of human cognition in Media Alive have their media history confused…
There has never been a human mind that had such a privileged status in media events.
Consider the fictional alien in Nicolas Roeg’s 1976 film, The Man Who Fell to Earth.
Throughout the film, we see how, sitting in front of banks of TV screens, Thomas Newton, experiences all the events of the analogue media world.
It drives him crazy!
‘Get out of my mind, all of you… Leave my mind alone,” he screams at the TVs.
But, aside from Bowie’s convincing extra-terrestrial performance, humans are not aliens of this kind…
We cannot detach experiences of media objects from the entangled thickness of media events.
Human experience of media does not operate like Newton’s command post – experiencing everything in the here and now!
Minds are not absent from the matter-flow of events.
We should not therefore be too concerned with trying to re-insert human consciousness into the hidden operational level of computation.
We have never had a command module in the technological nonconscious.
Finally, then, we might have to settle for the sleepwalker’s mere foothold in the durational viscosity of Media Alive.
VIRALITY 