I have been adapting some of the work from Virality into material relating to a critique of human computer interaction (HCI). I am particularly interested in what has been referred to as the 3rd paradigm of HCI (thanks to Matt Fuller for the prompt) and the ideas it takes from neuroscience and cognitive psychology relating to cognition and affect. My work kind of follows on from Crary’s work on attentive subjectivation. Here I’m interested in the managerial aims of HCI relating to subjectivity and workspaces. Below is a transcript (not referenced) from a recent talk I gave on this subject at the Centre for Cultural Studies Research, UEL.

A Glint in the Eye of the Consumer: From attentive subjects to neuro-consumers

My question: What are the lines of flight between the 2nd and 3rd paradigm of managerial HCI?

To answer this I will explore the blending of attentive and neurophysiological technologies

• Relations established between attention, the eye and the mind.
• Move from ocularcentric sciences of the mind to new ideas from neuroscience related to affect and cognition.
• Adoption of these methods of persuasion in product design and marketing

To think through HCI and subjectivation

Using a Tardean microsociology to think through the 3rd paradigm approach to the consumer. To ask what kind of subjectivities are being made here.

A starting point…

The Three Paradigms of HCI (greatly adapted from the work of Harrison, Senger and Tatar…

HCI Paradigm	Workspace Organization Managerial aims	Subjectivation	Methods
Ergonomic•(engineering and early comp science)	Explicitly Taylorist •The most efficient fit between human and machine (coupling)	Cog in a Wheel	•Testing•Behavioural•Empirical•Atheoretical
Cognitive•(cognitive turn)	Management of cognition •How to get often distracted users to pay attention and guide conscious decision-making processes. •Management of perception, attention, memory and action (decisions)	Cognitive/Attentive Subject	•Testing of concepts•Mental models, cognitive walkthroughs etc.•Task orientated usability testing.•Time spent on task, errors made.
Experience?•(pervasive computing – – ideas from neuroscience)	•Ubiquitous computing (“Everyware”). •Management of emotional, affective, •neurological, and noncognitive interactions.	The Neuro-Consumer (the consumer’s experience)	•Non task orientated.•The affective priming of consumer experience.•Exploring relations between conscious and unconscious states•Neuro-usability •Neuromarketing! •New methods of persuasion

In his book Downcast Eyes, Martin Jay describes how sight was regarded as the noblest of the senses – from Plato to Descartes.

The eye was supposed to delineate the objects we perceive from the stream of subjective affects we absorb in the atmosphere.

As Kant contended, the visual apparatus was, “the purest intuition – since it gives an immediate representation of an object without admixture of noticeable sensation.”

As Jonathon Crary argues, it was imperative for thinkers of all kinds to discover what faculties, operations, or organs, produced the complex coherence of conscious attentive states.

Attention was to be determined by screening out meaningless reverie and distractions that disrupt reason.

Despite the affect programs in the 1970s and a noncognitive approach developed in the 1980s, mainstream sciences of the mind, including HCI, continued to neglect these disruptions to attention… Preferring to stick to behavior and cognition rather than the stuff of emotions, feelings and affects.

That was arguably until the mid 1990s, when the neuroscientist Antonio Damasio’s book Descartes’ Error prompted much interest in the relation between cognition and emotion.

Damasio claims that:

Reason may not be as pure as most of us think it is… or wish it were.

Emotions and feelings may not be intruders in the bastion of reason at all… they may be enmeshed in its very networks.

Following Damasio the significance of the relation between embodied (somatic) emotions and cognitive decision making has spread throughout the sciences of the mind.

It also supports much of the conceptual frame of this third paradigm of HCI research.

Moreover, the relation between cognition and emotion has been grasped by business enterprises keen to tap into the experiences and decisions of the consumer

Particularly in neuromarketing, experience and emotional design, and now so-called neuro-usability.

Indeed, the emotion/cognition relation has played a key role in supporting the neuromarketers’ claim to new methods of persuasion.

To better understand what is meant by persuasion in this context, I want to focus here on the techniques of neuromarketing…

Particularly the combination of eye tracking software, along with neurophysiological devices like EEG and GSR.

Although the supporting hypothesis behind eye tracking is fundamentally Kantian – as part of the neuromarketers tool bag it measures more than conscious attention.

The eye is not merely a representational mirror of the mind.

New methods move on from simply seeing ideas and images as shapers of decisions.

They factor in the sensations experienced during attention.

Eye tracking quite literally measures what is attended to – by following a glint in the eye of the consumer. That is to say, an infrared light is reflected onto the eye and tracked.

Attention is analyzed according to fixations, saccades and scanpaths.

Fixations are measured according to duration (lasting between 250–500 milliseconds)

Saccades measure movement from one fixation to another (lasting 25 – 100 milliseconds).

When fixations and saccadic movements are linked together, they form scanpaths.

Most eye tracking research presents data in the form of a heat map, which shows the regions of most attention as hot and less attended areas as cold.

What is ostensibly being measured is thought attention, — generally linked to Just and Carpenters 1976 eye—mind” hypothesis, which states that “there is no noticeable lag between what is fixated and what is processed.”

“What a person is looking at is assumed to indicate the thought on top of the stack of cognitive processes.

Eye—movement recordings can thus provide a dynamic trace of where a person’s attention is being directed in relation to static or moving images.

Following this distinctly ocularcentric notion then, the eye—mind hypothesis traces a direct pathway between what enters the mind via the eye.

However, there are a number of anomalies, which suggest that the relation between sight and mind is not as direct or indeed conscious as it would seem.

Notably, the study of pupillometrics relates the unconscious reception of multiple sensory stimuli, triggered by smell, taste, touch and hearing, to pupil dilations…

This suggests that the journey between what is seen and what is thought is not an uninterrupted pathway.

By combining dilation with blinking and gazing, one eye tracking manufacturer claims that pupillometrics can reveal the eyes interpretation of affective valence… “going behind the cognitive curtain” and tapping into emotional engagement.

Another  software innovation by the Danish company, iMotions, flags a similar turn to affect in eye tracking.

Rather than presupposing that what is being looked at equates to what is being thought, the Emotion Tool works with eye tracking to analyze the relation between cognitive and emotional consumption.

Distinct from older methods that tend to measure user responses according to either voluntary attention or involuntary inattention, the Emotion Tool is intended to tap into the relation between these two states.

Older systems made a distinction between

(a) voluntary responses … associated with what is attracting the eye…

For example,

bodily gestures and orientation,

voice intonation, or eye contact and evasion.

And (b) involuntary responses

For example, increases in heart, pulse and breathing rates and body temperature

The Emotion Tool considers instead the relation between the implicit, unconscious part of the brain (the limbic system) – widely recognized as hardwired to the nervous system…

… and the physical reactions of the explicit, conscious system (the frontal cortex).

It is the somatic memory, physical responses and emotions of the implicit system which is now assumed to guide the explicit system.

As iMotions claim…

“It is now generally accepted that emotions dominate cognition: the ability to think, reason and remember.”

“Therefore, there is increasing interest in methods that can tap into these mostly subconscious emotional processes, in order to gain knowledge and understanding of consumer behavior.”

The Emotion Tool tracks facial expressions, particularly those that occur around the eyes, the amount of blinking, the duration of the gaze, and pupil dilation.

It also incorporates an algorithmic assessment of two dimensions of the emotional response: emotional strength and valence.

The first is gauged by the level of excitement an external stimulus provokes in the consumer,

The second, measures the feelings that follow the stimulus — the degree of attraction or aversion that an individual feels toward a specific object or event.

Scores are calculated from a range of

Pleasant to Unpleasant

… or neither pleasant nor unpleasant.

High scores are defined as affective, low scores unaffective.

Neuromarketers increasingly use eye tracking in combination with EEG and GSR in an effort to link attention to what a consumer “feels about a product.”

They claim to substitute the biased inaccuracies of self—reporting surveys, with objective measurements of eye movement, electrical activity in the brain, heart rate, skin conductance and temperature

The intention is to prime consumer experiences by arousing certain feelings which are somatically absorbed and moved to memory.

As the enthusiastic CEO of the Berkley based company Neurofocus puts it, these techniques help the marketer to look beyond conscious consumer engagement and actively seek out what unconsciously attracts.

“Absorption is the ideal,” he claims.

Because it “signifies that the consumer’s brain has not only registers your marketing message or your creative content, but that the other centers of the brain that are involved with emotions and memory have been activated as well.”

Along these lines, persuasion and engagement become the watchwords of neuromarketing.

There are, it seems, 3 requirements  necessary to prime emotional experiences

First, attention is drawn by “cultivating the ability to change what is focused on by intervening directly in perception.”

What draws attention is informed by research into Attention Deficit Disorder.

Second,

Using research from obsessive compulsive disorders and manias…

…the neuromarketer endeavours to stimulate and fascinate the consumer’s emotional responses to the brand stimulus.

Third,

market researchers no longer need to survey memory retention. Using research from memory disorders, they claim to be able to anticipate purchase intention, and steer it toward predetermined points of fascination.

Persuading the consumer by appealing to the subconscious is of course nothing new.

Many brands are already saturated in emotional experiences and celebrity narrative. But neuroscience might just suggest how unconscious consumption actually works.

As one emotional design guru asserts, attention needs to be grabbed at the visceral level of experience processing. Brands are all about emotions which draw the consumer towards the product, influencing purchase intent.

Not surprisingly perhaps, these methods are not limited to the consumer of brands.

A similar neurological approach has recently spilled over into the arenas of political campaigning.

In theUSalready, researchers have measured neural activity in the brains of Democrats and Republicans viewing the faces of presidential candidates in 2004.

By studying emotion regions in the brain, they were able to compare neural activity to self—reporting of feelings felt for particular candidates.

When a voter views a politician’s face… cognitive control networks, which regulate emotional reactions, are activated.

The emotional responses of voters to the candidates in 2008 were also tested, including swing voters and Democrats exposed to Bush’s campaign commercial featuring the events of 9/11.

Here we see George Lakoff’s concern about the easy to manipulate, neo-conservative nature of the political unconscious.

The War on Terror becomes “a misleading and destructive idea introduced under conditions of trauma and then repeated so often that it is forever in your synapses.

As Thrift argues,

“political consultants now understand enough of the dynamics of brain—body chemistry to be able to make reasonably predictable interventions in the political unconscious.”

As a result it is “possible to tug on the behavior of voters by transferring certain narratives into the political domain as forms of habitual response which the individual voter is plainly susceptible to.”

Ok, so what kind of subjectification does neuro-persuasion imply?

If indeed emotions, feelings and affect are coupled to persuasion in this way, we are perhaps seeing the same subjectivity in the making described by the 19th Century contagion theorist, Gabriel Tarde.

Tarde’s project asked what is society?

He answered that society is imitation-suggestibility.

Indeed, social agency is, for Tarde, a dream of action.

It is a reverie in which the social somnambulist is under the influence of the magnetic action-at-a-distance of points of fascination and the intoxicating glory of celebrity

Like Tarde’s neuron-level contagion of example and suggestion, the neuro-consumer’s vulnerability to persuasion is not merely satisfied by mental images, but achieved by way of sub representational flows of sensation, desire and belief.

Tarde would have certainly grasped the jargon of neuromarketing.

His mind contagions explicitly refer to a relation he established between desire and belief – as one and the same as that established between affect and cognition.

The object of desire, Tarde argued, is belief…

A Tardean rethinking of persuasion theory returns us to a much older sociological spat he had with Durkheim.

On one hand, Durkheim grasped the social as distinct from psychology & biology…

“… every time the social is explained in that way, he contended, we may rest assured that the explanation is false.”

On the other, and in the words of Lazzarato, Tarde provides an… ‘understanding of social ‘associations’ … with no distinction made between Nature and Society’

“Nothing, is less scientific, Tarde argued, than the establishment of this absolute separation. Of this abrupt break. Between the voluntary and the involuntary. Between the conscious and the unconscious.

Do we not pass by insensible degrees, he asked, from deliberate volition to almost mechanical habit?

The somnambulist “unconsciously and involuntarily reflects the opinion of others, or allows an action of others to be suggested to him.”

Like the neuromarketer, Tarde’s persuasion theory occurs at an intersection point between

The culture of attraction

and

A biologically hardwired inclination to imitate.

It is possible to follow Tarde’s line of flight to a number of new sciences and business enterprises. There are indeed similarities between Tarde’s contagion theory and the new science of networks, for example.

Particularly theories of herding and cascading networks.

Others have related his work to memetics – the underpinning theory of viral marketing.

But I suggest that Tarde is neither a network theorist nor a Neo-Darwinist.

The somnambulist is not made of networks or memes.

Tarde offers instead a neurological explanation of subjectivity in the making: a production of a porous neurological relation with others.

Where Stanley Milgram attributed conformity and imitation to agentic social proof, Tarde’s herd cannot escape the swash of affect, feeling and emotion.

Here Tarde’s imitation thesis further intersects with the fairly recent mirror neuron hypothesis: a brain circuitry that

… fires when we either perform a given action or see someone else perform the same action.”

Like this, Thrift argues that the mirror neuron is

“. . . a plausible neurophysiological explanation for the means by which the existence of the other is etched into the brain so that we are able to intuit what the other is thinking – we are able to “mind read”—not only because we see others’ emotions but because we share them.”

It is perhaps the volatility of this encounter that makes Tarde’s somnambulist open to imitation—suggestibility.

Indeed, the potential for marketers and political consultants to “mind read” consumers and voters marks a potential shift from managerial aims of cognitive persuasion toward the steering of noncognitive feelings.

Following the noncognitive psychologist Robert Zajonc, we could say that neuro-persuasion works on beliefs… not by way of appealing directly to the eye or mind, but rather by way of the gut or feelings that bypass or influence cognitive processes.

This is not an unthinking model of persuasion, far from it: It is rather, as Zajonc argues, the idea that feelings might have a mind of their own.

This is a marked shift away from the cognitive user in HCI, in which the inputs and outputs of the black box mind are managed and put to work, toward the neurological management of the unconscious, increasingly monitored and absorbed into a Tardean-like dream of action.

No longer a model of the mind—as—digital—computer, but an over stimulated twenty—first—century neuro—somnambulist: someone who confuses what they believe, desire, and decide on, with what they dream about.