“The free citizen will show his face, say his name, and have an address.” – Thomas de Maizière (German Minister of the Interior), 14 Thesen zu den Grundlagen einer gemeinsamen Netzpolitik der Zukunft [“14 Theses on the Fundamentals of a Common Internet Policy for the Future”] (2009)

“Biometric face recognition has seen a number of advances since 2006, driven by the trend and popularity of social networking sites, the prevalence of mobile smart phone applications, and successful implementation in visa applications and in criminal and military investigations. Media giants such as Google, Apple, and Facebook now include face recognition in their products, and the commercial development of low-cost ‘face sensors’ (cameras with built-in face detection) is underway.” – National Science and Technology Council (NSTC), Subcommittee on Biometrics and Identity Management, The National Biometrics Challenge (2011)

I. Augmented ID

He must have had a long night. Now, at a quarter past eight in the morning, the young man is standing in front of the mirror with disheveled hair and a crooked tie and preparing himself for another day at the office. Beyond typical bathroom activity, however, this routine also entails that the man has to turn on his smart phone and switch his public profile from “party mode” to “work mode.” Once this has been done, whoever wants to learn any information about him will not be directed to his Facebook profile or Twitter feed but rather to his business card or to his latest professional presentation. This is a scene from a promotional video made by the Swedish company TAT, which designs user interfaces for smart phones. The concept being promoted is called “Augmented ID,” and TAT advertises the idea with the slogan “Adding a digital layer to the real world to present yourself.”1 The video goes on: While the young man is clicking through a PowerPoint presentation at a meeting, his audience members are able point their phone cameras at him, and a face-recognition program calls up his profile onto their screens. With one of the icons that is floating around his face on the touchscreens, whoever so desires can look up his contact information and even rate and comment on his presentation. One member of the audience decides to give it four stars (even though the presentation has not even ended). In the final scene, a panoramic view of the room shows how Augmented ID will surround everyone there with a halo of icons. In other words, everyone present will be able to learn as much as they want about each other – business contacts, favorite songs on Last.fm, current Facebook statuses, and so on – without even having to ask. Of course, this also means that everyone gathered in the room will have to go through same identity management routine that the protagonist has performed in his bathroom. Everyone will have to decide, again and again, which digital mask to put on and which digital mask to take off.

Even though Augmented ID has not yet been fully developed, its promotional video offers a rather credible illustration of the convergence of social networks, mobile technologies, and face recognition. Essentially, the vision is one of an “augmented reality” in which every individual is constantly surrounded by a cloud of profile information and every social interaction can be linked to data operations. In this situation, every face is technologically recognizable and functions as a link to a public or semi-public profile that, for its part, is composed of various individual profiles on various social networks.

It is clear from several examples that there is a market for such applications and, moreover, that the necessary technology is largely available. In the summer of 2011, Facebook activated a highly controversial new feature that involved automatic face recognition. Previously, users of the social network themselves had to tag and identify pictures of their friends in order to assign names and profiles to one image or another, but now this process was supposed to take place (semi-)automatically. The new software compared pictures, grouped them together, and made suggestions about which recurring faces should be associated with which names.

More than two-hundred million pictures are uploaded onto Facebook every day, and thus the social network has become the largest database of images in the world. To the extent that more and more pictures are being tagged, and thus more and more people can be identified by name, Facebook has developed into the “registration office of the web,” as Konrad Lischka has written for Spiegel-Online.2 As more and more clearly identified photographs become available, face recognition software accordingly becomes more and more reliable. The company Face.com, which developed such software for web applications before being purchased by Facebook in 2012, claims that the probability of correctly identifying someone will increase considerably if at least ten pictures of this person have already been tagged.3

Facebook was compelled to deactivate the feature after users had mounted large-scale protests and after data protection agencies had applied a considerable amount of legal pressure, at least in Europe. Yet the sheer number of pictures posted on social networks, which are relatively easy to access (not only due to inadequate security settings), has made them a valuable resource for automated identification programs that are unaffiliated with the networks themselves. In 2011, in fact, a research team at Carnegie Mellon University demonstrated quite clearly how easily the data on Facebook and other sites could be exploited by external systems. With commercial smart phones, web cams, and software from the company Pitt Patt, which was bought out by Google in the summer of that year, the researchers were able to identify the personal Facebook pages of a great number of people who were supposedly using a certain dating site “anonymously.” From a representative sample of students on their campus, they were able to identify a third of them, using only unprotected and accessible profiles on social networks, and in several cases they were even able to access social security numbers and other sensitive data. The researchers summarized their findings as follows: “Your face is the veritable link between your offline identity and your online identit(ies). Data about your face and your name is, most likely, already publicly available online.”4 Faces have become the key to associating scattered and even anonymous profiles with one another.

It is thus no surprise that this form of cloud-powered face recognition has also been tested by the police. Since 2009, the European Union has supported a research project with the title INDECT, which stands for “Intelligent Information System Supporting Observation, Searching, and Detection for the Security of Citizens in Urban Environments.” One goal of the project is to develop processes for connecting publicly accessible internet profiles with data from security cameras and police databases. In the near future, it is possible that face recognition techniques and behavioral pattern analysis will be implemented in drones and security cameras to identify suspicious people and potentially criminal behavior in urban areas.5 It had been rumored that such technology was already to be used as early as 2012 at the European Championship games in Poland and the Ukraine, but this has been denied by authorities.6 Of course, this does not mean that the European Championship did not serve as a testing ground for advanced security systems. Upon entering the National Stadium in Warsaw, for instance, all of the attendees were recorded on camera, and images of their faces were compared with the information on their personalized admission tickets. Throughout the game, 370 surveillance cameras monitored the stands, and face-recognition software was supposedly able to identify each and every fan unambiguously at any given time.7

The use of face-recognition software at large sporting events has a prehistory. One of the largest tests of face recognition in a public (or semi-public) space was conducted at Super Bowl XXXV in Tampa, which took place in January of 2001. Without their knowledge or consent, more than seventy thousand attendees were recorded by security cameras as they entered the stadium. A program known as Viisage then compared their facial features – the width of their mouths, the position of their eyes and nostrils, and so on – with mug shots collected in police databases. The result at the time was somewhat sobering for the authorities: The software raised several alarms, mostly false, and no arrests were made on the basis of its findings.8

The year 2001 was important to the history of biometric recognition for another reason as well. In response to the attacks on 9/11, the United States and Europe instituted new security measures for air travel and border crossings, and these measures included the mandated use of electronically scannable passports for the sake of biometric identification. As early as January of 2001, the MIT Technology Review named biometrics one of the “top ten technologies that will change the world.”9 The introduction of biometric-compatible passports in the European Union, which occurred in 2005, can be understood as a form of state investment in the development of a promising branch of industry; according to Otto Schily, who was the German Minister of the Interior at the time, “The passports will be economically important as well. We will be able to show that Germany has the know-how and innovative capabilities to set the standards in the new sector of biometrics.”10

Given that biometric identification techniques have broken into the mass market, these investments seem to have paid off. What was at first developed and advanced by police forces, militaries, and secret services is now being used for purposes quite different from identifying criminals, monitoring the streets, and securing borders. Such technology has rather become an increasingly marketable aspect of consumer electronics: USB fingerprint readers are now used to secure access to sensitive data on laptops; Picasa, an image organizer owned by Google, allows users to browse through their private pictures with a face-recognition program; and game consoles equipped with 3-D cameras are able to distinguish between the various players standing in front of the television screen.

A more recent phenomenon known as “gigatagging” is predicated on the fact that, in certain cases, the desire of users to identify themselves in pictures is so strong that automatic face recognition is not even necessary. In addition to panoramic views of landscapes, the website gigapixel.com also contains photographs of sporting events and urban crowds. Thousands of hockey fans who watched game seven of the Stanley Cup finals in Vancouver can be seen in a photograph taken on June 15, 2011. The picture, which at first resembles a chaotic blur, can be enlarged like the images on Google Earth until each individual face is clearly distinguishable.11 This mass panorama is in fact not a singular image but rather a composite of more than two hundred individual photographs that were taken over the course of about fifteen minutes. Yet it is only when the picture is fully enlarged – and under close inspection – that one can see the “seams” where the individual pictures have been sewn together. Visitors to the site are encouraged to tag themselves and their friends, that is, to provide each face with a link to a corresponding Facebook profile. Hundreds of people have already participated, and in doing so they have probably also identified a number of acquaintances who were not even with them at the event in question (Thomas Thiel has referred to this phenomenon a “manhunt among friends”).12 The company Orange Mobile had done something similar for the Glastonbury Festival in 2010. According to their website, nearly ten thousand faces were tagged in crowd photographs, which was supposedly a world record at the time.13

In theses cases, the crowd identifies itself – though retrospectively, after it has dissolved. The virtual gathering of festival-goers consists of hundreds of individual profiles, while the crowd photographs serve as the interface with which the individuals in question can subsequently locate themselves within a mass experience. The commercial value of this for Orange Mobile lies in so-called post-event marketing and in prolonging the consumer experience and its associations with the sponsoring company. However, the fact that such large-scale instances of self-identification might also be of interest to security forces is made clear by a somewhat off-putting caption above the crowd picture taken in Vancouver: “Before the riot.” After the home team had lost – to recall – the peaceful and joyful scene captured in the image gave way to massive riots in the streets. During the subsequent police investigations, digital images of the crowds happened to be quite useful to law enforcement agencies. In order to identify offenders, the authorities scoured through vast amounts of video material that fans and passers-by had recorded on their mobile phones and made available to the police.14

II. Unlikely Repetitions

Since the onset of modernity, crowds and the act of identification have been closely associated with one another. Identification has always been a means to cope with contingency. It serves to filter out the repetition of significant patterns from streams of people, data, and images, and it makes use of such recurrences to differentiate and address individual subjects. Essentially, biometrics is thus a means to filter and select on the basis of repetitions and probabilities. It identifies configurations of features that repeat themselves and allows for such features to be associated with a name, a profile and a person. Since the nineteenth century, the process of identifying people for police records has been concerned with masses and crowds – with anonymous urban throngs, with real or imaginary masses of undetected repeat offenders, but also with the increasingly expansive amounts of data collected by the police itself.

From the perspective of security forces and the apparatuses of control, masses and crowds have been a problem of assigning correct addresses, and this is as true today as it was then. In this regard, the crowd is something like the opposite of the archive, namely a disorderly gathering in physical space. Identification, on the contrary, means isolation, registration, and designation, and it is based on the production of standardized conditions that allow for the comparison of repeating features. Such was the case as early as 1882, when, in Paris, Alphonse Bertillon developed the method of Bertillonage or anthropometric identification. Instead of relying on vague information about a suspect’s stature, hair color, and physique, Bertillon based his technique on measurable data that could be recorded alongside photographs on standardized index cards. This required an anthropological approach to identification that was grounded in statistics. He would begin his measurements of a suspect by recording the length of his head, which he would then sort into three categories: average, above average, or below average. The methodological basis of this classification was the statistical notion of normal distribution (each of the three segments of a bell curve represents a subset of equal size). These categories were further classified according to the width of a suspect’s head, the length of his (or her) middle finger and forearm, and so on. With such a system in place, it was possible to organize an archive of thousands of index cards into manageable sections with no more than a dozen entries per drawer.15

Today, four categories are used to classify biometric features: universality, distinctiveness, robustness, and measurability. In order to serve as a distinguishing trait, a physical characteristic therefore has to be present in all people, slightly different in everyone, unchanging over time, and quantifiable by means of one technology or another. The biometric criteria of distinctiveness and robustness, however, are relative quantities. Instead of a certain feature being understood as entirely unique and absolutely permanent, the quantitative extents to which a physical characteristic varies within a given population or changes over time become the “criteria to distinguish the qualities of biometric systems.”16 Bertillon’s measuring procedures were insufficient in both of these respects, and at the beginning of the twentieth century his methods were therefore superseded by fingerprint identification. Although Bertillon was able to narrow down his searches quite well with his archive, his final proof for identifying someone still depended on photographs and the special “discriminating features” of the suspects. With fingerprints, however, identities could be determined with remarkable certitude, and Francis Galton and the other founders of dactyloscopy were able to provide statistical evidence for the uniqueness of papillary ridges. The arrangement and distribution of so-called “minutiae,” which are the characteristic points at which individual ridges come to an end or branch off, show such a high level of variance that their recurrence in nature is supposed to be extremely unlikely.17 To this day, “nature does not repeat itself” serves as an axiom of biometric identification.18

III. Identity Without the Person

Over the past few years, one of the most vocal critics of biometric identification has been Giorgio Agamben. In response to the stringent entry requirements initiated by the United States after 9/11, he wrote a widely publicized short essay in 2004 with the title “Bodies Without Words: Against the Biopolitical Tattoo.” Because of his refusal to have his fingerprints taken, which foreign visitors to the country are now obligated to do, Agamben resigned from his faculty position at New York University. In his essay, he objected to “the appropriation and registration of the most private and unsheltered element, that is the biological life of bodies.” Through this reduction to “bare life,” to a mute “body without words,” he believed that individuals would be politically incapacitated, and he did not hesitate to compare this practice to the compulsory tattoos given at concentration camps.19

In a later essay, Agamben took it upon himself to elaborate upon his critique of biometric registration. At the heart of his discussion is the concept of the person. His diagnosis is that an “identity without the person” has come to replace personal identity, a process that began with the police records taken around the year 1900 and has culminated in the biometric systems of the present day. Agamben argues that the goal of biometrics is not the recognition (Anerkennung) of the person – which is always a social act – but rather the mechanical recognition (Wiedererkennung) of features and data records. The word persona originally denoted a “mask,” and the person is thus a social role whose identity is produced by society. People become persons when they are recognized as such by others, and recognition as a person comes with certain rights and privileges. As a moral person, my relationship to this role is ambivalent, in that I can choose to adopt it or I can distance myself from it. First developed in the nineteenth century, police methods of identification have served to disconnect identity from the person and its recognition; the societal mask has been set aside, and identity has been fused with the biological features and characteristics of the body. To Agamben, this “identity without the person” is no longer engendered by society and adopted by individuals; it is rather defined in terms of biology and specified by means of technology. Instead of identifying myself with a social mask, which I can wear without being reduced to it, I have been reduced to impersonal data, data about my biological existence that is technologically recorded and processed.20

There is a literary precedent to Agamben’s notion of “identity without the person.” In The Man Without Qualities, Robert Musil has his protagonist Ulrich experience something quite similar, namely the “statistical disenchantment of his person.”21 Ulrich, the man without qualities, happens to be arrested in the fortieth chapter of the novel. The occasion for the arrest is almost as trivial as it is random, and yet before Ulrich is able to explain matters, he finds himself at the police station. The arrest resembles an encounter with a Doppelgänger, for the description of himself that Ulrich confronts at the station is unquestionably a description of him. However, although this description may be proof of his identity, it is in no way identical to him – on the contrary: “His face counted only from the point of view of ‘description.’ He had the feeling that he had never before thought about the fact that his eyes were grey eyes, belonging to one of the four officially recognized kinds of eyes in existence of which there were millions of specimens. His hair was fair, his build tall, his face oval, and his special peculiarities were none, although he himself was of a different opinion on this score.”22 Interestingly enough, Ulrich is able to derive a good deal of fascination from the disenchantment of his person: “The most wonderful thing about it was that the police could not only dismantle a human being in this way so that nothing remained of him, but they would also put him together again out of these trifling components, unmistakably himself, and recognize him by them.”23

What happens at the police station could be called a transformation of characteristics into features. Characteristics lose their intrinsic qualities and become generally available. Identification does not operate with singular peculiarities, for such things cannot be compared. Isolated details such as one’s eye color and body size, which “mean” nothing in themselves, become identifying features when their combination with other impersonal traits is highly unlikely to be confused with another person’s combination of features. The “impersonal” aspect of identity is not the body reduced to its bare biological factuality, as Agamben supposed; it is rather the mass existence of the features with which it can be identified. As probable or improbable differentiators, these features always exist outside of the individual persons whom they make recognizable. They represent repetitions or deviations within statistically normalized multitudes. Every act of identification entails the conjuring of virtual and latent masses of people, the inestimable masses of other bodies. The crowd is not only the counter-image of biometric databases and identity indexes; it is also their prerequisite, and this is because biometric identity is always relational, based as it is on significant differences within large-scale distributions. The statistical disenchantment of the person locates these differences within pre-recorded sets of comparative data, and its most impersonal element is not the biological body but rather the masses of other bodies with which the latter is compared.

IV. Anonymous Masses

Urban crowds have engendered fascination and horror since the nineteenth century, if not before. They are a particular product of the modern city, of acceleration, anonymity, and the fragmentation of all aspects of life.24 While the masses can simply be regarded as a quantitatively determinable multitude of each and everyone, from a bourgeois perspective they have long represented the “other” – the rabble, hoi polloi, the “dangerous classes.” The discourse of crowd psychology, which reached its zenith in the last decades of the nineteenth century, is an expression of bourgeois anxiety about the unpredictability of such masses.25 According to this discipline, the masses are the milieu in which human beings revert to a violent natural condition that was thought to have been overcome. In this light, crowd psychology is not so distantly related to criminal anthropology, and thus it was above all the potentially criminal and seditious masses that interested the likes of Scipio Sighele and Gustave Le Bon during the 1880s and 1890s. To them, the masses are anonymous and in fact cannot be named. They have no identity of their own; individual identities dissolve in them. For Le Bon, crowds seem to be a force of nature that causes personality to disappear, allows unconscious instincts to have the upper hand, and hypnotizes or infects individuals to move in a common direction.26 As revenants of a prehistoric era, Le Bon’s crowds collectively revert to an atavistic and primitive stage, and thus the discourse of crowd psychology is closely connected to the general fear of degeneration that prevailed during the late nineteenth century.27 More than anything else, crowd psychology is the psychopathology of crowds.28 Then again, not a few of its ascriptions seem like the phantasmatic opposite of police efforts to maintain order and identify wrongdoers. Crowds were thought to be formless and indistinguishable, and their prevailing logic was believed to be one of similarity, imitation, and infection. This logic seemed to make it nearly impossible to differentiate individuals with distinguishable characteristics, and thus it seemed likewise impossible to ascribe individual blame to any of these individuals or to hold any one of them responsible for any given act.

The threatening and disorderly masses of the nineteenth century lost a good deal of their abhorrence by the 1920s. During the years between the wars, authors as various as Siegfried Kracauer and Ernst Jünger noticed the emergence of new, disciplined, and Fordistic mass formations. To them, such formations no longer represent the opposite of order but rather the indispensable bearer of orderliness itself. Published in 1927, Kracauer’s essay “The Mass Ornament” begins with a discussion of the staged and synchronized mass formations of gymnasts in a stadium and dancers on a stage. In mass ornaments such as these, individuals are merely building blocks – exchangeable elements or “fractions of a figure.”29 Kracauer regards the impression of abstract figures onto living bodies as a visible expression of the capitalist production process: “[I]t is only as a tiny piece of the mass that the individual can clamber up charts and can service machines without any friction. A system oblivious to differences in form leads on its own to the blurring of national characteristics and to the production of worker masses that can be employed equally well at any point on the globe.”30 In his book Der Arbeiter (1932), Ernst Jünger similarly remarks that modern technology has transformed the potentially dangerous masses of the nineteenth century into a disciplined army of workers. This was accomplished, he thought, by means of the Tayloristic optimization of the work process as well as by the complex networks and infrastructures of bureaucracy, technology, and transportation.31 The new, Fordistic mass formations of the 1920s are not defined by a common desire; they are rather an expression of an external and rigid order. In Jünger’s description, such masses are made to resemble a cinematographic sequence of shocks and sensations. The masses, he writes, “are perceived as interlaced ribbons and concatenated streaks of faces that rush by in a flash, and also as ant-like colonies whose forward motion is no longer subject to rampant desire but rather to an automatic form of discipline.”32

To Le Bon and others, the dreaded masses of the nineteenth century evinced the raw and uncontrollable animal nature of human beings. To Kracauer and Jünger, on the contrary, the masses of the 1920s called to mind a higher and abstract level of order. In both cases, however, the masses in question consist of homogeneous, indistinguishable, and faceless elements that are fully subordinate to the crowd itself, regardless of whether it is compulsive or disciplined.

V. Crowds in Augmented Space

Police identification was and remains an attempt to differentiate these seemingly faceless mobs, namely by finding statistically significant features that enable individuals to be addressed unequivocally. However, it is not only the job of the police records department to differentiate the masses on a statistical basis. This is rather a central concern of the social sciences as well, including their commercial applications in the fields of advertising, public relations, and marketing. Whenever these fields turn their attention from determining target groups or market segments to establishing individualized client relations, the primary issue becomes the proper attribution of consumer decisions. While the “salaried masses” of employees described by Kracauer in 1930 – not to mention the “other-directed” people that constituted David Riesman’s “lonely crowd” in the 1950s – were already treated by advertisers and the culture industry as individual consumers, it was only in exceptional circumstances that they were addressed as identifiable economic agents.33 Their individual preferences gained relevance only in the form of huge quantities: sales numbers, circulation rates, and radio or television ratings. Today, on the contrary, every single consumer decision creates added value; each decision contributes more information to a consumer’s profile, which can in turn be treated like a commodity in itself. Consumer feedback, moreover, is now being processed more and more quickly and directly by distributors, marketers, and manufacturers.34 Consumers are in fact rewarded for participating in such feedback loops – with product recommendations, rebates, and so on – and at the same time it is becoming more and more inconvenient not to participate in them.35 Data collection has become strikingly excessive, and this is especially so because it has become more and more inexpensive to gather, save, process, and transfer information. No longer sporadic and partial, the collection of data is now taking place more or less continuously.36

If the person, in Agamben’s sense of the word, had once been a stabile social mask whose statistical disenchantment was essentially experienced as a reduction to abstract and technologically detectable features, “identities without the person” have now been developed and expanded into complex profiles. Such profiles are the result of countless individual activities, decisions, transactions, and movements; and they define themselves against the immense background of data that can be statistically and sociometrically evaluated.37 As David Joselit has written: “We are all profiles: data records collected from innumerable consumer decisions, about whose consequences we are scarcely aware.”38 Although profiles can easily be distinguished from one another, their form at any given time is dependent on the type of access that leads to them. The elements of a profile are weighted differently, that is, depending on the search terms that are used to find it and the interests (or profiles) of those wishing to access it. Profiles collect previous consumer decisions and expressions of interest in order to identify potential customers, and they collect previous examples of suspicious behavior in order to identify potential threats. Although they are open-ended collections of data that can be supplemented and expanded in any given way, profiles can nevertheless be understood as definable unities, at least to the extent that only information that can be related to a single, identifiable person becomes part of his or her profile. Such profiles, therefore, are simultaneously open and closed quantities. Admittedly, individual profiles on Facebook or other sites are centrally managed, but even in this regard it is clear that the amount of data that is saved internally far exceeds the complex of data that appears on the user interface. In that users maintain multiple accounts and profiles on various sites, and in that these can be associated with one another and with other data complexes by means of a user’s own links or by a third party’s targeted search, volumes of data emerge that, as a whole, seem to have no fixed location and no fixed address even though each of their elements remains discretely addressable. Identity dissolves into a network of relations, a scattered and diffuse multitude with blurry boundaries. The result is that we have permanently lost control over the data that we produce and over the data that we collect. And yet, as “agents and authors” of our own profiles, we are nevertheless encouraged to keep this information up to date and even to associate ourselves with it.39 The presumably clear separation of social recognition (Anerkennung) and technological identification (Erkennung), which has been the impetus behind Agamben’s argument, is in fact becoming more and more blurry as social networks such as Facebook have begun to formalize, technologically, the very structures that underlie the rituals of social recognition.

Individual profiles now seem to be subsets of larger sets of data, while collective entities are treated as something that emerges from incessant operations of gathering and linking individual pieces of information. This is true not only for the latent and scattered masses that are brought together by social media networks; it also applies to the manifest and concentrated crowds of people who occupy seats in stadiums or who gather anywhere in urban spaces. One of the theses of this essay is that it has become more and more difficult to distinguish between these two types of crowds, let alone call them opposites. Rather, massive efforts are being made to establish stabile connections between latent and manifest crowds and to fuse them together into a single continuum. Throughout this process, the face is able to serve as one of several possible links that are used to connect physical bodies with digital profiles. It is generally more suitable for this than other biometric features because its digitally reproduced image is readily available in both spheres, assuming that such images are already circulating on the internet and have been recorded by security cameras in actual situations. The availability of facial images is opposed by the unavailability of the face itself, which is difficult to exchange, remove, or conceal – a fact that explains the immense interest in face-recognition systems.40 Face recognition, however, is only one the possible ways to achieve such ends; mobile telephones, for instance, have been far more effective in this regard. Whereas the technological detection of faces generates complex amounts of highly variable data that can only be made operational by means of rather error-prone algorithms, smart phones emit unambiguous address data.

As soon as each and every individual in a crowd can be recorded, identified, and associated with a data profile – by whatever means – a new mass formation will emerge that will have hardly anything in common with the anonymous crowds in modern cities. In Lev Manovich’s terms, it might then be possible to speak of “augmented crowds” or of masses operating in an “augmented space,” where physical gatherings are superimposed with huge amounts of digital information.41 The descriptions of the masses in the nineteenth and twentieth centuries underscored the moment of homogeneity, the leveling of differences produced by human crowds; individuals, in other words, were thought to lose themselves in crowds and become indistinguishable. Augmented crowds, on the contrary, are highly differentiated and scalable masses. They are a product of the technologically realized possibility of shifting one’s view from single elements of a crowd to the crowd as a whole, from individuals to nearly any given micro-gatherings, and from there to subgroups and immense multitudes. And they are product of being able to do all of this while ensuring that each level can be addressed and quantified. This new image of the crowd differs from previous conceptions in a second way as well. Because every physical gathering is limited by space and time, crowds were always regarded as fleeting and unstable formations. However, now that physical gatherings have become the object of comprehensive data processing, their temporality has changed accordingly; the information that refers to them can now be retrieved now or later, here or elsewhere. According to crowd psychology, the crowd was a collective without memory. Its members would literally forget themselves in order to act as though in a trance. With our new technological capabilities, the previously fleeting masses now seem to be archivable. What is more, today’s crowds even make records of themselves.42

Whereas the process of gigatagging provides information about large gatherings after the fact, engineers working on crowd monitoring are attempting to analyze them in real time. In this case it is typically mobile phones – rather than faces – that are used to locate individuals in a crowd. During the summer of 2012, for instance, the German Research Center for Artificial Intelligence and the London Police Department collaborated on a project that enabled authorities to “follow the streams of visitors during the Olympic Games ‘live’ in their borough, […] recognize potentially hazardous situations early on, and inform the visitors directly.”43 For this to work, voluntary participants allowed the coordinates of their smart phones to be monitored. On the basis of this data, so-called “heat maps” were created that, like infrared images, used a scale of colors from light blue to red in order to indicate “in which direction and at what speed the crowds are moving and where conglomerations of people can reach critical dimensions.”44 The participants in this project seem to have played two strange roles at once. Their behavior was treated as being representative of the actual crowds, and projections and prognoses about overall crowd behavior were made on the basis of their activity. They were thus part of a potentially hazardous mass (and only as such were they of interest to the security forces), and yet at the same time they were addressed as responsible individuals who were actively participating in the circumvention of such dangerous situations: “For example, if the system shows congestion at an underground station, it will recommend people to go to the nearest alternative station via its built-in messaging function.”45

This sort of ambivalence seems to be typical of such crowd monitoring systems, and perhaps of augmented crowds in general. Every action undertaken by a crowd as a whole, by its subsets, and by its individual components is continuously classified as more or less threatening or cooperative, whereby cooperation is often synonymous with consumption. So much can be made clear by citing a final example. Since 2011, the San Francisco-based company CrowdOptic has offered analytic tools to organizers of large sporting events that allow them to track where spectators are focusing their attention. The software tracks mobile phones, with which fans take pictures and videos of the event, and it uses GPS and compass data to monitor the position of the phones and the direction that they are pointing. By means of triangulation, moreover, the system is also able to determine the precise locations within a stadium that are attracting the most attention: “Location is good, focus is better,” as CrowdOptic’s slogan maintains. At the same time, the company also offers augmented-reality services for individual users: “CrowdOptic knows where all the mobile phones in the crowd are looking as they aim their phones; it can not only transmit this insight to the enterprise, but can also display relevant content on the phones according to the action each fan is watching, all in real time.”46 This overlaid information contributes to the appeal of participating in the data collection itself; it becomes part of the very pictures and videos that are being taken and can be tracked when they are posted online. The service promises to provide “hyper-targeted in-venue advertising, broadcasting, and security,” and thus to offer several benefits for organizers and producers. First, they are able to receive live images of where the spectators are directing their focus, and this enables them to optimize the drama of a given event and to place advertisements in more strategic locations. Second, they are able to track the online circulation of videos after an event has concluded, which allows them to improve their post-event marketing and to monitor for potential copyright infringements. Finally there is the issue of safety and crowd control: “Using advanced algorithms, CrowdOptic alerts the enterprise in real-time to shifts in fan focus and momentum, as well as anomalous activity in the crowd.” According to this line of thinking, safety risks manifest themselves first of all as anomalous shifts of attention. Danger looms, that is, as soon as the public ceases to focus on the game. At any given point, depending on the fluctuations of the incoming data produced by the crowd, a shift can thus take place from merely observing consumers to controlling the crowd. Here, to some extent, the crowd functions to govern itself with what could be called “crowd-sourced crowd control.”

Such an augmented crowd is neither an atavistic force of nature nor an abstract mass ornament; it is rather a technologically enhanced and differentiable matrix of continuous data streams. Its consumption is productive to the extent that it incessantly provides more and more information about itself. Even its fluctuating attention and whimsical unpredictability can be statistically analyzed and exploited for profit. The responses of crowds are no longer thought to reflect the human instincts of times past; they are rather analyzed to make forecasts about the future. Impending disasters can supposedly be foretold by the teeming activity of the masses, whose behavioral patterns unwittingly anticipate future events.47 Thus, too, the opposition has vanished between compulsive instincts and discipline, between life and form – the very opposition that, in one way or another, had characterized previous descriptions of the masses. In the end, augmented crowds are simultaneously spontaneous and predictable, passionate and controlled.