{"id":1076,"date":"2016-04-11T21:08:50","date_gmt":"2016-04-11T21:08:50","guid":{"rendered":"http:\/\/designing.rutgers.edu\/?p=1076"},"modified":"2019-05-07T21:22:47","modified_gmt":"2019-05-07T21:22:47","slug":"a-sea-of-data-apophenia-and-pattern-mis-recognition","status":"publish","type":"post","link":"https:\/\/designing.rutgers.edu\/?p=1076","title":{"rendered":"A Sea of Data: Apophenia and Pattern (Mis-) Recognition"},"content":{"rendered":"\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n

Hito Steyerl<\/a>
Link to the e-flux page<\/a><\/h3>\n<\/div>\n<\/div><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/section>\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t<\/a>
This image from the Snowden files was captioned: \u201cA single frame of scrambled video imagery.\u201d<\/figcaption><\/figure>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n
\n\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t<\/a>
Rose Mary Woods, Nixon\u2019s lifelong secretary, demonstrates the \u201cRose Mary Stretch,\u201d a gesticulation that purportedly led to the erasure of a section of the Watergate tapes. The quality of noise in this section of the tapes has been throughly analyzed to understand if the omission was intentional. Photo: Wikimedia commons.<\/figcaption><\/figure>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n
\n\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t<\/a>
This photograph from June 6, 2012 shows a student pilot and sensor operator manning the controls of a MQ-9 Reaper in a ground-based cockpit during a training mission flown from Hancock Field Air National Guard Base, Syracuse, New York. Photo: AP Photo.<\/figcaption><\/figure>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n
<\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n
\n
\n
\n

This is an image from the Snowden files. It is labeled \u201csecret.\u201d1<\/a>Yet one cannot see anything on it.<\/p>\n

This is exactly why it is symptomatic.<\/p>\n

\u00a0<\/p>\n

Not seeing anything intelligible is the new normal. Information is passed on as a set of signals that cannot be picked up by human senses. Contemporary perception is machinic to large degrees. The spectrum of human vision only covers a tiny part of it. Electric charges, radio waves, light pulses encoded by machines for machines are zipping by at slightly subluminal speed. Seeing is superseded by calculating probabilities. Vision loses importance and is replaced by filtering, decrypting, and pattern recognition. Snowden\u2019s image of noise could stand in for a more general human inability to perceive technical signals unless they are processed and translated accordingly.<\/p>\n

But noise is not nothing. On the contrary, noise is a huge issue, not only for the NSA but for machinic modes of perception as a whole.<\/p>\n

Signal v. Noise\u00a0was the title of a column on the internal NSA website running from 2011 to 2012. It succinctly frames the NSA\u2019s main problem: how to extract \u201cinformation from the truckloads of data\u201d:<\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n
\n

It\u2019s not about the data or even access to the data. It\u2019s about getting information from the truckloads of data \u2026 Developers, please help! We\u2019re drowning (not waving) in a sea of data\u2014with data, data everywhere, but not a drop of information.2<\/a><\/p>\n<\/blockquote><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n

Analysts are choking on intercepted communication. They need to unscramble, filter, decrypt, refine, and process \u201ctruckloads of data.\u201d The focus moves from acquisition to discerning, from scarcity to overabundance, from adding on to filtering, from research to pattern recognition. This problem is not restricted to secret services. Even WikiLeaks Julian Assange states: \u201cWe are drowning in material.\u201d3<\/a><\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/section>\n

\n
\n
\n
\n
\n
\n
\n
\n
\n

Apophenia<\/h3>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n

But let\u2019s return to the initial image. The noise on it was actually decrypted by GCHQ technicians to reveal a picture of clouds in the sky. British analysts have been hacking video feeds from Israeli drones at least since 2008, a period which includes the recent IDF aerial campaigns against Gaza.4<\/a>\u00a0But no images of these attacks exist in Snowden\u2019s archive. Instead, there are all sorts of abstract renderings of intercepted broadcasts. Noise. Lines. Color patterns.5<\/a>\u00a0According to leaked training manuals, one needs to apply all sorts of massively secret operations to produce these kinds of images.6<\/a><\/p>\n

But let me tell you something. I will decrypt this image for you without any secret algorithm. I will use a secret ninja technique instead. And I will even teach you how to do it for free. Please focus very strongly on this image right now.<\/p>\n

Doesn\u2019t it look like a shimmering surface of water in the evening sun? Is this perhaps the \u201csea of data\u201d itself? An overwhelming body of water, which one could drown in? Can you see the waves moving ever so slightly?<\/p>\n

I am using a good old method called apophenia.<\/p>\n

Apophenia is defined as the perception of patterns within random data.7<\/a>\u00a0The most common examples are people seeing faces in clouds or on the moon. Apophenia is about \u201cdrawing connections and conclusions from sources with no direct connection other than their indissoluble perceptual simultaneity,\u201d as Benjamin Bratton recently argued.8<\/a><\/p>\n

One has to assume that sometimes, analysts also use apophenia.<\/p>\n

Someone must have seen the face of Amani al-Nasasra in a cloud. The forty-three-year-old was blinded by an aerial strike in Gaza in 2012 in front of her TV:<\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n
\n

\u201cWe were in the house watching the news on TV. My husband said he wanted to go to sleep, but I wanted to stay up and watch Al Jazeera to see if there was any news of a ceasefire. The last thing I remember, my husband asked if I changed the channel and I said yes. I didn\u2019t feel anything when the bomb hit\u2014I was unconscious. I didn\u2019t wake up again until I was in the ambulance.\u201d Amani suffered second degree burns and was largely blinded.9<\/a><\/p>\n<\/blockquote><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n

What kind of \u201csignal\u201d was extracted from what kind of \u201cnoise\u201d to suggest that al-Nasasra was a legitimate target? Which faces appear on which screens, and why? Or to put it differently: Who is \u201csignal,\u201d and who disposable \u201cnoise\u201d?<\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n
\n\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t<\/a>
The Russian TV station Zvezda claimed this flock of birds over New York City appeared to form the shape of President Vladimir Putin\u2019s face. YouTube video screenshot.<\/figcaption><\/figure>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n
<\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/section>\n
\n
\n
\n
\n
\n
\n
\n
\n
\n

Pattern Recognition<\/h3>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n

Jacques Ranci\u00e8re tells a mythical story about how the separation of signal and noise might have been accomplished in Ancient Greece. Sounds produced by affluent male locals were defined as speech, whereas women, children, slaves, and foreigners were assumed to produce garbled noise.10<\/a>\u00a0The distinction between speech and noise served as a kind of political spam filter. Those identified as speaking were labeled citizens and the rest as irrelevant, irrational, and potentially dangerous nuisances. Similarly, today, the question of separating signal and noise has a fundamental political dimension. Pattern recognition resonates with the wider question of political recognition. Who is recognized on a political level and as what? As a subject? A person? A legitimate category of the population? Or perhaps as \u201cdirty data\u201d?<\/p>\n

What are dirty data? Here is one example:<\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n

Sullivan, from Booz Allen, gave the example the time his team was analyzing demographic information about customers for a luxury hotel chain and came across data showing that teens from a wealthy Middle Eastern country were frequent guests.<\/p>\n

\n

\u201cThere were a whole group of 17 year-olds staying at the properties worldwide,\u201d Sullivan said. \u201cWe thought, \u2018That can\u2019t be true.\u2019\u201d11<\/a><\/p>\n<\/blockquote><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n

The demographic finding was\u00a0dismissed as dirty data\u2014a messed up and worthless set of information\u2014before someone found out that, actually, it was true.<\/p>\n

Brown teenagers, in this worldview, are likely to exist. Dead brown teenagers? Why not? But rich brown teenagers? This is so improbable that they must be dirty data and cleansed from your system! The pattern emerging from this operation to separate noise and signal is not very different from Ranci\u00e8re\u2019s political noise filter for allocating citizenship, rationality, and privilege. Affluent brown teenagers seem just as unlikely as speaking slaves and women in the Greek polis.<\/p>\n

On the other hand, dirty data are also something like a cache of surreptitious refusal; they express a refusal to be counted and measured:<\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n
\n

A study of more than 2,400 UK consumers by research company Verve found that 60%\u00a0intentionally provided wrong information\u00a0when submitting personal details online. Almost one quarter (23 percent) said they sometimes gave out incorrect dates of birth, for example, while 9 percent said they did this most of the time and 5 percent always did it.12<\/a><\/p>\n<\/blockquote><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n

Dirty data is where all of our refusals to fill out the constant onslaught of online forms accumulate. Everyone is lying all the time, whenever possible, or at least cutting corners. Not surprisingly, the \u201cdirtiest\u201d area of data collection is consistently pointed out to be the health sector, especially in the US. Doctors and nurses are singled out for filling out forms incorrectly. It seems that health professionals are just as unenthusiastic about filling out forms for systems designed to replace them, as consumers are about performing clerical work for corporations that will spam them in return.<\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n
\n\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t<\/a>
An animated gif shows a dirty data sandstorm.<\/figcaption><\/figure>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n

In his book\u00a0The Utopia of Rules<\/i>, David Graeber gives a profoundly moving example of the forced extraction of data. After his mom suffered a stroke, he went through the ordeal of having to apply for Medicaid on her behalf:<\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n
\n

I had to spend over a month \u2026 dealing with the ramifying consequences of the act of whatever anonymous functionary in the New York Department of Motor Vehicles had inscribed my given name as \u201cDaid,\u201d not to mention the Verizon clerk who spelled my surname \u201cGrueber.\u201d Bureaucracies public and private appear\u2014for whatever historical reasons\u2014to be organized in such a way as to guarantee that a significant proportion of actors will not be able to perform their tasks as expected.13<\/a><\/p>\n<\/blockquote><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n

Graeber goes on to call this an example of utopian thinking. Bureaucracy is based on utopian thinking because it assumes people to be perfect from it\u2019s own point of view. Graeber\u2019s mother died before she was accepted into the program.<\/p>\n

The endless labor of filling out completely meaningless forms is a new kind of domestic labor in the sense that it is not considered labor at all and assumed to be provided \u201cvoluntarily\u201d or performed by underpaid so-called data janitors.14<\/a>\u00a0Yet all the seemingly swift and invisible action of algorithms, their elegant optimization of everything, their recognition of patterns and anomalies\u2014this is based on the endless and utterly senseless labor of providing or fixing messy data.<\/p>\n

Dirty data is simply real data in the sense that it documents the struggle of real people with a bureaucracy that exploits the uneven distribution and implementation of digital technology.15<\/a>\u00a0Consider the situation at LaGeSo (the Health and Social Affairs Office) in Berlin, where refugees are risking their health on a daily basis by standing in line outdoors in severe winter weather for hours or even days just to have their data registered and get access to services to which they are entitled (for example, money to buy food).16<\/a>\u00a0These people are perceived as anomalies because, in addition to having the audacity to arrive in the first place, they ask that their rights be respected. There is a similar political algorithm at work: people are blanked out. They cannot even get to the stage to be recognized as claimants. They are not taken into account.<\/p>\n

On the other hand, technology also promises to separate different categories of refugees. IBM\u2019s Watson AI system was experimentally programmed to potentially identify terrorists posing as refugees:<\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n
\n

IBM hoped to show that the i2 EIA could separate the sheep from the wolves: that is, the masses of harmless asylum-seekers from the few who might be connected to jihadism or who were simply lying about their\u00a0identities \u2026<\/p>\n

IBM created a hypothetical scenario, bringing together several data sources to match against a fictional list of passport-carrying refugees. Perhaps the most important dataset was a list of names of casualties from the conflict gleaned from open press reports and other sources. Some of the material came from the Dark Web, data related to the black market for passports; IBM says that they anonymized or obscured personally identifiable information in this set \u2026<\/p>\n

Borene said the system could provide a score to indicate the likelihood that a hypothetical asylum seeker was who they said they were, and do it fast enough to be useful to a border guard or policeman walking a\u00a0beat.17<\/a><\/p>\n<\/blockquote><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n

The cross-referencing of unofficial databases, including dark web sources, is used to produce a \u201cscore,\u201d which calculates the probability that a refugee might be a terrorist. The hope is for a pattern to emerge across different datasets, without actually checking how or if they correspond to any empirical reality. This example is actually part of a much larger subset of \u201cscores\u201d: credit scores, academic ranking scores, scores ranking interaction on online forums etc., which classify people according to financial interactions, online behavior, market data, and other sources. A variety of inputs are boiled down to a single number\u2014a superpattern\u2014which may be a \u201cthreat\u201d score or a \u201csocial sincerity score,\u201d as planned by Chinese authorities for every single citizen within the next decade. But the input parameters are far from being transparent or verifiable. And while it may be seriously desirable to identify Daesh moles posing as refugees, a similar system seems to have worrying flaws.<\/p>\n

The NSA\u2019s SKYNET program was trained to find terrorists in Pakistan by sifting through cell phone customer metadata. But experts criticize the NSA\u2019s methodologies. \u201cThere are\u00a0very few\u00a0\u2018known terrorists\u2019 to use to train\u00a0and\u00a0test\u00a0the model,\u201d explained Patrick Ball, a data scientist and director of the\u00a0Human Rights Data Analysis Group, to\u00a0Ars Technica. \u201cIf they are using the same records to train the model as they are using to test the model, their assessment of the fit is completely\u00a0bullshit.\u201d18<\/a><\/p>\n

Human Rights Data Analysis Group estimates that around 99,000 Pakistanis might have ended up wrongly classified as terrorists by SKYNET, a statistical margin of error that might have had deadly consequences given the fact that the US is waging a drone war on suspected militants in the country and between 2500 and four thousand people are estimated to have been killed since 2004: \u201cIn the years that have followed, thousands of innocent people in Pakistan may have been mislabelled as terrorists by that \u2018scientifically unsound\u2019 algorithm, possibly resulting in their untimely demise.\u201d19<\/a><\/p>\n

One needs to emphasize strongly that SKYNET\u2019s operations cannot be objectively assessed, since it is not known how it\u2019s results were utilized. It was most certainly not the only factor in determining drone targets.20<\/a>\u00a0But the example of SKYNET demonstrates just as strongly that a \u201csignal\u201d extracted by assessing correlations and probabilities is not the same as an actual fact, but determined by the inputs the software uses to learn, and the parameters for filtering, correlating, and \u201cidentifying.\u201d The old engineer wisdom \u201ccrap in\u2014crap out\u201d seems to still apply. In all of these cases\u2014as completely different as they are technologically, geographically, and also ethically\u2014some version of pattern recognition was used to classify groups of people according to political and social parameters. Sometimes it is as simple as,\u00a0we try to avoid registering refugees. Sometimes there is more mathematical mumbo jumbo involved. But many methods used are opaque, partly biased, exclusive, and\u2014as one expert points out\u2014sometimes also \u201cridiculously optimistic.\u201d21<\/a><\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n
\n\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t<\/a>
A plate of spaghetti meatballs return our gaze, courtesy of Google inceptionism. Source: Mary-Ann Russon, \u201cGoogle DeepDream robot: 10 weirdest images produced by AI \u2018inceptionism\u2019 and users online,\u201d International Business Times, July 6, 2015<\/figcaption><\/figure>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n
<\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/section>\n
\n
\n
\n
\n
\n
\n
\n
\n
\n

Corporate Animism<\/h3>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n

How to recognize something in sheer noise? A striking visual example of pure and conscious apophenia was recently demonstrated by research labs at Google:22<\/a><\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n
\n

We train an artificial neural network by showing it millions of training examples and gradually adjusting the network parameters until it gives the classifications we want. The network typically consists of 10\u201330 stacked layers of artificial neurons. Each image is fed into the input layer, which then talks to the next layer, until eventually the \u201coutput\u201d layer is reached. The network\u2019s \u201canswer\u201d comes from this final output layer.23<\/a><\/p>\n<\/blockquote><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n

Neural networks were trained to discern edges, shapes, and a number of objects and animals and then applied to pure noise. They ended up \u201crecognizing\u201d a rainbow-colored mess of disembodied fractal eyes, mostly without lids, incessantly surveilling their audience in a strident display of conscious pattern overidentification.<\/p>\n

Google researchers call the act of creating a pattern or an image from nothing but noise \u201cinceptionism\u201d or \u201cdeep dreaming.\u201d But these entities are far from mere hallucinations. If they are dreams, those dreams can be interpreted as condensations or displacements of the current technological disposition. They reveal the networked operations of computational image creation, certain presets of machinic vision, its hardwired ideologies and preferences.<\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n
\n

One way to visualize what goes on is to turn the network upside down and ask it to enhance an input image in such a way as to elicit a particular interpretation. Say you want to know what sort of image would result in \u201cBanana.\u201d Start with an image full of random noise, then gradually tweak the image towards what the neural net considers a banana. By itself, that doesn\u2019t work very well, but it does if we impose a prior constraint that the image should have similar statistics to natural images, such as neighboring pixels needing to be correlated.24<\/a><\/p>\n<\/blockquote><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n

In a feat of genius, inceptionism manages to visualize the unconscious of prosumer networks: images surveilling users, constantly registering their eye movements, behavior, preferences, aesthetically helplessly adrift between\u00a0Hundertwasser\u00a0mug knockoffs and Art Deco friezes gone ballistic. Walter Benjamin\u2019s \u201coptical unconscious\u201d has been upgraded to the unconscious of computational image divination.25<\/a><\/p>\n

By \u201crecognizing\u201d things and patterns that were not given, inceptionist neural networks eventually end up effectively identifying a new totality of aesthetic and social relations. Presets and stereotypes are applied, regardless of whether they \u201capply\u201d or not: \u201cThe results are intriguing\u2014even a relatively simple neural network can be used to over-interpret an image, just like as children we enjoyed watching clouds and interpreting the random shapes.\u201d26<\/a><\/p>\n

But inceptionism is not just a digital hallucination. It is a document of an era that trains smartphones to identify kittens, thus hardwiring truly terrifying jargons of cutesy into the means of production.27<\/a>\u00a0It demonstrates a version of corporate animism in which commodities are not only fetishes but morph into franchised chimeras.<\/p>\n

Yet these are deeply realist representations. According to Gy\u00f6rgy Lukacs, \u201cclassical realism\u201d creates \u201ctypical characters,\u201d insofar as they represent the objective social (and in this case technological) forces of our times.28<\/a><\/p>\n

Inceptionism does that and more. It also gives those forces a face\u2014or more precisely, innumerable eyes. The creature that stares at you from your plate of spaghetti and meatballs is not an amphibian beagle. It is the ubiquitous surveillance of networked image production, a form of memetically modified intelligence that watches you in the shape of the lunch that you will Instagram in a second if it doesn\u2019t attack you first. Imagine a world of enslaved objects remorsefully scrutinizing you. Your car, your yacht, your art collection observes you with a gloomy and utterly desperate expression.\u00a0You may own us, they seem to say,\u00a0but we are going to inform on you. And guess what kind of creature we are going to recognize in you!29<\/a><\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n
<\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n
\n
\n
\n
<\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n
\n\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t<\/a>
CGI acupuncture: Face Robot, a general-purpose animation system, promises efficiency in motion capturing actor\u2019s faces through this 32-point system.<\/figcaption><\/figure>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n
<\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n
<\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n
\n\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t<\/a>
Previously unknown archaeological monuments have been revealed as of September 2015 by the Stonehenge Hidden Landscapes Project. The findings include new information about the world\u2019s largest \u201csuper henge\u201d and include ritual monuments such as the mortuary building pictured above in a 3-D reconstruction. Copyright: LBI ArchPro, Joachim Brandtner<\/figcaption><\/figure>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/section>\n
\n
\n
\n
\n
\n
\n
\n
\n
\n

Data Neolithic<\/h3>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n

But what are we going to make of automated apophenia?30<\/a>\u00a0Are we to assume that machinic perception has entered its own phase of magical thinking? Is this what commodity enchantment means nowadays: hallucinating products? It might be more accurate to assume that humanity has entered yet another new phase of magical thinking. The vocabulary deployed for separating signal and noise is surprisingly pastoral: data \u201cfarming\u201d and \u201charvesting,\u201d \u201cmining\u201d and \u201cextraction\u201d are embraced as if we lived through another massive neolithic revolution31<\/a>\u00a0with it\u2019s own kind of magic formulas.<\/p>\n

All sorts of agricultural and mining technologies\u2014that were developed during the neolithic\u2014are reinvented to apply to data. The stones and ores of the past are replaced by silicone and rare earth minerals, while a Minecraft paradigm of extraction describes the processing of minerals into elements of information architecture.32<\/a><\/p>\n

Pattern recognition was an important asset of neolithic technologies too. It marked the transition between magic and more empirical modes of thinking. The development of the calendar by observing patterns in time enabled more efficient irrigation and agricultural scheduling. Storage of cereals created the idea of property. This period also kick-started institutionalized religion and bureaucracy, as well as managerial techniques including laws and registers. All these innovations also impacted society: hunter and gatherer bands were replaced by farmer kings and slaveholders. The neolithic revolution was not only technological but also had major social consequences.<\/p>\n

Today, expressions of life as reflected in data trails become a farmable, harvestable, minable resource managed by informational biopolitics.33<\/a><\/p>\n

And if you doubt that this is another age of magical thinking, just look at the NSA training manual for unscrambling hacked drone intercepts. As you can see, you need to bewitch the files with a magic wand. (Image Magick is a free image converter):<\/p>\n

\u00a0<\/p>\n

The supposedly new forms of governance emerging from these technologies look partly archaic and partly superstitious. What kind of corporate\/state entities are based on data storage, image unscrambling, high-frequency trading, and Daesh Forex gaming? What are the contemporary equivalents of farmer kings and slaveholders, and how are existing social hierarchies radicalized through examples as vastly different as tech-related gentrification and jihadi online forum gamification? How does the world of pattern recognition and big-data divination relate to the contemporary jumble of oligocracies, troll farms, mercenary hackers, and data robber barons supporting and enabling bot governance, Khelifah clickbait and polymorphous proxy warfare? Is the state in the age of Deep Mind, Deep Learning, and Deep Dreaming a Deep State\u2122? One in which there is no appeal nor due process against algorithmic decrees and divination?<\/p>\n

\u00a0<\/p>\n

But there is another difference between the original and the current type of \u201cneolithic,\u201d and it harks back to pattern recognition. In ancient astronomy, star constellations were imagined by projecting animal shapes into the skies. After cosmic rhythms and trajectories had been recorded on clay tablets, patterns of movement started to emerge. As additional points of orientation, some star groups were likened to animals and heavenly beings. However, progress in astronomy and mathematics happened not because people kept believing there were animals or gods in space, but on the contrary, because they accepted that constellations were expressions of a physical logic. The patterns were projections, not reality. While today statisticians and other experts routinely acknowledge that their findings are mostly probabilistic projections, policymakers of all sorts conveniently ignore this message. In practice you become coextensive with the data-constellation you project. Social scores of all different kinds\u2014credit scores, academic scores, threat scores\u2014as well as commercial and military pattern-of-life observations impact the real lives of real people, both reformatting and radicalizing social hierarchies by ranking, filtering, and classifying.<\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n
<\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n
\n
\n
\n
<\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n
\n\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t<\/a>
Could this image be a representation of the neo-neolithic? Source: Mary-Ann Russon, \u201cGoogle DeepDream robot: 10 weirdest images produced by AI \u2018inceptionism\u2019 and users online,\u201d International Business Times, July 6, 2015<\/figcaption><\/figure>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n
\n\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t<\/a>
Source: Anh Nguyen, Jason Yosinski, and Jeff Clune, \u201cDeep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images,\u201d cv-foundation.org, 2015<\/figcaption><\/figure>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/section>\n
\n
\n
\n
\n
\n
\n\t\t\t\t\t\t\t\t<\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
\n
\n
\n
\n
\n
\n

Gestalt Realism<\/h3>\n<\/div>\n<\/div>\n

But let\u2019s assume we are actually dealing with projections. Once one accepts that the patterns derived from machinic sensing are\u00a0not\u00a0the same as reality, information definitely becomes available with a certain degree of veracity.<\/p>\n

Let\u2019s come back to Amani al-Nasasra, the woman blinded by an aerial attack in Gaza. We know: the abstract images recorded as intercepts of IDF drones by British spies do\u00a0notshow the aerial strike in Gaza that blinded her in 2012. The dates\u00a0don\u2019t\u00a0match. There is\u00a0noevidence in Snowden\u2019s archive. There are\u00a0no\u00a0images of this attack, at least as far as I know of. All we know is what she told Human Rights Watch. This is what she said: \u201cI can\u2019t see\u2014ever since the bombing, I can only see shadows.\u201d34<\/a><\/p>\n

So there is one more way to decode this image. It\u2019s plain for everyone to see. We see what Amani\u00a0cannot\u00a0see.<\/p>\n

In this case, the noise must be a \u201cdocument\u201d of what she \u201csees\u201d now: \u201cthe shadows.\u201d<\/p>\n

Is this a document of the drone war\u2019s optical unconscious? Of it\u2019s dubious and classified methods of \u201cpattern recognition\u201d? And if so, is there a way to ever \u201cunscramble\u201d the \u201cshadows\u201d Amani has been left with?<\/p>\n

\u00d7<\/p>\n

Acknowledgments: The initial version of this text was written at the request of Laura Poitras, who most generously allowed access to some unclassified documents from the Snowden archive, and a short version was presented during the opening of her show \u201cAstro Noise\u201d at the Whitney Museum. Further thanks to Henrik Moltke for facilitating access to the documents, to Brenda and other members of Laura\u2019s studio, to Linda Stupart for introducing me to the term \u201capophenia,\u201d and to Ben Bratton for fleshing it out for me.<\/small><\/p>\n

\u00a9 2016 e-flux and the author<\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n

\n
\n
\n\t\t\t\t\t\t\t\t<\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/section><\/div>\n<\/p><\/div>\n<\/p><\/div>\n","protected":false},"excerpt":{"rendered":"

Hito SteyerlLink to the e-flux page This image from the Snowden files was captioned: \u201cA single frame of scrambled video imagery.\u201d Rose Mary Woods, Nixon\u2019s lifelong secretary, demonstrates the \u201cRose Mary Stretch,\u201d a gesticulation that purportedly led to the erasure of a section of the Watergate tapes. The quality of noise in this section of … Read moreA Sea of Data: Apophenia and Pattern (Mis-) Recognition<\/span><\/a><\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[135,6,52],"tags":[],"class_list":["post-1076","post","type-post","status-publish","format-standard","hentry","category-apophenia","category-art","category-data-visualization"],"_links":{"self":[{"href":"https:\/\/designing.rutgers.edu\/index.php?rest_route=\/wp\/v2\/posts\/1076"}],"collection":[{"href":"https:\/\/designing.rutgers.edu\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/designing.rutgers.edu\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/designing.rutgers.edu\/index.php?rest_route=\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/designing.rutgers.edu\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1076"}],"version-history":[{"count":11,"href":"https:\/\/designing.rutgers.edu\/index.php?rest_route=\/wp\/v2\/posts\/1076\/revisions"}],"predecessor-version":[{"id":1906,"href":"https:\/\/designing.rutgers.edu\/index.php?rest_route=\/wp\/v2\/posts\/1076\/revisions\/1906"}],"wp:attachment":[{"href":"https:\/\/designing.rutgers.edu\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1076"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/designing.rutgers.edu\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1076"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/designing.rutgers.edu\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1076"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}