governance of technology and governance through technology
Class Eight: Surveillance Capitalism
For each quote, we plan on discussing it, placing it into its context, and understanding how it fits into the broader discussion about bias.
- Surveillance Capitalism
- “Surveillance capitalism unilaterally claims human experience as free raw material for translation into behavioral data. Although some of these data are applied to product or service improvement, the rest are declared as a proprietary behavioral surplus, fed into advanced manufacturing processes known as “machine intelligence,” and fabricated into prediction products that anticipate what you will do now, soon, and later. Finally, these prediction products are traded in a new kind of marketplace for behavioral predictions that I call behavioral futures markets. ”
- “Competitive pressures produced this shift, in which automated machine processes not only know our behavior but also shape our behavior at scale. With this reorientation from knowledge to power, it is no longer enough to automate information flows about us; the goal now is to automate us.”
- “Instrumentarian power knows and shapes human behavior toward others’ ends. Instead of armaments and armies, it works its will through the automated medium of an increasingly ubiquitous computational architecture of “smart” networked devices, things, and spaces.”
- “Nonetheless, the danger of closing doors to rooms that will no longer exist is very real. The unprecedented nature of surveillance capitalism has enabled it to elude systematic contest because it cannot be adequately grasped with our existing concepts. We rely on categories such as “monopoly” or “privacy” to contest surveillance capitalist practices. And although these issues are vital, and even when surveillance capitalist operations are also monopolistic and a threat to privacy, the existing categories nevertheless fall short in identifying and contesting the most crucial and unprecedented facts of this new regime.”
- “For all of Google’s technological prowess and computational talent, the real credit for its success goes to the radical social relations that the company declared as facts, beginning with its disregard for the boundaries of private human experience and the moral integrity of the autonomous individual. ”
- “These invasive claims were nurtured by the absence of law to impede their progress, the mutuality of interests between the fledgling surveillance capitalists and state intelligence agencies, and the tenacity with which the corporation defended its new territories.”
- “Google and its expanding universe of competitors enjoy extraordinary new asymmetries of knowledge and power, unprecedented in the human story. I argue that the significance of these developments is best understood as the privatization of the division of learning in society, the critical axis of social order in the twenty-first century.”
- “Here we explore this new reality business, as all aspects of human experience are claimed as raw-material supplies and targeted for rendering into behavioral data. Much of this new work is accomplished under the banner of “personalization,” a camouflage for aggressive extraction operations that mine the intimate depths of everyday life.”
- “Totalitarianism was a transformation of the state into a project of total possession. Instrumentarianism and its materialization in Big Other signal the transformation of the market into a project of total certainty, an undertaking that is unimaginable outside the digital milieu and the logic of surveillance capitalism”
- “The first was the migration from the virtual to the real world. The second is a shift of focus from the real world to the social world, as society itself becomes the new object of extraction and control. Just as industrial society was imagined as a well-functioning machine, instrumentarian society is imagined as a human simulation of machine learning systems: a confluent hive mind in which each element learns and operates in concert with every other element. In the model of machine confluence, the “freedom” of each individual machine is subordinated to the knowledge of the system as a whole.”
- “I consider the implications of these developments for a second elemental right: the right to sanctuary. The human need for a space of inviolable refuge has persisted in civilized societies from ancient times but is now under attack as surveillance capital creates a world of “no exit” with profound implications for the human future at this new frontier of power.”
- “For example, calls to break up Google or Facebook on monopoly grounds could easily result in establishing multiple surveillance capitalist firms, though at a diminished scale, and thus clear the way for more surveillance capitalist competitors.”
Feedback (5 minutes)
- Did we like the readings?
- Too long? Too short?
- Did we like using Github? Github issues?