China’s Dystopian Tech Could Be Contagious

China’s Dystopian Tech Could Be Contagious

“The social-credit system was based explicitly on a familiar, Western model: the credit score. As a de facto reputation index, your credit score strongly conditions where you can rent, what kind of jobs or educational opportunities you’ll be eligible for, even what mode of travel you use to get around. This one number—formulated by obscure means, by largely unaccountable organizations, then used as a gating mechanism by a profusion of third parties, mostly in secret—has become what it was never meant to be: a general proxy for trustworthiness.

The Chinese genius, if you can call it that, was to take credit scoring as a tool of social discipline to its logical conclusion, building a formal public-private partnership around it. This move extends dominion across the entire range of interactions any member of modern society is more or less compelled to pursue by the very style and structure of contemporary life.

These disciplinary measures first reach individuals in the form of soft incentives, of the type familiar to Westerners from corporate loyalty programs. Citizens with higher social-credit scores enjoy discounts or upgrades on products and services, like hotel rooms or internet connectivity. Those who wear virtue on their sleeves further—perhaps by taking public transit consistently instead of driving to work, taking out the recycling regularly, or even denouncing a misbehaving neighbor—might enjoy new benefits, like being able to rent a flat with no deposit, or earning the right to send their children to exclusive schools. This hardly sounds like authoritarianism run amok, and to a certain degree, patriotic Chinese netizens are right to complain when Western critics conflate such nudges toward preferred behavior with actual tyranny.

But the system provides abundantly for sticks as well as carrots. Attend a “subversive” political meeting or religious service, for example, or frequent known haunts of vice, or do under-the-table business with an unregistered, informal enterprise, and the idea is that the network will know about it and respond by curtailing one’s privileges. The state wants its citizens to believe that there’s little point in trying to evade detection of such acts, especially when they are strongly correlated with suspicious sites, either by mobile-phone location data or by China’s extensive national network of facial-recognition-equipped surveillance cameras. Once detected, the system promises to pass judgment on the things a citizen is and is not permitted to do, buy, or access. And with no recourse in real time, no ability to appeal, and nowhere to turn for help.”