‘Welcome to China, where AI is being pressed into service as handmaiden to an authoritarian government. For many critics, this seems fraught with danger: an Orwellian world where “Big Brother” is always watching, able to spy on anyone from human rights lawyers to political dissidents and persecuted minorities. For supporters, it is near utopian: a land where criminals and miscreants are easily weeded out, where no one can cheat, where good behaviour is rewarded and the bad punished. The latter vision is the Chinese government’s stated aim. By 2020, a national video surveillance network will be “omnipresent, fully networked, always working and fully controllable”, according to an official paper released in 2015.’
The horror and the brilliance of Jeremy Bentham’s panopticon prison – designed to create in in-mates the sense of constant surveillance – makes, for the liberal minded, a disturbing but perplexing thought experiment. Its first lesson is to remind us that the context of ideas powerfully determine how willing we are to consume them. Originally inspired by the Russian method to efficiently monitor and instruct unskilled workers – not too far from any number of modern ‘onboarding’ techniques or ‘efficiency’ models (Amazon’s warehouses digitally track how the pace of each packers’ individual’s footsteps) which we stomach daily – the panopticon concept becomes ghoulish when transplanted into a system of punishment and reform. Yet to Bentham, a celebrated social architect, the self-regulation induced by omnipresent eyes would cure the ills of society:
‘Morals reformed-health preserved-industry invigorated-instruction diffused-public burthens lightened-Economy seated, as it were, upon a rock-the gordian knot of the Poor-Laws not cut, but untied-all by a simple idea in Architecture!’
The prison design remained theoretical – though the state of our prisons today has little to recommend the alternative approach. In China, where the totality and imminence of its vision for a surveillance state is lately laid bare in the FT, panopticon-paranoia is no thought experiment.
Fuelled by the phenomenal new capability of surveillance tech – of which China is the world’s leading investor – xueliang, or “sharp eyes” (the infamous slogan of the old party) is once again becoming its most powerful social currency; but this time, it’s machines, not just neighbours, who’ll be the watching. And, thanks to the devices in over 50% of China’s pocket, they know where exactly to find you. From schools, where facial recognition is being used to identify slackers and monitor nutrition, to the streets of Beijing, where 850,000 “informants” patrol, to the police of Zhengzhou, now sporting AI powered smart glasses with facial recognition technology, the grip of the state appears increasingly vice-like.
Now, with the launch of its new social credit system, past misdemeanours –traffic offences, court appearances – will determine future possibility in a more extensive way than ever imagined. A poor social credit score,’ the FT writes, ‘could jeopardise a university place, rule out certain jobs and even limit travel: more than 10.5 million people have been barred from buying airline or high-speed train tickets, according to the Supreme Court, since a debtors blacklist was launched.’ Good behaviour will be the difference between an easy life or a hard one. And for the Chinese, that is exactly the point of government: ‘a strong empowered state… [that] drag[s] the nation forward’. In this context, surveillance becomes merely a ‘logical extension’ of the duty of care.
For liberal democracies, as the leading Chinese scholar observes, the point of government is to put power in the hands of the people. We are not wired to be surveillance state. But we face endemic crises of educational attainment, crippling obesity, a failing criminal justice system. We suffer the complex web of inequity, entitlement and failures of accountability at every level of society. If were to transpose this idea – of social credit, of early student intervention – in our own context, what would we find? And would it be worse than the current status quo which – in so many insidious, intangible ways – sentences people to lives of hardship through intangible, but equally devastating, forms of ‘social credit’?
