The APDCAT warns of privacy risks to facial recognition artificial intelligence systems

The Catalan Data Protection Authority (APDCAT) has organized a debate this Wednesday to highlight the risks of facial recognition systems for the protection of personal data. Named ‘Artificial Intelligence Applied to Facial Recognition. Main risks to Privacy’, the session was held at the Four Years From Now (4YFN) event, organized as part of the Mobile World Congress. About eighty people attended, following the safety and hygiene regulations linked to COVID-19.

The debate included the participation of experts in the field: M. Àngels Barbarà, director of the APDCAT; Simona Levi, co-founder of Xnet; Carles Fernández, technical director of Herta Security, and Josep Domingo-Ferrer, director of the UNESCO Chair in Data Protection at the URV. Jordi Soria, coordination of Technology and Information Security of the APDCAT was in charge of moderating the event.

During the presentation, the director M. Àngels Barbarà, mentioned several applications of facial recognition and stressed that its use in mass video surveillance by public authorities is especially questionable. She recalled that from a social point of view, facial recognition technology faces strong opposition, particularly in the Western world, and argued that "these fears have a solid foundation". In this regard, she cited as an example the use of facial recognition in the Chinese social credit system, which assesses people's daily behavior, to reward or punish them.

Barbarà acknowledged that facial recognition, "like many other technologies, has beneficial and harmful applications" and that each application has its own tradeoff between risks and benefits. She added that "facial recognition in mass surveillance is probably the most controversial application".

The director reviewed the use of this technology in mass surveillance in several Western countries. Specifically in Europe, he referred to the published draft AI regulation. The European data protection authorities have just issued a joint opinion, recalling that the massive use of Biometric recognition in public spaces not only affects the rights and freedoms of individuals, but can also create a severe and irreversible effect of loss of anonymity in public spaces on the population as a whole, negatively and directly affecting the exercise of important rights and freedoms in a democratic society.

Barbarà lamented that "reliability remains a problem with current systems, and is 10 times less reliable when it comes to women, young people, blacks and other ethnic groups".

Two-way debate

The time for debate came with Simona Levi and Carles Fernández, who talked about the characteristics of this technology and the regulation on this aspect in the European framework.

Levi recalled that the GDPR offers us a principle that should guide any public or private data processing: the principle of minimization, which is essential to prevent our privacy from being violated. In this sense, she pointed out that "it is difficult to find assumptions in which facial recognition is compatible with this principle". 

For his part, Carles Fernández focused his speech on how facial recognition has advanced enormously in recent years, boosted by advances in Deep Learning technologies. Fernández said that technical improvements have been much faster than the corresponding legislation, and this is where the recent artificial intelligence regulation draft proposed by the European Comission comes into play.

The aim of the proposal is not to ban a technology to prevent it from being used against citizens' right to privacy, but rather to define the criteria that will allow a responsible use: permitted uses (emergencies, social threats, missing persons, etc.), procedural requirements (proportionality between privacy and the protection of other rights, judicial authorization, etc.), and quality of solutions (elimination of bias, high precision, cybersecurity, human supervision, etc.).

For Carles Fernández, this proposal aims to be the most demanding in the world, and “it wants to ensure that facial recognition in public spaces in Europe is an opportunity to protect rights, and that European solutions can be technically the best of the world".

Final considerations

For his part, Josep Domingo-Ferrer brought the session to an end, providing insights into the privacy risks of facial recognition. In this regard, he stressed that real-time biometric identification in public spaces (including facial recognition) is listed as a practice of artificial intelligence to be banned in the proposed EU regulation on artificial intelligence ( Artificial Intelligence Act, Section 5, paragraph 2), made public on April 21, 2021.

In fact, he recalled that "in this proposal, the use of these technologies by the police is only allowed if they are strictly necessary to search for potential victims of a crime, to deactivate an imminent and specific threat of terrorism, or to prosecute convicted criminals”, and that “even if the biometric identification is not in real time, the proposed regulation considers it high risk, and points out that its use should be very controlled”.

Therefore, he concluded that "not everything that can be done technically is ethically acceptable in our societies".