Swarms of Drones, Piloted by Artificial Intelligence, May Soon Patrol Europe’s Borders

EUROPE, WHISTLEBLOWING - SURVEILLANCE, ARTIFICIAL INTELLIGENCE-AI, TECHNOLOGY, 13 May 2019

Zach Campbell – The Intercept

Illustration: Soohee Cho/The Intercept

11 May 2019 – Imagine you’re hiking through the woods near a border. Suddenly, you hear a mechanical buzzing, like a gigantic bee. Two quadcopters have spotted you and swoop in for a closer look. Antennae on both drones and on a nearby autonomous ground vehicle pick up the radio frequencies coming from the cell phone in your pocket. They send the signals to a central server, which triangulates your exact location and feeds it back to the drones. The robots close in.

Cameras and other sensors on the machines recognize you as human and try to ascertain your intentions. Are you a threat? Are you illegally crossing a border? Do you have a gun? Are you engaging in acts of terrorism or organized crime? The machines send video feeds to their human operator, a border guard in an office miles away, who checks the videos and decides that you are not a risk. The border guard pushes a button, and the robots disengage and continue on their patrol.

This is not science fiction. The European Union is financing a project to develop drones piloted by artificial intelligence and designed to autonomously patrol Europe’s borders. The drones will operate in swarms, coordinating and corroborating information among fleets of quadcopters, small fixed-wing airplanes, ground vehicles, submarines, and boats. Developers of the project, known as Roborder, say the robots will be able to identify humans and independently decide whether they represent a threat. If they determine that you may have committed a crime, they will notify border police.

President Donald Trump has used the specter of criminals crossing the southern border to stir nationalist political sentiment and energize his base. In Europe, two years after the height of the migration crisis that brought more than a million people to the continent, mostly from the Middle East and Africa, immigration remains a hot-button issue, even as the number of new arrivals has dropped. Political parties across the European Union are winning elections on anti-immigrant platforms and enacting increasingly restrictive border policies. Tech ethicists and privacy advocates worry that Roborder and projects like it outsource too much law enforcement work to nonhuman actors and could easily be weaponized against people in border areas.

“The development of these systems is a dark step into morally dangerous territory,” said Noel Sharkey, emeritus professor of robotics and artificial intelligence at Sheffield University in the U.K. and one of the founders of the International Committee for Robot Arms Control, a nonprofit that advocates against the military use of robotics. Sharkey lists examples of weaponized drones currently on the market: flying robots equipped with Tasers, pepper spray, rubber bullets, and other weapons. He warns of the implications of combining that technology with AI-based decision-making and using it in politically-charged border zones. “It’s only a matter of time before a drone will be able to take action to stop people,” Sharkey told The Intercept.

Roborder’s developers also may be violating the terms of their funding, according to documents about the project obtained via European Union transparency regulations. The initiative is mostly financed by an €8 million EU research and innovation grant designed for projects that are exclusively nonmilitary, but Roborder’s developers acknowledge that parts of their proposed system involve military technology or could easily be converted for military use.

Much of the development of Roborder is classified, but The Intercept obtained internal reports related to ethical considerations and concerns about the program. That documentation was improperly redacted and inadvertently released in full.

In one of the reports, Roborder’s developers sought to address ethical criteria that are tied to their EU funding. Developers considered whether their work could be modified or enhanced to harm humans and what could happen if the technology or knowledge developed in the project “ended up in the wrong hands.” These ethical issues are raised, wrote the developers, when “research makes use of classified information, materials or techniques; dangerous or restricted materials[;] and if specific results of the research could present a danger to participants or to society as a whole.”

Roborder’s developers argued that these ethical concerns did not apply to their work, stating that their only goal was to develop and test the new technology, and that it would not be sold or transferred outside of the European Union during the life cycle of the project. But in interviews with The Intercept, project developers acknowledged that their technology could be repurposed and sold, even outside of Europe, after the European project cycle has finished, which is expected to happen next year.

Beyond the Roborder project, the ethics reports filed with the European Commission suggest a larger question: When it comes to new technology with the potential to be used against vulnerable people in places with few human rights protections, who decides what we should and should not develop?

Roborder won its funding grant in 2017 and has set out to develop a marketable prototype — “a swarm of robotics to support border monitoring” — by mid-2020. Its developers hope to build and equip a collection of air, sea, and land drones that can be combined and sent out on border patrol missions, scanning for “threats” autonomously based on information provided by human operators, said Stefanos Vrochidis, Roborder’s project manager.

The drones will employ optical, infrared, and thermal cameras; radar; and radio frequency sensors to determine threats along the border. Cell phone frequencies will be used to triangulate the location of people suspected of criminal activity, and cameras will identify humans, guns, vehicles, and other objects. “The main objective is to have as many sensors in the field as possible to assist patrol personnel,” said Kostas Ioannidis, Roborder’s technical manager.

The end product will be tested by border police in several European countries, including Portugal, Hungary, and Greece, but the project has also generated considerable interest in the private sector. “Eventually, we have companies that would certainly like to exploit this commercially,” Vrochidis told The Intercept. “They might exploit the whole outcome or part of the outcome, depending. They can exploit this in Europe but also outside of Europe.”

In their grant agreement, Roborder’s developers told the European Commission that they did not foresee any exports of their technology outside of the EU. In interviews, however, developers told The Intercept that the companies involved would be open to selling their technology beyond Europe. According to a spokesperson from the grant program funding Roborder, Horizon 2020, there is nothing Roborder’s EU backers can do to control where or how the technology they bankrolled is eventually used.

The documents obtained by The Intercept show Roborder responding to some ethical concerns about the project but not about the technology itself. In their grant application, Roborder’s developers conceded that their research “may be exploited by criminal organizations and individual criminals when planning to perpetrate acts of serious crime or terrorism” but wrote that the consortium of public and private companies developing the technology would work to keep their data safe. That group includes drone manufacturing companies, several national police departments, two national guards, a defense ministry, a port authority, a cyberdefense company, a company that specializes in developing equipment for electronic warfare, and another that provides “predictive analytics” for European police forces.

As for the technology’s possible modification for future clients, the answers were less clear. The developers would not comment on the potential for military sales after the project cycle ends. Developers added that their work is delayed because one of Roborder’s key consortium partners, Portuguese drone manufacturer Tekever, has left the project. Spokespeople for Roborder, Tekever, and Horizon 2020 would not explain the rationale for Tekever’s departure.

Horizon 2020 supports many security-oriented projects but maintains that “only research that has an exclusive focus on civil applications is eligible for funding.” The grant program previously funded a project that uses artificial intelligence to detect whether travelers are lying as they pass through automated border crossings.

Yet the documents obtained by The Intercept highlight inconsistent statements about Roborder’s potential military uses. According to one report, the project has no potential for “dual use,” or both military and civil deployment. Ten pages later, asked whether their work involved items that could be considered dual-use by European standards, Roborder’s developers wrote that it did.

Roborder hired a consultant, Reinhard Hutter, as an external ethics adviser to the project, according to another document from the Horizon 2020 program that was inadvertently released in full by the European Commission. In his report, Hutter wrote that “Roborder involves technology with military potential,” and that “the results of this project have the potential to be used back in the defense sector.” The technology involved, Hutter wrote, had “some dual-use potential but no dual-use activity on the project.” In other words, it could be used for military purposes but wouldn’t be used that way within the scope of Roborder.

Hutter declined to speak to The Intercept.

This blurring of lines between military and civilian development by the EU funding program might be deliberate. In a 2014 guidebook on European funding for dual-use projects, the European Commission notes that the regulation establishing Horizon 2020 mandates that all funded projects have “an exclusive focus” on civilian development, but the document also says that “substantial parts of the research funded is of relevance for defense and can lead to technologies that will be used by defense actors.”

The authors of a 2016 study commissioned by the security and defense sub-committee of the European Parliament went further, arguing that Horizon 2020’s clause on exclusive civilian development should be reinterpreted to include defense research. In order to compete with U.S. technological development, the study’s authors advocated for the creation of a European equivalent of DARPA, the U.S. Defense Advanced Research Projects Agency, whose work contributed to the development of the internet, GPS, and other technologies. In a 2017 speech, French President Emmanuel Macron echoed that, calling for, “a European agency for disruptive innovation in the same vein as” DARPA.

The 2016 report does not represent the views of the European Parliament or its security and defense sub-committee, and was not used to develop any specific legislation, a spokesperson for the European Parliament said. A spokesperson for Horizon 2020 rejected the idea that there was any ambiguity in what kind of projects the European Union would fund.

“The European Commission does not fund research intended for military use,” she said.

The drones Roborder plans to deploy are common technology. What would be groundbreaking for the companies involved is a functional system that allows swarms of drones to operate autonomously and in unison to reliably identify targets. AI threat detection is often inaccurate, according to robotics researchers, so any system that could correctly and consistently identify people, cars, and weapons, among other things, would be a substantial and lucrative advancement.

Drone cameras will not use facial recognition technology within the scope of the project, explained Ioannidis, Roborder’s technical manager, nor will they be able to determine any human characteristics, such as height, weight, age, skin color, or perceived gender. “The system will only identify that ‘this object is human,’” he added, “nothing more.”

Still, Ioannidis admitted that adding facial recognition to the Roborder system later on would be “technologically possible.” What about weaponizing the Roborder system to act against humans? “No,” he said, firmly. “The robots don’t have any authority to take any action against anyone. It’s just monitoring and giving alerts.”

But Sharkey, the U.K. robotics professor, argues that there is a thin line between using robots to monitor a border and using them to enforce one. Weaponizing a drone is relatively easy, he said, citing the 2015 case of the Connecticut teenager who equipped a drone with a handgun and a flamethrower. Sharkey worries about the implications of developing autonomous systems to patrol borders, including how the system could be used by a country coping with a large influx of people.

“The question is, where is this going?” Sharkey asked. “The current project does not propose to weaponize them, but it would just be too tempting for countries if a tipping point were to happen with mass migration.”

Hannah Couchman, a researcher at the U.K. human rights organization Liberty, agrees. “There are deep human rights and civil liberty concerns with this technology,” she told The Intercept. “Once this tech is developed, it’s seen as a solution, as a response to austerity, and a way to do a job efficiently with a lower cost, all rolled out without proper consultation and legislative scrutiny.”

“It’s not just about mitigating the human rights risk,” Couchman said. “It’s about whether we should use the technology in the first place.”

_____________________________________________________

Related:

• Pentagon Says All of Google’s Work on Drones Is Exempt From the Freedom of Information Act
  

• Cameras Linked to Chinese Government Stir Alarm in U.K. Parliament
  

• Inside the Video Surveillance Program IBM Built for Philippine Strongman Rodrigo Duterte
  

• Google’s Sidewalk Labs Plans to Package and Sell Location Data on Millions of Cellphones

Zach Campbell – zachcampbell@​pm.me

Go to Original – theintercept.com

Share this article: email mastodon facebook 🔗 copy link

DISCLAIMER: The statements, views and opinions expressed in pieces republished here are solely those of the authors and do not necessarily represent those of TMS. In accordance with title 17 U.S.C. section 107, this material is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. TMS has no affiliation whatsoever with the originator of this article nor is TMS endorsed or sponsored by the originator. “GO TO ORIGINAL” links are provided as a convenience to our readers and allow for verification of authenticity. However, as originating pages are often updated by their originating host sites, the versions posted may not match the versions our readers view when clicking the “GO TO ORIGINAL” links. This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of environmental, political, human rights, economic, democracy, scientific, and social justice issues, etc. We believe this constitutes a ‘fair use’ of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information go to: http://www.law.cornell.edu/uscode/17/107.shtml. If you wish to use copyrighted material from this site for purposes of your own that go beyond ‘fair use’, you must obtain permission from the copyright owner.

Comments are closed.