Governments that want to know whether their algorithms discriminate (indirectly) can soon use a new tool that the Central Planning Bureau (CPB) has developed. If it were up to the planning agency, the central government would ensure that the so-called Selectivity scan becomes available to all government organizations.
Governments and public agencies are increasingly using computer algorithms to make decisions, the CPB writes in the announcement. There is a risk that these algorithms are designed in such a way that they cause discrimination.
Indirect discrimination
A well-known example is the algorithm that a fraud team from the Education Executive Agency (DUO) used to detect potential fraudsters. That program discriminated unintentionally: it looked at, among other things, the age, the type of education (MBO, HBO, University) and the living situation of students. However, of 376 students who ended up in court before DUO, 97 percent turned out to have a migration background.
It is a typical case of ‘indirect discrimination’. “In such a case, an algorithm seems neutral, but it disadvantages a specific group,” says Mark Kattenberg, Data Science program leader at CPB.
According to Kattenberg, making a distinction between people is not necessarily discrimination – after all, making a distinction is what algorithms are intended for. “Indirect discrimination is allowed. But you must be able to justify it. If you cannot, then there is discrimination.” The latter also turned out to be the case at DUO: a total of ten thousand students who were identified as fraudsters by the algorithm received compensation at the end of last year.
Algorithm register
To prevent the use of discriminatory algorithms, want to the government requires government organizations to report the algorithms they use in the algorithm register. Now this is still done voluntarily. The European AI law, which will come into effect next August, also requires algorithms to be checked for discrimination and arbitrariness.
But such control is difficult with indirect selection. This is due to the Privacy Act, which requires organizations to collect and store as little data about citizens as possible. If you do not know whether people in your algorithm have a migration background, you cannot assess whether the algorithm discriminates against them.
The CPB tool should offer a solution to that dilemma. The program works with so-called microdata from Statistics Netherlands (CBS): anonymized data at personal level about, for example, income, assets, debts, age and gender.
After the algorithm has determined which people in the group have a higher chance of being a fraudster, for example, the developer shares this selection with CBS, Kattenberg explains. “Then you get back: this is your selection, so many percent of it has a migration background, that is so many percent higher or lower than the average in the entire group. So you only see the percentages, not the data of the people.”
New insights
The CPB tested the tool on an algorithm from the UWV. This shows that the method works, the CPB writes in its report: “The Selectivity Scan shows that it is possible to safely test algorithms with data that the algorithm developer cannot use himself.” According to the planning agency, the test “provides new insights” and limits “the risk of misuse of personal data”.
The CPB therefore strongly advises the government to make a tool “such as the Selectivity Scan” available to government organizations that work with algorithms. According to Kattenberg, CBS, as the owner of the microdata, could offer the service itself, but no agreements have yet been made about this.
The CPB itself will probably not offer, further develop and maintain the tool, says Kattenberg: “That falls outside our scope. core business as a planning agency.” Why did the CPB build the program after all? “Because it is a very economic subject. Algorithms are used to make selection processes more efficient, but there are disadvantages. Efficiency clashes with justice here. That is a truly classic economic theme, it would be good if economists pay more attention to it. That is also the reason why we did that.”
Also read
AI should be a top political priority
NEW: Give this item as a gift
As an NRC subscriber you can subscribe every month 10 items give as a gift to someone without an NRC subscription. The recipient can read the article directly, without a paywall.
The journalistic principles of NRC