AI Use in Refugee Resettlements Raises Risks of Discrimination

Resettlement agencies are responsible for ensuring refugees thrive in their new environment and making relocation decisions “without regard to race, religion, nationality, sex, or political opinion.” Unfortunately, these agencies’ use of automated decision-making systems in refugee resettlement could be inconsistent with this requirement.

ADM systems are a type of artificial intelligence that use algorithms and data to produce decisions typically made by humans. Of the 10 agencies that oversee refugee resettlement, at least half turn to this technology for assistance in resettlement decisions.

In 2018, the refugee resettlement agency HIAS partnered with researchers to create Annie MOORE, one of the first ADM systems to aid in resettlement. This system recommends placements that maximize employment chances for refugees entering the country.

However, Annie MOORE’s developers said limited data hampered the system’s prediction abilities, forcing researchers to primarily rely on data that measured short-term employment outcomes of refugees 90 days after their arrival.

The system didn’t account for refugees’ long-term employment, physical and mental health, education, or household earnings. Arguably, these factors are equally as important in measuring the success of refugee resettlement as short-term employment outcomes.

Although HIAS since implemented an updated ADM system called RUTH, the relative lack of data remains a concern. Notably, the system’s developers say, “while HIAS has data on hundreds of thousand[s] of refugees, they only have data on 15 fiscal years, which is, moreover, incomplete and smaller-scale in earlier years.”

ADM technologies can be very effective in expediting the refugee resettlement process. However, when developers base their algorithms on limited data, they can also perpetuate inequality. This disparity becomes evident when analyzing how ADM refugee resettlement decisions vary across demographics.

Authors of a February 2023 working paper from Harvard Business School found that the impact of current resettlement algorithms can differ substantially depending on a refugee’s age, education, and country of origin. They detail the legal issues arising from such variations and say “the use of a refugee assignment algorithm can be jeopardized from a legal standpoint if it has disparate impacts on refugees from different origin countries.”

The researchers also point to Recital 71 of the EU’s General Data Privacy Regulation, which requires individuals overseeing ADM systems to prevent discriminatory effects based on a person’s racial or ethnic origin.

Although Congress has yet to pass such comprehensive data privacy legislation, Title 8 of the US Code provides the government with sufficient jurisdiction to regulate resettlement agencies’ decision-making processes if they discriminate against individuals based on nationality.

In showing that ADM systems disfavor refugees based on their country of origin, researchers have observed a flaw that harms individuals and risks putting resettlement agencies at odds with federal law.

Case law illuminates other potential problems that could emerge for resettlement agencies using ADM systems. For example, in Bauserman v. Unemployment Insurance Agency, plaintiffs alleged that the Michigan Unemployment Insurance Agency’s ADM system automatically categorized applications as fraudulent, then collected fines without due process.

In July 2022, the Michigan Supreme Court held that the plaintiffs could seek monetary damages from the state based on an alleged violation of the state’s constitution. The state of Michigan approved a $20 million settlement for individuals wrongly accused of fraud by the ADM system.

While ADM systems used in refugee resettlement vary from those used by Michigan’s Unemployment Insurance Agency, the resettlement agencies should ensure their programs both comply with due process requirements and don’t have discriminatory effects.

Refugee resettlement agencies have one of the most crucial roles in the refugee process. They must relocate individuals who experienced unknowable hardships and work to enable them to prosper in the US. These aren’t simple tasks, so it’s understandable that agencies would turn to new technologies for assistance.

However, these agencies must ensure the systems they employ don’t disadvantage the very people they’re intended to help.

This article does not necessarily reflect the opinion of Bloomberg Industry Group, Inc., the publisher of Bloomberg Law and Bloomberg Tax, or its owners.

Author Information

Clarisa Russenberger is a student at Wayne State University Law School in Detroit.

Write for Us: Author Guidelines