The Lensa AI image generator and ChatGPT language model have captured the public’s imagination. These innovations in artificial intelligence demonstrate why AI needs to be on the radar of every government decision-maker. AI technologies such as machine learning (ML) and neural networks are entering our daily lives – and agencies need to understand the potential consequences.

Of particular concern is whether bias and inequity can purposefully or inadvertently be built into AI algorithms, trained into AI models, or layered onto AI outputs. Here’s one example. Lensa AI allows users to create digital avatars based on personal pics. But some people who uploaded photos got back renderings that changed their appearance in ways that were, let’s say, not work-appropriate – and that seemed to be biased based on gender and race.

Agencies, meanwhile, are deploying AI in an exploding number of use cases, from routing grant applications in the Department of Health and Human Services to conducting sentiment analysis in the Department of Homeland Security. One increasingly common application of AI is for fraud detection.

Any agency involved in approving or issuing benefits needs effective ways of identifying and preventing fraud. AI offers a powerful solution. But just as bias and inequity can emerge unexpectedly in AI-powered image generation, there’s a potential for them to appear in AI-enabled fraud detection.

Before agencies can root out bias and inequity in their fraud detection, however, they need to explore some fundamental questions. How do bias and inequity emerge in AI? What would it mean to be more fair or equitable? The answers aren’t always clear or simple. But here’s what your agency should be thinking about today.

Understanding Sources of Bias and Inequity

Fraud is a perennial challenge for agencies that provide benefits to the public. For example, over the past three years, anywhere from 6% to 44% of unemployment payments were reported by states as “improper.” Such fraud has a negative impact on agency budgets and programs, and on the residents those programs are intended to serve.

Fraud has become so fluid, and fraudsters have become so well-funded and sophisticated, that traditional methods of identifying fraud are no longer viable. ML algorithms offer an effective solution by rapidly analyzing large volumes of data and spotting anomalies or patterns that indicate fraud.

But the data analyzed by fraud-identifying AI can include information such as age, gender, ZIP code, and financial history. As a consequence, there’s a potential for bias to creep into fraud detection.

Inequity can emerge in fraud detection when decisions about investigation or enforcement affect different population categories differently. At a high level, this can occur in two ways:

• Disparate treatment – Decisions are applied to demographic groups differently.
• Disparate impact – Decisions harm or benefit demographic groups differently.

Note that decisions can be applied equally but still result in disparate impacts. If a fire department requires that firefighters be able to lift 100 pounds, female job applicants might be at a disadvantage. The fire department should then demonstrate that the requirement is job-related and necessary.

Why You Can’t Just Look the Other Way

Agencies might try to avoid bias through simple unawareness. Let’s say you’re developing an ML model to uncover tax fraud, and you don’t want the model to be biased based on a legally protected attribute such as age. You might think the solution would be to just omit age from the data.

But other attributes can act as a proxy for age. For instance, if the dataset included the number of past tax returns filed, that could be a proxy for age. Depending on the constituencies your agency serves and the use case for your ML model, attributes such as ZIP code, income, consumption of government benefits, and criminal justice interactions could be proxies for legally-protected attributes.

What’s more, if you train your model using historical data, the model and its outputs could be affected by bias baked into the original dataset. For instance, Lensa AI generates avatars by using an open source model built on a dataset of images scraped from the internet. Even if the model omits attributes such as gender and race, if the images gathered from the internet skew in certain ways, the model’s output will skew accordingly.

A Commitment to Equity

Of course, there are situations where an agency might want to purposefully target AI inputs or outputs to benefit a particular community. If your agency wants to analyze data to better serve populations such as human trafficking victims, drug users, or families with low incomes, say, you might need to include sensitive attributes. Can that be considered bias or inequity? There isn’t a simple answer.

What’s important is that agencies be aware of these issues and commit to avoiding bias and pursuing equity when deploying AI. One way to achieve that goal is by working with a provider of AI solutions that has a proven track record driving positive AI-powered outcomes across government organizations.

Another way is by aligning AI efforts with the Blueprint for an AI Bill of Rights issued by the White House in October 2022. The document establishes five principles to guide design and use of automated systems:

• Safe and effective systems – Systems should be developed in consultation with diverse communities to identify risks and potential impacts.
• Algorithmic discrimination protections – Algorithms should not subject the public to discrimination, and systems should be designed and used in an equitable way.
• Data privacy – Built-in safeguards should protect the public from abusive data practices, and individuals should have agency over how data about them is used.
• Notice and explanation – Developers and deployers of systems should inform the public how systems are being used and how outcomes will affect them.
• Human alternatives, consideration, and fallback – Individuals should be able to opt out of systems when possible.

AI is providing governments with extraordinary benefits, from more efficient use of resources to more effective delivery of services. Advanced analytics are enabling agencies to identify, stop, and proactively prevent fraud, efficiently and at scale. As these technologies enter more aspects of our daily lives, understanding the role of fairness and equity in AI will help agencies optimize their investments and achieve positive outcomes.