Ryan Chiao, Senior Photographer

Three Yale scientists are on a mission to produce objective machine learning algorithms from inherently biased training data.

In the modern world, questions of who will pay back their loans and who should qualify for insurance are increasingly decided by computer programs. These algorithms are used under the assumption that they are impartial; however, biases often become ingrained in machine learning programs through training methods and data, according to Amin Karbasi, professor of electrical engineering and computer science, and Ehsan Kazem, former Yale postdoctoral fellow. But now, researchers at the Yale School of Management have designed a novel “train then mask” technique for supervised learning to help eliminate these biases from algorithms and ensure computers do not repeat societal discimination patterns. 

“I don’t think there is any universal solution to [implicit bias in algorithms] at this point,” said professor of marketing Soheil Ghili, one of the three researchers who pioneered this new technique. “There could be multiple solutions depending on what your main objective is. The objective here is to reduce the disparity in treatment of different gender or racial groups while maintaining accuracy of your prediction to the extent possible and most importantly while maintaining the feature that two individuals who are otherwise identical, which by otherwise I mean with the exception of their sensitive features, will be treated equally.”

Kazemi, a current Google researcher, explained that implicit biases can often sneak into algorithms even if not explicitly programmed in. As an example, he pointed to height and gender. Because men are often taller than women, if an algorithm is trained to prioritize height, it may also implicitly be prioritizing men over women. 

Complications arise when attempting to eliminate discrimination by controlling the information given to an algorithm. Even though gender is not used in training the model, it is incorporated in the importance of height to the final prediction. This means that while gender is not explicitly present, the algorithm can still discriminate with respect to gender if its training data is skewed in that direction, Kazemi explained.

The pioneers of the new “train then mask” technique — Ghili, Kazemi and Karbasi — argue that the solution to removing bias in algorithms is not to remove classifications such as gender, but rather to initially include it. 

“Let’s say you want to classify basketball players.” Kazemi told the News. “Although you don’t want to favor men over women, height is important. You want to keep that feature. You keep the explicit feature (gender) in the model when you train a model, so that you make sure the other part, the contribution of the height, comes from the importance of the height itself, not from historical discrimination or from acting as a proxy for gender. [Then] at the time of running the model, you assume all people have the same gender, so there is no discrimination.”

“Train then mask” prediction algorithms are made in two stages. First they are trained with all data, including sensitive features, to limit the formation of implicit biases. Then, when run, sensitive features are masked to ensure discrimination does not play a role in decision making. Essentially, every individual is run with the same race and gender and then their true characteristic features determine the outcome.

This means that the “train then mask” algorithm is fair with respect to each individual. If two loan applicants with identical features except for race come into a bank, they will be given identical reliability scores, and both accepted, or both denied.

The “train then mask” definition of fairness does not necessarily represent all definitions of fairness. Some believe that specific groups should be given priority, as with affirmative action, in order to counteract societal discrimination. However, others worry that prioritizing historically marginalized racial groups in algorithms could potentially lead to legal disputes.

The definition of fairness continues to evolve from individual to individual and situation to situation, which is why Ghili says there is no “universal solution” to biases in algorithms at this point in time.

“There is a trade off between accuracy and fairness,” Karbasi said. “If you want to be very accurate about your predictions, you have to use all the features, but sometimes that is not the right thing to do, because then you are going to add bias towards a group of people.” 

While some accuracy is relinquished in favor of fairness through the “train then mask” technique, the method has a minimal impact on success rate, according to the data reported in a November 2018 Yale study. The scientists employed their “train then mask” on real world data to make three predictions: an individual’s income status, the reliability of a credit card applicant and whether a criminal will reoffend. 

The results were promising. In predicting income status, the unconstrained algorithm which was given all data points was correct 82.5 percent of the time. In contrast, the “train then mask” algorithm was correct 82.3 percent of the time and without implicit bias.

A recent investigation in “The Markup” reported that with conventional mortgage-approval algorithms, Black loan applicants are 80 percent more likely to be rejected than similar white applicants. “Train then mask” algorithms can help add fairness to the industry, Karbasi said. 

According to Karbasi, the future of “train then mask” algorithms is bright. 

“For simple models, this is provably the best thing you can do,” he said.

An article describing the technique, “Eliminating Latent Discrimination: Train Then Mask,” was published in 2019 in the Proceedings of the AAAI Conference on Artificial Intelligence.