In a statement, Microsoft said the company has "already taken steps to improve the accuracy of our facial recognition technology", while IBM responded saying it has "several ongoing projects to address dataset bias in facial analysis - including not only gender and skin type, but also bias related to age groups, different ethnicities, and factors such as pose, illumination, resolution, expression and decoration".
MIT researchers found that three facial recognition programs had an error rate of up to 0.8 percent when white men were involved.
The findings raise questions about how today's neural networks, which learn to perform computational tasks by looking for patterns in huge data sets, are trained and evaluated. In the paper, the researchers name data sets-IJB-A and Adience-as examples of commonly-used image databases that contain a large majority of light-skinned faces. This method can be used to determine someone's gender and identify a person in cases of criminal suspect or for unlocking a phone and it is not just about computer vision. As noted by Georgetown University's Center for Privacy and Technology, these gender and racial disparities could, in the context of airport facial scans, make women and minorities more likely to be targeted for more invasive processing such as manual fingerprinting.
Buolamwini says that the benchmark data set, composed of 1,270 images of people's faces that are labeled by gender as well as skin type, is the first data set of its kind, created to test gender classifiers, that also takes skin tone into account.
Providing evidence that both personal and institutional biases can be reflected in software and artificial intelligence, a new study found that facial recognition technology has a much higher rate of failure when analyzing darker-skinned people, especially women.
Each face was then assigned a rating for skin type based on the six-point Fitzpatrick rating system, which dermatologists use as "the gold standard" for classifying different shades of skin, the paper notes.
Gender was misidentified in less than one percent of lighter-skinned males; in up to seven percent of lighter-skinned females; up to 12 percent of darker-skinned males; and up to 35 percent in darker-skinner females.
Wall Street ends higher for third straight session
U.S. stocks are extending gains from late-Friday, putting the market on track for a strong start to the new week. In overseas trading, stock markets across the Asia-Pacific region moved mostly higher during trading on Tuesday.
Such bias is sadly nothing new, with Google's image recognition in its Photos app making a massive classification gaff that highlights a problem of bias that seems to get programmed into AI systems.
For darker-skinned women - those assigned scores of IV, V, or VI on the Fitzpatrick scale - the error rates were 20.8 percent, 34.5 percent, and 34.7.
Two years ago, The Atlantic reported on how facial recognition technology used for law enforcement purposes may "disproportionately implicate African Americans". The final set contained more than 1,200 images.
"To fail on one in three, in a commercial system, on something that's been reduced to a binary classification task, you have to ask, would that have been permitted if those failure rates were in a different subgroup?"
Explaining further, Buolamwini said she used 1,270 faces as samples for the research. This is because of the data sets provided to the systems and the condition in which the algorithms were created.
"It takes time for us to do these things", he adds. "She was bringing up some very important points, and we should look at how our new work stands up to them".