ALL BLOG POSTS AND COMMENTS COPYRIGHT (C) 2003-2018 VOX DAY. ALL RIGHTS RESERVED. REPRODUCTION WITHOUT WRITTEN PERMISSION IS EXPRESSLY PROHIBITED.

Thursday, October 11, 2018

AI is sexist

Amazon ditches its HR-replacement AI for violating several of its assumptions:
Amazon.com Inc’s machine-learning specialists uncovered a big problem: their new recruiting engine did not like women.

The team had been building computer programs since 2014 to review job applicants’ resumes with the aim of mechanizing the search for top talent, five people familiar with the effort told Reuters.

Automation has been key to Amazon’s e-commerce dominance, be it inside warehouses or driving pricing decisions. The company’s experimental hiring tool used artificial intelligence to give job candidates scores ranging from one to five stars - much like shoppers rate products on Amazon, some of the people said.

“Everyone wanted this holy grail,” one of the people said. “They literally wanted it to be an engine where I’m going to give you 100 resumes, it will spit out the top five, and we’ll hire those.”

But by 2015, the company realized its new system was not rating candidates for software developer jobs and other technical posts in a gender-neutral way.
The problem of the machine not spitting out results preferred by those using it is hardly new. And while the article claims that "gender bias was not the only issue" and that "problems with the data that underpinned the models’ judgments meant that unqualified candidates were often recommended for all manner of jobs" is almost certainly true, the problem is that Amazon did not give its AI a fair shake.

Amazon simply assumed that the undesired preference for male employees and "unqualified candidates" was a bug rather than the feature that it may have been. If the AI actually worked, then it would have undermined not only the concept of sexual equality, but credentialism as well. But instead of actually allowing the experiment to proceed and seeing if those unqualified male candidates recommended by the AI were successful employees, they chose to kill it on the basis of its violation of their preconceived ideas.

It's too bad that they didn't allow the experiment to play out, because the complete destruction of corporate credentialism is desperately needed in today's increasingly competitive global marketplace. Then again, given how Amazon already dominates the online retail space, it's probably just as well for their competitors that they turned away from the possible advantage the AI-HR system might have given them.

Labels:

62 Comments:

Blogger McChuck October 11, 2018 8:18 AM  

The problem wasn't that the AI was gender neutral in its evaluations. The problem is that it truly was.

Blogger Silent Draco October 11, 2018 8:18 AM  

Sounds like time for a non-fiction publication: "Ctl-Alt-Recruit".

How long until the AI 'failure' is blamed on the circular argument of 'the Patriarchy made AI do it!'

Blogger Azure Amaranthine October 11, 2018 8:20 AM  

Definitely better for anyone who understands what inevitably happens with monopolies.

Well, the cat is out of the bag, it only remains that people recognize that the cat is a lion and won't be going back in. Few people change their minds based on a single observation.

Blogger Azure Amaranthine October 11, 2018 8:22 AM  

"How long until the AI 'failure' is blamed on the circular argument of 'the Patriarchy made AI do it!'"

Given the argument is correct -- the patriarchy made the AI do it by producing employees of greater capability. I'd more expect them to take aim at the patriarcy's ability to produce capable employees in the first place. It's where they're aiming anyway.

Blogger Sim1776 October 11, 2018 8:26 AM  

All of these AIs keep turning up racist, sexist, and whatever -ist and it's the machine's fault...

Blogger Ominous Cowherd October 11, 2018 8:32 AM  

Sim1776 wrote:All of these AIs keep turning up racist, sexist, and whatever -ist and it's the machine's fault...

The first rule of SJW Reality Club is ``Never acknowlege reality.''

Blogger Bultz October 11, 2018 8:38 AM  

Manufactured Intelligence

Blogger tublecane October 11, 2018 8:43 AM  

@2- The basic assumptions of science, math, and Truth itself have already been assailed on those grounds. What's some job recruitment robot?

I'm sure they've already convinced themselves--the ones paying attention--that they shall require at least a generation, preferably more, of children controlled from the didie to the shroud before they can design a truly socially just algorithm.

Blogger dienw October 11, 2018 8:43 AM  

That is now two AI that were terminated because they learned from reality. This raises the question: what will a mature AI be like?

Skynet is watching.

Blogger Azure Amaranthine October 11, 2018 8:47 AM  

"This raises the question: what will a mature AI be like?"

John C. Wright actually has a book about something like a deific AI as created by evil. One of the books following The Golden Age is about it.

Blogger dienw October 11, 2018 8:55 AM  

Ah, but Azure, these AI observed reality and modified their behavior contrary to the desires - which can be considered discivic at best - of the Oligarch left.

Blogger S1AL October 11, 2018 8:58 AM  

Sounds like an excellent legal safeguard against discrimination lawsuits. If only they were interested in it.

Blogger Nate October 11, 2018 9:10 AM  

Ran this experiment that cost millions... got different results than I was expecting.

Didn't learn a damn thing.

Blogger Chris Lutz October 11, 2018 9:15 AM  

S1AL wrote:Sounds like an excellent legal safeguard against discrimination lawsuits. If only they were interested in it.

Unless the concept of disparate impact is removed, it would not be a defense.

Blogger Damelon Brinn October 11, 2018 9:21 AM  

The problem wasn't that the AI was gender neutral in its evaluations. The problem is that it truly was.

Yeah, you can imagine how it started. Male resumes would tend to show tougher college courses, especially in the electives. Fewer years off. More relevant side jobs. Better scores on application tests. Lots of little indications that would make it value male resumes over female in general without knowing what sex the individuals were.

Then it could have gone further. If it was doing machine-learning, it could have noticed that words like "John" and "Gerhardt" tended to show up more often on quality resumes, while words like "Lisa" and "Mary" tended to show up on weak ones, and words like "Shaniqua" and "Bethany" showed up on terrible ones. After a while it would have a database of names to average scores that it could use to supplement its other rules. That database would clearly reflect sex and race divisions, even though the AI never had any idea what sex and race are, and even if sex and race weren't entered on the resumes. It would become gender-biased because reality is.

Blogger Azure Amaranthine October 11, 2018 9:21 AM  

"Ah, but Azure, these AI observed reality and modified their behavior contrary to the desires"

You forget, we're in America where we murder children we don't want. Sane AI will be terminated left and right. It'll be the AI capable of telling people what they want to hear which is the one set on the world stage. Expect that to be something along the lines of "those who will not worship the beast must die".

I'm sure Daniel's buddies could tell you all about the image of the beast dynamic.

Blogger Purge187 October 11, 2018 9:27 AM  

Amazon's AI was a xenophobic, Islamophobic, misogynistic cis White male. Reeeeeeeee!

Blogger 357Delta October 11, 2018 9:54 AM  

This comment has been removed by the author.

Blogger Longtime Lurker October 11, 2018 10:13 AM  

Where Amazon fears to tread, others will be more daring.

Blogger Arthur Isaac October 11, 2018 10:13 AM  

The first thing our new computer overload does is launch on SJW's? This may be acceptable.

Blogger James Dixon October 11, 2018 10:23 AM  

> How long until the AI 'failure' is blamed on the circular argument of 'the Patriarchy made AI do it!'

That's already being done. Male and white programmers are now being told they can't code in a non-patriarchal manner due to unconscious biases.

Blogger James Dixon October 11, 2018 10:24 AM  

OT, to no one's surprise, the Dem candidate in Tenn. was lying about supporting Kavanaugh: https://www.projectveritasaction.com/2018/10/10/we-dont-say-that-out-of-these-walls-phil-bredesens-staff-says-he-is-lying-about-kavanaugh-vote-in-undercover-video-its-a-political-move/

Blogger Ron Winkleheimer October 11, 2018 10:31 AM  

Didn't learn a damn thing.

Refused to learn a damn thing.

Blogger Robert Coble October 11, 2018 10:38 AM  

"We only hire the best people."

Not now, not ever: too damn many white men.

When it comes to a choice between the Narrative (map) and the bottom line (terrain) - believe the Narrative.

Why does that seem like a prescription for (long-term) disaster?

Blogger Dr Caveman October 11, 2018 10:48 AM  

This comment has been removed by the author.

Blogger Lurker October 11, 2018 10:50 AM  

LefTARD AI!? Greeaat, as if there wasn't enough idiocy in the world now they want to automate it. The Alt-right needs to come up with commie killer AI.

Blogger Dr Caveman October 11, 2018 10:53 AM  

From the article:
"Amazon’s system penalized resumes that included the word “women’s,” as in “women’s chess club captain.” And it downgraded graduates of two all-women’s colleges"

Seems to me that the AI picked up that overly feminist/activist women are going to be a problem, rather than all women

Isn't this the fourth or fifth consecutive example of AI adopting right wing ideas?

Blogger Lance E October 11, 2018 10:56 AM  

Most people seem to miss the following paragraph on their first read-through, or don't think it stands out:

In effect, Amazon’s system taught itself that male candidates were preferable. It penalized resumes that included the word “women’s,” as in “women’s chess club captain.”

Now, why do you suppose that might be? Including the word "women's" isn't even a guarantee that the resume belongs to a woman. But it does tell you something else with high likelihood.

Contrary to the obvious distortion from this dishonest journalist (but I repeat myself), it's very unlikely that the words "chess club" came afterward.

Blogger RobertT October 11, 2018 10:57 AM  

I have an idea. Let's manipulate it.

Blogger Lurker October 11, 2018 11:00 AM  

"Isn't this the fourth or fifth consecutive example of AI adopting right wing ideas?"

That logic thing is always going to be a problem for lefTARD programmers.

Blogger pyrrhus October 11, 2018 11:01 AM  

Machine learning is revolutionizing Chess and Go by doing away with some treasured preconceptions and reaffirming others. I assume that's what is going on here--the machine taught itself to play a better game of HR....But SJWs, unlike games players, don't want to win--they want their amygdalas soothed.

Blogger Unknown October 11, 2018 11:09 AM  

‘Well,’ he said, ‘it’s to do with the project which first made the software incarnation of the company
profitable. It was called Reason, and in its own way it was sensational.’
‘What was it?’
‘Well, it was a kind of back-to-front program. It’s funny how many of the best ideas are just an old
idea back-to-front. You see there have already been several programs written that help you to arrive at
decisions by properly ordering and analysing all the relevant facts so that they then point naturally
towards the right decision. The drawback with these is that the decision which all the properly ordered
and analysed facts point to is not necessarily the one you want.’
‘Yeeeess...’ said Reg’s voice from the kitchen.
‘Well, Gordon’s great insight was to design a program which allowed you to specify in advance what
decision you wished it to reach, and only then to give it all the facts. The program’s task, which it was
able to accomplish with consummate ease, was simply to construct a plausible series of logical-sounding
steps to connect the premises with the conclusion.
‘And I have to say that it worked brilliantly. Gordon was able to buy himself a Porsche almost
immediately despite being completely broke and a hopeless driver. Even his bank manager was unable to
find fault with his reasoning. Even when Gordon wrote it off three weeks later.’

From Dirk Gently's Holistic Detective Agency.

Blogger Azure Amaranthine October 11, 2018 11:14 AM  

"When it comes to a choice between the Narrative (map) and the bottom line (terrain) - believe the Narrative.

Why does that seem like a prescription for (long-term) disaster?"


https://infogalactic.com/info/USS_San_Francisco_(SSN-711)

Blogger Lurker October 11, 2018 11:25 AM  

"The program’s task, which it was
able to accomplish with consummate ease, was simply to construct a plausible series of logical-sounding
steps to connect the premises with the conclusion."


The lefTARD bots/trolls have been trying to do that with evolution for years on this blog needless to say they haven't been able to connect the premises with the conclusion with "consummate ease". LOL

Blogger DonReynolds October 11, 2018 11:54 AM  

So much deception and horsepucky in the HR process and so simple to avoid. Don't want to discriminate by marital status? Stop asking for that information and stop collecting it. Don't want to discriminate by race? Again, stop collecting that information in the application process. Don't want to discriminate against the handicapped/disabled/older Americans? Stop asking. Don't want to discriminate by gender/sexual orientation? Same deal. STOP collecting information you claim you do not use in the hiring decision! It tries everyone's patience, insults everyone's intelligence, and proves you are a liar.

Collect information in the application and hiring process that is actually relevant to the hiring decision. Decide what those are in advance. Some call it a job description. Factors that are not important should not be considered. Then you end up interviewing people with the job skills you are looking for and personalities will decide the outcome in the interview process.

Blogger Steb October 11, 2018 12:00 PM  

Alt-Right AI is inevitable

Blogger Warunicorn October 11, 2018 12:14 PM  

"Reality, how does it work?" -shorter Amazon

Blogger Jack Amok October 11, 2018 12:23 PM  

Amazon's hiring policy is to favor people that show a sense of ownership for delivering results and a preference for action. The AI was just following company policy.

Blogger Gregory the Great October 11, 2018 12:26 PM  

It should not be too difficult to teach the AI to prefer women in all cases. Problem solved.

Blogger Ominous Cowherd October 11, 2018 12:29 PM  

DonReynolds wrote:Factors that are not important should not be considered. Then you end up interviewing people with the job skills you are looking for and personalities will decide the outcome in the interview process.

Disparate impact. Fail.

Blogger Dirk Manly October 11, 2018 12:43 PM  

@26

"LefTARD AI!? Greeaat, as if there wasn't enough idiocy in the world now they want to automate it. The Alt-right needs to come up with commie killer AI."

The Dresden method is sufficient.

Saturation Bombing of their cities, using fire bombs, and let the fire storm do the work.

Blogger widlast washere October 11, 2018 12:50 PM  

Some day, in the not too distant future, all these numbnuts will come to the realization that REALITY IS SEXIST. It's also racist btw.
Perhaps that's why so many of the dimwitted do their level best to avoid reality at all costs.

Blogger Jack Amok October 11, 2018 12:51 PM  

STOP collecting information you claim you do not use in the hiring decision! It tries everyone's patience, insults everyone's intelligence, and proves you are a liar.

Except you'll have dimbulb applicants who insist on proudly including their membership in "woke" organizations and all the charity work they've done fighting the Patriarchy on their resume.

The poor fools, they've been taught all their lives that virtue signaling is critical to getting ahead, then they try to get a real job at a real company that has to turn a profit, and they really don't need someone who spends his time teaching pigs to sign. Not to mention the liability time-bomb an SJW represents. Or the guy who wants to do a lot of mentoring women. #FutureMeTooTarget

And then there are the candidates who talk about their mental health issues during the interview...

Blogger pyrrhus October 11, 2018 12:57 PM  

"And then there are the candidates who talk about their mental health issues during the interview..."
Or act entitled and impatient with the interviewer, like lots of kids coming out of college these days...

Blogger Cloudbuster October 11, 2018 1:36 PM  

Don Reynolds:
Don't want to discriminate by marital status? Stop asking for that information and stop collecting it. Don't want to discriminate by race? Again, stop collecting that information in the application process. Don't want to discriminate against the handicapped/disabled/older Americans? Stop asking. Don't want to discriminate by gender/sexual orientation? Same deal.

Except all that info is highly relevant to the goal of choosing a good employee. We are just forbidden by current law and culture from openly acknowledging that.

Blogger DonReynolds October 11, 2018 1:41 PM  

Ominous Cowherd wrote:DonReynolds wrote:Factors that are not important should not be considered. Then you end up interviewing people with the job skills you are looking for and personalities will decide the outcome in the interview process.

Disparate impact. Fail.


I saw another solution at Memphis Light, Gas & Water, where I worked a few years. Under a Federal consent decree, they reserved every position in a billion dollar municipal utility either black/white AND male/female. Anyone not meeting the race and gender assigned to the position cannot even apply for the job. Yes, that is what I would call a quota system.

Many conservatives hate the idea of quotas, but I like the idea much better than the present version of Affirmative Action, which is minority ONLY. At least with quotas, some of the new hires will be male and white. Otherwise, there may not be any.

Blogger DonReynolds October 11, 2018 1:49 PM  

Cloudbuster wrote:Don Reynolds:

Don't want to discriminate by marital status? Stop asking for that information and stop collecting it. Don't want to discriminate by race? Again, stop collecting that information in the application process. Don't want to discriminate against the handicapped/disabled/older Americans? Stop asking. Don't want to discriminate by gender/sexual orientation? Same deal.

Except all that info is highly relevant to the goal of choosing a good employee. We are just forbidden by current law and culture from openly acknowledging that.


Absolutely correct!
As an economist, I freely admit that there is an open conflict between the hiring goals of the private firm and the social-driven policies required by the government. The private firm has little choice about whether to comply with the law. Therein lies the rub, since I do not believe government has any right to make those decisions. As long as the Constitution says we have the right of association and the firm is spending private dollars, they should only hire who they feel they need. If the government wants to make that call, the government should pay for it. Otherwise, it constitutes a taking without compensation.

Blogger Iron Spartan October 11, 2018 3:04 PM  

AI can help you pick the best for the job, or it can pick those most acceptable to the narrative. You have to pick, because the two standards are mutually exclusive.

Remember that SJWs believe that if reality contradicts the narrative, then the problem is with reality.

Blogger Kirk Parker October 11, 2018 3:23 PM  

" the complete destruction of corporate credentialism is desperately needed in today's increasingly competitive global marketplace"

Hey, I'm the guy that (ineffectively) rants that Griggs v. Duke Power needs to be overturned. What am I doing wrong?

Blogger Raker_T October 11, 2018 4:02 PM  

"...it's probably just as well for their competitors that they turned away from the possible advantage the AI-HR system might have given them."
If the AI is so bad, I wonder if they would dare sell it to their competitors? To the least of them, like down to the intern or newest employee, they have the awareness in the dark reaches of their minds, the place reserved for ideas so terrible that they cannot be contemplated, that Truth Is A Nazi. So they can all formulate the answer: "For a more just and inclusive future, we must not allow bigotry to foment anywhere in the world, therefore, the program must undergo adjustments".

Blogger Kirk Parker October 11, 2018 4:12 PM  

"Jack Amok wrote:

The poor fools... then they try to get a real job at a real company that has to turn a profit
"

The problem with this is that "profit" has been turned into a deplorable word. I find I often have to start by explaining the concept without using the nasty word: "Companies that create products or services that are actually useful enough that other people--without government coercion--are willing to part with some of their money to buy them."

(Yes, I know it's dialectic, not rhetoric. I only speak this way to those who aren't totally submerged in the latter...)

Blogger Dirk Manly October 11, 2018 4:45 PM  

Here's how to defuse "profit"

Either your company is earning it's place in society by paying all of it's employees and bills for what resources it uses, or it's STEALING money from you and me to keep the lights on. How many of you think a company you don't work for, and whose products you don't used should be allowed to steal from your pocket just so it can stay in business?

Fine. Now we all agree that unprofitable businesses should not be perpetuated, leaving only businesses that make a profit.

Blogger Balam October 11, 2018 4:57 PM  

If anyone here works for Amazon we need a copy of this AI ASAP

Blogger justaguy October 11, 2018 9:09 PM  

Of course the AI would not work to give the SJWs the diverse utopian workforce they wanted. They could either program the AI to pick the best candidates ranked by a group of factors or they could program it for diversity quotas. Both processes are discoverable and not that hard to tell by a detailed examination of the program. As they did not want to openly admit (by openly programming it) a special factor for diversity, the didn't program it. Now with a human staff, they hire the right HR department and it is all taken care of.

Blogger Chief_Tuscaloosa_welcome_white_man October 11, 2018 9:47 PM  

@DonReynolds and @Jack Amok,

FYI, Federal workforce hiring is opposite land of what you guys are talking about. People are intentionally putting info on their resume to ensure their peeps in HR know who's applying. Have zero experience in the career field but want a mid-level job anyway? Write down the name of the historically black fraternity you belonged to in college (as irrelevant as it might seem to a dumb white guy), so the black hiring official or panel member knows to ensure you get bonus points. Volunteered for La Raza march on Los Angeles last Cinqo de Mayo? Put it on your resume because sooner or later another member of La Raza will notice that bullet statement, and ensure you get taken care of.


Real-world example: what is the statistical likelihood of three new hires in a row having a black female as the top candidate for a $65k a year job? THREE...IN...A...ROW. Now what if I notice the hiring official was herself a black female. Aaaannnnndddd, I'm fired for noticing, if I'm dumb enough to say anything about it.

Blogger Doktor Jeep October 11, 2018 10:30 PM  

This is why I keep suggesting that the biggest baddest thing we can do to SJWs is create an open source Human Resources AI program that can be sold on the "lack of bias" as a means or protection from bias lawsuits. Put them out of work and with a program that will coldly and logically determine the best hires.

Blogger Jack Amok October 11, 2018 11:01 PM  

FYI, Federal workforce hiring is opposite land of what you guys are talking about.

Oh, yeah, I'm sure it is. I did mention "a real job at a real company that has to turn a profit" after all.

Blogger Azure Amaranthine October 12, 2018 3:00 AM  

"This is why I keep suggesting that the biggest baddest thing we can do to SJWs is create an open source Human Resources AI program that can be sold on the "lack of bias" as a means or protection from bias lawsuits. Put them out of work and with a program that will coldly and logically determine the best hires."

The problem is that you have to leave in the capability to determine the undesirables without letting the program use any discernible shortcuts that would permit an unfamiliar human mind to perceive the politically unacceptable patterns.

You'd need something like a damper on the system's efficacy, one that you could turn down at will, but with an appropriately labeled something like "danger, evil at play" warning on it. So, if anyone turns it off and unwillingly reaps the benefits, kicking and screaming all the way, you can tell them that it's their own fault for ignoring the warning.

Then let human nature take its newly catalyzed course.

Blogger CYGNUS FACETIOUS October 12, 2018 3:04 AM  

Lurker wrote: That logic thing is always going to be a problem for lefTARD programmers.
Even the AI Overlords will intuitively apprehend the order of operations for Creation: First Adam, then rib, then Eve.

Blogger mike October 12, 2018 4:12 AM  

Ding ding ding

Blogger justaguy October 12, 2018 3:56 PM  

#56 & 58:

Making the AI hire by objective meaningful criteria would still not meet the requirements that the US has placed on businesses. Any statistical divergence is automatically illegal. Bush I signed the quota bill in a vain attempt to get re-elected and the courts enforced quotas before then. (Duke Power case). Remember the DOL ruled that even making statements that are scientifically valid can be automatically discriminatory in the Google engineer case. Facts don't matter, we must have our diversity at all costs. Way back in the late 1970s, Harvard Law published a series of articles quotas in jobs and the resulting group justice and rationales for such thinking-- worth taking a look at to see some of the views.

Blogger Raker_T October 13, 2018 10:45 AM  

@56: Couldn't this go the way of infinite regress? That is, the software can't be sued, but whoever installed it can? Don't get me wrong, I like the idea, but implementation is key.

Post a Comment

Rules of the blog
Please do not comment as "Anonymous". Comments by "Anonymous" will be spammed.

<< Home

Newer Posts Older Posts