Wednesday, May 21, 2014

Dun Mares and Black Stallions: Another Look at Applicant Tracking Systems

Allow me to start from afar. Take a look at the screenshot of a Google ad that kept popping up in my Gmail inbox. It appears that, having been sifting through thousands of my emails for eight or nine years now, the world's #1 search engine and online advertiser "thinks" I am a flirty plus-size female who likes knitwear, has anger issues, and may be a little slow in the head:
I guess I don't need to worry about my privacy.

On the other hand... a Taoist tale many of us are familiar with thanks to J. D. Salinger comes to mind (in case you don't remember, see below):
    Duke Mu of Chin said to Po Lo: "You are now advanced in years. Is there any member of your family whom I could employ to look for horses in your stead?" Po Lo replied: "A good horse can be picked out by its general build and appearance. But the superlative horse - one that raises no dust and leaves no tracks - is something evanescent and fleeting, elusive as thin air. The talents of my sons lie on a lower plane altogether; they can tell a good horse when they see one, but they cannot tell a superlative horse. I have a friend, however, one Chiu-fang Kao, a hawker of fuel and vegetables, who in things appertaining to horses is nowise my inferior. Pray see him." Duke Mu did so, and subsequently dispatched him on the quest for a steed. Three months later, he returned with the news that he had found one. "It is now in Shach'iu" he added. "What kind of a horse is it?" asked the Duke. "Oh, it is a dun-colored mare," was the reply. However, someone being sent to fetch it, the animal turned out to be a coal-black stallion! Much displeased, the Duke sent for Po Lo. "That friend of yours," he said, "whom I commissioned to look for a horse, has made a fine mess of it. Why, he cannot even distinguish a beast's color or sex! What on earth can he know about horses?" Po Lo heaved a sigh of satisfaction. "Has he really got as far as that?" he cried. "Ah, then he is worth ten thousand of me put together. There is no comparison between us. What Kao keeps in view is the spiritual mechanism. In making sure of the essential, he forgets the homely details; intent on the inward qualities, he loses sight of the external. He sees what he wants to see, and not what he does not want to see. He looks at the things he ought to look at, and neglects those that need not be looked at. So clever a judge of horses is Kao, that he has it in him to judge something better than horses." When the horse arrived, it turned out indeed to be a superlative animal.
      -- J. D. Salinger, Raise High the Roof Beam, Carpenters

Joking aside, what I am trying to say - yet again - is that computers, although impressively fast, are still disappointingly dumb, and that they perform very poorly when it comes to processing human language, which is inherently ambiguous. To make things even harder for computers, humans often make it even more ambiguous.

In fact, in June 2011, having introduced Schema.org (initiated by Google), the world's four leading search engines pretty much admitted that they do not have production-ready technology capable of "understanding" what a web page really means and tried to off-load part of the job onto webmasters (humans) by encouraging them to add semantic markup to their web pages. I have written about it before, so I am not going to go into detail here again.

So much for the introduction.