This is one of my 3 dogs, she has
taught me an invaluable lesson that reinforces my preference for non-"black-box"
types of models/algorithms.
Each one of my dogs has her own
personality and behavioural patterns, the brown one (shown above) likes to pee after a meal,
the white one before a meal; the black one has no set patterns I can detect,
but she takes her time to eat. So if I am not in the vicinity of the girls when
they are eating, the white one will aggressively try to steal the black one’s
food.
That allows the brown one the
freedom to pee where she chooses. And often she chooses to pee where she knows
she is not allowed to (the no-pee zone). The other two have learnt, through
positive and negative reinforcement where they are and are not allowed to pee.
However, the brown one figured
out that, since she only gets punished for peeing in the no-pee zone if she
gets caught, the solution is not “do not pee here”, but “do not to get caught
peeing here”.
Hence, nowadays, she often
chooses to have her post-meal pee behind a low wall (in the no-pee zone), I
can’t see her doing it, and therefore can’t catch her in the act and punish her.
What does that have to do with
models/algorithms being explainable?
I see my dog as a brown box. She
received data:
1 the experiments when she had a
pee in the right places and was rewarded,
2 the experiments where she had a
pee where she was not allowed and was punished, and
3 the experiments when I was not
there to see her, and she was neither punished nor rewarded.
And from this data her algorithm came up with: “Do not get caught peeing in the wrong place”. That's obviously not what I expected.
How can I change that outcome?
The usual solution is to feed her
more data. More cases when she is rewarded or punished, basically I have to
watch her more often. Also, I could install cameras and fit her collar with
some device, so when she is found to be peeing where she shouldn’t, she is punished.
Basically I would build a model/algorithm to recognise the peeing position,
teach it where peeing is allowed and not, and run a rule whereby if peeing is
taking place in an area where it is not allowed, then punish the dog.
It can be done; it is expensive,
may be morally reprehensible, (and even worse if my model doesn’t recognise the
peeing position accurately enough confuse my dog), but in any case, I can’t be
sure what the brown box will come up with. May be she’ll find the blind spot of
the camera...
I’m not the only one who’d try to
feed more experiments to the box. This is what google did after their
self-driven car crashed earlier this year. They could not know for sure what
caused the crash, assumed it was because the car expected the incoming large
vehicle to slow down so as to allow the self driving vehicle to go past; it
didn’t. Hence more examples of large vehicles not slowing down were fed to the
self driving vehicle. The hope is that the issue was diagnosed correctly and
that the new data used to further train the machine would remedy the issue. But
we can’t know for sure, since we do not know what decisions were actually
taken, it’s hope: "From now on, our cars will more deeply
understand that buses (and other large vehicles) are less likely to yield to us
than other types of vehicles, and we
hope to handle situations like this more gracefully in the future." (highlight not in original text) (http://www.bbc.com/news/technology-35692845)
Basically you can’t
expect what issues might occur, you just have to deal with the consequences,
and try to train the algorithm again. But isn't it better to know before things go awry? Knowing what's inside would help.
Other issues with "black box" type models
Another issue I have
with “black box” type models, is that since you don;t know what drives the
algorithm, you cannot anticipate the reaction to a change; the parameter that
changes may have no impact on the algorithm, or might have a huge impact,
unless it’s something that has happened in the past, it is very hard to know
what will happen. Contrast this with a model with clearly defined drivers and
coefficients; the model might become less accurate, but the general impact can
be ascertained in advance, and prepared for. Or you could actually change a
parameter, in order to get a desirable outcome.
For example, let’s say that the
distance between my brown dog and myself is important in her decision to pee or
not (the closer I am the less likely she is to pee in the no-pee-zone), then I could influence her
behaviour by moving closer to her. But the brown box doesn’t reveal her
decision making mechanism, nor the drivers of her behaviour.
A third reason is that
when algorithms leave the ‘lab’ and encounter the real world, there are many
more factors to deal with. And, especially if you are trying to use your model
to select people to receive a specific offer, it makes sense to positively
influence the delivery of the message. I saw this for real in one of my
previous roles. We had rolled out a model but found that the response, while
very good, was still below expectations. At that time, the outcome of the model
was rolled out simply via a set of customers each call centre agent would call,
given a script. The change we made was to actually brief the people doing the
selling about the main drivers of the model, convince them that it made sense,
and their conviction helped convince the prospects and sales increased.
This third reason
doesn’t apply in my case, I just have to deal with the consequences of the
algorithm using a mop and pail.
No comments:
Post a Comment