A few days ago I was asked to prepare a piece that would
help people who have not really been exposed to thinking using data on how to
go about doing it.
Since the presentation did not take place, I decided that
rather than let the slides go to waste, I would share it here. Please note
though that my presentation style is very spoken with just visual cues on the
slides rather than full explanations, so do forgive the short comings.
Why left handed? Because many people believe that
left-handed people are more creative, and I believe that analysis is a creative
endeavour, at least analysis that creates and impact.
And why a samurai? Well I hope that by the end of
this post, you should have your own answer.
Analysis/Analytics is
a way of thinking
Everyone analyses things before making a decision, whether
consciously or not. So this is nothing new.
In a business context however, it is worth thinking whether
there are business implications to the idea, whether it may have some benefit.
There is no point, in a business context, to do things just for the sake of
doing it, RoI and so on.
Once the potential business implications have been cleared
up, the next step is to decide upon a metric or set of metrics we would use to
decide whether the idea works or not. The metrics are usually very closely
linked to the business objective; if the idea we are testing is to do with
increasing sales, then it makes more sense to use sales as a metric, or sales
growth rather than employee satisfaction for example.
Once the metric is chosen, or a list in order of preference
and including some proxies if the actual preferred metric would not be
available, then the next step is to look for the data needed for analysis; this
includes both history and granularity.
Next is the number crunching, based on the algorithms
chosen.
And finally we reach a certain conclusion.
There are a couple of things worth pointing out.
- The first step always has to be the business context; the assumption is that analytics is designed to help the business. This also means that there is need for some level of subject matter expertise, some creativity, at a minimum to translate the business issues into something solvable by analysis of data
- I deliberately chose metrics, data, and algorithm before any analysis starts and in this order, We need to have a clean and objective view of the situation, pick the best, second best (... or more) approaches and tackle them in that order. It is not difficult to find a combination that gives us the answer we'd like to have, but that would not be real analysis, that would be fishing, not analysing. We let numbers tell their story, we do not torture them until they say what we want to hear.
- While I have described the process, or at least one crank of the wheel as linear, it is not necessarily so. For example, if we had chosen to use year on year growth as our metric, but find out that we only have 6 months of history, then we should go back and evaluate whether we can have a good enough analysis leading to a conclusion if we use only 6 months of data, in which case we should amend the metric to monthly or quarterly sales growth for example.
But analysis cannot
exist on its own.
Analysis is part of a process, cycle of having hypotheses,
analysing and assessing their viability then reaching a conclusion.
The process can then be repeated so as to get better and
better answers, either by testing different things or by refining ideas.
There are 2 points to bear in mind:
- The key is the approach of experimentation; we cannot expect to have the perfect answer the first time, every time. Another way of looking at it is that "no" is a good answer too. It may dent our ego if the hypothesis was "dear to us", but in the spirit of experimentation, a conclusive "no" is as important as a conclusive "yes". And this brings us to the second point.
- It is worth reiterating (I touched on it earlier), is to be as objective as possible, While a hypothesis may be "dear to us", they should all be analysed and evaluated objectively. I am not saying passion should be excluded, passion is great in hypothesis forming, but hypothesis testing ( even predicting) should be done with a cold heart and mind.
What about the process of
discovery you could ask, just blindly letting the data tell its story? I think
it is perfectly acceptable, but as we interpret the data, a story will be
formed, whether externally (say from experience or the past) or as coming from
the data, and this will lead to a hypothesis, and so the cycle begins.
It is also important to remember
again, that analysis, at least in a
business context, cannot be purely for the sake of analysis.
I keep saying it, but analysis
should be done to help the business, and we should avoid analysis-paralysis.
If, as a result of the analysis,
given our choice of metrics and methodology there is no conclusive answer, then
we should review. For example, we may decide to rephrase the business issue, or
decide to collect more data to allow for a more definitive answer.
If we have a conclusive answer
then we can exit the process and go to a next step; remember there is nothing
wrong with deciding that the analysis of data does not support the hypothesis,
experimentation is about learning and moving on.
It may not be easy to switch to an experimentation driven methodology,
but the rewards are worth it.
The ultimate aim of analysis is to
take action; analysis, in a business context, only truly comes to life when
there is a resulting action, the experiment is tested in real life, and to me
that’s one of the most exciting times, when you really get to see whether your
analysis has been accurate and how much it actually helps the business.
The other side of the coin is that
if action is taken without proper analysis, then it can be likened to gambling.
Of course people with experience can use this to help guide their actions, but
what is experience if not an accumulation of data. The danger is that, given
how more lasting/easier to recall memories in the human brain are usually
associated with emotion, we may be remembering a distorted view which only an
objective analysis may reveal. Also, situations change, and in fluid
environments, data analysis is invaluable as a tool to guide decision making
and a precursor to action.
But just taking action, is not enough; the result matters.
Furthermore, analysis of results
can lead to new hypotheses, refinement of hypotheses and kick starts a virtuous
cycle of improvement.
Still, why the Samurai?
Because it is very important to
have the right attitude when analysing the data and the may be romanticised
image of a Samurai as someone who is zen-like but decisive helps. It is
important, when analysing data, to be able to put everything aside and focus in
what the data is saying.
