Tale of the left handed samurai

A few days ago I was asked to prepare a piece that would help people who have not really been exposed to thinking using data on how to go about doing it.

Since the presentation did not take place, I decided that rather than let the slides go to waste, I would share it here. Please note though that my presentation style is very spoken with just visual cues on the slides rather than full explanations, so do forgive the short comings.

Why left handed? Because many people believe that left-handed people are more creative, and I believe that analysis is a creative endeavour, at least analysis that creates and impact.

And why a samurai? Well I hope that by the end of this post, you should have your own answer.

Analysis/Analytics is a way of thinking

Everyone analyses things before making a decision, whether consciously or not. So this is nothing new.

In a business context however, it is worth thinking whether there are business implications to the idea, whether it may have some benefit. There is no point, in a business context, to do things just for the sake of doing it, RoI and so on.

Once the potential business implications have been cleared up, the next step is to decide upon a metric or set of metrics we would use to decide whether the idea works or not. The metrics are usually very closely linked to the business objective; if the idea we are testing is to do with increasing sales, then it makes more sense to use sales as a metric, or sales growth rather than employee satisfaction for example.

Once the metric is chosen, or a list in order of preference and including some proxies if the actual preferred metric would not be available, then the next step is to look for the data needed for analysis; this includes both history and granularity.

Next is the number crunching, based on the algorithms chosen.

And finally we reach a certain conclusion.

There are a couple of things worth pointing out.

1. The first step always has to be the business context; the assumption is that analytics is designed to help the business. This also means that there is need for some level of subject matter expertise, some creativity, at a minimum to translate the business issues into something solvable by analysis of data
2. I deliberately chose metrics, data, and algorithm before any analysis starts and in this order, We need to have a clean and objective view of the situation, pick the best, second best (... or more) approaches and tackle them in that order. It is not difficult to find a combination that gives us the answer we'd like to have, but that would not be real analysis, that would be fishing, not analysing. We let numbers tell their story, we do not torture them until they say what we want to hear.
3. While I have described the process, or at least one crank of the wheel as linear, it is not necessarily so. For example, if we had chosen to use year on year growth as our metric, but find out that we only have 6 months of history, then we should go back and evaluate whether we can have a good enough analysis leading to a conclusion if we use only 6 months of data, in which case we should amend the metric to monthly or quarterly sales growth for example.

But analysis cannot exist on its own.

Analysis is part of a process, cycle of having hypotheses, analysing and assessing their viability then reaching a conclusion.

The process can then be repeated so as to get better and better answers, either by testing different things or by refining ideas.

There are 2 points to bear in mind:

1. The key is the approach of experimentation; we cannot expect to have the perfect answer the first time, every time. Another way of looking at it is that "no" is a good answer too. It may dent our ego if the hypothesis was "dear to us", but in the spirit of experimentation, a conclusive "no" is as important as a conclusive "yes". And this brings us to the second point.
2. It is worth reiterating (I touched on it earlier), is to be as objective as possible, While a hypothesis may be "dear to us", they should all be analysed and evaluated objectively. I am not saying passion should be excluded, passion is great in hypothesis forming, but hypothesis testing ( even predicting) should be done with a cold heart and mind.

What about the process of discovery you could ask, just blindly letting the data tell its story? I think it is perfectly acceptable, but as we interpret the data, a story will be formed, whether externally (say from experience or the past) or as coming from the data, and this will lead to a hypothesis, and so the cycle begins.

It is also important to remember again, that analysis, at least in a business context, cannot be purely for the sake of analysis.

I keep saying it, but analysis should be done to help the business, and we should avoid analysis-paralysis.

If, as a result of the analysis, given our choice of metrics and methodology there is no conclusive answer, then we should review. For example, we may decide to rephrase the business issue, or decide to collect more data to allow for a more definitive answer.

If we have a conclusive answer then we can exit the process and go to a next step; remember there is nothing wrong with deciding that the analysis of data does not support the hypothesis, experimentation is about learning and moving on.

It may not be easy to switch to an experimentation driven methodology, but the rewards are worth it.

The ultimate aim of analysis is to take action; analysis, in a business context, only truly comes to life when there is a resulting action, the experiment is tested in real life, and to me that’s one of the most exciting times, when you really get to see whether your analysis has been accurate and how much it actually helps the business.

The other side of the coin is that if action is taken without proper analysis, then it can be likened to gambling. Of course people with experience can use this to help guide their actions, but what is experience if not an accumulation of data. The danger is that, given how more lasting/easier to recall memories in the human brain are usually associated with emotion, we may be remembering a distorted view which only an objective analysis may reveal. Also, situations change, and in fluid environments, data analysis is invaluable as a tool to guide decision making and a precursor to action.

But just taking action, is not enough; the result matters.

Finally, once action is taken and the experiment is run, it is critical to gather the results and compare them to what is expected and learn whether things went as expected.

Furthermore, analysis of results can lead to new hypotheses, refinement of hypotheses and kick starts a virtuous cycle of improvement.

Still, why the Samurai?

Because it is very important to have the right attitude when analysing the data and the may be romanticised image of a Samurai as someone who is zen-like but decisive helps. It is important, when analysing data, to be able to put everything aside and focus in what the data is saying.

So who wants to be a left-handed Samurai?