Recently a conversation with a
client caused me to re-think this situation. Yes, I fundamentally believe
analytics/”Data science” is a comparative advantage an organisation can
possess. However this need not mean an FTE (Full Time Employee) army, or does it?
As in most such questions, the
answer is… "it depends".
To make things simple, I’ll
consider 3 points of view:
A "Data Scientist"'s point of view
B Hiring Manager's point of view
C Organisation's point of view
Of course the ideal situation is
where all 3 parties’ interests are in agreement.
I simplify the issue by measuring
the status on 2 axes:
X. Number of “data science” projects undertaken in
a year
This is an
indication of how often the skills of the “data scientist” are being used. This
had to be projects that go beyond BI, need to have a predictive component. A
“data scientist” is an expensive resource and it is a waste from all parties
point of view to utilise a “data scientist” to build dashboards most of the
time for example. There are other people much more skilled at this than “data
scientists”.
Y. Variety of projects undertaken in a year
The variety of
projects is an indication of how far down the path of utilising “data science”
or how broad the adoption or experimentation with “data science” is an
organisation.
These 2 axes can be thought as
components of analytical maturity – necessary but not sufficient conditions (I
will discuss about analytical maturity in a subsequent blog). The more “data
science’ projects you undertake in a year, the more likely you are to be using
them. More importantly, the more varied they type of problems you are trying to
solve using “data science”, the more likely it is that the adoption of “data
science’ or the attempts at adoption permeate the organisation.
However, this looks at only the
production side of things, not the consumption. There are many organisations
out there who adopt a “if you build it…” and end up with white elephants.
At the end of this blog, I’ll
describe an Occam’s Razor, but for now let’s look at things a bit more
management school style.
A “Data
Scientist”'s Point of view
The bottom right is where the
“Data Scientist” is kept busy on projects that require his/her skills and
knowledge, but these projects are repetitive. This can lead to boredom, and the
palliative situation is to build strong feedback loops to keep improving and
challenging the “Data Scientist”. Else it might make sense to rotate “data
scientists’ and bring in new pairs of eyes to try and bring quantum
improvements rather than continual polishing.
The top left is where the “Data
scientist” gets a variety of projects but is under-utilised, a stop-start
adventure. This is likely the case of an organisation who is starting with data
science operationalisation, or is not mature enough to exploit “data science”
fully. In this case, it might be better to hire specialist “data scientist” on
a as-needed basis. This would be much better use of resources and make better
use of “data scientists’” skills.
The bottom left is where no “data
scientist” would want to tread; apparently he/she did not ask the right
questions in the interview.
B Hiring Manager's Point of View:
I was going to write an analysis
of a hiring manager’s point of view, but I think it is enough to say that if
they are not aligned to that of the organisation (principal agent problem) then
it doesn’t matter what is really going on, all that matters is the impression
you can give; it is an ego or resume padding trip and no reasoning can be
applied.
C Organisation’s
point of view
The top-right is the sweet spot
for the organisation; the “data scientist” is involved in many projects (well
utilised) and a variety of projects (organisational analytical maturity),
chances are organisations in this quadrant are able to generate sufficient RoI (Returns on Investment) from their “data scientist”. But does that mean that organisations in this
quadrant should not hire data scientists on gigs? The answer depends what the
“data scientist” can bring.
The bottom right is where the
“data scientist” keeps doing the same things. Basically, at every iteration of
a model, unless there have been structural changes, there will be incremental
improvements. The question is whether this generates sufficient RoI for an FTE.
One of the questions clients
(both when I was FTE and on gigs) often ask me is when they know a model needs
to be relooked at. My usual answer is all to do with metrics. In the same way
as I insist of proper performance metrics of models to be agreed at the start
of a project (to ensure RoI), I also encourage clients to have acceptable
variability in results. It’s a bit like having a point estimate and a band of
acceptable intervals (CI). So only if the results degrade below an acceptable
level would it be worth looking at (you cannot assume that the degradation is
due to random fluctuations and the impact on returns is too negative).
So chances are, unless each
application of the model generates huge returns, it might make more sense not
to have a “data scientist” on the payroll but to use a “data scientist” on gigs
(1).
The top left is where the “data
scientist” is engaged on a variety of projects but not many projects. This is a
clear case that a “data scientist” by gig is better. Not only do you utilise
resources only when you need them, but you can ensure you get the best resource
for the project.
The bottom left shows and
organisation that has no use for a full time data scientist, but should explore
“data science” via gigs.
Conclusion:
It is quite clear that the best
case scenario for both the “data scientist” and the organisation is when the “Data scientist” is fully
engaged, building a variety of models/algorithms to solve different business
use cases and generating RoI.
In the rest of the cases, the
situation is unlikely to last. In cases where there is either variety or
quantity but not both, the “data scientist” is likely to feel lack of growth
and leave, causing the organisation to go through the expensive hiring process
again and again, further impacting RoI.
In the case when there is neither
quantity nor variety, there is no point engaging a full time “data scientist”.
This is a very clear-cut case for having “data scientists” on a gig basis. The
approach should be one of proof of concept, prove the RoI that can be obtained,
using “data scientists” on gigs; not only is the cost overall lower, but the
organisation can get specialists.
Simlarly, when there is variety
but lack of quantity, “data scientists” on gigs offer the possibility of
specialist help, and help only when needed. When you do not have projects
requiring skills of a “data scientist” then why pay someone?
When there is no variety, the
challenge is different. It is likely that this is the case when the analytical development
of the organisation has stalled; for example, “data science” is used only in
one area of the organisation, hence a lack of variety. Here again it might make
more sense to only get “data scientists” on gig basis, to try and expand the
variety and increase RoI by opening new avenues for returns. Also, another way
of increasing RoI in this case if to hire “data scientists” for prototyping,
but leaving maintenance to less expensive resources (2).
Finally, does that mean that if
you have both variety and quantity you do not need “data scientists” on gigs?
Well, it depends how well your RoI is doing. In the corporate world, where
performance has to increase over time, using specialist help for prototyping,
having a “different set of eyes” looking at business issues can be a solution.
Of course you may choose to rotate your “data scientists” to ensure that
freshness, but if could come at the cost of returns.
In a nutshell:
In a nutshell it all depends on
the RoI. When I first took up a contract, it was very exciting as someone in
the analytics field. My headcount was directly funded by the business and I had
to justify my existence every year, I had specific RoI targets. That made me so
alive. I guess that’s why I am believe strongly in RoI.
Apart of vanity and bragging
rights on the part of management, why would an organisation spend on a resource
that is generating low or no RoI?
Organisations have to measure the
returns on their investments; that includes human capital. As long as the RoI
is met, then hiring a FTE “data scientist” makes sense. Else, at the least,
until the organisation matures enough to be able to allow the “data scientist”
to generate that RoI, it should stick to “data scientists” on gigs.
1 There have been cases when
clients want to put data scientists on a gig but also pay a retainer. This is
sort of a compromise between the 2 models. However, retainers may not work.
From the organisations perspective there will be an incentive to use the “data
scientist” for non-“data science” work.
2 Another question I often get is
“when do I need to review my models?”. I believe that the model metrics should
not be decided just for a one time use, but also for on-going performance. For
example, a targeted response rate of 15%, where review will take place if the
response rate dips below say 12% for 2 consecutive runs.
No comments:
Post a Comment