How governance should take advantage of technology/"Big Data" and stop making excuses.

How governance should take advantage of technology/"Big Data" and stop making excuses.
( Why the 30000foot defense shouldn't be allowed in this world of "Big Data")

“Big Data” has been touted as the panacea, personally I think it is important to understand what is worth doing and what is not, to discuss what we should allow to be done, and what not, and be allowed to change our minds as time goes by as we see the consequences of our decisions.

One topic where, I believe, “Big Data” is very useful is in that of governance. 

The amount of data that is captured within an organization, together with the technologies that democratise the ability to extract or pursue meaning in a mass of data should make it impossible for a governance body to claim the 30000 foot defense: “we are too high up to have known”. No, sorry; you should have known (especially when you are paid very handsomely to simply ensure governance).

What can big data and appropriate visualisations allow you to see? Everything down to the minute details should you choose to:

With text mining you can scan through megabytes of documents and instantly get the gist of it all, allowing you to query further (yes, you can have ‘experts’ to do the querying if you want, but you cannot say you had no idea). For example the summarization of the Senate report on torture:

All you have to do when you see something you don’t find kosher is to ask: “tell me more about this”, and you drill down further.

Or if you want to understand how people’s or organisations’ behaviour varied, you could visually see how they differ (the example refers to an earlier piece on whiskies rather than people or organisations, but the idea is the same, comparing attributes across individuals):

And with some mouse work:

Ask “how does Mr. GlenFiddich differ from the rest?”.

Or what do the relationships between my clients/relationship managers look like?

And what is this interesting relationship in the centre?

In sum, with “Big Data”, there’s very little going on, especially within an organization, that can be hidden from people whose mandate is to ensure everything goes according to expected standards and values; unusual behaviour leaves traces that are so visible; you don’t need to be a rocket scientist to uncover them.

This blog post was inspired by: “http://www.theguardian.com/commentisfree/2015/mar/13/bankers-hsbc-rona-fairhead-firewall”