Why Are We Suddenly So Bad At Predicting the Future?

Imagine that a monkey got into the control room of the universe and spent the year clicking random buttons. Imagine him hopping about on the ‘one musician less’ button, stomping on the ‘auto-destruct’ lever and gurgling while he thumped repeatedly on the ‘introduce chaos’ switch. Close your eyes and picture him dropping a giant poo on the bright red panel marked ‘do not touch under any circumstances’. That my friends is the only way to think about 2016 – after all, it was the year of the monkey in the Chinese zodiac. It was the year when rational thinking took a beating, when meritocracy became a bad word, when liberalism escaped from the battlefield to a cave in the mountains to lick its wounds. And not surprisingly, a year when projections, predictions and polls made as much sense in the real world as an episode of Game of Thrones on steroids.
Given much of our lives are spent in productively engaging with the future and making decisions based on big and small decisions about the possible future, this last point is more important than just the schadenfreude of laughing at pollsters and would be intellectuals. The present passes too quickly, so really every decision you’ve ever made in your life is counting on future events to turn out in ways that are favourable. Getting this wrong is therefore injurious to health, to put it mildly. And yet our ability to predict the future has never been under such a cloud in living memory. Why is this so?

Fundamentally, we’re wired to think linearly in time, space and even line of sight. We are taught compound interest but we get it intellectually rather than viscerally. When you first encounter the classic rice grains and chessboard problem, as a smart person, you know that it’ll be a big number, but hand on heart, can you say you got the order of magnitude right? i.e. the total amount of rice on the chessboard would be 10x the world’s rice production of 2010? Approximately 461,168,602,000 metric tons? This problem of compounding of effects is incredibly hard to truly appreciate, even before you start to factor in all the myriad issues that will bump the rate of change up or down, or when the curve hits a point of inflexion. The Bill Gates quote  – ‘we over-estimate the impact of technology in 2 years, and under-estimate the impact over 10’ – is a direct reframing of this inability to think in a compound manner.

Then there’s the matter of space and line of sight. The way the future unfolds is dramatically shaped by network effects. The progress of an idea depends on it’s cross fertilisation across fields, geographies and disciplines, across any number of people, networks and collaborations. These collaborations can be engineered to a point or are the result of fortuitous clustering of minds. In his book ‘Linked’ – Ablert-Lazlo Barabasi talks about the mathematician Erdos who spent his life nomadically, travelling from one associates’ home to another discussing mathematics and ironically, network theory. Not surprisingly, a lifestyle also practiced for many years by a young Bob Dylan, if you substitute mathematics for music. Or consider the story of the serial entrepreneur in Rhineland in the 1400s, as told by Steven Johnson, in ‘Where Good Ideas Come From’. Having failed with a business in mirrors, he was working in the wine industry, where the mechanical pressing of grapes had transformed the economics of winemaking. He took the wine press, and married it with a Chinese invention – movable type, to create the worlds first printing press. His name of course, was Johannes Gutenberg. This kind of leap is not easy to predict, not just for the kind of discontinuity they represent (more on that later), but also because of these networked effects. Our education system blinkers us into compartmentalised thinking which stays with us through our lives. Long ago, a student of my mothers once answered a question about the boiling point of water by saying “in Chemistry, it’s a 100 degrees Centigrade, but in Physics, I’m not sure”. We are trained to be specialists, becoming more and more narrow as we progress through our academic career, ending up more or less as stereotypes of our profession. Yet human progress is driven by thousands of these networked, collaborative, and often serendipitous examples. And we live in a world today with ever expanding connections, so it’s not surprising that we have fallen behind significantly in our ability to understand how the network effects play out.

If you want to study the way we typically make predictions, you should look no further than sport. In the UK, football is a year round sport, so there are games every weekend for 9 months and also mid week for half the year. And with gambling being legal, there is an entire industry around football gambling. Yet, the average punter, fan or journalist makes predictions which are at best wilfully lazy. There is an apocryphal story about our two favourite fictitious sardars – Santa Singh and Banta Singh, who decide to fly a plane. Santa, the pilot, asks Banta, the co-pilot to check if the indicators are working. Banta looks out over the wing and says “yes they are, no they aren’t, yes they are, no they aren’t…” – this is how a lot of predictions are made in the world of premier league football today. Any team that loses 3 games is immediately in a ‘crisis’ while a team that wins a couple of games are deemed to be on their way to glory. Alan Hansen, an otherwise insightful pundit and former great player, will always be remembered for his one comment “You can’t win anything with Kids” – which he made after watching a young Manchester United side lose to Aston Villa in the 1995-96 season. Manchester United of course went on to win the season and dominate the league for the next decade and a half. Nobody predicted a Leicester City win in 2016 of course, but win they did. The continuous and vertiginous increase in TV income for football clubs has led to a relatively more equal playing field when it comes to global scouting networks, so a great player can pop up in any team and surprise the league. Yet we find it hard to ignore all the underlying trends and often find ourselves guilty of treating incidents as trends.

The opposite, is amazingly, also true. We are so caught up with trends that we don’t factor in the kinks in the curve. Or to use Steve Jobs’ phrase – the ding in the universe. You can say that an iPhone like device was sure to come along sooner or later. But given the state of the market – with Nokia’s dominance and 40% global market share, you would have bet your house on Nokia producing the next breakthrough device eventually. Nobody saw the iPhone coming, but when it did it created a discontinuous change that rippled across almost every industry over the next decade. The thing is, we like trends. Trends are rational and they form a kind of reassuring continuity so that events can fit our narratives, which in turn reaffirm our world view. And unless we’re close to the event, or perennial change seekers and nomads ourselves, it’s hard to think of countercyclical events. It’s now easy to see how in 2016 we were so caught up in the narrative of progressive liberalisation and unstoppable path to globalisation, we failed to spot those counter-cyclical events and cues that were right there in our path.

In fact there are any number of cognitive biases we are guilty of – on an everyday basis. This article just lists a dozen of them. My favourites in this list are the confirmation bias and the negativity bias. Both of these are exacerbated by social media and digital media. While social media has led us to the echo-chambers – the hallmarks of 2016, our projection bias is also accentuated by our ability to choose any media we want to consume, in the digital world, where access is the easy part. Similarly, bad news spreads faster on social networks and digital media today than at any time before in history. Is it possible that despite knowing and guarding against these biases in the past, we’ve been caught out by the spikes in the impact and incidence of a couple of these, in the digital environment we live in today?
To be fair, not everybody got everything wrong. Plenty of people I know called the Donald Trump victory early in the game. And amongst others, John Batelle got more than his share of predictions right. There is no reason to believe that 2017 will be any less volatile or unpredictable than 2016, but will our ability to deal with that volatility improve? One of the more cynical tricks of the prediction game is to make lots of predictions at many different occasions. People won’t remember all your bad calls, but you can pick out the ones you got right, at leisure! This is your chance, then, to make your predictions for 2017. Be bold, be counter-cyclical. And shout it out! Don’t be demure. The monkey is history, after all. This is the year of the rooster!

Enterprise Social: Your Future Neural Network

If you’re reading this in the last months of 2014, it is likely that you belong to a company which has pondered enterprise-social platforms. And it’s likely that many head scratching moments have occurred and that the answers so far fall into broadly two camps.
External use and mining of social media is now well understood as a means to build customer engagement and feedback. Plenty of people still get this wrong but tools like Clarabridge are strong players in this area.
The more perplexing piece is the use of social platforms inside the business and here, most businesses have explored rolling out Yammer or Chatter and are often wondering ‘what next’? Many are actually struggling to justify what it is that they are getting out of this?
In the rest of this piece, I’d like to suggest that the Enterprise Social Platform is an idea whose time has come and that in 2015 we will all be flipping to the new model of enterprise social.
First though, let me highlight some of the problems of communications inside large organisations:
Information flows in big companies typically represent hierarchies. So information typically flows much more smoothly up and down hierarchies and quite poorly across these lines. This is just a truism of corporate cultures and represents a significant inefficiency.
Second, reward systems in most companies are typically not aligned to avowed corporate values. Many companies try to enshrine values such as collaboration into their work culture but have no means in place to recognise or reward such behaviour. Consquently, collaboration, or other such values, are often a road-kill in the stampede of those behaviours which are actually rewarded – often to do with sales targets and hard numbers. This is arguably detrimental to the business in the longer run, and this is specifically the problem that WorkAngel, one of the start-ups in this space is seeking to solve.
And how exactly do we measure the internal transaction costs of doing business? For example to create a proposal or to deliver a project, you might need to work with up to a dozen people from different teams and departments. Often the two kinds of transaction costs you encounter are (a) time to locate the right person and (b) the time and effort-cost of the transaction, given that the said person may have no idea about who you are and may have no motivation to support you. Again, start-ups such as ProFinda and WorkAngel look to solve these everyday problems.
Another challenge you face as you grow larger and more global, is the tendency of localised problem solving. Would you even know if the problem you’re trying to solve (say, a branch specific problem in a Bank or a retail outlet), has been solved somewhere else in the organisation? Or that the expertise exists elsewhere? Also known as the re-inventing the wheel syndrome.
Many people have written in recent times about the role of culture – often more important than strategy as a critical contributor to success of an organisation. Where does your culture reside? Where is it rooted? How do you create a crucible for capturing this?
Even more so when you go through mergers or acquisitions and you are trying to quickly forge a new culture, and all the earlier challenges reappear in starker ways.
And what about all the soft information that you increasingly need to hold and manage but have no repository of? Somebody in the team worked as a volunteer at a hospital and has the insight which could make a big difference to your healthcare project. But it’s not in her CV, or in your corporate systems. And all the conversations between people that hold ideas and sparks for new products. And that brainstorming session that could have been a product, but everybody got busy with a bunch of other projects? Where is all of that intellectual capital?
And a last but important challenge: we are today in the world of bring your own self. This is a term I first heard used by Ade McCormack while we were discussing BYOD. But the idea is ubiquitous – increasingly we are blurring, in positive ways, the line between personal and professional. We no longer have to leave our personalities and hobbies at home because increasingly those can represent pools of capabilities or knowledge that can add value to our work and to our businesses. We are being encouraged to bring our whole selves to work. And our information systems are inadequate to reflect and represent our whole selves.
So Where Do We Go From Here?
We know that we’re moving from systems of record to systems of engagement in most businesses. Implicit in this is a move from structured to unstructured information. And from data and transaction centric to relationship and conversation centric information. It also implies the move from desktop to mobile and from process based to context based information. John McCarthy and his colleagues at Forrester have a report on this.
Seen in this context the enterprise social platform is that new and complementary layer which holds this new conversation centric, context based, relation and human oriented information which seeks to address the problems we outlined earlier. There are two key words here. (a) Complementary – nobody is suggesting we junk our transactional and structured data in favour of the warm and fuzzy. and (b) seeks – there is no guarantee. A lot will still depend on that pesky notion of implementation.
Social platforms are inherently non-hierarchical. Although there are some like Loyakk, which are designed around certain types of hierarchies, most are designed to engineer those serendipitous or interest based discussions, or perhaps around specific problem domains which are independent of hierarchy. On a social platform it’s not just the HIPPO who get’s his/her way. (Highest Paid Person in the Organisation). So cross hierarchical and also cross departmental conversations become quite common on social platforms. I have found Yammer perfectly good for this kind of discussion. For example when I wanted to know how to create an internal enterprise blog, I had 3 good answers in about 30 mins of posting a 1 line question.
I’m excited talking to next generations social platforms, such as ProFinda and WorkAngel because they go to the core of the social challenge outlined above. WorkAngel, for example has a mechanism for explicitly identifying and rewarding those behaviours which align with the avowed organisational values, through a social feedback mechanism. What’s more the rewards can be quick and even financial. ProFinda’s secret sauce is the algorithm that goes to work on all your data both structured and unstructured, internal and external, to create relationship patterns which tradition tools don’t or can’t get. Both of these are softwares you license and install, so we mustn’t confuse Social Technologies with Social Media. Both of these have strong mobile platforms and analytics, so critical to today’s needs. As does Broadvision, with it’s Vmoso mobile solution which sits on top of the Clearvale social platform.
Beyond Platforms To Results
Needless to say, the best platforms will not be good enough to deliver success if you just treat them as the answer, by themselves. Some of the key, and often obvious guidelines:
It’s always useful to have clear and measurable business objectives. Following a discussion with one of our clients who wanted to implement a well known social tool, but didn’t have a clear business case, we drew up this mind map. The idea was simply to identify the different types of problems that could be solved by the tool in question, from providing support on HR issues or driving cross functional teams.
I would urge taking this further till you can see a clear path to money – either money earned or saved. You may not get clear answers but it will still provide clarity of thinking and enable you to get the right data going forward.
What we should avoid, is what the authors Macdonald and Bradley, in their book “The Social Organisation”, call ‘provide and pray’. Don’t just invest in the platform and hope some good will come of it.
But Can’t We Experiment With New Models?
Does this sound contradictory with the notion that with new technologies, you don’t always have a clear business case, and you need to try stuff? The key here, I believe, is that if you’re trying it out, it should be (a) a controlled experiment and (b) with specific learning objectives. So the learning becomes the business objective and backed by an understanding of the metrics of that learning.
Things To Avoid
(1) This is not an IT solution
(2) The product/ platform is not the answer by itself
(3) Avoid buzzwords like ‘gamification’ if they are not delivering your business objective.
Some Other Issues To Think About:
Can social experiences truly be engineered? Probably not a hundred percent. But they can be curated. And the right platform, with the right implementation underpinned by clarity of goals, will go a long way in getting you there.
Can true collaborations across hierarchies actually take place in large companies? Probably less so in high power-distance cultures. How you enable the voice of the junior most people in the business is definitely a part of the answer.
What about the work/ non-work balance?
This goes back to the point earlier about bring your own self. This is going to be one of the key debates as we move away from traditional office environments with clear time and place separation of work and personal. New mores and guidelines will evolve. It is entirely possible that people in err in both directions. This space needs watching and again, it won’t harm to gather data which can help to create those guiding principles.
In Conclusion:
Unless there is a specific need for point to point communications between people who need it, the publish subscribe model is a far more efficient way of communicating. Facebook has made us comfortable with the notion of a feed and thanks to social media platforms, we are all able to post, follow, engage and respond as we feel best. No education is required here.
If done right, the social layer will be the neural network of your enterprise. Not getting it right, though, will lock your business into the past and history is very unkind to anybody who doesn’t evolve.

Surviving The Information Age

I was recently asked (as I often am), “what devices will we use in x years?” Usually, in this kind of question,  x can be 2 years, 5 years or 10 years, depending on the ambition of the question? This is often a cue for animated debates on Google Glasses, Apple Watches and the next big thing in wearable computing and whether you would want a phone chip installed in your ear. This kind of argument, whichever flavour of device your rooting for, misses the point completely about the real challenge facing us – how to survive the information age. In fact, to stretch a point, this is like being faced with an energy crisis and arguing whether the batteries we use should be square or round. 
I am personally petrified of how ill equipped I am in dealing with this information driven era we are increasingly finding ourselves in. By all apparent measures, you would think I’m reasonably§ information savvy. All my files are on dropbox and accessible over the cloud. My phones are always backed up. My photographs are on Flickr, Picassa or on Apple’s photostream. My music is on Spotify or on iTunes. I use Google docs extensively to collaborate professionally. I maintain 4 different levels of passwords to keep my data safe. And yet, I would give myself a 3 out of 10 in terms of being ready for the information world. 
In my last blog I touched about the problem of “Dom’s MacBook in Iran“. That was just one example. Professor Gerd Kortuem of the Lancaster University’s High Wire program spoke at a session I attended a few years ago about an example where they added little meters to the drills used by roadworks teams, in order to measure the levels of vibration and to alert the supervisor if it was above safety limits. This created a huge problem for supervisors, as they now had new information they needed to act on. Earlier, they just took a gut call and that was it. Now they had to review the information and decide what to do if the readings were too high. Stop work? Look for alternatives? Inform the office? Increasingly, we find ourselves in possession of information which actually creates new challenges for decision making.  
On the other hand, this superabundance of information creates a responsibility of it’s own. It is an act of negligence today to not do the “due diligence” on any important decision. Whether it’s researching a hotel you want to book for a holiday, or perhaps the person you are going to meet. Whether you are unwell and need to know about the symptoms and possible causes, or whether you’re checking nutritional values of the things you’re buying at the super market, small and big decisions are now made much easier by information availability. The bottom line is you can make better decisions and a series of better decisions should lead to a higher quality of life and work. 
Which brings us to the first of my list of three key survival skills, which collectively explain why all of us aren’t yet equally good at handling and using this information. I’m talking about the ability to find information effectively. How to use Google (or any other) search effectively. How to find information on lean thinking without being flooded by results for lean meat. Information search skills should be taught in primary school. 
A personal peeve of mine is the phrase “new research shows…” a term often heard in television news programs. Usually it’s accompanied by fresh perspectives on whether something is or isn’t good for health, and runs contrary to previously held ideas. However, very rarely are we told the source of the data, and more critically the source of the sponsorship of the research. We know well that it’s easy for interest groups to “create” research with favourable outcomes. So if research suggesting that broccoli can cure the common cold is backed by the Broccoli Growers Association, you would do well to dig deeper into the evidence. Mostly this kind of “newsworthy” research suffers from the sponsorship bias or often just the news bias – the need to make a story. The recent story in the BBC about surviving on £ 1 per day is riddled with palpably bad research and poor homework,  as well documented herebut it made for a good story. Of course if you aren’t up to date with the Daily Mail’s ongoing obsession with things that cause and/or cure Cancer, you can get a quick summary here.
The second information survival skill therefore, that every 10 year old should know today, is to be able to validate the source of data. It’s likely that you can get a slew of answers to almost any question, on the internet. But which one do you trust and how do you establish the quality of the source? Or remove sponsorship bias? Equally, when the fall back option for most people is Wikipedia, it’s important to note what is and isn’t best crowdsourced. To put it simply, in a quiz show, if you were asked a question about a character on a soap, it’s a good question for an audience poll. Not so much if the question refers to, say, the isotopes of Neon. 
Sooner or later, that pesky question pops up again – what is information, and how does it differ from data? My favourite answer is context. Let’s take 2 people – Mary and Max. Max is navigating his way through a jungle, with no access to provisions except what he can find and eat in the jungle. Mary is playing football in a tournament and about to take a penalty. Both are given 2 pieces of data each. First, that most goalkeepers tend to dive to the left or right, so statistically, hitting a penalty straight down the middle has the highest chance of scoring. Second, that if you dig a hole in a muddy area, it takes 20 minutes for the sediments to settle, and the water to become drinkable. Now, clearly for Max and Mary, one of these pieces of data is information. The other, irrelevant. To see the ludicrousness of information without context, see this Fry & Laurie sketch)
But Max and Mary might find themselves in each other’s shoes at some point of time.  Will they still retain the “irrelevant” information they were given? 
Which means that the third key survival skill is an ability to continuously build and reference your data gathering so that your personal library and signposts enable you to marry information to context all the time? Our education system was historically built to provide information you had to store in your head and use for the rest of your life. That has obviously changed, but do you have a reliable library system to replace it? Plenty of tools (such as bit.ly) for example enable tagging and marking of content and a combination of ubiquitous access and smart devices make this library always accessible. You are your own librarian. Pay attention to your filing system. 
To put this in the enterprise perspective, the role of all systems is to deliver the right information in the right context. Whether it’s customer data to a sales person, or risk information to a project manager. This is the bottom line. Once you strip away all the IT jargon and the systems-speak, this is the simple objective. Every time your business doesn’t deliver the relevant information at the point of decision making, it’s an area of improvement for information technology. 
But of course, you can only deliver the data you have, so data capture becomes the next challenge. The best source of enterprise data is transactions (and the worst are probably areas where employees are expected to make an extra effort just to provide the data). What’s very interesting is that as more and more activities go digital, we’re seeing the emergence of the digital trail that comes straight out of the transaction. 
Increasingly every activity, from using an oyster card in the tube, to a meter reading and from checking your bank account to measuring your blood pressure is a digital activity and leaves a digital trail. This is a trail of data which is is currently divergent and disaggregated. But if harnessed, they could be extremely powerful. Even within your business, the ability to mine the digital trail creates a new source of information. The HBR article Exploiting the Virtual Value Chain is a must read to understand how this could work. 
Sometime in the late 90s, I read a very interesting story about a transport company in India which was having trouble tracking the vehicles as they made their way across the vast hinterland of the company, with no real communication system in place to track them. Some of the journeys were over 7 days long each way. The company hit upon the idea of doing a deal with a specific set of petrol pumps where in exchange for a the volume of business, the company got some benefits and information on every truck that refuelled at any of the stations. In one step, the company had created an information network, which would report back on each truck whenever they refuelled. A great example of the digital trail at work. 
The Hailo taxi application is another great example of this. Hailo digitises the process of calling a taxi and even paying for it. But it creates a digital information trail which allows you track which taxi you used and when. The company now markets as a feature that you can always trace back if you left something in the cab. 
However, one of the great unanswered questions which is sure to be debated hotly over the next few years is about ownership of the information. As we generate digital trails about our purchases, our health, our travel, our energy consumption, this creates a huge and valuable information cloud. Who owns this?
In case you’re not convinced about how valuable information can be, consider the fact that smart meters can reveal detailed information about all devices and appliances being at home, including when and for how long. Which in turn provides very meaningful clues into the lifestyles of people in the house – including the number of people, when they are at home and what they do at home. Clearly this is a gold mine for marketers. 
So is it us as individual owners of our own data? Is it each service provider? Will there be a role for an information intermediary who can hold our data and monetize it on our behalf? Who should this be? Google? The Government? Cooperatives? Richard Seymour, founder of Seymour Powell has an idea about a digital surrogate who is on “our side of the glass” who is the repository and identity manager for us. But there is also aggregation value of the information which needs to be realised. 
To summarise, we need to individually build the 3 basic skills of finding information, verifying the source and creating our own reference & library system. We also, as companies need to tap into the digital trail of transactions and find creative ways of extracting value and meaning from the digital trail, and delivering information in the right context. Finally as a society, we need to answer questions about the ownership, curation and exploitation of data at an individual and collective level. For me these are the basics of survival in the information age.