denotes premium content | Jun 20 2013 Feature
posted 10 Mar 2008 in Volume 11 Issue 6
Masterclass part II: prediction markets
Co-creating an organisation's future
Knowledge sharing and collaboration tools are driving interest in prediction markets. Historically organisations have difficulty getting the 'truth' out of employees regarding critical business questions like 'how soon will this product be ready to go?'.
Overview
Prediction is the nirvana for business. Every business leader wants to reduce risk and decision-making uncertainty. Prediction markets hold the allure of optimising new product development but are not without issues. Wisdom of Crowds author James Surowiecki,1 whose book popularised the notion that ‘the many are smarter than the few’, is the first to admit, “prediction markets aren’t a crystal ball, but they are almost always better than any existing forecasting method”.2
This article defines prediction markets and cites case examples revealing their value. It addresses the variety of situations in which to adopt – or not, criteria for establishing and lessons learnt. It suggests essential steps to successful implementation and the business benefits that can flow from connecting intelligence and co-creating the future using prediction markets in organisations.
Introduction
Business needs for improved forecasting in highly competitive markets, knowledge sharing and collaboration are driving interest in prediction markets. While past slow adoption rates point to implementation issues, just released Google study findings3 and media coverage of Bet2give4 forecasts (predicting the Boeing 787 delivery delay) reveal growing use.
Andrew McAfee’s5 inclusion of prediction markets in his
Why organisations want prediction markets
Historically, organisations do poorly getting at the truth about their employees’ thinking on critical business questions. What is the likelihood a product or service will get to market on time, be a success in the market, be sold in large enough quantity to generate hoped for revenue, be met with customer praise or even meet their company’s key performance goals?
Communication up and down, as well as between, collaborating divisions is not always optimal despite cross-functional teams, project management processes and open feedback systems to ensure forthright information flows. Overconfidence, optimism, Wall Street pressures, sales quotas, pleasing the boss, fear of failure, risk aversion, shooting the messenger bearing bad news all conspire to reduce the truth. The product or service will make or miss the mark.
The business press regularly reports stories about information shortages and sharing failures catching senior executives off guard at the worst possible times. Boeing’s failure to meet its 787 delivery schedule is a recent example. Chairman and Chief Executive Jim McNerney’s statement on the delayed delivery tells all: problems, including a serious lack of documentation on the work remaining to complete the first airplanes, drove the company’s decision this month to delay the first 787 deliveries for six months and to replace the head of the plane’s development efforts.
Noting that Boeing was “surprised on the physical reality” of the condition of the first plane, McNerney said officials “realised we really need to work with [suppliers] to make sure we have better visibility” on the manufacturing process.8
For Boeing, this is just the beginning as the extent of the problem unfolds and delays mount along with customer concern. But tapping into the collective intelligence of the employees, suppliers, and leadership connected with 787 Dreamliner production may have saved Boeing both financial repercussions and market confidence. The public Bet2Give9 prediction market stock in whether or not Boeing would meet its May 2008 delivery target had been trading way below 50 per cent, mostly between 20 to 30 per cent in October of 2007, months before the missed delivery date announcement.
David Perry,10 co-founder prediction markets platform provider Consensus Point, confirms, “Yes, markets are early warning systems for many things, they give you a sense of what your people know and do not know.” Top managers at Boeing would have benefited from employees’ knowledge that delivery dates were going to be missed.
While knowledge alone does not bridge the doing-something-about-it gap, often knowledge of a problem is a precursor to more systemic issues. In any event, early information can lead to course corrections. Ever narrowing profits and a highly competitive business environment are driving use of ‘quants’; as Jed Cristiansen11 told us, “[they] are becoming more highly prized because the ‘easy’ profit in many industries is gone. As an industry matures, it needs more information and better analysis to keep profitability, thus the need for quants. I think this is certainly the case for prediction markets”.
What is a prediction market?
A prediction market is a place where information is aggregated via market (or other) mechanisms for the primary purpose of forecasting events, or the probability that an event will occur.12
Aggregation is the key word. What is being aggregated and are there potential biases?
A common starting point is aggregating employee information about a predictable event. The ‘will we meet a delivery date?’ question taps into the wisdom of the crowd – all of the employees who have knowledge. As David Perry describes: “Prediction markets are not necessary if everyone has perfect knowledge; markets are designed to tell people what they do know, to be quiet when they do not know and are designed to get around the problem of getting everyone to contribute what they do know.”
Beyond an aggregation platform, Wisdom of Crowds author James Surowiecki recommends two other key factors for tapping collective intelligence in an organisation:
-
Diversity – having access to a lot of different perspectives, sources of information and sets of knowledge is more valuable than individual IQ or expertise;
-
A prediction market has all these components. It is a market for buying, selling and trading shares for a price. Markets by their nature are dynamic and take place over time, allowing for prices to fluctuate depending on traders’ confidence. Perry notes, “the longer the markets run, the more informed. They are like wine, they get better with time”.
Markets do require some basic trading knowledge that may inhibit adoption and skew participation for those who are more experienced and or comfortable with numbers. Both issues can be addressed in the prediction market design and implementation plan.
Prediction markets as decision making tool?
Prediction markets expand people’s connections in organisations. Through market participation, employees from disparate parts of organisations discover unknown people with similar interests and unexpected talents. Market activity becomes a thread in employee conversations. Previously unrecognised expertise emerges through successful trading and listing on leader boards.
Smart companies are exploring use and adopting prediction markets to connect intelligences and improve decision-making and forecasting. But doing so demands that leadership not be threatened by what the collective wisdom tells them, especially when the information shared is not what they want to hear. Organisations are notoriously resistant to processes that upset traditional power and control structures. Individuals hired for their expertise may resist no longer being the sole source of information.
Successful prediction market adoption: what does it take?
1. Three Rs of participation
Pioneering prediction market provider Newsfutures CEO Emile Servan-Schreiber focuses on the importance of participation. His three Rs for participation13 include:
-
Rewards – any kind of material prize, for example, weekend holiday, cash prize;
-
Recognition – how much the company will recognise participants for good forecasts;
-
Relevance – how much my participation is going to help me do my job.
Experience with a pharmaceutical industry market engaging doctors to forecast volume of prescriptions that will be written for different drugs reveals the three Rs at work. Participating in the market is relevant, because it informs a doctor’s daily work. Doctors are rewarded by increased and timely industry knowledge. Being invited to participate recognises their professional standing and taps intrinsic motivation.
Qualcomm’s senior director, business development, Ricardo dos Santos14 believes “collective intelligence and internal markets are key to every step of our decentralised innovation process, from idea generation through selection, development and ultimate implementation of the top concepts”. Qualcomm’s prediction or preference markets address rewards, recognition and relevance in highlighting new product ideas and corresponding entrepreneurs for the annual Venture Fest. “The top entrepreneurs, on which the market bets are recognised, form a mock-company, receive seed funding, and are given the opportunity to pitch their business case to the C-suite.”
2. Keep it simple
Growing prediction market experience indicates not everybody will be an enthusiastic participant. Even at Google, a company known for hiring quantitative talent, participation skewed towards software programmers and those with more quantitative backgrounds. For trading newcomers, prediction market participation can be challenging. Technology providers are developing new formats to simplify the predicting process.
3. Promote geographic trader spread
A key finding from investigating Google’s prediction markets was the impact of close physical proximity in influencing trading decisions. Surprising in this age of telecommuting and geographically dispersed teams was discovering those physically co-located had more similar trades than even those with close social networks.
4. Pay close attention to your regulatory environment
Google is not alone in using prediction markets but few companies seem willing to openly reveal their initiatives. Prediction Markets Cluster founder John Maloney explains regulation is the “elephant in the room”. Especially in highly regulated industries and for companies operating under compliance legislation like Sarbanes-Oxley, corporate legal and communications departments are protective. Compliance is a real business issue to address with management in proposing any prediction market initiative.
5. Match market type to business problem
Newsfutures CEO Servan-Schreiber sees a trend in businesses moving from testing prediction markets to applying to strategic problems. Still, he warns there are a variety of means beside a prediction market to reach organisational grassroots knowledge. These can be ‘idea pageants’ or ‘competitive forecasting’. Intercontinental Hotels’ idea pageant culls 20 ideas from hundreds down to the top few. A wise council of senior managers allocates funding and resources. A prediction market is a good choice when there is a knowable problem to be addressed within a fixed time.
A preference market, David Perry explains, is “for opinions or beliefs for instance what features and benefits to put into a product.” This approach can replace a typical focus group. A power market can be used if you need an answer tomorrow and want to see how the market shifts over a limited period of time.
To select the best market design, factor in business issues at hand and the organisation’s ability to incorporate the resulting information into its decision-making processes. Watch for resistance to using what might be considered ‘gaming, gambling or betting’.
6. Market efficiency
Questions arise about the efficiency of the market and whether it can be gamed. Perry responds, “Gaming the market is irrational – buying a lot goes against the collective intelligence of the market. It is a lot more difficult to game than people think. It encourages the uninformed to go in. It ultimately self-corrects.”
Gaming is also addressed by the design of the market where weighting is used to balance out results from different demographics.
7. Design to avoid biases
The Google markets study found four biases:
-
Overpricing of favourites;
-
Short aversion;
-
Optimism;
-
Under pricing of extreme outcomes.
The behaviour of newer and less experienced employees being more optimistic than more experienced was a factor. Overall as the market progressed, the biases were less, coinciding with David Perry’s observation that markets get better with time. Too much focus on the trading leader boards can become an issue if they are perceived as the experts, potentially erasing the benefit of the market to tap the many, not only the few.
8. Integrate prediction market projects to your collaboration ecosystem
For companies ranging from Google to Best Buy, Qualcomm and Arcelor Mittal, prediction markets are proving an invaluable business intelligence tool.
Still, they are just one decision support tool. In company-wide planning, ensure learning gained is distributed and integrated with other initiatives. Prediction markets are very useful for assessing project management, sales forecasting and generating ideas for innovation.
9. Strategically use prediction markets to engage stakeholders
As a knowledge sharing tool, prediction markets provide an opportunity to capture insights in, across and between organizations. Markets can engage employees operating only within the organizations as Google does. Companies like Corning15 have tested prediction markets to tap partner and supplier knowledge to predict LCD screen adoption.
For more the complex beyond the organisation applications, recognise that answers to simple time defined questions may inadequately reflect the complexity of the business system at this level.
10. Integrate ways to tap the thinking processes too
Experience with the Long Bets project, www.longbets.org (focused on bets that are of societal or scientific importance), reveals that it’s also important to capture what leads to thinking, the argument behind the bet. Then if the bet wins or loses, the thought processes are also visible.
Include in your prediction market design a means of promoting and capturing dialogue around the bet maker’s rationale. Gold lies in the trader’s thought processes, not just the outcome.
Final note
Prediction markets, like any social media or enterprise 2.0 tools, are only as good as the organisational cultures using them. Even with the data generated from an open system tool, leaders need to be able to execute and implement. Data may tell you employees are not confident in meeting a goal but only effective management skills will enable a change.
Good organisational cultures beget better cultures. In our experience, and those interviewed for this article, high performing companies good at adopting and implementing new processes will benefit the most from prediction markets. Companies need to be ready, have a business need and be willing to make the management changes to support tapping their stakeholders’ collective intelligence to co-create an organisation’s future.
In closing, our two part article on connecting intelligence in organisations through broadcasting innovation (Part 1) and prediction markets (Part 2), you are invited to visit the wiki where we gathered article resources and add your contributions. Please join us at the Connected Intelligence wiki – http://connectedintelligence.wikispaces.com/.
Victoria Axelrod is principal of Axelrod Becker Consulting, Inc., US, providing innovation and growth strategies from stakeholder networks. Jenny Ambrozek is founder of SageNet LLC, US, a consulting practice helping companies architect participation.
References and sources
Surowiecki, J., Wisdom of Crowds, Random House, 2004.
Christiansen, J., 2007, notes from the European Prediction Markets Summit, http://kmblogs.com/public/blog/186847.
Cowgill, B., et al, 2008, ‘Using Prediction Markets to Track Information Flows: Evidence from Google’, http://googleblog.blogspot.com/2008/01/flow-of-information-at-googleplex.html.
https://bet2give.com/b2g/market/linear/market.html?symbol=Boeing787onTime.
McAfee, A., ‘How to Hit the Enterprise 2.0 Bullseye’, blog post 03 November, 2007, http://blog.hbs.edu/faculty/amcafee/index.php/faculty_amcafee_v3/how_to_hit_the_enterprise_20_bullseye/
Hanson, R., ‘A 1990 Corporate Prediction Market’, 2006, blog post retrieved 31 January, 2008, http://www.overcomingbias.com/2006/11/first_known_bus.html.
http://hanson.gmu.edu/ppt/PredMkt%20Quick%20Review.ppt
McNerney, J., Wall Street Journal, ‘Boeing Focuses on Dreamliner’, 25 October, 2007, http://online.wsj.com/article/SB119326230811970396.html,~
https://bet2give.com/b2g/market/linear/market.html?symbol=Boeing787onTime, Emile Servan-Schreiber, CEO Newsfutures, Prediction Markets Google Group post extracted 26 January, 2007, http://groups.google.com/group/Prediction-Markets/browse_thread/thread/c9cfa4d8e0ea27e7#.
David Perry, CEO Consensus Point, www.consensuspoint.com, phone interview, 24 January, 2008.
Jed Christiansen, catalyst, Mercury Research & Consulting
http://www.mercury-rac.com/about-us.html, e-mail interview January 2008
Alex Kirtland – Partner, Usable Markets, LLC, 25 March 2007, Information Association Summit power point presentation http://www.usablemarkets.com/?p=69.
Servan-Schreiber, E., phone interview 25 January, 2008, Newsfutures, www.newsfutures.com.
Dos Santos, R., phone interview 29 January, 2008.
New York Times, 03 March, 2006, retrieved from http://hanson.gmu.edu/PAM/press/NYT-3-11-06.htm.
