I always enjoy witnessing a good debate. And I mean the type of debate where one person is given a thesis to defend, while the other person speaks in favour of the anti-thesis. Sometimes – when smart people really get into it – seeing two debaters line up the arguments and create the strongest possible defence can really clarify the pros and cons in my mind and hence make me understand the issue better.
For example – be it one in a written format – recently my good friend and colleague at the London Business School, Costas Markides, was asked by Business Week to debate the thesis that “happy workers will produce more and do their jobs better”. Harvard’s Teresa Amabile and Steven Kramer had the (relatively easy) task of defending the “pro”. I say relatively easy, because the thesis seems intuitively appealing, it is what we’d all like to believe, and they have actually done ample research on the topic.
My poor London Business School colleague was given the hapless task to defend the “con”: “no, happy workers don’t do any better”. Hapless indeed.
In fact, in spite of receiving some hate mail in the process, I think he did a rather good job. I am giving him the assessment “good” because indeed he made me think. He argues that having happy, smiley employees all abound might not necessarily be a good sign, because it might be a signal that something is wrong in your organisation, and you’re perhaps not making the tough but necessary choices.
As said, it made me think, and that can’t be bad. Might we not be dealing with a reversal of cause and effect here? Meaning: well-managed companies will get happy employees, but that does not mean that choosing to make your employees happy as a goal in and of itself will get you a better organisation? At least, it is worth thinking about.
In spite that perhaps to you it might seem a natural thing to have in an academic institution – a good debate – it is actually not easy to organise one in business academia. Most people are simply reluctant to do it – as I found out organising our yearly Ghoshal Conference at the London Business School – and perhaps they are right, because even fewer people are any good at it.
I guess that is because, to a professor, it feels unnatural to adopt and defend just one side of the coin, because we are trained to be nuanced about stuff and examine and see all sides of the argument. It is also true that (the more naïve part of) the audience will start to associate you with that side of the argument, “as if you really meant it”. Many of the comments Costas received from the public were of that nature, i.e. “he is that moronic guy who thinks you should make your employees unhappy”. Which of course is not what he meant at all. Nor was it the purpose of the debate.
Yet, I also think it is difficult to find people willing to debate a business issue because academics are simply afraid to have an opinion. We are not only trained to examine and see all sides of an argument, we are also trained to not believe in something – let alone argue in favour of it – until there is research that produced supportive evidence for it. In fact, if in an academic article you would ever suggest the existence of a certain relationship without presenting evidence, you’d be in for a good bellowing and a firm rejection letter. And perhaps rightly so, because providing evidence and thus real understanding is what research is about.
But, at some point, you also have to take a stand. As a paediatric neurologist once told me, “what I do is part art, part science”. What he meant is that he knew all the research on all medications and treatments, but at the end of the day every patient is unique and he would have to make a judgement call on what exact treatment to prescribe. And doing that requires an opinion.
You don’t hear much opinion coming from the ivory tower in business academia. Which means that the average business school professor does not receive much hate mail. It also means he doesn’t have much of an audience outside of the ivory tower.
Showing posts with label Research. Show all posts
Showing posts with label Research. Show all posts
Monday, April 2, 2012
Monday, March 19, 2012
Research by Mucking About
I am a long standing fan of the Ig Nobel awards. The Ig Nobel awards are an initiative by the magazine Air (Annals of Improbable Research) and are handed out on a yearly basis – often by real Nobel Prize winners – to people whose research “makes people laugh and then think” (although its motto used to be to “honor people whose achievements cannot or should not be reproduced" – but I guess the organisers had to first experience the “then think” bit themselves).
With a few exceptions they are handed out for real research, done by academics, and published in scientific journals. Here are some of my old time favourites:
• BIOLOGY 2002, Bubier, Pexton, Bowers, and Deeming.“Courtship behaviour of ostriches towards humans under farming conditions in Britain” British Poultry Science 39(4)
• INTERDISCIPLINARY RESEARCH 2002. Karl Kruszelnicki (University of Sydney). “for performing a comprehensive survey of human belly button lint – who gets it, when, what color, and how much”
• MATHEMATICS 2002. Sreekumar and Nirmalan (Kerala Agricultural University). “Estimation of the total surface area in Indian Elephants” Veterinary Research Communications 14(1)
• TECHNOLOGY 2001, Jointly to Keogh (Hawthorn), for patenting the wheel (in 2001), and the Australian Patent Office for granting him the patent.
• PEACE 2000, the British Royal Navy, for ordering its sailors to stop using live cannon shells, and to instead just shout “Bang!”
• LITERATURE 1998, Dr. Mara Sidoli (Washington) for the report “farting as a defence against unspeakable dread”. Journal of analytical psychology 41(2)
To the best of my knowledge, there is (only) one individual who has not only won an Ig Nobel Award, but also a Nobel Prize. That person is Andre Geim. Geim – who is now at the University of Manchester – for long held the habit of dedicating a fairly substantial proportion of his time to just mucking about in his lab, trying to do “cool stuff”. In one of such sessions, together with his doctoral student Konstantin Novoselov, he used a piece of ordinary sticky tape (which allegedly they found in a bin) to peel off a very thin layer of graphite, taken from a pencil. They managed to make the layer of carbon one atom thick, inventing the material “graphene”.
In another session, together with Michael Berry from the University of Bristol, he experimented with the force of magnetism. Using a magnetized metal slab and a coil of wire in which a current is flowing as an electromagnet, they tried to make a magnetic force that exactly balanced gravity, to try and make various objects “float”. Eventually, they settled on a frog – which, like humans, mostly consists of water – and indeed managed to make it levitate.
The one project got Geim the Ig Nobel; the other one got him the Nobel Prize.
“Mucking about” was the foundation of these achievements. The vast majority of these experiments doesn’t go anywhere; some of them lead to an Ig Nobel and makes people laugh; others result in a Nobel Prize. Many of man’s great discoveries – in technology, medicine or art – have been achieved by mucking about. And many great companies were founded by mucking about, in a garage (Apple), a dorm room (Facebook), or a kitchen and a room above a bar (Xerox).
Unfortunately, in strategy research we don’t muck about much. In fact, people are actively discouraged from doing so. During pretty much any doctoral consortium, junior faculty meeting, or annual faculty review, a young academic in the field of Strategic Management is told – with ample insistence – to focus, figure out in what subfield he or she wants to be known, “who the five people are that are going to read your paper” (heard this one in a doctoral consortium myself), and “who your letter writers are going to be for tenure” (heard this one in countless meetings). The field of Strategy – or any other field within a business school for that matter – has no time and tolerance for mucking about. Disdain and a weary shaking of the head are the fates of those who try, and step off the proven path in an attempt to do something original with uncertain outcome: “he is never going to make tenure, that’s for sure”.
And perhaps that is also why we don’t have any Nobel Prizes.
With a few exceptions they are handed out for real research, done by academics, and published in scientific journals. Here are some of my old time favourites:
• BIOLOGY 2002, Bubier, Pexton, Bowers, and Deeming.“Courtship behaviour of ostriches towards humans under farming conditions in Britain” British Poultry Science 39(4)
• INTERDISCIPLINARY RESEARCH 2002. Karl Kruszelnicki (University of Sydney). “for performing a comprehensive survey of human belly button lint – who gets it, when, what color, and how much”
• MATHEMATICS 2002. Sreekumar and Nirmalan (Kerala Agricultural University). “Estimation of the total surface area in Indian Elephants” Veterinary Research Communications 14(1)
• TECHNOLOGY 2001, Jointly to Keogh (Hawthorn), for patenting the wheel (in 2001), and the Australian Patent Office for granting him the patent.
• PEACE 2000, the British Royal Navy, for ordering its sailors to stop using live cannon shells, and to instead just shout “Bang!”
• LITERATURE 1998, Dr. Mara Sidoli (Washington) for the report “farting as a defence against unspeakable dread”. Journal of analytical psychology 41(2)
To the best of my knowledge, there is (only) one individual who has not only won an Ig Nobel Award, but also a Nobel Prize. That person is Andre Geim. Geim – who is now at the University of Manchester – for long held the habit of dedicating a fairly substantial proportion of his time to just mucking about in his lab, trying to do “cool stuff”. In one of such sessions, together with his doctoral student Konstantin Novoselov, he used a piece of ordinary sticky tape (which allegedly they found in a bin) to peel off a very thin layer of graphite, taken from a pencil. They managed to make the layer of carbon one atom thick, inventing the material “graphene”.
In another session, together with Michael Berry from the University of Bristol, he experimented with the force of magnetism. Using a magnetized metal slab and a coil of wire in which a current is flowing as an electromagnet, they tried to make a magnetic force that exactly balanced gravity, to try and make various objects “float”. Eventually, they settled on a frog – which, like humans, mostly consists of water – and indeed managed to make it levitate.
The one project got Geim the Ig Nobel; the other one got him the Nobel Prize.
“Mucking about” was the foundation of these achievements. The vast majority of these experiments doesn’t go anywhere; some of them lead to an Ig Nobel and makes people laugh; others result in a Nobel Prize. Many of man’s great discoveries – in technology, medicine or art – have been achieved by mucking about. And many great companies were founded by mucking about, in a garage (Apple), a dorm room (Facebook), or a kitchen and a room above a bar (Xerox).
Unfortunately, in strategy research we don’t muck about much. In fact, people are actively discouraged from doing so. During pretty much any doctoral consortium, junior faculty meeting, or annual faculty review, a young academic in the field of Strategic Management is told – with ample insistence – to focus, figure out in what subfield he or she wants to be known, “who the five people are that are going to read your paper” (heard this one in a doctoral consortium myself), and “who your letter writers are going to be for tenure” (heard this one in countless meetings). The field of Strategy – or any other field within a business school for that matter – has no time and tolerance for mucking about. Disdain and a weary shaking of the head are the fates of those who try, and step off the proven path in an attempt to do something original with uncertain outcome: “he is never going to make tenure, that’s for sure”.
And perhaps that is also why we don’t have any Nobel Prizes.
Tuesday, February 21, 2012
“The Best Degree for Start-up Success”
“So you want to start a company. You've finished your undergraduate degree and you're peering into the haze of your future. Would it be better to continue on to an MBA or do an advanced degree in a nerdy pursuit like engineering or mathematics? Sure, tech skills are hugely in demand and there are a few high-profile nerd success stories, but how often do pencil-necked geeks really succeed in business? Aren't polished, suited and suave MBA-types more common at the top? Not according to a recent white paper from Identified, tellingly entitled "Revenge of the Nerds."
Interested? Yes, it does sound intriguing, doesn’t it? It is the start of an article, written by a journalist, based on a report by a company called “Identified”. In the report, you can find that “Identified is the largest database of professional information on Facebook. Our database includes over 50 million Facebook users and over 1.2 billion data points on professionals’ work history, education and demographic data”.
In the report, based on the analysis of data obtained from Facebook, under the header “the best degree for start-up success”, Identified presents some “definitive conclusions” about “whether an MBA is worth the investment and if it really gets you to the top of the corporate food chain”. Let me no longer hold you in suspense (although I think by now you do see this one coming from a mile or two, like a Harry and Sally romance), the definitive conclusion is: “that if you want to build a company, an advanced degree in a subject like engineering beats an MBA any day”.
So I have read the report…
[insert deep sigh]
and – how shall I put it – I have a few doubts… ( = polite English euphemism)
Although Identified has “assembled a world class team of 15 engineers and data scientists to analyse this vast database and identify interesting trends, patterns and correlations” I am not entirely sure that they are not jumping to a few unwarranted conclusions. ( = polite English euphemism)
So, when they dig up from Facebook all the profiles of anyone listed as “CEO” or “founder”, they find that about ¾ are engineers and a mere ¼ are MBAs. (Actually, they don’t even find that, but let me not get distracted here). I have no quibbles with that; I am sure they do find what they find; after all, they do have “a world class team of 15 engineers and data scientists”, and a fact is a fact. What I have more quibbles with is how you get from that to the conclusion that if you want to build a company, an advanced degree in a subject like engineering beats an MBA any day.
Perhaps it may seem obvious and a legitimate conclusion to you: more CEOs have an engineering degree than an MBA, so surely getting an engineering degree is more likely to enable you to become a CEO? But, no, that is where it goes wrong; you cannot draw this conclusion from those data. Perhaps “a world class team of 15 engineers and data scientists [able] to analyse this vast database and identify interesting trends, patterns and correlations” are superbly able at digging up the data for you but, apparently, they are less skilled in drawing justifiable conclusions. (I am tempted to suggest that, for this, they would have been better off hiring an MBA, but will fiercely resist that temptation!)
The problem is, what we call, “unobserved heterogeneity”, coupled with some “selection bias”, finished with some “bollocks” (one of which is not a generally accepted statistical term) – and in this case there is lots of it. For example – to start with a simple one – perhaps there are simply a lot more engineers trying to start a company than MBAs. If there are 20 engineers trying to start a company and 9 of them succeed, while there are 5 MBAs trying it and 3 of them succeed, can you really conclude that an engineering degree is better for start-up success than an MBA?
But, you may object, why would there be more engineers who are trying to start a business? Alright then, since you insist, suppose out of the 10 engineers 9 succeed and out of the 10 MBAs only 3 do, but the 9 head $100,000 businesses and the three $100 million ones? Still so sure that an engineering degree is more useful to “get you to the top of the corporate food chain”? What about if the MBA companies have all been in existence for 15 years while all the engineering start-ups never make it past year 2?
And these are of course only very crude examples. There are likely more subtle processes going on as well. For instance, the same type of qualities that might make someone choose to do an engineering degree could prompt him or her to start a company, however, this same person might have been better off (in terms of being able to make the start-up a success) if s/he had done an MBA. And if you buy none of the above (because you are an engineer or about to be engaged to one) what about the following: people who chose to do an engineering degree are inherently smarter and more able people than MBAs, hence they start more and more successful companies. However, that still leaves wide open the possibility that such a very smart and able person would have been even more successful had s/he chosen to do an MBA before venturing.
I could go on for a while (and frankly I will) but I realise that none of my aforementioned scenarios will be the right one, yet the point is that there might very well be a bit going on of several of them. You cannot compare the ventures started by engineers with the ventures headed by MBAs, you can’t compare the two sets of people, you can’t conclude that engineers are more successful founding companies, and you certainly cannot conclude that getting an engineering degree makes you more likely to succeed in starting a business. So, what can you conclude from the finding that more CEOs/founders have a degree in engineering than an MBA? Well… precisely that; that more CEOs/founders have a degree in engineering than an MBA. And, I am sorry, not much else.
Real research (into such complex questions such as “what degree is most likely to lead to start-up success?) is more complex. And so will likely have to be the answer. For some type of businesses an MBA might be better, and for others an engineering degree. And some type of people might be more helped with an MBA, where other types are better off with an engineering degree. There is nothing wrong with deriving some interesting statistics from a database, but you have to be modest and honest about the conclusions you can link to them. It may sound more interesting if you claim that you find a definitive conclusion about what degree leads to start-up success – and it certainly will be more eagerly repeated by journalist and in subsequent tweets (as happened in this case) – but I am afraid that does not make it so.
Interested? Yes, it does sound intriguing, doesn’t it? It is the start of an article, written by a journalist, based on a report by a company called “Identified”. In the report, you can find that “Identified is the largest database of professional information on Facebook. Our database includes over 50 million Facebook users and over 1.2 billion data points on professionals’ work history, education and demographic data”.
In the report, based on the analysis of data obtained from Facebook, under the header “the best degree for start-up success”, Identified presents some “definitive conclusions” about “whether an MBA is worth the investment and if it really gets you to the top of the corporate food chain”. Let me no longer hold you in suspense (although I think by now you do see this one coming from a mile or two, like a Harry and Sally romance), the definitive conclusion is: “that if you want to build a company, an advanced degree in a subject like engineering beats an MBA any day”.
So I have read the report…
[insert deep sigh]
and – how shall I put it – I have a few doubts… ( = polite English euphemism)
Although Identified has “assembled a world class team of 15 engineers and data scientists to analyse this vast database and identify interesting trends, patterns and correlations” I am not entirely sure that they are not jumping to a few unwarranted conclusions. ( = polite English euphemism)
So, when they dig up from Facebook all the profiles of anyone listed as “CEO” or “founder”, they find that about ¾ are engineers and a mere ¼ are MBAs. (Actually, they don’t even find that, but let me not get distracted here). I have no quibbles with that; I am sure they do find what they find; after all, they do have “a world class team of 15 engineers and data scientists”, and a fact is a fact. What I have more quibbles with is how you get from that to the conclusion that if you want to build a company, an advanced degree in a subject like engineering beats an MBA any day.
Perhaps it may seem obvious and a legitimate conclusion to you: more CEOs have an engineering degree than an MBA, so surely getting an engineering degree is more likely to enable you to become a CEO? But, no, that is where it goes wrong; you cannot draw this conclusion from those data. Perhaps “a world class team of 15 engineers and data scientists [able] to analyse this vast database and identify interesting trends, patterns and correlations” are superbly able at digging up the data for you but, apparently, they are less skilled in drawing justifiable conclusions. (I am tempted to suggest that, for this, they would have been better off hiring an MBA, but will fiercely resist that temptation!)
The problem is, what we call, “unobserved heterogeneity”, coupled with some “selection bias”, finished with some “bollocks” (one of which is not a generally accepted statistical term) – and in this case there is lots of it. For example – to start with a simple one – perhaps there are simply a lot more engineers trying to start a company than MBAs. If there are 20 engineers trying to start a company and 9 of them succeed, while there are 5 MBAs trying it and 3 of them succeed, can you really conclude that an engineering degree is better for start-up success than an MBA?
But, you may object, why would there be more engineers who are trying to start a business? Alright then, since you insist, suppose out of the 10 engineers 9 succeed and out of the 10 MBAs only 3 do, but the 9 head $100,000 businesses and the three $100 million ones? Still so sure that an engineering degree is more useful to “get you to the top of the corporate food chain”? What about if the MBA companies have all been in existence for 15 years while all the engineering start-ups never make it past year 2?
And these are of course only very crude examples. There are likely more subtle processes going on as well. For instance, the same type of qualities that might make someone choose to do an engineering degree could prompt him or her to start a company, however, this same person might have been better off (in terms of being able to make the start-up a success) if s/he had done an MBA. And if you buy none of the above (because you are an engineer or about to be engaged to one) what about the following: people who chose to do an engineering degree are inherently smarter and more able people than MBAs, hence they start more and more successful companies. However, that still leaves wide open the possibility that such a very smart and able person would have been even more successful had s/he chosen to do an MBA before venturing.
I could go on for a while (and frankly I will) but I realise that none of my aforementioned scenarios will be the right one, yet the point is that there might very well be a bit going on of several of them. You cannot compare the ventures started by engineers with the ventures headed by MBAs, you can’t compare the two sets of people, you can’t conclude that engineers are more successful founding companies, and you certainly cannot conclude that getting an engineering degree makes you more likely to succeed in starting a business. So, what can you conclude from the finding that more CEOs/founders have a degree in engineering than an MBA? Well… precisely that; that more CEOs/founders have a degree in engineering than an MBA. And, I am sorry, not much else.
Real research (into such complex questions such as “what degree is most likely to lead to start-up success?) is more complex. And so will likely have to be the answer. For some type of businesses an MBA might be better, and for others an engineering degree. And some type of people might be more helped with an MBA, where other types are better off with an engineering degree. There is nothing wrong with deriving some interesting statistics from a database, but you have to be modest and honest about the conclusions you can link to them. It may sound more interesting if you claim that you find a definitive conclusion about what degree leads to start-up success – and it certainly will be more eagerly repeated by journalist and in subsequent tweets (as happened in this case) – but I am afraid that does not make it so.
Monday, January 23, 2012
Fraud and the Road to Abilene
Over the weekend, an (anonymized) interview was published in a Dutch national newspaper with the three “whistle blowers” who exposed the enormous fraud of Professor Diederik Stapel. Stapel had gained stardom status in the field of social psychology but, simply speaking, had been making up all his data all the time. There are two things that struck me:
First, in a previous post I wrote about the fraud, based on a flurry of newspaper articles and the interim report that a committee examining the fraud has put together, I wrote that it eventually was his clumsiness faking the data that got him caught. Although that general picture certainly remained – he wasn’t very good at faking data; I think I could have easily done a better job (although I have never even tried anything like that, honest!) – but it wasn’t as clumsy as the newspapers sometimes made it out to be.
Specifically, I wrote “eventually, he did not even bother anymore to really make up newly faked data. He used the same (fake) numbers for different experiments, gave those to his various PhD students to analyze, who then in disbelief slaving away in their adjacent cubicles discovered that their very different experiments led to exactly the same statistical values (a near impossibility). When they compared their databases, there was substantial overlap”. Now, it now seems the “substantial overlap” was merely a part of one column of data. Plus, there were various other things that got him caught.
I don’t beat myself too hard over the head with my keyboard about repeating this misrepresentation by the newspapers (although I have given myself a small slap on the wrist – after having received a verbal one from one of the whistlers) because my piece focused on the “why did he do it?” rather than the “how did he get caught”, but it does show that we have to give the three whistle blowers (quite) a bit more credit than I – and others – originally thought.
The second point that caught my attention is that, since the fraught was exposed, various people have come out admitting that they had “had suspicions all the time”. You could say “yeah right” but there do appear to be quite a few signs that various people indeed had been having their doubts for a longer time. For instance, I have read an interview with a former colleague of Stapel at Tilburg University credibly admitting to this, I have directly spoken to people who said there had been rumors for longer, and the article with the whistle blowers suggests even Stapel’s faculty dean might not have been entirely dumbfounded that it had all been too good to be true after all... All the people who admit to having doubts in private state that they did not feel comfortable raising the issue while everyone just seemed to applaud Stapel and his Science publications.
This reminded me of the Abilene Paradox, first described by Professor Jerry Harvey, from the George Washington University. He described a leisure trip which he and his wife and parents made in Texas in July, in his parents’ un-airconditioned old Buick to a town called Abilene. It was a trip they had all agreed to – or at least not disagreed with – but, as it later turned out, none of them had wanted to go on. “Here we were, four reasonably sensible people who, of our own volition, had just taken a 106-mile trip across a godforsaken desert in a furnace-like temperature through a cloud-like dust storm to eat unpalatable food at a hole-in-the-wall cafeteria in Abilene, when none of us had really wanted to go”
The Abilene Paradox describes the situation where everyone goes along with something, mistakenly assuming that others’ people’s silence implies that they agree. And the (erroneous) feeling to be the only one who disagrees makes a person shut up as well, all the way to Abilene.
People had suspicions about Stapel’s “too good to be true” research record and findings but did not dare to speak up while no-one else did.
It seems there are two things that eventually made the three whistle blowers speak up and expose Stapel: Friendship and alcohol.
They had struck up a friendship and one night, fuelled by alcohol, raised their suspicions to one another. And, crucially, decided to do something about it. Perhaps there are some lessons in this for the world of business. For example, Jim Westphal, who has done extensive, thorough research on boards of directors, showed that boards often suffer from the Abilene Paradox, for instance when confronted with their company’s new strategy. Yet, Jim and colleagues also showed that friendship ties within top management teams might not be such a bad thing. We are often suspicious of social ties between boards and top managers, fearful that it might cloud their judgment and make them reluctant to discipline a CEO. But it may be that such friendship ties – whether fuelled by alcohol or not – might also help to lower the barriers to resolving the Abilene Paradox. So perhaps we should make friendships and alcohol mandatory – religion permitting – both during board meetings and academic gatherings. It would undoubtedly help making them more tolerable as well.
First, in a previous post I wrote about the fraud, based on a flurry of newspaper articles and the interim report that a committee examining the fraud has put together, I wrote that it eventually was his clumsiness faking the data that got him caught. Although that general picture certainly remained – he wasn’t very good at faking data; I think I could have easily done a better job (although I have never even tried anything like that, honest!) – but it wasn’t as clumsy as the newspapers sometimes made it out to be.
Specifically, I wrote “eventually, he did not even bother anymore to really make up newly faked data. He used the same (fake) numbers for different experiments, gave those to his various PhD students to analyze, who then in disbelief slaving away in their adjacent cubicles discovered that their very different experiments led to exactly the same statistical values (a near impossibility). When they compared their databases, there was substantial overlap”. Now, it now seems the “substantial overlap” was merely a part of one column of data. Plus, there were various other things that got him caught.
I don’t beat myself too hard over the head with my keyboard about repeating this misrepresentation by the newspapers (although I have given myself a small slap on the wrist – after having received a verbal one from one of the whistlers) because my piece focused on the “why did he do it?” rather than the “how did he get caught”, but it does show that we have to give the three whistle blowers (quite) a bit more credit than I – and others – originally thought.
The second point that caught my attention is that, since the fraught was exposed, various people have come out admitting that they had “had suspicions all the time”. You could say “yeah right” but there do appear to be quite a few signs that various people indeed had been having their doubts for a longer time. For instance, I have read an interview with a former colleague of Stapel at Tilburg University credibly admitting to this, I have directly spoken to people who said there had been rumors for longer, and the article with the whistle blowers suggests even Stapel’s faculty dean might not have been entirely dumbfounded that it had all been too good to be true after all... All the people who admit to having doubts in private state that they did not feel comfortable raising the issue while everyone just seemed to applaud Stapel and his Science publications.
This reminded me of the Abilene Paradox, first described by Professor Jerry Harvey, from the George Washington University. He described a leisure trip which he and his wife and parents made in Texas in July, in his parents’ un-airconditioned old Buick to a town called Abilene. It was a trip they had all agreed to – or at least not disagreed with – but, as it later turned out, none of them had wanted to go on. “Here we were, four reasonably sensible people who, of our own volition, had just taken a 106-mile trip across a godforsaken desert in a furnace-like temperature through a cloud-like dust storm to eat unpalatable food at a hole-in-the-wall cafeteria in Abilene, when none of us had really wanted to go”
The Abilene Paradox describes the situation where everyone goes along with something, mistakenly assuming that others’ people’s silence implies that they agree. And the (erroneous) feeling to be the only one who disagrees makes a person shut up as well, all the way to Abilene.
People had suspicions about Stapel’s “too good to be true” research record and findings but did not dare to speak up while no-one else did.
It seems there are two things that eventually made the three whistle blowers speak up and expose Stapel: Friendship and alcohol.
They had struck up a friendship and one night, fuelled by alcohol, raised their suspicions to one another. And, crucially, decided to do something about it. Perhaps there are some lessons in this for the world of business. For example, Jim Westphal, who has done extensive, thorough research on boards of directors, showed that boards often suffer from the Abilene Paradox, for instance when confronted with their company’s new strategy. Yet, Jim and colleagues also showed that friendship ties within top management teams might not be such a bad thing. We are often suspicious of social ties between boards and top managers, fearful that it might cloud their judgment and make them reluctant to discipline a CEO. But it may be that such friendship ties – whether fuelled by alcohol or not – might also help to lower the barriers to resolving the Abilene Paradox. So perhaps we should make friendships and alcohol mandatory – religion permitting – both during board meetings and academic gatherings. It would undoubtedly help making them more tolerable as well.
Wednesday, January 11, 2012
Bias (or why you can’t trust any of the research you read)
Researchers in Management and Strategy worry a lot about bias – statistical bias. In case you’re not such an academic researcher, let me briefly explain.
Suppose you want to find out how many members of a rugby club have their nipples pierced (to pick a random example). The problem is, the club has 200 members and you don’t want to ask them all to take their shirts off. Therefore, you select a sample of 20 of them guys and ask them to bare their chests. After some friendly bantering they agree, and then it appears that no fewer than 15 of them have their nipples pierced, so you conclude that the majority of players in the club likely have undergone the slightly painful (or so I am told) aesthetic enhancement.
The problem is, there is a chance that you’re wrong. There is a chance that due to sheer coincidence you happened to select 15 pierced pairs of nipples where among the full set of 200 members they are very much the minority. For example, if in reality out of the 200 rugby blokes only 30 have their nipples pierced, due to sheer chance you could happen to pick 15 of them in your sample of 20, and your conclusion that “the majority of players in this club has them” is wrong.
Now, in our research, there is no real way around this. Therefore, the convention among academic researchers is that it is ok, and you can claim your conclusion based on only a sample of observations, as long as the probability that you are wrong is no bigger than 5%. If it ain’t – and one can relatively easily compute that probability – we say the result is “statistically significant”. Out of sheer joy, we then mark that number with a cheerful asterisk * and say amen.
Now, I just said that “one can relatively easily compute that probability” but that is not always entirely true. In fact, over the years statisticians have come up with increasingly complex procedures to correct for all sorts of potential statistical biases that can occur in research projects of various natures. They treat horrifying statistical conditions such as unobserved heterogeneity, selection bias, heteroscedasticity, and autocorrelation. Let me not try to explain to you what they are, but believe me they’re nasty. You don’t want to be caught with one of those.
Fortunately, the life of the researcher is made easy by standard statistical software packages. They offer nice user-friendly menus where one can press buttons to solve problems. For example, if you have identified a heteroscedasticity problem in your data, there are various buttons to press that can cure it for you. Now, note that it is my personal estimate (but notice, no claims of an asterisk!) that about 95 out of a 100 researchers have no clue what happens within their computers when they press one of those magical buttons, but that does not mean it does not solve the problem. Professional statisticians will frown and smirk at the thought alone, but if you have correctly identified the condition and the way to treat it, you don’t necessarily have to fully understand how the cure works (although I think it often would help selecting the correct treatment). So far, so good.
Here comes the trick: All of those statistical biases are pretty much irrelevant. They are irrelevant because they are all dwarfed by another bias (for which there is no life-saving cure available in any of the statistical packages): publication bias.
The problem is that if you have collected a whole bunch of data and you don’t find anything or at least nothing really interesting and new, no journal is going to publish it. For example, the prestigious journal Administrative Science Quarterly proclaims in its “Invitation to Contributors” that it seeks to publish “counterintuitive work that disconfirms prevailing assumptions”. And perhaps rightly so; we’re all interested in learning something new. So if you, as a researcher, don’t find anything counterintuitive that disconfirms prevailing assumptions, you are usually not even going to bother writing it up. And in case you’re dumb enough to write it up and send it to a journal requesting them to publish it, you will swiftly (or less swiftly, dependent on what journal you sent it to) receive a reply that has the word “reject” firmly embedded in it.
Yet, unintended, this publication reality completely messes up the “5% convention”, i.e. that you can only claim a finding as real if there is only a 5% chance that what you found is sheer coincidence (rather than a counterintuitive insight that disconfirms prevailing assumptions). In fact, the chance that what you are reporting is bogus is much higher than the 5% you so cheerfully claimed with your poignant asterisk. Because journals will only publish novel, interesting findings – and therefore researchers only bother to write up seemingly intriguing counterintuitive findings – the chance that what they eventually are publishing is BS unwittingly is vast.
A recent article by Simmons, Nelson, and Simonsohn in Psychological Science (cheerfully entitled “False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant”) summed it up prickly clearly. If a researcher, running a particular experiment, does not find the result he was expecting, he may initially think “that’s because I did not collect enough data” and collect some more. He can also think “I used the wrong measure; let me use the other measure I also collected” or “I need to correct my models for whether the respondent was male or female” or “examine a slightly different set of conditions”. Yet, taking these (extremely common) measures raises the probability that what the researcher finds in his data is due to sheer chance from the conventional 5% to a whopping 60.7%, without the researcher realising it. He will still cheerfully put the all-important asterisk in his table and declare that he has found a counterintuitive insight that disconfirms some important prevailing assumption.
In management and strategy research we do highly similar things. We for instance collect data with two or three ideas in mind in terms of what we want to examine and test with them. If the first idea does not lead to a desired result, the researcher moves on to his second idea and then one can hear a sigh of relief behind a computer screen that “at least this idea was a good one”. In fact, you might only be moving on to “the next good idea” till you have hit on a purely coincidental result: 15 bulky guys with pierced nipples.
Things get really “funny” when one realises that what is considered interesting and publishable is different in different fields in Business Studies. For example, in fields like Finance and Economics, academics are likely to be fairly skeptical whether Corporate Social Responsibility is good for a firm’s financial performance. In the subfield of Management people are much more receptive to the idea that Corporate Social Responsibility should also benefit a firm in terms of its profitability. Indeed, as shown by a simple yet nifty study by Marc Orlitzky, recently published in Business Ethics Quarterly, articles published on this topic in Management journals report a statistical relationship between the two variables which is about twice as big as the ones reported in Economics, Finance, or Accounting journals. Of course, who does the research and where it gets printed should not have any bearing on what the actual relationship is but, apparently, preferences and publication bias do come into the picture with quite some force.
Hence, publication bias vastly dominates any of the statistical biases we get so worked up about, making them pretty much irrelevant. Is this a sad state of affairs? Ehm…. I think yes. Is there an easy solution for it? Ehm… I think no. And that is why we will likely all be suffering from publication bias for quite some time to come.
Suppose you want to find out how many members of a rugby club have their nipples pierced (to pick a random example). The problem is, the club has 200 members and you don’t want to ask them all to take their shirts off. Therefore, you select a sample of 20 of them guys and ask them to bare their chests. After some friendly bantering they agree, and then it appears that no fewer than 15 of them have their nipples pierced, so you conclude that the majority of players in the club likely have undergone the slightly painful (or so I am told) aesthetic enhancement.
The problem is, there is a chance that you’re wrong. There is a chance that due to sheer coincidence you happened to select 15 pierced pairs of nipples where among the full set of 200 members they are very much the minority. For example, if in reality out of the 200 rugby blokes only 30 have their nipples pierced, due to sheer chance you could happen to pick 15 of them in your sample of 20, and your conclusion that “the majority of players in this club has them” is wrong.
Now, in our research, there is no real way around this. Therefore, the convention among academic researchers is that it is ok, and you can claim your conclusion based on only a sample of observations, as long as the probability that you are wrong is no bigger than 5%. If it ain’t – and one can relatively easily compute that probability – we say the result is “statistically significant”. Out of sheer joy, we then mark that number with a cheerful asterisk * and say amen.
Now, I just said that “one can relatively easily compute that probability” but that is not always entirely true. In fact, over the years statisticians have come up with increasingly complex procedures to correct for all sorts of potential statistical biases that can occur in research projects of various natures. They treat horrifying statistical conditions such as unobserved heterogeneity, selection bias, heteroscedasticity, and autocorrelation. Let me not try to explain to you what they are, but believe me they’re nasty. You don’t want to be caught with one of those.
Fortunately, the life of the researcher is made easy by standard statistical software packages. They offer nice user-friendly menus where one can press buttons to solve problems. For example, if you have identified a heteroscedasticity problem in your data, there are various buttons to press that can cure it for you. Now, note that it is my personal estimate (but notice, no claims of an asterisk!) that about 95 out of a 100 researchers have no clue what happens within their computers when they press one of those magical buttons, but that does not mean it does not solve the problem. Professional statisticians will frown and smirk at the thought alone, but if you have correctly identified the condition and the way to treat it, you don’t necessarily have to fully understand how the cure works (although I think it often would help selecting the correct treatment). So far, so good.
Here comes the trick: All of those statistical biases are pretty much irrelevant. They are irrelevant because they are all dwarfed by another bias (for which there is no life-saving cure available in any of the statistical packages): publication bias.
The problem is that if you have collected a whole bunch of data and you don’t find anything or at least nothing really interesting and new, no journal is going to publish it. For example, the prestigious journal Administrative Science Quarterly proclaims in its “Invitation to Contributors” that it seeks to publish “counterintuitive work that disconfirms prevailing assumptions”. And perhaps rightly so; we’re all interested in learning something new. So if you, as a researcher, don’t find anything counterintuitive that disconfirms prevailing assumptions, you are usually not even going to bother writing it up. And in case you’re dumb enough to write it up and send it to a journal requesting them to publish it, you will swiftly (or less swiftly, dependent on what journal you sent it to) receive a reply that has the word “reject” firmly embedded in it.
Yet, unintended, this publication reality completely messes up the “5% convention”, i.e. that you can only claim a finding as real if there is only a 5% chance that what you found is sheer coincidence (rather than a counterintuitive insight that disconfirms prevailing assumptions). In fact, the chance that what you are reporting is bogus is much higher than the 5% you so cheerfully claimed with your poignant asterisk. Because journals will only publish novel, interesting findings – and therefore researchers only bother to write up seemingly intriguing counterintuitive findings – the chance that what they eventually are publishing is BS unwittingly is vast.
A recent article by Simmons, Nelson, and Simonsohn in Psychological Science (cheerfully entitled “False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant”) summed it up prickly clearly. If a researcher, running a particular experiment, does not find the result he was expecting, he may initially think “that’s because I did not collect enough data” and collect some more. He can also think “I used the wrong measure; let me use the other measure I also collected” or “I need to correct my models for whether the respondent was male or female” or “examine a slightly different set of conditions”. Yet, taking these (extremely common) measures raises the probability that what the researcher finds in his data is due to sheer chance from the conventional 5% to a whopping 60.7%, without the researcher realising it. He will still cheerfully put the all-important asterisk in his table and declare that he has found a counterintuitive insight that disconfirms some important prevailing assumption.
In management and strategy research we do highly similar things. We for instance collect data with two or three ideas in mind in terms of what we want to examine and test with them. If the first idea does not lead to a desired result, the researcher moves on to his second idea and then one can hear a sigh of relief behind a computer screen that “at least this idea was a good one”. In fact, you might only be moving on to “the next good idea” till you have hit on a purely coincidental result: 15 bulky guys with pierced nipples.
Things get really “funny” when one realises that what is considered interesting and publishable is different in different fields in Business Studies. For example, in fields like Finance and Economics, academics are likely to be fairly skeptical whether Corporate Social Responsibility is good for a firm’s financial performance. In the subfield of Management people are much more receptive to the idea that Corporate Social Responsibility should also benefit a firm in terms of its profitability. Indeed, as shown by a simple yet nifty study by Marc Orlitzky, recently published in Business Ethics Quarterly, articles published on this topic in Management journals report a statistical relationship between the two variables which is about twice as big as the ones reported in Economics, Finance, or Accounting journals. Of course, who does the research and where it gets printed should not have any bearing on what the actual relationship is but, apparently, preferences and publication bias do come into the picture with quite some force.
Hence, publication bias vastly dominates any of the statistical biases we get so worked up about, making them pretty much irrelevant. Is this a sad state of affairs? Ehm…. I think yes. Is there an easy solution for it? Ehm… I think no. And that is why we will likely all be suffering from publication bias for quite some time to come.
Monday, December 12, 2011
Most People Don't Know Their Business (so asking them is useless)
I’ll admit it; I am rapidly becoming a skeptic when it comes to interview-based data. And the reason is that people (interviewees) just don’t know their business – although, of course, they think they do.
For example, in an intriguing research project with my (rather exceptional) PhD student Amandine Ody, we asked lots of people in the Champagne industry whether different Champagne houses paid different prices for a kilogram of their raw material: grapes. The answer was unanimously and unambiguously “no”; everybody pays more or less the same price. But when we looked at the actual data (which are opaque at first sight and pretty hard to get), the price differences appeared huge: some paid 6 euros for a kilogram, others 8, and yet other 10 or even 12. Thinking it might be the (poor) quality of the data, we obtained a large sample of similar data from a different source: supplier contracts. Which showed exactly the same thing. But the people within the business really did not know; they thought everybody was paying about the same price. They were wrong.
Then Amandine asked them which houses supplied Champagne for supermarket brands (a practice many in the industry thoroughly detest, but it is very difficult to observe who is hiding behind those supermarket labels). They mentioned a bunch of houses, both in terms of the type of houses and specific named ones, who they “were sure were behind it”. And they quite invariably were completely wrong. Using a clever but painstaking method, Amandine deduced who was really supplying the Champagne to the supermarkets, and she found out it was not the usual suspects. In fact, the houses that did it were exactly the ones no-one suspected, and the houses everyone thought were doing it were as innocent as a newborn baby. They were – again – dead wrong.
And this is not the only context and project where I have had such experiences, i.e. it is not just a French thing. With a colleague at University College London – Mihaela Stan – we analyzed the British IVF industry. One prominent practice in this industry is the role of a so-called integrator; one medical professional who is always “the face” towards the patient, i.e. a patient is always dealing with one and the same doctor or nurse, and not a different one very time the treatment is in a different stage. All interviewees told us that this really had no substance; it was just a way of comforting the patient. However, when we analyzed the practice’s actual influence – together with my good friend and colleague Phanish Puranam – we quickly discovered that the use of such an integrator had a very real impact on the efficacy of the IVF process; women simply had a substantially higher probability of getting pregnant when such an integrator, who coordinates across the various stages of the IVF cycle, was used. But the interviewees had no clue about the actual effects of the practice.*
My examples are just conjectures, but there is also some serious research on the topic. Olav Sorenson and David Waguespack published a study on film distributors in which they showed that these distributors’ beliefs about what would make a film a success were plain wrong (they just made them come true by assigning them more resources based on this belief). John Mezias and Bill Starbuck published several articles in which they showed how people do not even know basic facts about their own companies, such as the sales of their own business unit, error rates, or quality indicators. People more often than not were several hundreds of percentages of the mark, when asked to report a number.
Of course interviews can sometimes be interesting; you can ask people about their perceptions, why they think they are doing something, and how they think things work. Just don’t make the mistake of believing them.
Much the same is true for the use of questionnaires. They are often used to ask for basic facts and assessments: e.g. “how big is your company”, “how good are you at practice X”, and so on. Sheer nonsense is the most likely result. People do not know their business, both in terms of the simple facts and in terms of the complex processes that lead to success or failure. Therefore, do yourself (and us) a favor: don’t ask; get the facts.
For example, in an intriguing research project with my (rather exceptional) PhD student Amandine Ody, we asked lots of people in the Champagne industry whether different Champagne houses paid different prices for a kilogram of their raw material: grapes. The answer was unanimously and unambiguously “no”; everybody pays more or less the same price. But when we looked at the actual data (which are opaque at first sight and pretty hard to get), the price differences appeared huge: some paid 6 euros for a kilogram, others 8, and yet other 10 or even 12. Thinking it might be the (poor) quality of the data, we obtained a large sample of similar data from a different source: supplier contracts. Which showed exactly the same thing. But the people within the business really did not know; they thought everybody was paying about the same price. They were wrong.
Then Amandine asked them which houses supplied Champagne for supermarket brands (a practice many in the industry thoroughly detest, but it is very difficult to observe who is hiding behind those supermarket labels). They mentioned a bunch of houses, both in terms of the type of houses and specific named ones, who they “were sure were behind it”. And they quite invariably were completely wrong. Using a clever but painstaking method, Amandine deduced who was really supplying the Champagne to the supermarkets, and she found out it was not the usual suspects. In fact, the houses that did it were exactly the ones no-one suspected, and the houses everyone thought were doing it were as innocent as a newborn baby. They were – again – dead wrong.
And this is not the only context and project where I have had such experiences, i.e. it is not just a French thing. With a colleague at University College London – Mihaela Stan – we analyzed the British IVF industry. One prominent practice in this industry is the role of a so-called integrator; one medical professional who is always “the face” towards the patient, i.e. a patient is always dealing with one and the same doctor or nurse, and not a different one very time the treatment is in a different stage. All interviewees told us that this really had no substance; it was just a way of comforting the patient. However, when we analyzed the practice’s actual influence – together with my good friend and colleague Phanish Puranam – we quickly discovered that the use of such an integrator had a very real impact on the efficacy of the IVF process; women simply had a substantially higher probability of getting pregnant when such an integrator, who coordinates across the various stages of the IVF cycle, was used. But the interviewees had no clue about the actual effects of the practice.*
My examples are just conjectures, but there is also some serious research on the topic. Olav Sorenson and David Waguespack published a study on film distributors in which they showed that these distributors’ beliefs about what would make a film a success were plain wrong (they just made them come true by assigning them more resources based on this belief). John Mezias and Bill Starbuck published several articles in which they showed how people do not even know basic facts about their own companies, such as the sales of their own business unit, error rates, or quality indicators. People more often than not were several hundreds of percentages of the mark, when asked to report a number.
Of course interviews can sometimes be interesting; you can ask people about their perceptions, why they think they are doing something, and how they think things work. Just don’t make the mistake of believing them.
Much the same is true for the use of questionnaires. They are often used to ask for basic facts and assessments: e.g. “how big is your company”, “how good are you at practice X”, and so on. Sheer nonsense is the most likely result. People do not know their business, both in terms of the simple facts and in terms of the complex processes that lead to success or failure. Therefore, do yourself (and us) a favor: don’t ask; get the facts.
Wednesday, December 7, 2011
The Lying Dutchman: Fraud in the Ivory Tower
The fraud of Diederik Stapel – professor of social psychology at Tilburg University in the Netherlands – was enormous. His list of publications was truly impressive, both in terms of the content of the articles as well as its sheer number and the prestige of the journals in which it was published: dozens of articles in all the top psychology journals in academia with a number of them in famous general science outlets such as Science. His seemingly careful research was very thorough in terms of its research design, and was thought to reveal many intriguing insights about fundamental human nature. The problem was, he had made it all up…
For years – so we know now – Diederik Stapel made up all his data. He would carefully review the literature, design all the studies (with his various co-authors), set up the experiments, print out all the questionnaires, and then, instead of actually doing the experiments and distributing the questionnaires, made it all up. Just like that.
He finally got caught because, eventually, he did not even bother anymore to really make up newly faked data. He used the same (fake) numbers for different experiments, gave those to his various PhD students to analyze, who then in disbelief slaving away in their adjacent cubicles discovered that their very different experiments led to exactly the same statistical values (a near impossibility). When they compared their databases, there was substantial overlap. There was no denying it any longer; Diederik Stapel, was making it up; he was immediately fired by the university, admitted to his lengthy fraud, and handed back his PhD degree.
In an open letter, sent to Dutch newspapers to try to explain his actions, he cited the huge pressures to come up with interesting findings that he had been under, in the publish or perish culture that exist in the academic world, which he had been unable to resist, and which led him to his extreme actions.
There are various things I find truly remarkable and puzzling about the case of Diederik Stapel.
• The first one is the sheer scale and (eventually) outright clumsiness of his fraud. It also makes me realize that there must be dozens, maybe hundreds of others just like him. They just do it a little bit less, less extreme, and are probably a bit more sophisticated about it, but they’re subject to the exact same pressures and temptations as Diederik Stapel. Surely others give in to them as well. He got caught because he was flying so high, he did it so much, and so clumsily. But I am guessing that for every fraud that gets caught, due to hubris, there are at least ten other ones that don’t.
• The second one is that he did it at all. Of course because it is fraud, unethical, and unacceptable, but also because it sort of seems he did not really need it. You have to realize that “getting the data” is just a very small proportion of all the skills and capabilities one needs to get published. You have to really know and understand the literature; you have to be able to carefully design an experiment, ruling out any potential statistical biases, alternative explanations, and other pitfalls; you have to be able to write it up so that it catches people’s interest and imagination; and you have to be able to see the article through the various reviewers and steps in the publication process that every prestigious academic journal operates. Those are substantial and difficult skills; all of which Diederik Stapel possessed. All he did is make up the data; something which is just a small proportion of the total set of skills required, and something that he could have easily outsourced to one of his many PhD students. Sure, you then would not have had the guarantee that the experiment would come out the way you wanted them, but who knows, they could.
• That’s what I find puzzling as well; that at no point he seems to have become curious whether his experiments might actually work without him making it all up. They were interesting experiments; wouldn’t you at some point be tempted to see whether they might work…?
• Truly amazing I also find the fact that he never stopped. It seems he has much in common with Bernard Madoff and his Ponzi Scheme, or the notorious traders in investments banks such as 827 million Nick Leeson, who brought down Barings Bank with his massive fraudulent trades, Societe Generale’s 4.9 billion Jerome Kerviel, and UBS’s 2.3 billion Kweku Adoboli. The difference: Stapel could have stopped. For people like Madoff or the rogue traders, there was no way back; once they had started the fraud there was no stopping it. But Stapel could have stopped at any point. Surely at some point he must have at least considered this? I guess he was addicted; addicted to the status and aura of continued success.
• Finally, what I find truly amazing is that he was teaching the Ethics course at Tilburg University. You just don’t make that one up; that’s Dutch irony at its best.
For years – so we know now – Diederik Stapel made up all his data. He would carefully review the literature, design all the studies (with his various co-authors), set up the experiments, print out all the questionnaires, and then, instead of actually doing the experiments and distributing the questionnaires, made it all up. Just like that.
He finally got caught because, eventually, he did not even bother anymore to really make up newly faked data. He used the same (fake) numbers for different experiments, gave those to his various PhD students to analyze, who then in disbelief slaving away in their adjacent cubicles discovered that their very different experiments led to exactly the same statistical values (a near impossibility). When they compared their databases, there was substantial overlap. There was no denying it any longer; Diederik Stapel, was making it up; he was immediately fired by the university, admitted to his lengthy fraud, and handed back his PhD degree.
In an open letter, sent to Dutch newspapers to try to explain his actions, he cited the huge pressures to come up with interesting findings that he had been under, in the publish or perish culture that exist in the academic world, which he had been unable to resist, and which led him to his extreme actions.
There are various things I find truly remarkable and puzzling about the case of Diederik Stapel.
• The first one is the sheer scale and (eventually) outright clumsiness of his fraud. It also makes me realize that there must be dozens, maybe hundreds of others just like him. They just do it a little bit less, less extreme, and are probably a bit more sophisticated about it, but they’re subject to the exact same pressures and temptations as Diederik Stapel. Surely others give in to them as well. He got caught because he was flying so high, he did it so much, and so clumsily. But I am guessing that for every fraud that gets caught, due to hubris, there are at least ten other ones that don’t.
• The second one is that he did it at all. Of course because it is fraud, unethical, and unacceptable, but also because it sort of seems he did not really need it. You have to realize that “getting the data” is just a very small proportion of all the skills and capabilities one needs to get published. You have to really know and understand the literature; you have to be able to carefully design an experiment, ruling out any potential statistical biases, alternative explanations, and other pitfalls; you have to be able to write it up so that it catches people’s interest and imagination; and you have to be able to see the article through the various reviewers and steps in the publication process that every prestigious academic journal operates. Those are substantial and difficult skills; all of which Diederik Stapel possessed. All he did is make up the data; something which is just a small proportion of the total set of skills required, and something that he could have easily outsourced to one of his many PhD students. Sure, you then would not have had the guarantee that the experiment would come out the way you wanted them, but who knows, they could.
• That’s what I find puzzling as well; that at no point he seems to have become curious whether his experiments might actually work without him making it all up. They were interesting experiments; wouldn’t you at some point be tempted to see whether they might work…?
• Truly amazing I also find the fact that he never stopped. It seems he has much in common with Bernard Madoff and his Ponzi Scheme, or the notorious traders in investments banks such as 827 million Nick Leeson, who brought down Barings Bank with his massive fraudulent trades, Societe Generale’s 4.9 billion Jerome Kerviel, and UBS’s 2.3 billion Kweku Adoboli. The difference: Stapel could have stopped. For people like Madoff or the rogue traders, there was no way back; once they had started the fraud there was no stopping it. But Stapel could have stopped at any point. Surely at some point he must have at least considered this? I guess he was addicted; addicted to the status and aura of continued success.
• Finally, what I find truly amazing is that he was teaching the Ethics course at Tilburg University. You just don’t make that one up; that’s Dutch irony at its best.
Wednesday, November 16, 2011
What's wrong with senior executive pay – lots in my view
There are three things I do not like about top management pay: 1) they usually get paid too much, 2) way too large a part is flexible, performance-related pay, 3) often, a very sizeable chunk of it is paid through stock options.
I used to think - naively - that high top management pay was high simply due to supply and demand: these smart people with lots of business acumen and experience are hard to come by; therefore you have to pay them lots. These grumpy anti-corporates claiming their pay is too high are just envious and naive. Turns out I was (maybe not envious, but certainly naive).
Pay level
Because digging into the rigorous research on the topic - and there is quite a bit of it - I learned that there is really not much of a relationship between firm performance and top management pay. These guys (mostly guys) get paid a lot whether or not their company's performance is any good. Moreover, I learned what sort of factors push up top managers' remuneration - and it ain't supply and demand. It has much more to do with selecting the right company directors (to serve on your remumeration committee) and making sure you are well networked and socialized into the business elite.* Now I have to conclude: top management pay is generally too high, and quite a bit too high.
Flexible pay
Secondly: where does this absurd idea come from that 80+ percent of these guys' remuneration has to be performance related?! "To reward them for good performance and stimulate them to act in the best interest of the company and its shareholders" you might say? To which I would reply "oh, come on!?" If your CEO is the type of guy who needs 90 percent performance-related pay or otherwise he won't act in the best interest of the company, I would say the perfect time to get rid of him is yesterday. You and I do not need 90 percent performance related pay to do our best, do we? So why would it be allowed to hold for top managers? As Henry Mintzberg put it: "Real leaders don't take bonuses".
Moreover, one should only pay performance-related remuneration if you can actually measure the person's performance. And that is - especially for top managers - actually pretty darn hard to do. The strategic decisions one takes this year will often only be felt 5 or 10 years from now, if not longer. Moreover, the performance of the company - which we always take to proxy the CEO's performance - is influenced by a whole bunch of other things; many not under a CEO's control. Hence, short term financial performance figures are a terrible indicator of a top manager's performance in the job and long-term performance contracts all but impossible to specify. If you can't reliably measure performance, don't have performance-related pay, and certainly not 80+ percent of it. We know from ample research that humans start manipulating their performance when you tie their remuneration to some strange metric and, guess what, CEOs are pretty human (at least in that respect); they do too.
Options
Finally: stock options... Once again, I have to say "oh, come on...". We pretty much take for granted that we pay top managers by awarding them options, but don't quite realize any more why. When I ask this question to my students or the executives in my lecture room ("why do we actually pay them in options...?") usually a stunned silence follows after which someone mumbles "because they are cheap to hand out...?". I usually try to remain polite after such an answer but why would they be cheap; cheaper than cash, or shares for that matter? True, it does not cost you anything out of pocket if you give them an option to buy shares for say 100 one year from now, while your present share price is 90, but if the share price by that time is 150 it does cost you 50. Moreover, you could have sold that stock option to someone who would have happily paid you good money for it, so in terms of opportunity costs it is realy money too. No, stock options are not cheaper than cash, shares, or whatever.
We give them options to stimulate them to take more risk. "Risk?! We want them to take more risk?!" thou might think. Yes, that's what you are doing if you give them options. If the share price is 150 at the time the option expires, the CEO can buy the shares at 100 and thus make 50. However, if the share price is 90 the option is worthless, and the CEO does not make anything. However, the trick is that the CEO then does not care whether the share price is 90 or, say, 50 - in either case he does not make any money; worthless is worthless. As a consequence, when his options (i.e. the right to buy shares at 100) are about to expire and the company's share price is still 90, he has a great incentive to quickly take a massive amount of risk. Going to a roulette table would already be a rational to do.
Because if you placed the company's capital on red, and the ball hits red, share price may jump from 90 to 130, and suddenly your options are worth a lot of money (130-100 to be precise). However, if your bet fails, the ball hits black and you lose a ton of money, who cares; the share price may fall from 90 to 50, but your options were worthless anyway. Hence, options give a top manager the upside risk, as we say, but do not give them the downside risk. Therefore, we incentivize them to take risk. You might think "I seldom see herds of CEOs in a casino by the time options expire, so this grumpy Vermeulen guy must be exaggerating" but I'd reply we have seen quite a lot of casino-type strategy in various businesses lately (e.g. banks). More importantly, we know from research that CEOs do take excessive risk due to stock options (see for instance Sanders and Hambrick, 2007; Zhang e.a. 2008). I think it would be naive to think that we give CEOs 90 percent performance related pay and most of it in stock options, and then think that they will not start acting in the way the remuneration system stimulates them to do. Of course it influences their decisions, and if it didn't, there would be no reason left to make their pay flexible and based on options, now would there?
Therefore, I would say, out with the performance-related pay for top managers (a good bottle of wine at Christmas and, if you insist, a small cheque like the rest of us would do). And while we're at it, let's try to reduce the level as well.
I used to think - naively - that high top management pay was high simply due to supply and demand: these smart people with lots of business acumen and experience are hard to come by; therefore you have to pay them lots. These grumpy anti-corporates claiming their pay is too high are just envious and naive. Turns out I was (maybe not envious, but certainly naive).
Pay level
Because digging into the rigorous research on the topic - and there is quite a bit of it - I learned that there is really not much of a relationship between firm performance and top management pay. These guys (mostly guys) get paid a lot whether or not their company's performance is any good. Moreover, I learned what sort of factors push up top managers' remuneration - and it ain't supply and demand. It has much more to do with selecting the right company directors (to serve on your remumeration committee) and making sure you are well networked and socialized into the business elite.* Now I have to conclude: top management pay is generally too high, and quite a bit too high.
Flexible pay
Secondly: where does this absurd idea come from that 80+ percent of these guys' remuneration has to be performance related?! "To reward them for good performance and stimulate them to act in the best interest of the company and its shareholders" you might say? To which I would reply "oh, come on!?" If your CEO is the type of guy who needs 90 percent performance-related pay or otherwise he won't act in the best interest of the company, I would say the perfect time to get rid of him is yesterday. You and I do not need 90 percent performance related pay to do our best, do we? So why would it be allowed to hold for top managers? As Henry Mintzberg put it: "Real leaders don't take bonuses".
Moreover, one should only pay performance-related remuneration if you can actually measure the person's performance. And that is - especially for top managers - actually pretty darn hard to do. The strategic decisions one takes this year will often only be felt 5 or 10 years from now, if not longer. Moreover, the performance of the company - which we always take to proxy the CEO's performance - is influenced by a whole bunch of other things; many not under a CEO's control. Hence, short term financial performance figures are a terrible indicator of a top manager's performance in the job and long-term performance contracts all but impossible to specify. If you can't reliably measure performance, don't have performance-related pay, and certainly not 80+ percent of it. We know from ample research that humans start manipulating their performance when you tie their remuneration to some strange metric and, guess what, CEOs are pretty human (at least in that respect); they do too.
Options
Finally: stock options... Once again, I have to say "oh, come on...". We pretty much take for granted that we pay top managers by awarding them options, but don't quite realize any more why. When I ask this question to my students or the executives in my lecture room ("why do we actually pay them in options...?") usually a stunned silence follows after which someone mumbles "because they are cheap to hand out...?". I usually try to remain polite after such an answer but why would they be cheap; cheaper than cash, or shares for that matter? True, it does not cost you anything out of pocket if you give them an option to buy shares for say 100 one year from now, while your present share price is 90, but if the share price by that time is 150 it does cost you 50. Moreover, you could have sold that stock option to someone who would have happily paid you good money for it, so in terms of opportunity costs it is realy money too. No, stock options are not cheaper than cash, shares, or whatever.
We give them options to stimulate them to take more risk. "Risk?! We want them to take more risk?!" thou might think. Yes, that's what you are doing if you give them options. If the share price is 150 at the time the option expires, the CEO can buy the shares at 100 and thus make 50. However, if the share price is 90 the option is worthless, and the CEO does not make anything. However, the trick is that the CEO then does not care whether the share price is 90 or, say, 50 - in either case he does not make any money; worthless is worthless. As a consequence, when his options (i.e. the right to buy shares at 100) are about to expire and the company's share price is still 90, he has a great incentive to quickly take a massive amount of risk. Going to a roulette table would already be a rational to do.
Because if you placed the company's capital on red, and the ball hits red, share price may jump from 90 to 130, and suddenly your options are worth a lot of money (130-100 to be precise). However, if your bet fails, the ball hits black and you lose a ton of money, who cares; the share price may fall from 90 to 50, but your options were worthless anyway. Hence, options give a top manager the upside risk, as we say, but do not give them the downside risk. Therefore, we incentivize them to take risk. You might think "I seldom see herds of CEOs in a casino by the time options expire, so this grumpy Vermeulen guy must be exaggerating" but I'd reply we have seen quite a lot of casino-type strategy in various businesses lately (e.g. banks). More importantly, we know from research that CEOs do take excessive risk due to stock options (see for instance Sanders and Hambrick, 2007; Zhang e.a. 2008). I think it would be naive to think that we give CEOs 90 percent performance related pay and most of it in stock options, and then think that they will not start acting in the way the remuneration system stimulates them to do. Of course it influences their decisions, and if it didn't, there would be no reason left to make their pay flexible and based on options, now would there?
Therefore, I would say, out with the performance-related pay for top managers (a good bottle of wine at Christmas and, if you insist, a small cheque like the rest of us would do). And while we're at it, let's try to reduce the level as well.
Wednesday, September 28, 2011
Can countries benefit from having their domestic firms acquired by foreign companies?
When a foreign company acquires a domestic firm, it often leads to outcries of indignation, nostalgia (“another of our once great companies in foreign hands”), and calls for legislation to prevent any more foreign poaching. Politicians and union leaders proclaim that the foreign owners may not be dedicated to keep up investment in the subsidiary, and that the take-over threatens national jobs and other economic interests. “Most governments are reluctant to see their corporate treasures fall into foreign hands”, the BBC wrote in an article devoted to the topic.
But is all this (slightly xenophobic) fear justified? Well, maybe not; at least not on all dimensions. Because we have increasing evidence that foreign ownership of a firm may actually also benefit firms, specifically in terms of their innovativeness. And this increased innovativeness may clearly benefit the host country.
Professor Annique Un, from Northeastern University in Boston, for example, did a pointy study. She collected data on 761 manufacturing firms operating in Spain, examined which ones were foreign hands and what their innovation output was in terms of new products introduced in the market. And the answer was pretty clear: foreign owned firms were more innovative than purely domestic firms.
Interestingly, Annique also corrected her models for the amount of R&D investments spent in the companies, and it turned out that this was not what was driving it; foreign owned companies were not just more innovative because they were investing more. Instead, they were more innovative irrespective of R&D. As a matter of fact, they were able to generate more product innovations for the same level of investment; meaning that they were simply better at it.
The study’s results suggested that they were better at it for two reasons. First, foreign parents seemed to use their domestic subsidiary to channel innovation into the country. Put differently, it seemed a foreign-owned company could tap into its parent’s superior repository of innovative stuff, and most of them gratefully made ample use of that option. Secondly, the foreign-owned companies were simply also better at coming up with new stuff on their own, in comparison to their domestic counterparts. Apparently, something about them being foreign-owned stimulated them to be more agile and creative, which resulted in more product introductions.
Whatever the reason behind this foreign-driven surge in innovation, the host country was better off for it; the evidence clearly showed that the foreign mercenaries stimulated diversity in the markets, giving customers more choice, while raising the bar for everyone. And this is not a benefit we hear many politicians, newspapers, and union leaders proclaim and acknowledge, when yet another foreign corporation is eyeing up their country’s corporate treasures.
But is all this (slightly xenophobic) fear justified? Well, maybe not; at least not on all dimensions. Because we have increasing evidence that foreign ownership of a firm may actually also benefit firms, specifically in terms of their innovativeness. And this increased innovativeness may clearly benefit the host country.
Professor Annique Un, from Northeastern University in Boston, for example, did a pointy study. She collected data on 761 manufacturing firms operating in Spain, examined which ones were foreign hands and what their innovation output was in terms of new products introduced in the market. And the answer was pretty clear: foreign owned firms were more innovative than purely domestic firms.
Interestingly, Annique also corrected her models for the amount of R&D investments spent in the companies, and it turned out that this was not what was driving it; foreign owned companies were not just more innovative because they were investing more. Instead, they were more innovative irrespective of R&D. As a matter of fact, they were able to generate more product innovations for the same level of investment; meaning that they were simply better at it.
The study’s results suggested that they were better at it for two reasons. First, foreign parents seemed to use their domestic subsidiary to channel innovation into the country. Put differently, it seemed a foreign-owned company could tap into its parent’s superior repository of innovative stuff, and most of them gratefully made ample use of that option. Secondly, the foreign-owned companies were simply also better at coming up with new stuff on their own, in comparison to their domestic counterparts. Apparently, something about them being foreign-owned stimulated them to be more agile and creative, which resulted in more product introductions.
Whatever the reason behind this foreign-driven surge in innovation, the host country was better off for it; the evidence clearly showed that the foreign mercenaries stimulated diversity in the markets, giving customers more choice, while raising the bar for everyone. And this is not a benefit we hear many politicians, newspapers, and union leaders proclaim and acknowledge, when yet another foreign corporation is eyeing up their country’s corporate treasures.
Saturday, September 17, 2011
Don’t be mistaken, bankers kill (but they give life too)
"In terms of power and influence, you can forget the church, forget politics. There is no more powerful institution in society than business” the equally famous as illustrious CEO and founder of the BodyShop – the late Dame Anita Roddick – said. And of course she was right. The most comprehensive and dominant institution in today’s society is business.
Business is more influential than people often realize, simply because it creates – or destroys – wealth. And wealth impacts pretty much anything we care about. Whether you analyze crime rates in a particular country, malnutrition, happiness, or infant mortality; a huge influence is how wealthy the particular society is. And wealth is created by business.
As a consequence, for example, the 2008 banking crisis undoubtedly killed people. Infant mortality is closely related to wealth and consequently an economic crisis will among others lead to a surge in infant mortality, somewhere, in some country down the road. It also means that the strategic business choices made by CEOs such as Lehman’s Richard Fuld or RBS’s Fred Goodwin indirectly but significantly influence the survival chances of some baby boy or girl born on the outskirts of London, Cairo, or Detroit. And therefore, whether you like it or not, bankers kill.
But let’s not forget that they give life too. The inverse of “bankers kill” is true too. If banks make wise choices, given their pivotal role in our economies, they can trigger a huge boost to the prosperity of many industries. And the profits, employment, and general wealth created through this boost will really improve the health and survival chances of the baby cradled by her mother somewhere on the outskirts of London, Cairo, or Detroit.
Given the research we have on the link between economic prosperity and infant mortality it would not even be too onerous to come up with some estimate of the direct relationship between Royal Bank of Scotland’s balance sheet and the probability of a baby surviving. We could relatively easily calculate the link between profit and the number of lives saved. I could even imagine that the computer terminals that give live updates of a company’s fluctuating share price – which many corporations have dotted across their entrance halls and offices for everyone to see – would be reprogrammed to display the number of children’s lives saved. Traders walking over to their lunch break could have an immediate update of how many baby lives the deal they just closed saved – or destroyed.
A ridiculous thought? Why? Don’t you care (even) more about the life or death of a baby than your company’s fluctuating share price? I am guessing you do. And you know these bankers aren’t so different from (other) human beings. Your company’s performance also creates wealth, and wealth saves lives. Why then only monitor its financial performance? I tell you, the sandwich you’re having for lunch will taste a whole lot better, knowing that this morning you just saved some unknown baby’s life, somewhere on the outskirts of London, Cairo, or Detroit.
Business is more influential than people often realize, simply because it creates – or destroys – wealth. And wealth impacts pretty much anything we care about. Whether you analyze crime rates in a particular country, malnutrition, happiness, or infant mortality; a huge influence is how wealthy the particular society is. And wealth is created by business.
As a consequence, for example, the 2008 banking crisis undoubtedly killed people. Infant mortality is closely related to wealth and consequently an economic crisis will among others lead to a surge in infant mortality, somewhere, in some country down the road. It also means that the strategic business choices made by CEOs such as Lehman’s Richard Fuld or RBS’s Fred Goodwin indirectly but significantly influence the survival chances of some baby boy or girl born on the outskirts of London, Cairo, or Detroit. And therefore, whether you like it or not, bankers kill.
But let’s not forget that they give life too. The inverse of “bankers kill” is true too. If banks make wise choices, given their pivotal role in our economies, they can trigger a huge boost to the prosperity of many industries. And the profits, employment, and general wealth created through this boost will really improve the health and survival chances of the baby cradled by her mother somewhere on the outskirts of London, Cairo, or Detroit.
Given the research we have on the link between economic prosperity and infant mortality it would not even be too onerous to come up with some estimate of the direct relationship between Royal Bank of Scotland’s balance sheet and the probability of a baby surviving. We could relatively easily calculate the link between profit and the number of lives saved. I could even imagine that the computer terminals that give live updates of a company’s fluctuating share price – which many corporations have dotted across their entrance halls and offices for everyone to see – would be reprogrammed to display the number of children’s lives saved. Traders walking over to their lunch break could have an immediate update of how many baby lives the deal they just closed saved – or destroyed.
A ridiculous thought? Why? Don’t you care (even) more about the life or death of a baby than your company’s fluctuating share price? I am guessing you do. And you know these bankers aren’t so different from (other) human beings. Your company’s performance also creates wealth, and wealth saves lives. Why then only monitor its financial performance? I tell you, the sandwich you’re having for lunch will taste a whole lot better, knowing that this morning you just saved some unknown baby’s life, somewhere on the outskirts of London, Cairo, or Detroit.
Monday, August 29, 2011
Boards and fraud – who gets the sack and who gets to stay?
We have seen lots of corporate scandals over the past decade, and in many of these cases the boards of directors were up for some heavy criticism. Whether it was Enron, Tyco, WorldCom, or one of the toppled investment banks, their boards took some flack, since of course they are ultimately responsible for the corporation’s actions.
But what happens to such directors? What happens to these people in the business elite when their company, for example, is caught being involved in financial fraud? Well, perhaps not surprisingly – and this may come as a relief – they often get the sack (as research by Professor Arthaud-Day from from Kansas State University and colleagues convincingly showed). Directors associated with financial misrepresentations are often dismissed from the board of their fraudulent company but, interestingly, subsequently they also regularly get the boot at another board. As you may know, outside directors often serve on the boards of multiple companies and a study by Professor Srinivasan from the Harvard Business School showed that they lose about 25 percent of these (rather lucrative) jobs if one of the companies in their portfolio is caught up in fraud.
Yet, this also implies that 75 percent of companies retain a particular board member, even though he or she is compromised having served on the board of another company while it was committing fraud. And that begs the question, what firms decide to retain such a tainted board member, and which ones decide give them the sack?
Professors Amanda Cowen and Jeremy Marcel from the University of Virginia decided to examine this. They managed to collect data on 277 directors who served on multiple boards concurrently, one of which was associated with financial fraud. Their statistical analysis showed that companies that were covered by more equity analysts and governance-rating agencies were more likely to dismiss compromised board members; up to twice as likely. These external observers apparently serve as a bit of watchdog. However, surprisingly, when a company had a relatively large number of public pension fund investors amongst its shareholders, they were less likely to dismiss a compromised board member. Cowen and Marcel speculated that this was because these pension fund shareholders do the monitoring themselves, so that they don’t care much about the company’s directors; tainted or not.
You also have to realize who does the firing; and that is the rest of the board. Cowen and Marcel’s research also showed that very prestigious, well-networked boards were less likely to fire their tainted fellow director. It is well known that boards of directors form a rather cliquish corporate elite. It is not easy to find your way into this world, but once your solidly in, not even a little financial fraud is going to convince your corporate buddies to throw you out.
But what happens to such directors? What happens to these people in the business elite when their company, for example, is caught being involved in financial fraud? Well, perhaps not surprisingly – and this may come as a relief – they often get the sack (as research by Professor Arthaud-Day from from Kansas State University and colleagues convincingly showed). Directors associated with financial misrepresentations are often dismissed from the board of their fraudulent company but, interestingly, subsequently they also regularly get the boot at another board. As you may know, outside directors often serve on the boards of multiple companies and a study by Professor Srinivasan from the Harvard Business School showed that they lose about 25 percent of these (rather lucrative) jobs if one of the companies in their portfolio is caught up in fraud.
Yet, this also implies that 75 percent of companies retain a particular board member, even though he or she is compromised having served on the board of another company while it was committing fraud. And that begs the question, what firms decide to retain such a tainted board member, and which ones decide give them the sack?
Professors Amanda Cowen and Jeremy Marcel from the University of Virginia decided to examine this. They managed to collect data on 277 directors who served on multiple boards concurrently, one of which was associated with financial fraud. Their statistical analysis showed that companies that were covered by more equity analysts and governance-rating agencies were more likely to dismiss compromised board members; up to twice as likely. These external observers apparently serve as a bit of watchdog. However, surprisingly, when a company had a relatively large number of public pension fund investors amongst its shareholders, they were less likely to dismiss a compromised board member. Cowen and Marcel speculated that this was because these pension fund shareholders do the monitoring themselves, so that they don’t care much about the company’s directors; tainted or not.
You also have to realize who does the firing; and that is the rest of the board. Cowen and Marcel’s research also showed that very prestigious, well-networked boards were less likely to fire their tainted fellow director. It is well known that boards of directors form a rather cliquish corporate elite. It is not easy to find your way into this world, but once your solidly in, not even a little financial fraud is going to convince your corporate buddies to throw you out.
Subscribe to:
Posts (Atom)