After decades of struggling to boost its profile beyond that of a commuter college, the University of Missouri-Kansas City finally could call itself a global leader.
“UMKC ranked No. 1 in the World,” a 2011 university news release said.
An academic study had ranked UMKC’s business school ahead of Harvard, Stanford and other top colleges in innovation management research — the study of how entrepreneurs turn good ideas into big bucks and jobs.
“Oh my, have we made a big score,” Chancellor Leo Morton told a crowd at the formal announcement.
But a Kansas City Star investigation raises questions about that score and other rankings achieved by UMKC’s Henry W. Bloch School of Management. The Star found a pattern of exaggerations and misstatements that polished the school’s reputation as it sought to boost enrollment and open donors’ checkbooks.
At the newspaper’s request, independent experts analyzed the No. 1-in-the-world study, which was published in the Journal of Product Innovation Management. One concern the experts cited: a previously undisclosed relationship between the university and the study’s Chinese authors.
In addition, the experts said, it appears the study may have been structured in such a way to ensure that the Bloch School received the top ranking.
At the time the paper was written, the two authors were working on the UMKC campus as visiting scholars at the Bloch School. Their yearlong visit was at the invitation of then-dean Teng-Kee Tan and professor Michael Song.
The two women had come from the same Chinese university where Song had been a part-time professor the previous four years. At UMKC, the three shared the same office number. And, The Star has learned, the authors consulted with Song about the paper before its publication.
Both UMKC and officials at the Journal of Product Innovation Management, or JPIM, say those relationships do not negate the study’s findings.
But the undisclosed relationship and the study’s unorthodox methodology raised red flags for several experts. At a school named for Henry W. Bloch, an entrepreneur whose motto is “no shortcuts,” the revelations sting, some students and faculty members say.
“They named themselves the No. 1 in innovation technology,” said former Ph.D. candidate Xian Cao, “but I don’t really believe it.”
A professor who feared that speaking out would harm his career said he and his colleagues were skeptical even as the study’s results were announced.
“We all knew that this was bullshit,” he said. “We knew that UMKC was not better than MIT and Stanford.”
Another professor, Richard Arend, demanded an investigation. Despite assurances to the contrary from UMKC and the journal’s editors, he remains convinced that the study was contrived, with help from people within the university, to deliver UMKC its top standing.
“Science is about investigating strange outcomes,” Arend told The Star. “For example, when UMKC is ranked above more-established, better-funded, private institutions like Stanford, MIT and Harvard in an area of knowledge that they are world renowned for, there are questions.”
In a written statement, UMKC called Arend’s suspicions groundless and dismissed him as “a disgruntled Bloch School faculty member whom Michael Song declined to recommend for promotion.”
But he was not alone. Former Bloch School assistant professor H. Dennis Park, now at Drexel University in Philadelphia, said he left UMKC in part because of what he thought were inflated rankings.
“I was a little concerned. ‘What if these things got out?’” he said. “It is sort of like these people who were working for Enron before the disaster happened.”
At The Star’s request, the editors of two academic journals analyzed the JPIM study, as did the co-founder of a website that highlights flaws in academic research.
Like Arend, all either questioned the JPIM study’s validity or cited potential shortcomings in the methodology and data collection.
“I just think this paper is fatally flawed,” said Ivan Oransky, co-founder of RetractionWatch.com.
Oransky was among dozens of people inside and outside UMKC interviewed for this story. The Star also reviewed thousands of pages of internal UMKC documents obtained through an open-records request, as well as information from other sources.
From those interviews and records, reporters found a number of other embellishments that boosted the Bloch School’s reputation in recent years.
Among them were inaccuracies and mischaracterizations of fact in the data the university supplied to the Princeton Review, which has awarded UMKC’s entrepreneurship program high rankings in four of the past five years.
UMKC denies it “engaged in ‘a pattern of exaggerations’ or took ‘short cuts’ on a path to achieving national and global recognition and rankings.”
Tan stepped down as dean last year for health reasons and sent word that he was unable to respond to requests for comment.
Song denied that he inflated the accomplishments of the entrepreneurship program he founded at UMKC. He dismissed as “nonsense” allegations that the JPIM paper was rigged to deliver UMKC the top world ranking, and he said he couldn’t recall what if any role he had in its publication.
“Did I see a version of their paper? I don’t really remember,” he said.
Perhaps the only person to whom the Bloch School rankings were more important than Tan and Song was Henry Bloch himself. The co-founder of the HR Block tax preparation firm had long pushed for recognition for a school he considered one of his legacies to Kansas City.
In 2011, he finally made a donation that UMKC had long sought to add a building at the growing Bloch School.
The $32 million gift, which came a few months before the No. 1 ranking, was the largest in UMKC history. And it was one, Bloch told The Star, that he doubts he would have made absent the other rankings that were coming the school’s way.
‘A virtuous cycle’
The Bloch School had few rankings to brag about before Tan left Nanyang Technological University in Singapore for UMKC in 2009.
The Bloch School appeared on none of the lists ranking the best business schools in the nation. Not Bloomberg/Businessweek. Not The Economist. Not Forbes or U.S. News World Report.
UMKC wanted Tan to change that.
On top of his base salary of $410,000 — more than Morton was being paid to run the entire university — Tan was promised an annual bonus of up to $50,000 if he could boost the school’s profile, enlist community support and raise money.
Tan understood rankings’ role in reaching all three goals.
“Top ranking results,” Tan wrote a friend several months ago, “is the trigger of a virtuous cycle for building brand equity and excellence and sustainable resources for today’s business schools.”
Without top rankings, the Bloch School had struggled for years to raise money to hire new faculty and add degree programs.
One example: the failure to match a $12.5 million challenge grant that the Kauffman Foundation pledged in the mid-2000s toward development of the Bloch School’s then-fledgling entrepreneurship education program.
“If it is matched, it will give the Bloch School a tremendous lift,” a report by the Greater Kansas City Community Foundation said before the September 2007 deadline.
“On the other hand, failure to match will be seen as a significant failure of leadership by the university, the Bloch School and the philanthropic community.”
Failure came, and UMKC left roughly half of the Kauffman money on the table.
The next year, when the university sought replacements for both the chancellor and the dean of the business school, fundraising ability was high on the lists of job requirements.
The Bloch School needed someone who could get the business and philanthropic community excited, and Tan promised to do just that.
“I faced the search committee,” Tan told a reporter soon after moving into the wood-paneled dean’s suite at Oxford Hall. “I said, ‘My job is to make your dreams come true.’”
Tan began to make that happen by expanding the executive education certificate program, which provides training for employees of area companies. He established ties with overseas universities, especially in China, as a way to attract foreign students.
And he set an ambitious goal to double Bloch School enrollment within five years.
It wouldn’t be easy, but that’s where top rankings can be helpful, said John Byrne, who in 1988 developed the first regular business school rankings system while at BusinessWeek.
“It’s the No. 1 criteria an applicant uses to choose a business school,” said Byrne, who runs the graduate business education website Poets Quants.
But not all rankings carry the same weight. Those handed out by U.S. News and Bloomberg/Businessweek, for example, are held in higher regard than those awarded by the Princeton Review, known for its “Top 20 Party Schools.”
Criticism of the Princeton Review, which has no association with Princeton University, stems from the company’s refusal to disclose its methodology.
“We don’t know the weight of any of the 40 data points,” Byrne said. That’s why he thinks the rankings are “dramatically flawed.”
Still, Tan was thrilled in 2009 when the Bloch School entered the list at No. 25 for its entrepreneurship program.
“It will lift the entire standard of the business school as a whole,” he said, “and attract nationally the top student candidates. They will come here, start ventures and take root here in Kansas City.”
‘We wanted a builder’
Song came to UMKC in 2004 with the understanding that he would build a “nationally pre-eminent” entrepreneurship program for the Bloch School, based in part on his reputation as a top academic.
“Michael had the most impressive research portfolio,” said Lee Bolman, the interim dean at the time. “He had been a very productive scholar and, second, he exudes energy and enthusiasm. Michael is an entrepreneur himself. He’s a builder, and we wanted a builder.”
By the time Tan arrived, Song was well on his way to shaping the program that he and his employers had envisioned, the Institute for Entrepreneurship and Innovation.
His sales pitch to potential donors emphasized how Kansas City’s economy would benefit from an institute that did cutting-edge research and provided hands-on training to budding entrepreneurs.
“We cannot make UMKC into Stanford,” Song said. But by focusing on a single niche, training entrepreneurs how to succeed, the Bloch School could distinguish itself, he said.
Tan embraced that vision, and together the two would go on to aggressively promote the institute, known as IEI.
That initial top 25 Princeton Review ranking wasn’t treated merely as welcome recognition for the Bloch School. Tan called it a signature event for UMKC as a whole.
“My goal is to put Bloch on the map by making us known for what we do best,” he said. “Rising waters lift all boats, and along that journey, we bring up the standards of everything we do at Bloch.”
It was slow going at first. In 2010, the Bloch School fell off the Princeton Review list.
On occasion, embellishment was a stand-in for accomplishment. In a 2010 video uploaded to YouTube, Song claimed the Bloch School had won accolades from The New York Times.
“We have been featured in The New York Times as one of the two national best models for entrepreneurial education,” Song said in that video greeting to guests at a university event.
In fact, that 2005 article in The Times made no such endorsement. Rather, the one-sentence mention said the Bloch School had “a strong emphasis on entrepreneurship” and was “adding a program in January in which student teams create and run their own companies for six months.”
Questioned about it recently, Song acknowledged the “error,” attributing it to “a poor choice of words.”
In its formal response, UMKC said the IEI “as an entity has never made a claim of endorsement by The New York Times.”
But Song was not alone. Morton repeated the claim, documents show, as did the IEI in its successful 2012 application for an award from the U.S. Association for Small Business and Entrepreneurship.
Even today, The New York Times logo appears at the bottom of the institute’s home page, alongside logos for awards and rankings the IEI has received.
The JPIM paper
Although rankings were hard to come by in 2010, the next year saw two major announcements.
The first was September’s pairing of Henry Bloch’s $32 million gift for a new building with news that not only was the graduate entrepreneurship program back on Princeton Review’s top 25 list, but that the undergraduate program had made it too.
Then in December came the announcement of the school’s top global ranking.
With an advance copy of the journal study in hand, UMKC administrators had spent weeks that fall planning what they called the “Big Bang Event.”
“The ranking,” said an internal university memo, “will have a significant impact on the reputation of UMKC as a global research university … as well as Kansas City as it positions itself as the ‘entrepreneurial capital of the world.’”
It was Tan who alerted UMKC officials to the ranking months before its publication and pushed for the splashy announcement to a crowd of 300 business leaders and other potential donors.
Yet when Tan recited the biographies of the study’s authors at the event, he gave no clue that they had spent the previous year at UMKC as visiting scholars.
Nor did the journal article mention those ties.
But The Star has learned that the women, PianPian Yang and Lei Tao, arrived at Song’s invitation and that Tan had signed off on visa request forms for them, according to the university system’s assistant custodian of records.
Yang was a Ph.D. candidate and Tao an assistant professor at the Management School of Xi’an Jiaotong University. Song had been a specially appointed professor there from 2006 to 2010 while working full time at UMKC.
Yang’s resume says she was at UMKC from August 2010 to August 2011. At UMKC, she and Tao shared the same office number as Song, and he described himself as “their professor.”
Because the authors did not respond to requests for comment, it’s unclear why they chose to write the academic paper that ranked UMKC No. 1 on a list of the “world’s top innovation management universities” and Song as the globe’s most prolific researcher.
But the paper’s aim, its introduction stated, was to “extend” a 2007 JPIM rankings article by Jeffrey Thieme.
Thieme’s work ranked the top scholars in innovation management. Yang and Tao did that, but they also ranked the top universities where those scholars worked.
Unbeknown to readers of JPIM, there was a common thread between the two studies beyond the fact that both found Song to be the No. 1 researcher.
Song also played a role in each.
It was Song, Thieme said, who suggested he write that first paper for Anthony DiBenedetto, then JPIM’s editor. DiBenedetto also was a friend of Song and collaborator on two dozen papers.
“Michael did say something about it and said that Tony (DiBenedetto) would probably be interested in it,” said Thieme, one of Song’s former doctoral students and now a professor at the University of Memphis. “So I started looking into it and thought, ‘Yeah, there might be something there.’”
Thieme counted the number of articles Song and other scholars published in 14 academic journals over a 15-year period. He chose those 14, he said, because they had been identified in previous studies as the top-ranked journals in the fields of marketing, management and technology innovation management.
The 15-year period he surveyed mirrored the years Song had been publishing papers at the time.
Thieme acknowledged that “counting articles” like his are subjective by nature because the author chooses the journals to survey and the time frame to measure.
“But I didn’t put any subjectivity in mine,” Thieme said, adding that Song clearly deserved the top ranking.
“All I did was lay it out. How the chips fell, that’s how they fell.”
Like Thieme, the two Chinese scholars also ranked innovation management scholars based on how many articles they had published in certain journals.
Except when they did it, five years later, the journal mix was different. They counted articles in 10 journals rather than Thieme’s 14. And instead of 15 years, the time period was 20 years — again mirroring Song’s years of writing journal articles.
The result was clearly beneficial to UMKC and its faculty.
In addition to Song retaining his No. 1 status with 53 articles in those 10 journals, his UMKC colleague and mentor, Mark Parry, rose from 13th place to fourth with 18 papers. Song’s ex-wife and UMKC colleague, Lisa Zhao, who hadn’t made the earlier list, was now ranked 50th.
Meanwhile, some of their peers at other universities saw their rankings drop. Fewer of their articles were counted because of the change in journals selected.
One example: Elko Kleinschmidt, professor emeritus at McMaster University in Ontario, dropped from fifth place to 19th. Thieme’s study credited him with 17 articles during a 15-year period; Yang and Tao’s study credited him with only nine in 20 years.
“I can only think that the inclusion of these (other) journals would have given a somewhat changed top listing,” said Kleinschmidt, who claimed it was “misleading” for Yang and Tao to say the journals they selected were similar to the ones used in the 2007 article.
But for those who would later raise questions about the study, the university rankings were oddest of all.
Traditionally, universities are ranked using a method created by the University of Texas-Dallas to rank the research output at top U.S. business schools. Yang and Tao acknowledged that in their paper.
That method credits scholars’ papers to the universities where they worked at the time the papers were published. The thinking is that approach better reflects the school’s research climate over the long term.
But Yang and Tao chose instead to award credit for all of an author’s papers, no matter where they were written, to the university where the researcher was working at the time the Tao-Yang study was submitted for publication.
They wrote that it was “more appropriate” to rank universities that way and cited a single journal article to support that argument.
In fact, the article they referenced said the method they had chosen tends to distort rankings.
“Some institutions,” wrote Michael Jay Polonsky, a marketing professor at Deakin University in Australia, “have ‘bought’ high profile academics (sometimes even for short periods) through paying high salaries to work for the institution and raise its ranking.”
Between the journal selection and the counting method Yang and Tao used, UMKC came out on top, followed by MIT and Michigan State University.
How would UMKC have done if the study had used the traditional counting method — assigning credit for researchers’ work to the institution where they were employed at the time their papers were published?
Readers were left to guess. But in an email sent to Arend and obtained by The Star, Yang acknowledged that UMKC wouldn’t have made the top 10.
“However, as space is limited,” Yang wrote, “we cut these pages from this research.”
The benefactor
Henry Bloch didn’t know how the Bloch School’s rankings were achieved, who had written the JPIM paper or the other details behind it.
But he was happy that his business school was finally No. 1 in at least one niche.
Bloch had once hired a New York public relations man to get top rankings for the school, but he got nothing for the $50,000 fee.
“The end of the year he said, ‘We are resigning the account. We can’t do it,’” Bloch told The Star. “It is more difficult to get rankings than he thought it was.
“You just can’t buy ’em, you have to earn them.”
That rang true with Bloch, who with his brother Richard turned HR Block into a global phenomenon in the last half of the 20th century.
They took no shortcuts, but as Henry Bloch neared his 90th birthday, he was growing impatient at how long it was taking the school that bore his name to gain national prominence.
Since the initial endowment in the mid-1980s, he’d given the Bloch School millions. He was willing to keep writing checks, but he wanted progress in the rankings as proof that he was making a wise investment.
“He wanted accountability, which any donor would want,” said Guy Bailey, former UMKC chancellor.
UMKC officials responded by keeping Bloch updated on the school’s progress with emails and personal visits. Occasionally, Tan would send small gifts. A box of apples for Henry Bloch. Salmon from Seattle for son Tom Bloch.
To the school’s students, Tan held up Bloch as someone to model themselves after, producing a one-hour documentary film tribute titled “No Shortcuts: The Entrepreneurial Life of Henry Bloch.”
“I am forever grateful to your dad, your self and your family’s generosity and support for the Bloch School,” Tan wrote to Tom Bloch the week before the $32 million gift was announced. “I know we are attracting higher quality students to come study at the Bloch School each year as our ranking continues to excel.”
Henry Bloch told The Star that he had repeatedly turned down university officials’ pleas to pay for a new building at the business school. He said no even after officials talked about how enrollment was growing and UMKC needed more space.
Eventually it was that argument that convinced him. But without rankings, he told The Star, he doubts he would have given UMKC the millions for the new Bloch School building that opened last fall.
“No, I don’t think I would have,” Bloch said.
The skeptics
Soon after the Big Bang celebration in December 2011, Arend began acting on his hunch that the No. 1 ranking was fabricated.
Arend, a professor of strategy and entrepreneurship, had grown suspicious of Song’s approach to rankings. He’d sat in meetings where Song suggested ways to boost the school’s Princeton Review rankings that Arend thought stretched the truth.
After dissecting the JPIM paper and finding what he thought were irregularities with the journal selections and counting methods, Arend began sending a torrent of emails to the paper’s authors, the journal’s editors, its publisher and outside experts. He peppered them with questions and turned up what he thought was some damning information.
From DiBenedetto, who edited both ranking papers, he learned that the Chinese scholars didn’t disclose their affiliation with UMKC when they submitted their paper for publication.
According to emails obtained by The Star, DiBenedetto didn’t see that as a problem, as long as the methodology used was objective, and he said he assumed it was.
As it turned out, DiBenedetto also came out well in the study. He ranked No. 6 worldwide and his university, Temple, was 11th.
From Yang, Arend learned in an email dated May 6, 2012, that DiBenedetto wasn’t the only one who reviewed the paper before its publication. Song had too.
But the bigger revelation from his correspondence with Yang, Arend thought, was her admission that UMKC wouldn’t have made the top 10 had the standard counting method been used.
As Arend pressed for answers, Yang pleaded with Song to intervene.
In a June 27, 2012, email to Morton, Tan and others, Song quoted Yang:
“I have received some very strange communications from your faculty, Richard Arend. At the beginning, I thought that he was interested in learning from our article. … However, he has escalated recently in harassing me with a lot of accusation which caused me significant stress in my life.”
Song told them that he, too, was concerned about Arend’s persistence.
“As you will see from the email exchanges,” Song wrote, “Dr. Arend’s actions have important ramifications, and could result in potential damage to the Bloch School, UMKC, and Kansas City communities.”
Specifically, Song said, Arend was out to damage “my reputation and that of UMKC as the world’s top innovation management university.”
That more or less is UMKC’s official stance when it comes to Arend, who has tenure and, therefore, a good deal of job protection.
“This person has been ‘shopping’ his complaints broadly for years within the academic realm,” according to a statement from the university, “and has been rebuffed at every turn.”
Contrary to UMKC’s assertions that his campaign grew out of a grudge against Song for fighting his promotion, Arend said he started complaining about the ranking before he was up for a full professorship.
Not embarrassed
To evaluate his concerns, The Star sought out independent experts, made them aware of the key circumstances behind the paper’s publication and asked for their analysis.
Two editors of academic business journals and Oransky, the co-founder and curator of RetractionWatch.com, reviewed the paper.
All agreed that the choice of journals used to count articles and the ranking method were curious. All commented critically on the co-authors’ undisclosed affiliation with Song and UMKC.
“If it indeed produces different results,” said Ray Bagby, editor of the journal Entrepreneurship Theory and Practice, then that “is suspicious.”
Oransky, Bagby and Ben Martin, the editor of the journal Research Policy, said the appearance of a potential conflict of interest should have at least been disclosed so that readers could judge for themselves whether the paper was biased.
“A good test in such cases,” Martin said, “is, would the authors now feel embarrassed if the previously undeclared interest were made public?”
UMKC says it is not embarrassed. Disclosure policies vary widely from one journal to the next, said university spokesman John Martellaro. What’s most important, he said, is that the data can stand on its own when scrutinized by fellow scholars.
“The universal standard is still blind review,” Martellaro said. “The editors and peer reviewers do not know the identity of the authors when they review the work, in order to keep personalities, relationships, etc., from influencing their conclusions.”
The paper ranking UMKC No. 1 was not peer reviewed ahead of publication.
However, JPIM’s publisher did assemble what it called an “independent peer review board” after questions about the article arose.
That board was untroubled by the authors’ undisclosed affiliation with the university. All three of its unnamed members said the paper could have been improved, but they would have published it.
Further, all agreed that the data collected involved a certain amount of subjectivity. But as one reviewer said, that’s always the case with rankings articles “and therefore they will always be subject to criticism.”
The potential for manipulating results is why several journal editors contacted by The Star said they refuse to publish rankings papers.
“Our journal is not involved in that and will not get involved in that,” said Richard Bettis, a co-editor at Strategic Management Journal.
Others say the culture of self promotion within academia has colleges always on the lookout for rankings and studies that put them in the best light. Sometimes they stretch the limits.
“People come up with new and different ways to ‘count’ publications and their own schools often come out quite well,” said Angelo DeNisi, former editor of the Academy of Management Journal.
The methods and relationships that allowed UMKC to come out well in the JPIM article do not trouble Dave Donnelly, who succeeded Tan as dean.
“That is what everybody does,” he said.
These days he is talking less about entrepreneurship and innovation and more about the Bloch School’s other business disciplines.
“The JPIM ranking has put us as the top program in innovation, but after a few years you’ve got to move on,” Donnelly said. “That is why we talk now about finance, executive education, and we continue to talk about the new things we are doing.”
After hearing The Star’s findings last week, Henry Bloch said in a written statement that the school’s leadership “continues to have my full confidence and support.”
He also wrote that he had invested in the school because of its growing enrollment and “across-the-board great performance.”
As for the two men who came to UMKC with a mission to raise the entrepreneurial profile of the Bloch School, neither holds an administrative post today.
As soon as Donnelly’s status changed from acting to permanent dean in March, he said, he told Song that he would need to step away from his position as director of IEI.
Between some family health issues and his consulting work, Song no longer had the time Donnelly thought was needed to run the institute.
Now, after Song pulls into his reserved space outside the new Bloch Executive Hall for Entrepreneurship and Innovation, where he once commanded a spacious office, he treks to a small one next door in Oxford Hall. This fall, he’s scheduled to teach two classes.
As for Tan, he remains a member of the faculty. But because of his health concerns, it’s unclear whether he will teach at the Bloch School again.
Under Tan, Bloch School enrollment did not double, but it did go up 25 percent to 1,925 undergraduate and graduate students. Since his arrival, the school has amassed $48 million in gifts, pledges and planned donations.
For his achievements, a plaque in the atrium of the Bloch School’s new building bears his name.
On his watch, it reads, the school accomplished many things. Among them, “international recognition.”
How Chinese scholars’ choice affected rankings
Their methodology: Scholars PianPian Yang and Lei Tao gave credit for researchers’ studies to the university where the researchers were currently working, not to the university that employed them when they did the work. Here are their top 10 universities for innovation management:
1. University of Missouri-Kansas City
2. Massachusetts Institute of Technology
3. Michigan State University
4. INSEAD (an international business school)
5. Harvard University
6. University of Pennsylvania
7. Northeastern University
8. Texas AM University
9. Stanford University
10. Delft University of Technology in the Netherlands
Traditional method: If the scholars had used the traditional method — giving credit to the university that employed researchers when they did their work — UMKC wouldn’t have made the top 10.
1. Michigan State University
2. Massachusetts Institute of Technology
3. University of Michigan
4. University of Pennsylvania
5. Harvard University
6. Delft University of Technology
7. INSEAD
8. Northeastern University
9. University of Texas at Austin
10. University of North Carolina at Chapel Hill and Carnegie Mellon
UMKC"s misleading march to the top
Không có nhận xét nào:
Đăng nhận xét