Open wide for some soccer arguments.

News

Opinion: Deloitte’s Pro/Rel Study is Useless

by on 21 November 2016

Promotion and Relegation Is A Thing, Say Consultants Paid To Say So

At midnight Eastern time, on Monday the 21st of November, the English consulting firm Deloitte released a summary of a study — commissioned by Miami FC owner Riccardo Silva — on the topic of promotion and relegation in US Soccer. It’s hard to convey in writing the absurdity of the whispered buzz that this embargoed report generated after its existence was leaked to the general public on Saturday. Suffice to say, if you knew nothing about it until now, your life was better for it because it is a hypeburger packed with empty calories.

The summary report mainly catalogs the common arguments for and against promotion and relegation (from here on out, referred to as “pro/rel”), proposes the obvious remedies to the challenges of implementation, and displays the results of a survey of 1,000 American soccer fans through a collage of cheaply made infographics. (As a student in a design school, understand that I am an authority on cheaply made infographics.) It’s as if the study authors had never encountered the pro/rel debate before and believe what they’ve uncovered is new.

What is missing from the document is any kind of hard-won knowledge or analysis. Perhaps that can be found in the full report itself, which may not be released, but there’s no indication from the summary that the rest of the study goes much deeper. Even low-hanging fruit goes unpicked. Consider the canard — regurgitated in the executive summary — that “Promotion and relegation introduces competition at the top and the bottom of leagues, increasing the number of matches in a season with something ‘at stake’.”

The idea that leagues with pro/rel contain more meaningful games is dubious and is a great example of something that could use some clarification by paid consultants. In MLS, much of the league is still in playoff contention in the final weeks of the season. Further up, the top group of teams are maneuvering for a first-round bye and the best in the league are competing for the Supporters Shield. In contrast, over in England, teams like Stoke might enter March or April with a rough idea that they will finish out of the European places but clear of relegation and thus have nothing to play for. In MLS, they’d be battling for playoffs. Is it so cut and dry that leagues with pro/rel produce more meaningful contests down the stretch? Perhaps, but perhaps not, and this study doesn’t attempt to lay some ground-rules, total up the matches, and find an answer.

More egregious is the study’s lack of curiosity about some of the central claims of the pro/rel argument.

More egregious is the study’s lack of curiosity about some of the central claims of the pro/rel argument. To pick just one and attach no added importance to it, the study suggests that pro/rel could aid in youth development because; “Ambitious ownership at all levels may improve facility and coaching provision at a larger number of clubs.”

This strikes me as precisely the opposite of what would occur. In a pro/rel system, the overarching incentive for owners becomes the current first team and its success in staying clear of the drop. If pro/rel were suddenly instituted, would a bottom feeding MLS team like the Chicago Fire or an ambitious NASL team like the New York Cosmos invest each marginal dollar in an academy? Of course not! Investing in long-term benefits like an academy structure or training facilities requires stability. It may take five years for the first players truly incubated in your academy to come of age. If investing in the uncertain gains of an academy would jeopardize your first division status, no club would do it.

“This is not intended in any way to be a polemic or the final word…it’s very much intended to be a piece that helps inform the discussion around this topic.” – Dan Jones,
Head of Deloitte’s Sports Business Group

The Deloitte study was an opportunity to plumb the depths of questions like this one, where crucial insight could be gleaned by a study of English academy expenditures and production, or research from the field of behavioral economics. One of the critical flaws of the pro/rel discussion is that it is nearly impossible to develop good research design to test the effects of a pro/rel system against a closed system. To the extent that it is possible, you’d suspect that an international consulting firm would be the ones to do it. Instead, the consultants were content to merely reproduce talking points and proceed as if they were self-evident. On Neil Morris’ excellent Inverted Triangle Podcast (listen to this), Dan Jones, the head of Deloitte’s sports business group, says “this is not intended in any way to be a polemic or the final word…it’s very much intended to be a piece that helps inform the discussion around this topic.”

In what way to does this study inform the discussion?

Would pro/rel result in higher match attendance and TV audiences? Would pro/rel raise the profile of the league in the United States? Would pro/rel help professionalize the front offices in lower divisions? Would pro/rel attract more stable and committed ownership in lower divisions? Would pro/rel lead to more investment, overall, in the sport? Would it lead to less? What would pro/rel do to player salaries? Would pro/rel lead to a higher standard of play? Could pro/rel be compatible with a salary cap and, if not, what could be done to prevent the dominance of New York, Los Angeles, and other big markets? Why is pro/rel important to soccer’s success in the US and Canada when it has not inhibited the growth of professional football, baseball, basketball, and hockey?

I don’t want to suggest in this piece that I have a better handle on sports business than the Deloitte consultants. I certainly understand that they did not have access to the kind of data (from MLS teams, for instance) that would have aided a deeper analysis. And not all of these questions may be possible to answer. Certainly few can be answered with anything approaching certainty. Many of these questions may be answered favorably for the pro/rel system and many may have negative implications. But this much-touted study answers precisely none of them and, as a result, it does a disservice to us all.

The one thing it does do is provide us with the results of a survey of slightly more than 1,000 American soccer fans. This survey is interesting, but it’s also not very useful. Respondents, for example, report that they would be seven times more likely to watch more club soccer games on TV if pro/rel were introduced than they would be to watch less club soccer games on TV. What does this mean? What do we gain from this survey result? How would someone even say with any confidence what they would do, as a consumer, if pro/rel were implemented in the United States? I certainly have no idea how my soccer watching habits would change — do you? As a review of broad sentiments to the topic of pro/rel, the study is telling. But as an article that enriches the raging debate on the topic, it provides little.

To cut through American soccer’s byzantine pro/rel debate, what we need is a serious economic analysis from an impartial source. There exist ways to conduct original research on the subject. There can be no meaningful discussion of the topic without someone actually sitting down and doing the research. In that process, trade-offs may become clear. Do Americans want an open system if it stifles youth development and reduces the league to a system where the title is contested between four to five teams each year? Do Americans want a closed system if it perennially is killing off lower divisions and MLS increasingly resembles the no-fun NFL?

To arrive at these honest reckonings, we need some real analysis to build on. This isn’t it.

Tags: , , ,

  • Sean DiSesa

    If investing in an academy requires stability, then why does any club in an open system have an academy?

    • Alex Schieferdecker

      Many have academies, few all have good academies that they invest lots of money into. Only the big clubs do that.

      • Sean DiSesa

        So the likes of Southampton, Rennes, Feyenoord, Lens, Real Sociedad are “big clubs”? They might be top flight but they certainly aren’t Europe’s elite. In the lower divisions it’s more important to rely on your academy to either sell off players for a profit or play well enough to get promoted.

        • Alex Schieferdecker

          They are big clubs. They are fairly secure in their placement in the top divisions. That security allows them to continue to invest in their academies (which return a profit through player sales). What this report really could’ve looked into was how clubs like Derby County, Nottingham Forest, and Le Havre are investing their money.

          I don’t know the answer, I’d love to see.

          • Sean DiSesa

            That’s not true though. Just of the examples I listed, Lens are currently in the second division, while Southampton and Real Sociedad have been up and down in the past decade. Most of those clubs are closer to fighting relegation than fighting for Champions League. And there are many more examples like that throughout Europe. There just isn’t any practical evidence to support the claim that closed leagues produce superior academies.

          • Alex Schieferdecker

            “There just isn’t any practical evidence to support the claim that closed leagues produce superior academies.”

            Precisely. Nor is there evidence that open leagues produce superior academies. What the topic needs is a serious analysis, and that was not provided by this report.

          • Sean DiSesa

            What? The best academies in the world exist in the open league format. They are far superior to any academy that exists in the closed league format. There’s your evidence.

          • Alex Schieferdecker

            Almost every academy in the world exists in an open league. Those that exist in a closed league are barely a decade old.

            To prove something, you need to be able to isolate and manipulate an independent variable and track the response of the dependent variable. The inability to do that is what I meant when I wrote “One of the critical flaws of the pro/rel discussion is that it is nearly impossible to develop good research design to test the effects of a pro/rel system against a closed system.”

            In other words, because the best leagues in the world are open doesn’t mean that open leagues are best. In this case, it just means that almost ALL leagues are open.

  • Mitch

    It’s not useless, just not very useful. People want pro/rel because we want to see every club have the opportunity to grow in an open, merit based, system. The status quo is disenfranchising many clubs and fans. An open system is better for many reasons, imho.

    • Alex Schieferdecker

      Right. Which is why the study was useless. It did nothing more than restate those reasons, instead of investigating them and providing evidence for or against.

    • Dave Laidig

      An open system has benefits, but also has negatives. And some of the good things going on with the league right now (relative parity, attracting investment) will be harmed. So they key questions are “will the benefits of a change outweigh the harm”. Right now, Deloitte says no, P/R shouldn’t happen yet. But following from that is “If not now, then when?” And the study provides very little useful information on that part.

  • randomhookup

    I wonder if the consultants on the study were all Europeans or did they involve any of their American staff? The diversity of opinions would have perhaps given the study a broader understanding of the marketplace, if they didn’t include non-Europeans.

    • Dave Laidig

      It was Deloitte UK that issued the report

      • randomhookup

        Yes, but Deloitte is a global firm and it’s not uncommon to use assets from other countries. Only using UK based consultants would really bias the results.

  • Dave Laidig

    What gets lost in the hype is the report concluded “US club soccer is not immediately ready for promotion and relegation.” And in order to get to P/R, a series of complicated economic issues must be resolved among scores of stakeholders with competing interests. The report named three (but there is more) items that need to be addressed for P/R:

    (1) Decisions made on the optimum number of teams in the existing leagues;

    (2) The continued development and stability of a second tier competition to develop clubs capable in management and football terms of joining the first tier; and

    (3) Consideration of the mechanism by which long term league investors have their “equity” protected, at least in the short term, from relegation.

    These issues are contentious and have been debated for years. And this is what is so underwhelming about the report. It’s easy to paint a rosy picture when the hard part is stuffed into 3 bullet points at the end of the document.

    • Alex Schieferdecker

      Right on. Huge missed opportunity to get some outside expertise to navigate these longstanding questions and challenges.

  • Dave Laidig

    The poll and facts pulled by Deloitte are of questionable quality and utility. And when it makes these kinds of errors, one has to wonder if its overall conclusions are flawed as well.

    For example, the poll is of “1058 soccer fans.” After this election, it’s clear how polls should be interpreted with great caution. And this makes no explanation on how the respondents were found – and we can’t tell if they represent all soccer fans or not. Also, if soccer is going to be more successful in the U.S., more non-soccer fans need to be converted. Their voice is not part of this at all.

    Further, the poll has wacky results. If pro/rel, fans are 8X more likely to watch USMNT? What does that have to do with pro/rel? And that factor is larger than those more likely to watch club soccer. How does that make sense?

    • Alex Schieferdecker

      This is why I was so disappointed to only get the summary. I wanted to read the methodology of their poll.

      • Dave Laidig

        NERD!

        (I agree. I wonder where these respondents came from, the response rate, how they were screened as “US soccer fans”, how the people were contacted, and the script for the questions…among other things)

        • Alex Schieferdecker

          Particularly interested in the script, given how weird the wording is for some of the infographics.

  • Dave Laidig

    The Deloitte analysis itself shows signs of selecting data points to fit foregone conclusions.

    For instance, the study cherry picks attendance data to suggest MLS is stagnant. For example, it ignores 66% of the league and focuses on attendance for the 8 teams that started in ’96. And then the study inappropriately reports a “Compound Annual Growth Rate” instead of net increases, or avg attendances. Then, in a pretty cheeky twist, three of the expansion teams (can you guess who?) with high attendance are used to show promoted teams would improve attendance.

    Forget that the original teams are going to be capped by stadium size. And that comparing 20 years to 4 or 5 years is going to skew the numbers reported by Deloitte. We’ll also set aside the irony of using the opposite of pro/rel (league expansion in a closed system) as an example of promotion. Any reasonable analysis of the future of soccer in the U.S. needs to address the elephant in the room – TV revenue is not sufficient to maintain a top soccer league. Talking about attendance figures is a red herring when the need is eyeballs watching on TV.

    The only mention of TV viewership in the Deloitte infographic is the promotion playoffs in England. And what is the point there? All US Leagues already have playoffs, and we know that they get better ratings. If the point is that promotion can add a couple games to watch, that is easily undertaken by adding a playoff game (without the economic hardship of relegation), another competition or perhaps just showing the US Open Cup all the way through.

    Paid for by someone with a vested interest, this study has the earmarks of a paid-for publication.

  • Offensive Loons Fan

    Acadamies are a good example of the chicken and egg nature of promotion and relegation. A strong academy can help to ensure long term financial feasibility and a talent pipeline. It can help your club go from good to great. But you have to be great in order to meaningfully prioritize it. So what comes first?

    I wish a pro/rel advocate would step out and just say “look, pro/rel is a straight up gamble and I don’t know what would happen, but it feels right, it feels like soccer is stable enough in this country for it to succeed, and though I recognize it’s a gamble, I think it’s a great gamble to make.” At least then we are debating what this truly is: opinions and impressions. A quick way to weed someone out of your life is to block people who say they KNOW what would happen if pro/rel were introduced.