Healthy Skeptism of Medical Market Analysis

A recent article in The Lancet (“Promoting Healthy Scepticism of Health Statistics,” by Tim Parsons, Public Affairs Office, Johns Hopkins Bloomberg School of Public Health, Baltimore) pointed up the need to understand how specific statistics have been compiled, who the sources are and what the potential biases may be in order to make good decisions based on those statistics.  While the article focused on the use of health statistics by consumers and policymakers, the premise triggers us to highlight parallel issues in the generation and use of medical technology market data.  And although the Lancet article concentrated on the idea that consumers may be ill-prepared to understand the sources and biases of health statistics (and therefore have less scepticism about such data) and policymakers may be too prone to ignore biases, this tendency toward poor understanding of data and willingness to ignore bias are not problems limited to consumers and policymakers.  The medical technology industry and its stakeholders — comprised of medical product companies, venture capitalists, PR firms, publishers, consultants, etc. — have equal tendency to misunderstand or misuse data.

The data in question for the medical technology industry includes examples like the size and growth of a particular market segment, the “leading players”, and other measures.  Without a good understanding of sources, methodologies and other factors that influence the estimates that are available, bad decisions can clearly ensue.

Apples to Apples

By far, the most common source of confusion in analyses is what is being measured.  Does the market include equipment, devices, supplies, services or all of these?  What are the clinical applications for which this market is defined, or do the estimates encompass sales of the products used for all applications?  Different analysts may define the markets differently and without those definitions being clearly laid out, there can be limited value in the use of the resulting data, especially if the intention is to compare the estimates to another analyst’s estimates

The apples/apples problem is compounded by the “data inertia” problem.

Data Inertia 

We have previously coined and used the term “data inertia” to describe the tendency for published data estimates to stick in the minds of the industry, sometimes ignoring very real changes that have taken place in markets that necessarily change those estimates.  For example, the number of hysterectomies performed annually in the U.S. has previously been “stuck” in the industry’s mind at 650,000.  While there are some changes, such as the emergence of laparoscopically-assisted vaginal hysterectomy techniques, that have reduced the procedure’s associated trauma and made the procedure easier to perform (potentially increasing the number of candidates), the procedure continues to be a target of accusations that it is frequently performed unnecessarily, while at the same time technologies such as endometrial ablation have emerged to obviate the need for hysterectomy.  Consequently, the absolute number of hysterectomies has been in decline for several years, yet due to “data inertia”, the figure of 650,000 annual hysterectomies frequently appears in estimates. 

Data can be “sticky” in this way when the actual numbers behind estimate do not change significantly over time, when a well produced analysis gains lots of exposure or when an estimate is done in a narrow area in which few alternative analyses have been done.  In the end, the more critical the impact of an estimate in its ultimate use, the greater the challenge must be made in verifying the estimate.

Growth Rates: Compound, Annual, Average, Aggregate

Without going into a math lesson, the use of different growth rate measures can complicate the analysis and make its conclusions murky.  Obviously, qualitative terms are often used that are indistinct — high growth, flat market, replacement market, even “hockey stick” growth.  They must be defined to mean anything.  (A “flat” market can variously be described as zero growth, or <5% annual growth.  A “replacement” market is one in which sales are limited to replacement of units sold previously.  “Hockey stick” growth is that longed-for emerging market growth that goes from several years of no growth or low growth to a sudden introductory growth rate in double-digit-plus rate.)  But quantitative measures of growth can also be misused (intentionally or no) and therefore misunderstood.  Most conventionally, the use of “Compound Annual Growth Rate” (CAGR) is the accepted metric.  CAGR is the growth rate, if applied year-over-year, from Year 1 to Year X that will result in the market growth from year 1 to year x.  It does not speak of variation in growth within that term, just from Year  1 to X.  An alternative growth measure is “Average Growth Rate”, which simply averages the growth rates of each yearly growth from Year 1 to X.  CAGR and Average Growth Rate will, when the annual growth rate is linear, produce the same result, but if there is non-linear growth, the Average Growth Rate will understate the growth.

Ultimately, the question is how accurately one wants or needs to describe the growth.  Suffice it to say that the CAGR approach leaves no doubt, but sometimes the rule that applies is what Mark Twain said, “Never let the truth get in the way of a good story.”

Objective or Subjective.  Impartial or incentivized?

As the article in The Lancet noted, the ideal analysis has a good disconnection between the entity performing the analysis and those with a stake (especially financial) in the outcome of hte analysis.  Considering common sources for data in the medical technology industry — company sales/marketing departments, company CEOs/VPs/Directors/etc., venture capitalists, physicians and others — the analysts using these sources must temper each data point by knowledge of its source and its potential bias.  By doing so, and by “triangulating” the data points from multiple sources, the third party publisher (in particular) can generate an estimate that is reasonably defensible, because the case for objectivityis is stronger with the third party publisher.

Which measures are used, and the degree to which sources and methodologies are disclosed, says much more about those presenting the analysis than it does about the market or topic in question.  Be that as it may, the medical technology industry stakeholder must always keep in mind the purpose to which an estimate will be used when choosing among alternatives.  That choice can establish credibility, encourage optimism, inspire confidence, secure funding and achieve other goals, or it can cause quite the opposite.

From the March 2007 edition of MedMarkets.

Tags: medtech

Leave a Reply