Tuesday 22 May 2007

Sampling issues

When peddling their wares to analysts, vendors are often opposed an apparently unbeatable argument: the number of inquiries from users. We have seen Borg analysts using it in different ways, one being the small number of customer calls equates a slow of market acceptance.

There is apparently no come back -it's the A-bomb: Gartner is right because it has insight from its end-user client base.

However, this is somewhat simplistic and AR professionals should challenge analysts to explore the following avenues:

  • Customers can be calling for a number of reasons but more often than not when they have an issue. No problems, no calls. This means RAS analysts don't tend to hear from satisfied customers and may thus develop a tendency to paint a picture darker than reality.
  • The issues customers are calling about may be different than the analyst's assumptions, for instance they may be confused rather than not buying at all.
  • Finally Gartner's users customer base may be very different from a given vendor -they are less represented in SME's and Southern Europe for instance.

3 comments:

alan pelz-sharpe said...

Does this sort of thing not point to the need for a clear division between buy side and sell side analysts?

If your analysis is to help advise users/buyers like ours (CMS Watch) or Burton for example - surely you want to be clear that there is no conflict in motivation....."buy our wares because we talk to a lot of your users"....is a little questionable isn't it?

At some point I think the industry analyst community will have to become more transparent regarding conflicts of interest.

Anonymous said...

Re: "The small number of customer calls equates a slow of market acceptance" - I'm not sure this is a good place to start. There is an inherent assumption that the analyst role is all about shortlisting products, and therefore the above is around product enquiries... whereas we are seeing plenty of evidence that customers require assistance to frame the situations they are in, and these may not be linked directly to products. A simple example is governance/compliance/risk - plenty of interest, but which can't be defined in terms of market acceptance.

Meanwhile we are frequently talking to vendors who are doing good things for their customers, but who don't get visibility because their products don't fit the predefined models. I personally believe that while some established technologies can be defined quite nicely as quadrants or whatever, more leading edge solutions suffer from such mechanisms.

So, I don't believe Gartner has the A-bomb on this. It may be that its clients aren't talking about certain things, or it may just be that in certain circumstances, the analysts are asking the wrong questions, or prompting the wrong calls. I have no evidence of this obviously - I'm just stating it as a potential reason for fewer calls.

ARonaut said...

Can't agree more Jonno, we were just saying that the Borg uses the fact they have user insight in arguments.

In fact, they may just be talking to other customers or maybe it's only unsatisfied customers going to Gartner.