Tuesday, 28 February 2006

Gartner limits vendor briefings to 30 minutes

In a mass distribution e-mail today, Gartner announced a new policy effective March 31st that vendor briefings would be limited to 30 minutes.

Borg imposes 30mn briefing limit

So it looks like the Borg is about to limit briefings duration: Gartner limits vendor briefings to 30 minutes.

Vinnie, Dean and Jon have good comments on this, the two sides of the argument are:

  • Many vendors impose a death-by-powerpoint sentence to analysts with 3 hours and 54 charts briefing. This is BAD practice (see links below). A half an hour session should thus force vendors go straight to the point an be more productive.
  • Analyst briefings should be interactive ; 1800 seconds does not allow much of a two-way conversation.

  • ARmadgeddon is against unnecessary analyst cruelty and agrees that vendors could do a better job at scoping briefings to better address analysts needs. However, 30 mn is too short to allow for an interactive conversation. The fact that this new rule comes from the Borg is not a surprise as Gartner analysts tend to be more quiet and provide significantly less feedback than independent analysts during non-paid briefings. As a side note, arranging briefings through vendor relations takes Analyst Relations Managers anywhere from 1 hour to 2 days...

    POJ's wrote a nice take on this on GartnerWatch: You have the right to...
    His point that Gartner analysts would struggle to explain anything in less than 40 mn is spot on.


    More from ARmadgeddon on analyst briefings:
  • AR 101: Jon Collins on briefing analysts
  • [Monkchips:] No Time Wasters Please: On briefing industry analysts
  • Redmonk: how to brief analysts
  • Monday, 27 February 2006

    Will the Borg be dis-intermediated?

    Interesting and thought provoking post by former analyst Dana Gardner on why blogs and the information abundance is a threat to Gartner:

    Self-IT analysis is the wave of the future (Dana Gardner, ZDNet.com)

    His analysis is quite similar to Jame's (read this).

    There's of course the reverse argument: too much information kills information, there's a role for someone to package it for users and make money in the process (read this post on Vinnie's blog).

    We however remain sceptical about Gartner's consulting business (would be nice to hear from the chaps at Accenture, Deloitte, Cap, PWC, IBM, E&Y if they ever lost a deal to Gartner?)

    Friday, 24 February 2006

    The Governor, ancient Iraq and Gartner

    Like a modern David, James has decided to take on the Borg Goliath:


    frontpage hit counterIn this post (and before), he argues the cuneiform writing is on the wall for Gartner Borg:
    • Why would vendors continue to pay a ransom to a gatekeeper to enterprise purchasing?
    • They're under pressure to disclose their "methodologies" and clients
    • He quotes Jonathan Schwartz, COO (SUNW), as not being happy with (IT)
    His predication is that open source research will form a new ecosystem and make Gartner obsolete, just like an old mainframe. Vinnie says it's like Gulliver and the Lilliputs.

    Joe has already posted an analysis on Gartner Watch:
    ARmadgeddon's take: commoditisation of research is an inescapable trend and presents a serious threat to traditional content-based IT Analyst Research firms such as Gartner. However, open source research faces challenges before it can be seen as a credible alternative:
    • Reputation/credibility lies with individual analysts or firms (and not with the "ecosystem")
      **An AR professional emailed us that the main issue with "one-man-boutiques" for them was the lack of branding/the confusion in the marketplace.
    • The output of independent analysts varies considerably as it is not like a single source code (although the emergence of aggregators such as IT-Analysis.com is a major step forward). **As Richard Stiennon puts it in the comments, "open source research would imply a wikipedia like collaboration on market research, [...] the results however would be FREE to use by anyone." (see comments for more)**
    • Consulting to users remains the exception for independent analysts and it is not proven can the open source model can be financially viable for them. **Vendors value this user insight more than everything else, this is however currently the priviledge of Gartner, Forrester, Ovum, etc....**
    **27/2 edits**

    Wednesday, 22 February 2006

    Monday, 20 February 2006

    Is there an afterlife for analysts?

    Duncan asked yesterday Why vendors hire analysts.

    The underlying question lies around analysts career development.

    You see, analysts do have somewhat atypical jobs -which may explain why they are often peculiar themselves. They come from various backgrounds, either from IT departments, IT vendors (most frequent case) and less frequently from resellers or other channel partners, often ticking well over 15 years of experience.
    Their analyst job is often fascinating, both intellectually and for they have a privileged industry vantage point -sitting between so many constituencies (vendors, users, channel, etc...)

    So it's not a surprise if analysts tend to stick around and progress naturally within research firms, however even in Gartner there can be only so many VP's... Creating their own consultancy can then appear as an attractive option. Duncan describes these market dynamics here: From the 'water cycle' to the 'analyst cycle'.

    Another option is to move into AR. As James and Duncan point out, running AR for an IT vendor is sometimes a rather brutal landing. We also had some comments from analysts on being dealt with by ex-analysts (this may cause some clash of egos), on the other hand ex-analysts are well positionned to sell the value of AR to the business. Another option can be to work for or create an AR consultancy (KGC, LighthouseAR are run by former analysts) -but it's quite a niche market...

    Saturday, 18 February 2006

    Frost demands cash for press release

    In ARmadgeddon: AR 101: Analysts and press quotes
    we explained that analysts don't charge for quotes. However, itis worth adding a postscript: one reader has pointed out that there's one case where vendors do pay for a press release. This is what she writes:

    "I just wanted make a comment say that you have to be either green or naive to take this idea of paying for quotes seriously, or even think if could happen here in the States but not in Europe. However, a comment about Frost is on the money: they told us that we had won an award, so we said 'great'. They replied, if you want to tell people who have won, then you need to pay us for the package -- which included a trophy, a photo opp and a press release. We said, no keep the trophy: we'll just spread the news about the award ourselves. They told us that they would only anounce the award if we paid for it."

    We'd love to hear about other experiences like this.

    Friday, 17 February 2006

    IDC misses forecast by 1900%

    Interesting read on IDC un-prediction in today's El Reg:

  • Le Misancalculation: A one act Itanium tragedy by IDC (The Register 17/02/06)


  • Extracts:
    "As the artist's rendering points out, IDC does not have a sparkling track record where Itanium sales prediction are concerned. In fact, its 2004 prediction was 1,900 per cent above the actual sales mark. Some analyst firms can survive on results like that, and some can't."
    "IDC never explores these issues. Instead, it relies on flaky data and embarrassing conclusions to create a study that could outdo its 2004 effort - a feat we thought impossible."

    WOW, this seems to be a hard one to get away from...

    Wednesday, 15 February 2006

    AR 101 series: Analysts and press quotes

    David published Tuesday this most post after Tom Foremski of SiliconValleyWatcher wrote that vendors pay analysts for quotes:


    Quotes for hire, interesting concept...

    We obviously agree with David that this would be utterly un-ethical, but it's important to put the record straight:

    We've never, ever, heard of anyone asking taking payment for quotes. We are very keen to hear if Tom can substantiate his allegations. We bet he can't and thus should retract.

    PS (20/02/06): David in fact confused quotes and awards, which Frost & Sullivan charges for. Still not really ethical but not the same as charging for quotes.

    This brings us to the subject of this AR 101: how can you leverage analysts in the media?

    Analysts are, often rightly, viewed as thought leaders. Reporters value their insight, they are an invaluable ally to quickly nail down the issues around a topic and provide quotable material -all under tight deadlines. Journalists don't have the depth and experience of analysts, so a little help is welcome (an AR manager was heard saying recently that he believed the average VNU reporter age to be under 25...). So it's not a surprise that trade press often features analyst quotes.It's a good deal for analysts, they get exposure and it feeds their demanding ego. Some firms reward analysts for speaking to journalists, META was an example. This can lead to quote-happy analysts, some IDC analysts are particularly prolific as they produce so many cuts of the same data that every vendor can be a leader somewhere (read Give 'em all something - we need to sell reprints). Of course, too much goodness eventually hurts the analyst credibility...

    So, how can AR managers leverage analysts to help their employer's profile in the media?

    The most frequent tactics are:
    • To include analysts quotes in press releases. This is subject to approval, Gartner and IDC have formal processes ; Gartner and AMR only allow quotes from published material (but not FirstTakes or Symposium presentations).

    • To include analysts contact details in the press package sent to journalists, possibly with quotes.
    PS (20/02/06): we received comments from AR managers and analysts saying those practices were highly ineffective -some journalists tend to deliberately ignore proposed quotes and go for "really independent" quotes. In other terms "shoveling quotes down a journo's throat will backfire and damage both vendor and analyst reputations." We would be interested in your comments on this...

    Some additional tips:
    • Depending on the relationship the AR manager has with them, independent analysts will often be quite willing to provide quotes and be contacted by reporters.

    • It goes without saying that good AR managers will make sure that the analyst is briefed before reporters get the press release and that they have a (positive) opinion on the subject and contribute to the debate.

    • Good AR managers will also ensure that they use analysts that are consistent between what they tell reporters and what they publish.

    • Finally, some analysts are better at this game than others. A pedantic analyst speaking in 80 words sentences may not provide good quotable material to journalist and may even be mis-interpreted.

    Other post in the AR 101 series:

    Monday, 13 February 2006

    InformationWeek responds to comments about article, mentions ARmadgeddon

    Interesting that both Gartner responded to the InformationWeek article and now InformationWeek itself has responded to the comments about the article. Hopefully this will start a higher profile conversation with all industry stakeholders, especially IT managers, about the role, methodology and business practices of the IT analysts.

    IT Analysts Turn The Table And Analyze Us

    Ovum to IPO, Forrester loses sales VP

    Dave reports here that Ovum is about to float:
    Analyst Insight: Ovum announces float

    This was in the air for a long time but it's official now. We heard they stopped recruiting analysts/expanding because of this upcoming float, which is somewhat strange.
    Anyway, the Ovumites are about to make some paper money as they all will get some shares -which is probably good as Ovum as a reputation for not paying them very well....

    The other news today is that Forrester is loosing Tim Royston-Webb, its EMEA Sales VP (ex. Giga before), to Datamonitor. Both have a reputation of being pretty harmless at selling to vendors (check C'mon guys, sell to us!), so we think this change of ship is going to make no difference whatsoever.

    The press release can be found in Tekrati:
    Datamonitor Appoints Tim Royston-Webb as Managing Director, Technology Global

    C'mon guys, can't you write-up a little the stories instead of just reprinting what you've read elsewhere?

    Sunday, 12 February 2006

    KCG oppose, and offer, share of voice analysis

    Our friends at KCG have us modestly confused. An article in their newsletter is titled Under the Influence: Five Reasons to Worry About "Share of Voice" Metrics.

    Stephen explains that "we see most of our '“competition'” nowadays coming from PR firms. In attempting to transpose their typical volumetric 'Share of Voice'” measurement systems from the media business to the analyst influence business they regularly do our function a grave disservice."

    In essence, he feels the key weaknesses of the share of voice approach are:

    1. It is mistaken to measure share of voice inside firms; instead, track your top ten analysts.
    2. They mistakenly measure the media: instead search research, which is essential.
    3. Searching in the same way will produce different results at different firms; each firm needs to be searched differently to produce results that are consistent.
    4. Searching on company name doesn't work; use key words.
    5. Share of voice analysis does not only analyze quotes.
    The criticism seems over-blown, because KCG actually sells share of voice analysis itself, what it calls 'Coverage Metrics', and so do most of its competitor PR agencies and the AR consultancies. These studies also measure volume, even if they eliminate the media and most analysts from consideration. The fact that KCG itself supplies share of voice studies is not the only clue that those metrics can be useful.
    1. Sometimes it is appropriate to target firms and not only the key analysts. Much of the research about our firm is co-authored by a team of analysts. Following only one would be mistaken. It als would not fit the global spread of our business. We've learnt that analysts at the same firm but in other countries do not display the same preferences, and we need to get a top-level picture of the whole coverage by each firm, not just one who knows us best and is normally the most favorable. Even KCG stresses that its metrics "Measure coverage by target firm or target analyst list."
    2. To not track analysts in the media would be mad. Analysts have a huge impact on the media and their influence is reflected differently in the media from in their research. To ignore the media is lazy and woud get us killed internally. When PR and sales come back with negative media clippings, how can we say they should not be tracked?
    3. Searching is not that hard. I can recommend cut and paste: when I have the search right at a firm, I paste the search term into the back of the Excel sheet that our intern uses when she does the searches. It's very easy to be consistent if you think about it.
    4. Searching on company name can work if you and your key competitors are in the same industries. Yes, if you're in many businesses you need one or more key words to reduce the number of results. But that's not hard. Use the cut and paste tip I mentioned above.
    5. Quotes and share of voice are different things, but doesn't mean that only one of them can be useful. If my company is mentioned more in analyst research, or by analysts in the media, that's normally a better thing than being mentioned otherwise. Of course, it's also important to track quotes (although that means looking at the media, and breaking one of the KCG's commandments) but we do that by hand in a separate analysis.
    In short, share of voice metrics are very useful and, honestly, if they were not then KCG and its competitors would not sell them. I am sure that KCG has a point that some PR agencies get it wrong at times: but if KCG is telling its clients to not track the media, then they will also be getting it wrong in the eyes of some clients, whose customers follow the media.

    Thursday, 9 February 2006

    Has Forrester missed a golden opportunity?

    META Group had over 2,600 clients at the time that the acquisition by Gartner was announced. Gartner CEO Gene Hall in the earnings calls since the deal closed has consistently said that Gartner was retaining about 55% of the META clients. So roughly 1,170 META clients have been in play for the taking by Forrester, IDC, Ovum, et al.

    You would expect that Forrester as the nearest competitor in terms of product offerings and size would be the prime beneficiary of all these former META clients floating around. Well, you would be wrong.

    CEO George Forrester Colony responding to a financial analyst question during the February 1st earnings announcement said this:

    “And I would say that [the META campaign] was successful but not wildly successful. I think that we definitely got a general boost from Gartner-META coming together in 2005, and, you know, I would say it was -- we probably gained 10 to 12 accounts in Q4, somewhere in that range. You know, I think it was in the 10 to 12 range.”

    Not wildly successful? The Forrester ''Miss META? Find Forrester'' campaign was a complete bust. Looking at the earnings calls for 2005, George said that Forrester picked up about 28 former META clients all year. Only 28 out of 1,170 possible clients or about 2.4% of the potential opportunity. Even if one was to be kind and say that only half those META clients were really up for grabs, Forrester’s performance is still very pathetic.

    Forrester had a historic opportunity to dramatically expand its client base and become a much stronger competitor to Gartner. Unfortunately, George Colony was content to sit back and passively wait for a fraction of the opportunities to fall into his lap. With Gartner aggressively ramping up its sales force, the opportunity for Forrester is past and Colony will find his company increasingly considered a niche player in the analyst industry.

    Wednesday, 8 February 2006

    Gartner responds to InformationWeek article

    The Gartner Ombudsmen have posted a response to the InformationWeek article on their blog Credibility of Analysts: Gartner Put to the Test. A few points from the blog:

    • Vendor-sponsored white papers: No, it's unethical (An expanded comment would be "Not any longer" because Gartner use to do sponsored white paper until a few years ago)
    • Transparency of Gartner's research methodologies: Gartner has documented and published its methodologies for all to see (An expanded comment would be "We are working on it, not everything is documented yet." There are some deliverables like the Vendor Rating which are not documented nor transparent. The Magic Quadrant is documented but Gartner does not publish criteria weightings or how analysts inject subjective opinions into a MQ)
    • (R)esearch is ... influenced by ... investment in the SI Ventures II fund and by outside investors such as Silver Lake Partners: No (Don't have any quibbles on this one)

    The Omsbudsman blog does have "Comments" turned on, so go over and provide Gartner some direct input.

    More reactions to the Information Week Credibility of Analysts article

    Interesting take on InformationWeek's Credibility of analysts really bland article from Mike Rothman on his pseudo-blog (he does not allow comments):

    • Educated end users need to know the context of a report (is it sponsored or not?) and treat the information accordingly.
    • Having an analyst participate in vendor web casts, etc. is doing a service to the user community. Any analyst says a vendor's stuff is great on a own web cast is an idiot.
    • There was no substantiation of the "small vendors only get covered if they are clients" claim.
    IW had a great opportunity to educate their readers on how and when analysts can add value. Instead, they focused on old rumors and innuendo of bias. IW, if you are going to go on a witch hunt, you better find some witches. They are out there, and exposing clear instances of bias would have been interesting. They should have used the space to show clear examples of bias. That would have been interesting.

    Duncan also commented on the same subject: ANALYST EQUITY: Research suggests Karma beats conspiracy:
    • In outline form, Aberdeen's story is the same as in every Greek tragedy: pride; error; downfall; insight. Balancing between users and vendors is a difficult and dangerous job: few succeed for obvious reasons. As the Italians say, "He who serves two masters must lie to one of them."
    • So we think that InformationWeek is right to point out the tensions; and we think that most analyst firms that try to serve both buyers and sellers do so at their peril. However, we also think that buyers and vendors both wise up to that reality quickly.

    Tuesday, 7 February 2006

    Gartner should invest more in research, say former Gartnerians

    The Internet Research Group in its InternetAcceleration newsletter had this interesting article:

    X-Gartner Woes -- We've received another e-mail from a disgruntled X-Gartner employee complaining (again) about his former management's lack of ability to adequately fund his group. Unfortunately, this complaint by X-Gartner-ites is all too common. (Gartner has seen its share of red ink, its largest shareholder is a hedge fund that's mostly interested in quickly making money and its upper management team has been playing musical chairs.) It's our opinion that the large research firms can provide tremendous value to IT organization decision making. A dollar spent on research should yield fifty times that in terms of bottom line profits that result from improved IT strategies. But from where we sit, the state of things today doesn't seem to be so clear cut. Maybe it's time that Gartner should start taking the advice of its former employees (hey, you'd think that this would be a pretty good form of free advice from some people that were, how should we say this, Gartner analysts). The advice that we hear these former employees give is -- SPEND MORE, NOT LESS, ON RESEARCH. We've been in this business long enough to know that it's a simple correlation -- the more time you put into researching the topics that you're covering, the better your fact base and the greater the chance that you'll provide better advice. Pretty simple, huh? Why do we care? It's our trickle down theory. If Gartner can actually help IT organizations to improve their decision making they'll start improving their technology decision making, their companies will perform better and they'll be in a position to spend even more money expanding (including buying more IT stuff). Not only that, quite quickly, these companies will value research more and will start spending even more on research. So it gets down to this -- if Gartner really believes in the value of research, then it needs to change its direction. Even though it is losing money, now is the time to invest in the fundamentals of the business. If you'd like, perhaps we can whip up a Magic Quadrant covering large research companies to help illustrate our point.

    - - - - - - - - - - - -

    Comment on IRG's article:

    Unfortunately, Gartner CEO Gene Hall seems way more focused on expanding the Gartner Sales force than investing in research. In 2005 Hall bought META Group to eliminate a competitor and to get the sales force (he crowed in the Q3FY05 earnings call about the 84 sales reps he picked up). With new hires, Hall expanded Gartner sales headcount by more than 100 in 2005 -- but did not make a similar investment in analysts and research. In this morning's Q4FY05 earnings call, Hall again talked about adding sales headcount -- between 80 and 120 incremental new reps -- to go after $1bn+ corporations with mentioning an investment in research.

    Monday, 6 February 2006

    InformationWeek article "Credibility Of Analysts"

    The article "Credibility Of Analysts" mentioned in the previous post has been published. Rather long, about eight pages printed with the sidebar.

    While Greenemeier and McDougall raise some interesting questions, it seems rather soft and does not nail any of the firms on any issue.

    Some of the issues they raise are interesting, like the credibility / ethics issue: can you be credible when vendor have to pay-to-play?

    I love this quote too: A visitor to InformationWeek.com last week weighed in with a different complaint in a blog posting: "The problem I've seen with analyst firms is when analysts forget their primary role as trend watchers and market surveyors and think of themselves as market makers."

    The Forrrester Wave is also depicted as more transparent than the Gartner MQ, which is probably wrong in our opinion given the recent MQ process changes.

    The bottom line summarises quite well the cynicism about IT analysts:

    "So IDC helps seed the market, Gartner helps price it, and both get paid for doing so. Is everyone comfortable with that?"

    Thursday, 2 February 2006

    The Credibility Of Analysts [InformationWeek "blog"]

    Larry Greenemeier from InformationWeek is running an interesting poll on the Credibility Of Analysts.

    He plans to publish the results on the 6th of February.

    Although is blog is not a real blog, it raises interesting issues, like why does the Borg not put products they place in its Gartner Magic Quadrant to the test.

    Selected quotes:

  • "an analyst industry that's tapped into an almost desperate need that businesses have to stay on top of emerging technology and a community of IT vendors even more desperate to make the sale"

  • "IT vendors and their customers both became beholden to the big analyst firms"

  • "The reluctance that vendors and end users have in publicly questioning analyst firms tells us all we need to know about the clout that these firms have"
  • Wednesday, 1 February 2006

    Making it up While Drinking

    Amidst rumours of a Bloor sell-out by the Justin gang, the analyst cycle is in full motion in the UK, with independent analysts talking of getting together –some have obviously read this post talking about a federation against the Borg.
    This industry warming up, evaporation and condensation into new firm seems to be at Quocirca’s expense –who are actively recruiting new analysts to make up for the departure of Dale Vile and Jon Collins.

    This brings us to today’s news: the same Jon is joining MWD, MWD (formed by the two ex-Ovum Neils –Ward-Dutton and Macehiter) is partnering with Freeform Dynamics (itself founded by the two Viles –Dale and Helen) and Influencer50 (an agency trying to help vendors influencing the influencers –a conflict of interest here?).
    If you thought this was complicated, James announced here that he’ll be partnering with the two Viles… Menage a trois or swingers club?

    ARmadgeddon’s analysis: the re-combination of free-spinning stratospheric analysts creates a confusing picture but when the dust settles it will precipitate into an articulated and complementary competitor for Gartner (IT) as the established competitors choose to remain in their cosy niche -IDC, Forrester (FORR), prepare for IPO -Ovum, or stay confined to their region –Experton (.5 probability).