AR Classics Series: Buyer beware on analyst advice

Here's more decade-old wisdom we found on archive.org; this time it's an article by Efrem G. Mallach.

Admit it. You're an advice junkie. Need proof? The industry analyst business rakes in more than $1 billion annually, mostly from IS managers who seek advice on trends, vendors and products. Consultants take in even more for advice tailored to a specific situation.

If you spend that much on advice, you pay attention to it. Vendors know that. So U.S. vendors spend more than $100 million per year to get analysts and consultants in their corner. These efforts don't always work, but they help.

Why do vendors go to this expense? Because some parts of an analyst's advice are objective, but others aren't. It's one thing to say that vendor A's hub supports X ports at Y bit/sec. That's objective. It's another thing to say that vendor B is a technology laggard, or vendor C's strategy is flawed. That's where opinions come into play and where analysts' attitudes toward vendors matter.

What's more, feelings should matter. A vendor's treatment of analysts may not mirror its treatment ofusers precisely, but it's a guide. When vendors give analysts prompt, clear responses to questions, access to experts when needed and truly useful presentations, the analysts will conclude that users are likely to get similar attention. Analysts who get the opposite will conclude the opposite. The logic may not be perfect, but experience can't be denied.

Yet this situation of vendors courting analysts with varying effectiveness makes analyst evaluations inherently subjective. What's an IS manager to do?

• 1. Don't kick the habit. It's OK to need advice. Technology is complex and moves quickly. You can't spend all your time evaluating vendors and technologies. Go to people whose job it is, and pay them for what they know.
• 2. Remember you're at the end of a long information chain. Everyone in the chain helped shape what you hear. And everyone in the chain has an ax to grind, personal favorites or entrenched dislikes. They put their spin on the elusive ``truth.''
• 3. Get multiple analyst perspectives. If you can't subscribe to more than one service, read what others say in the press, pull summaries off World Wide Web sites or exchange views with colleagues.
• 4. Probe. Only a minority of analyst subscribers take full advantage of their call-in privileges. Join that minority. Find out what assumptions the analysts used, what their methodology was and how they chose the vendors to examine.
• 5. Be aware that analyst firms receive millions of dollars from vendor clients, too. Although few analysts will ``fudge'' evaluations to favor clients, they are more likely to know client firms in depth. This makes them more likely to write about clients and more aware of their products' true capabilities.
• 6. Check the record. What did this firm or even this particular analyst say about the vendor, product or technology a year ago? Two years ago? And what really happened? As mutual-fund advertisements say, past performance is no guarantee, but it's another calibration point.
• 7. Evaluate recommendations in the context of your firm's approach to technology. Is it leading-edge or conservative? Is management centralized or distributed? Does your business stress market share or current profits? The right recommendation for one company profile could be disastrous for another.

That's work. But if you don't do it, your analyst experience could be a bad trip.