Reviewing the Dynamics Product Reviews: One Expert's Take

How seriously should users take supposedly independent reviews of software products?

This is a thorny, yet important, question for users, especially as they try to make decisions about upgrades, new products, or even about switching products.

Product reviews are comforting to users when the reviews reinforce initial impressions. They are marketing tools for software companies and re-sellers, who re-print favorable reviews, and ignore unfavorable ones. But if they aren't credible, favorable reviews can lead users down a dangerous path, toward making poor choices.

One of the major challenges for users is assessing the credibility of reviewers. It's difficult for users to know whether a reviewer has a vested interest in one product or another-most obviously, whether a magazine's reviews are driven by how much advertising particular users or vendors are doing.

Overall, users are probably best off being highly skeptical, suggests Jason Carter, a consultant to many Dynamics partners and ISVs. In an interview and in postings on his blog, Carter makes his case by pointing to questionable reviews of Dynamics and Sage products contained in the publication, CPA Technology Advisor.

At the start of last year, he says, the publication provided highly favorable reviews of two Sage products, along with somewhat less sterling reviews of others...written by a Sage re-seller. Now, this year, the publication's reviews provided top scores to Dynamics GP and NAV, along with the two Sage products.

Yet nothing really changed, argues Carter. "The two Sage products got perfect scores last year. Surely scoring them something less than perfect this year would require quite an explanation. The products would not have regressed during the year (...