Blunt – The Conversation Agency

No matter how big your company is, real voice of customer data can be an immensely powerful source of business intelligence for your marketing team. According to Forrester Research, online voice of customer research is one of the fastest emerging areas within  all market research. If you’re looking to take the plunge into voice of customer research, running a website survey is usually the first place to start.

To help you in your decision making process, Blunt has outlined 5 points to consider. Follow these, and you’ll be on the right track to picking a great website survey tool.

1) Price

Price is the first hurdle to clear in the decision-making process. For the cost conscious, there are powerful free and low cost website survey solutions out there. Look up Kampyle, Crowd Science, QuestionPro, SurveyGizmo, SurveyMonkey, Zoomerang, or 4Q, and you’ll see what each tool has to offer. You’ll get the data cheaply, but you’ll incur some heavy lifting if you want to draw out meaningful insight. These applications are best used for quick and dirty surveying, but if used properly they can answer some important questions. Obviously, if you’re operating under strict budgetary constraints, this is where you’re going to start.

On the enterprise side, the most respected solutions include iPerceptions, Foresee Results, CRM Metrix, Opinion Lab, and Omniture Survey. The range and value of an offering  is typically commensurate with what you’re willing to pay. SAAS-style yearly servicing agreements can start as low as 15-20k, but once you add in all the goodies  (custom questionnaire design, multiple user accounts, pro services, a dedicated account manager), you’re probably running closer to 80k/year. Some solutions are offered on a  pilot or trial basis, and this can be a great way to kick the tires and determine whether the vendor’s services match the claims made in their marketing material.

2) Methodology

Ignore the white papers, the spin, and the pseudoscience, and opt for a survey solution that gives you the flexibility to craft the research you need to answer your business’ most burning questions. Demographics, psychographics, and surfographics are great starting points, but the tactical advantages that can be gained from a broader, more diverse stream of data can be immense. Branching logic is a bonus, because it means that you can target specific sets of questions to specific user segments. Balance quantatitative data points with raw, qualitative feedback to get at a full, holistic view of your visitors. Make sure you can vary the question types–rating, multi-select, drop box, matrix-style, single-select–as there’s nothing more boring from a user experience standpoint that filling out 25 straight Likert scale-style questions. If you go down that dark road, you’re begging for drop off, which can lead to bias and garbage data.

It’s really important that your survey has a persuasive invitation method. You want your data to be pouring in, not trickling in. There are various invitation methods employed by leading website survey vendors. Among these are: on-arrival overlay, on-exit pop-up, mid-stream intercept, and comment cards triggered by page links. Each of these methods has its pros and cons. Some yield higher response rates, but recruit very broad-based samples. Others provide more targeted samples, but are prone to systemic bias. Others are extremely unobtrusive, but suffer from low response rates. Some vendors, like Kampyle, smartly offer multiple invitation methods. Whatever the case, probe your vendor and make them lucidly explain why they’ve chosen a specific invitation method. Make them show you real response rate data from comparable websites.

3) Metrics

There are two dominant schools of thought here. On one side, you have the disciples of customer loyalty, on the other side you have the ardent proponents of customer satisfaction. The disciples of loyalty are married to the Net Promoter Score (NPS). Developed and popularized by Satmetrix, Bain & Company, and Fred Reichheld, NPS has been woven into the offerings of myriad vendors. The beauty of NPS is in its simplicity: you ask one question, “How likely is it that you would you recommend (company, product, site) to a friend or colleague?” on a 0-to-10 point rating scale and you bucket the respondents into Promoters, Passives, and Detractors. To calculate your NPS, you subtract your % of Detractors from your % of Promoters. Simple and beautiful, huh?

Not if you’re a proponent of customer satisfaction measurement. The University of Michigan-developed American Customer Satisfaction Index (ACSI) is currently employed by Foresee Results in all their website survey applications. The ACSI is widely recognized in many online verticals–particularly e-commerce–as the gold standard for measuring customer satisfaction. ForeSee Results claims that the ACSI is “a leading economic indicator measuring over 200 companies representing nearly 50% of U.S. Gross Domestic Product (GDP),” which is “backed by more than 14 years of empirical evidence showing the cause-and-effect relationship between customer satisfaction and financial performance.”

Don’t get caught up in the metrics arms race. While those two metrics are certainly useful, you can get just as far by measuring visitor intent and task completion.  In a blog post I wrote last year, I outlined five reasons why every website operator should be measuring visitor task completion. With the rise of the intention web, intimate knowledge of the minds of your visitors will be the next horizon for online marketers. Align your desired site outcomes with your visitor intent mix, and you’ve got a recipe for success.

4) Reporting tools

Both free and paid tools will offer automated reporting. There are four things to look for here: agile and flexible dashboards, drill-downs and segment filtering, data output, and natural language processing. The dashboard should succinctly summarize your KPI trends, alert you to problem issues that require immediate attention, and point out emerging themes. The reporting tool should allow you to select key segments based on a combination of response criteria and then data-mine the hell out of these segments. At some point, you’re going to want to export the raw data from the reporting tool for some analytical calisthenics in Excel or SPSS. Make sure the tool has export capability so you can get your hands dirty. Finally, you’re going to need some way to extract insight from the open-ended data you’re collecting. The good tools all employ some type of natural language processing to surface insight. iPerceptions’ new Interactive Dashboard does a great job of extracting core linguistic concepts and visualizing them using tag clouds and other funky tools.

Though the buzz has died a bit recently, it’s still fashionable for vendors to tout their ability to integrate with other silos of online business intelligence such as web analytics, ad servers, social media, or more traditional BI & CRM systems like Cognos or Siebel. Omniture tried to build a unified web intelligence platform with Genesis and its Survey offering plugs snugly into it. On the less sophisticated end, Kampyle advertises a very simple Greasemonkey script that allows users to inject survey data into Google Analytics. Within the past two months, iPerceptions has rolled out a more elaborate Google Analytics integration that provides access to session-level clickstream data, mashed up with survey response data. Be cautious when dealing with integrators, however. From experience, I can tell you that it’s a challenge to master even one stream of online intelligence; get that under your belt first before you worry about mashing data together.

5) Professional Services

If your budget has room for professional services, you’ll usually be met with a two-fold challenge. First off, the people who know the most about the ins and outs of your business–your marketing team–are probably all english and communications majors who talk a big game about being data-centric but in reality are scared shitless at the mere mention of means, samples, or statistical significance. Second, your vendor does have trained and competent analysts and statisticians on the payroll–who know about chi-square tests and stochastic regression–but they lack the passion and in-depth knowledge of the space that your own employees possess. So, like it or not, you’ll have to bridge two sources of intelligence: your own (tactical) and your vendor’s (statistical). Knowing this, look for vendors who are accessible, responsive, and willing to learn.

It’s sad, but website survey vendors simply don’t breed for the kind of in-depth sector specialization that exists in traditional market research and consultancy. I recall attending a strategy session run by an automotive client back when I was an analyst at iPerceptions. Analysts from Maritz were also in the room, and I was absolutely stunned at how well they knew the automotive industry. Nameplates, motors, pricing, options, marketing campaigns–you name it, and  they were on top of it. Needless to say, their contribution to the meeting (and they prestige in the eyes of the client, no doubt) dwarfed mine. It opened my eyes to the fact that I had been trained to crunch data, but not to understand the nuances of my client’s business. As a result, I couldn’t tell a compelling story. Ultimately, actionability stems from compelling stories, not data pukes.

Keep these five points in mind when you select a website survey tool, and you’ll be guaranteed to pick a good one, which will start you down the path towards customer centricity. You will gain great fame, untold riches, and be desired by women across the globe. OK, maybe not–but if you pick a great survey tool and use the insights to make your company more money, your bosses will notice and they’ll thank you with great enthusiasm (and compensation)!