|
HOME The Control chart is the fundamental tool of Statistical Process Control (SPC). |
An Article for All Seasons
This article appeared in different form
in NewMedia Magazine, April 99 There are a lot of reasons to put a survey on the web. It's fast and inexpensive, and you can do things with web surveys that aren't feasible with any other media. You can get a survey up on the web and be counting responses in a matter of days - even hours. People run surveys on their web sites for many different reasons. A commercial site may want to establish the demographics of those who visit in order to help sell advertising on their site. Some web surveys are trying to research the potential market for a new product or service, while others are establishing whether or not current customers are satisfied. Surveys and polls can be used to establish a sense of community for a site, and draw people back again and again to check the results. At Babynames.com., they run a daily poll on how well people like different names. Jennifer Moss, the site director says "When we get feedback on the site, a lot of people have replied that they like being able to vote on a name. They feel like they're being heard and their opinion counts." There are some potential problems though, the most notable
being biased results. Are the people who complete surveys on
the web the same as those who don't? Almost certainly not, but
depending on who you want to survey, it may not make any difference.
Dr. William Hammers of the Advanced General Aviation Transport
Experiment (AGATE) group decided on an Internet survey after
determining that "the profile of people who buy and fly
airplanes and the profile of people who are on the web are a
very close match." The next step is to decide who should be surveyed. For most web sites, this would be anyone who visited and was willing to fill out the survey. However, there are situations where you may want to limit who can take the survey. For example, the results of a survey to find out what the most popular on-line game is can be skewed by a few die-hard fans who take the survey repeatedly and vote for their favorite. If this is a potential problem for you, both Ann Ray of Apian software [They produce SurveyPro] and Bob Tortoro of Gallup advise limiting access by e-mailing respondents a unique password. Now, you'll need to decide on questions and design the survey itself. Style can be an issue. Ann Ray says "Something like black text on a white background is great. If you keep the graphics small it downloads faster. Keep the layout simple and clean." Bob Tortoro admitted that "Our own reaction at the beginning was to design a fairly fancy questionaire." but later realized that "you're much better off using a very plain questionaire." Generally, people are only going to spend a limited amount of time and effort to take a survey. The easier it is to respond, the better chance you have of the person completing the survey and hitting the send button rather than moving on to another web site. It's best to provide people with options to point and click on rather than having them type in a response. Ann Ray cautions "Be sure to match the entry mechanism to the question. Use radio buttons or drop down menus for a single response, checkboxes for multiple answers. It's useful to include an option for ‘other' responses, but few people will use it so it needs to be supplemental, not part of your core data." Radio buttons are best with 5 or fewer answers to choose from.
More than that, and a drop down menu is easier to use. For really
lengthy lists with short identifiers, such as the 50 states,
it's wise to allow respondents the option of typing in the 2
letter abbreviation rather than forcing them to scroll through
the list all way down to Wyoming. When setting up your survey, make sure that you don't accidently leave the first option as a default response. This can badly skew your results, as people often leave questions unanswered. If your survey picks up the first option as being selected by default, you'll get unrealistic estimates of the response rates. It's always a good idea to allow people the option of ‘Not applicable' as well, in order to distinguish between those who skipped the question and those who didn't feel they could answer it. When you have several items to compare, don't ask people to rank more than 5. If you need information on more choices than that, just ask them to indicate their top 3 choices. When deciding between using a scale or a yes/no type question, determine if anyone might want to respond with ‘sometimes'. If that's possible, use a scale. Open-ended comment questions take a great deal of time to
analyze, so be prepared for that. Generally, put such questions
at the end of the survey. It is a good idea to include at least
one though. If you have missed any important issues in your survey,
the respondents will tell you what they are. For example, the AGATE survey was developed to determine the market potential of a new type of small aircraft, designed to be easier to learn to fly and considerably less expensive than anything currently on the market. Pilots, former pilots, and potential pilots each receive a different set of questions. Another technique developed for this survey, was the use of random numbers to decide which questions respondents would receive. This was particularly useful in determining prices. When given a selection of prices and asked ‘what would you pay", people typically respond with answers at the low end of the price range. Sellers, on the other hand, wish to maximize their profits by charging as much as they can for a product or service while simultaneously selling as much of it as possible. Charge too much, they price some people out of the market. Charge too little, they reduce their potential profit. If you ask a yes/no question about a specific price, you only get information about that particular price, not the entire range which is needed to determine the optimum pricing strategy. What we did with the AGATE survey was to select 3 different prices, then present each respondent with a yes/no question listing only one of them. By looking at the percentage of positive responses to the different prices, we can build a curve showing what the market potential is at each price level. Prior to collecting responses from your survey, it needs to be tested. Ask a few people to take the survey and provide feedback on it, before publishing it. This allows you to check over the survey for confusing questions (sometimes something that seems clear to you has a different meaning for others), make sure that the questions and scales all match up properly (a common error), and time how long it takes to complete the survey. 5 to 10 minutes is about as much time as you can expect someone to donate answering your questions, unless you are offering some sort of incentive to complete the survey such as a coupon or gift certificate. Most surveys don't require much more than tallying up the answers by way of analysis. More detailed analysis involve comparison of different groups - for example, looking at income levels by sex. Do the proportions of men and women indicate they have the same income levels? A cross-tabulation will usually provide the data you want. Keep the end in mind as you design your questions to make sure you can get the charts and analysis you want from the data. TO DO: 1. Determine the goal of the survey |
