AUSTRIA— Rushing through long or difficult questions in online surveys harms data quality more than any other type of respondent behaviour, a new study suggests.

Psychologists from the University of Vienna and the University of Deusto in Bilbao, Spain, tested a Javascript tool which tracked respondents’ activity while they completed a 23-question online survey about their use of instant messaging (IM).

The researchers assessed the quality of the data collected by checking results for consistency, and by comparing demographic information from respondents who shared their IM user names with information found in online address books.

As well as looking at plausibility of answers and non-response, the Javascript tool allowed the researchers to study behaviour such as excessive clicking or mouse movements, time spent considering questions and whether respondents changed their mind before settling on an answer.

The biggest problem, they found, was respondents answering without reading questions properly. “Introduction texts were rarely read thoroughly,” they said, “and semantic differentials [matrix questions] showed higher levels of clicking through than other questions.”

On the basis of their findings the authors recommend that questionnaire designers

• keep introductory texts as short as possible
• only use matrix questions if absolutely necessary
• avoid questions that require calculations or other difficult tasks
• avoid putting more than one question on each page

The researchers said the project had shown that their tool, the UserActionTracer, can be implemented easily to detect problematic items in surveys. Further reserach, they said, might include looking at the positioning of radio buttons, and analysing the speed and duration of mouse movements.

The study was conducted by Stefan Stieger of the University of Vienna and Ulf-Dietrich Reips of the University of Deusto in Bilbao, Spain and is published in the journal Computers in Human Behavior.

'Cookies' Cause Bitter Backlash

Spate of Lawsuits Shows User Discomfort With Latest Innovations in Online-Tracking Technology

By JENNIFER VALENTINO-DEVRIESAnd EMILY STEEL

Tools that track users' whereabouts on the Web are facing increased regulatory and public scrutiny and prompting a flurry of legal challenges.

Since July, at least six suits have been filed in U.S. District Court for the Central District of California against websites and companies that create advertising technology, accusing them of installing online-tracking tools that are so surreptitious that they essentially hack into users' machines without their knowledge. All of the suits seek class-action status and accuse companies of violating the federal Computer Fraud and Abuse Act and other laws against deceptive practices.

In 2001 and 2003, courts ruled that websites could place small text files called "cookies" on machines. Cookies allow sites to remember users, so they don't have to log in user information on each visit. But they can also be used to track users across websites, compiling a profile of a user's browsing interests.

The earlier decisions said tracking across websites was legal. Since then, online tracking has become the foundation of the $23 billion online advertising industry. The industry says tracking tools help subsidize content, allowing many websites to be free to users.

The new lawsuits challenge the older rulings because modern tracking tools are more sophisticated than early cookies.

In one of the lawsuits, filed last week in the Central District of California, three California residents sued Cable News Network, Travel Channel and others over alleged tracking of Web surfing on mobile phones using technology that the suit says is particularly difficult to delete. A spokesman for Scripps Networks Interactive Inc., which controls the Travel Channel, said the company doesn't comment on pending litigation. Time Warner Inc., which owns CNN, declined to comment.

Another suit, filed earlier this month, accuses Fox Entertainment Group and the American idol.com website of using a new kind of cookie—known as a Flash cookie—that can "re-spawn" tracking files that users have deleted, without users' knowledge. News Corp., which owns Fox Entertainment Group as well as The Wall Street Journal, declined to comment on the litigation.

The tools cited in the suits are part of an "arms race" in tracking technologies, said Chris Hoofnagle, director of the Berkeley Center for Law & Technology's information-privacy programs. Some users, uncomfortable with tracking, now routinely block or delete cookies. "There are some in the industry who do not believe that users should be able to block tracking, so they are turning to increasingly sophisticated tools to track people," he said.

One such technology involves "Flash cookies," which use Adobe Systems Inc.'s popular Flash program to save a small file on a user's computer. Flash is the most common way to show video online. Flash cookies can be useful for remembering preferences, such as volume settings for videos. But marketers also can use Flash cookies to track users online.

Last year, researchers including Mr. Hoofnagle at Berkeley found that some Flash cookies were being used to re-create regular browser cookies that users had deleted. (The researchers included Ashkan Soltani, a technology consultant who later helped The Wall Street Journal analyze popular websites for its "What They Know" series, which has been documenting the increasing scope and intrusiveness of the tracking industry.)

Adobe and the Network Advertising Initiative, an industry group, condemn the practice of using Flash cookies to re-spawn deleted cookies.

The Flash-cookie suits allege that companies violated federal law by thwarting users' attempts to limit tracking. The most recent, which involved American idol.com, cites online advertising technology company Clearspring Technologies Inc. as the originator of these cookies. A similar lawsuit in August against Clearspring cited the Journal's "What They Know" series on the growing prevalence of tracking technologies.

In August, Clearspring CEO Hooman Radfar said in a statement on the company's blog that the company doesn't use Flash cookies for tracking. He said that it used Adobe Flash local storage to count visitors to sites and how long they spent on the site. The company says the claims in the suit are "factually inaccurate," adding, "Clearspring does not and never did collect, store, or sell Personally Identifiable Information."

Mobile tracking is also on the rise, as online advertisers attempt to reach consumers on smartphones. Lawyers involved in the suit over mobile websites say companies are using technology that makes it difficult for users to block or prevent tracking. That suit alleges that a company called Ringleader Digital Inc. tracks users of Apple Inc.'s iPhone by assigning each phone a unique ID number, similar to a cookie. If a user deletes the ID number, the suit claims, it respawns itself moments later.

Apple declined to comment on the use of the iPhone in the alleged practices, but Apple CEO Steve Jobs has previously said companies should ask consumers "every time" if they want to use their data.

Ringleader Digital says it is evaluating the lawsuit carefully and intends to defend its practices vigorously. The company says users can delete the ID number from their iPhones. It also says users can opt out of the tracking by going to the company's website.

"We are working on new ways for consumers to be able to verify for themselves that their opt-outs have taken effect," the company said in a statement.

Congress and regulators also are looking more closely at online tracking. Two bills have been introduced in the House of Representatives that would restrict the practice. The Federal Trade Commission is expected to issue new privacy guidelines by the end of the year and is considering a do-not-track registry, similar to the do-not-call list for telemarketing, that would allow consumers to opt out of behavioral targeting.

"We're examining these types of issues both as a policy matter and in connection with non-public investigations," said Jessica Rich, deputy director of the FTC's bureau of consumer protection.

—Shira Ovide contributed to this article.

Write to Jennifer Valentino-DeVries at jennifer.valentino@wsj.com and Emily Steel at emily.steel@wsj.com

For me the most contentious presentation at the recent ESOMAR Congress was one by Bill Blyth and Judith Passingham which addressed the issue of standardisation. Bill is a committed and articulate proponent of standardisation and initiatives such as ISO. I hope I am not doing Bill and Judith a disservice when I paraphrase their argument (I disagree with them fairly substantially, so I need to be careful not to create straw men).

Their case seemed to be:

a)    Most market research we do now is based on principles and thoughts that date back 40 to 70 years and that this adherence to traditional techniques and values has facilitated the development of a $30 billion global industry.

b)    In a modern global market MR needs to provide the sort of service that modern businesses demand, including standardised services that can delivered in a reliable way in, for example across 30 countries, and outputs that can be interfaced with other business intelligence systems within companies.

c)    Initiatives, such as research ISOs, are an essential part of creating a product that will separate ‘real’ market research companies from other suppliers of superficially similar information, in the context of large and or global clients (which is something like 80% of the MR business).

I think that opponents of this view need to realise that its proponents are concentrating on something like 80% of the current market research business, i.e. the top few hundred buyers, almost all of the research being international, and almost all of it quantitative or similar (e.g. automated data collection).

My view has moved on slightly as a result of hearing the presentation. My key thoughts are:

1)    The reasons market research is very similar to 40 and 70 years ago is because we have wasted chances to grow the business in the past. If MR had taken a bigger share of website testing, credit checking, analytics, benchmarking, and direct marketing when we had the chance we would today be a larger industry with a wider range of tools.

2)    If we narrow our definition of what is and is not market research, we will continue to lose opportunities. For a while this might enable MR to grow in absolute terms (in line with global economic growth) but MR will continue to shrink as a proportion of Business Intelligence.

3)    Standardisation is at its most effective when dealing with data collection and data processing, especially in connection with quantitative data. If a client buys a 30 country survey they want to be sure that the same standards are applied in each country, that the same checking has been applied, that the same processing methods have been used.

4)    It seems to me that the ISO and standardisation debate is about dividing the research process into the ‘mechanics’ and the insight and then concentrating the process heavily on the mechanics. This may suit the large agencies, for whom nearly all their money comes from the mechanics. But it is alien to research providers who concentrate on interpretation, insight, advice, and consultancy.

5)    It seems to me that in a research process standardisation runs the risk of not answering those problems that are amenable to standardisation. There are two views about measurement. A) If you can’t measure it then you can’t manage it. B) If you can measure it then it does not matter very much. I believe the truth is somewhere in between the two, but closer to B.

I think some elements of the push towards standardisation are fine, such as not cheating, accurate tabulation, ESOMAR’s questions for panel owners, codes of conduct etc. However, I think that beyond these relatively obvious areas they are at best a waste of effort and at worst they will limit the growth and change of market research.

I think that if the push for ISOs continues and is successful, most market research will be done by companies who are not part of the MR community and who will not be signing up to MR ethics, codes, or ISOs.

Here's a wake up call for the cable companies.

A new report from Credit Suisse says almost 30% of Netflix subscribers aged 18-24 are using Netflix in lieu of cable or satellite TV. (17% of all users are skipping cable for Netflix.)

The emergence of Netflix, and other internet TV options, led Credit Suisse to downgrade the entire entertainment industry.

"Netflix’s low cost, subscription streaming service (with improving content) is our biggest worry and could become ‘good enough’ for consumers with moderate income and TV usage to use as a substitute for pay TV," says Credit Suisse.

That's might be true, but there's a few caveats:

1. Of course younger people would skip cable. They're busy with school and drinking, so cable isn't that important. Also, they don't have a lot of extra cash laying around to spend on cable. Let's see what happens when they get older.

2. This could be bad news for Netflix. If this trend plays out, it's only going to be more expensive and harder for Netflix to secure deals with major content companies at a reasonable price. The cable companies have a lot of sway over content companies that supply live streaming to Netflix. They might try to block more shows from Netflix. Let's not forget Netflix's most recent deal cost it $1 billion over 5 years.

3. Netflix doesn't truly replace cable. There's nothing you can see on Netflix today that's on cable yesterday. In other words, Netflix already has a big delay in its programming. If you want to watch last night's TV show, or even last month's TV show, Netflix isn't a viable option. For people that actually like TV, Netflix isn't a great option.

(via CNBC)

Tom Ewing has a really nice post on the history of 'insight' over on his Blackbeard Blog, and there are some really good comments (and mine).

Here are a few thoughts from me on insight.

Most research needs are not met by insight, they are met by information, carefully analysed, filtered, and presented, but information. How many X are we selling, which of these versions of Y is going to sell best, what are housewives actually using Z for? These all examples of information, not insight, and these are the sorts of things clients need on a regular basis, they also sometimes need insight, but not so often and not in such large amounts - indeed most money is made from operationalising insight, not from having the insight.

The Concise Oxford has two definitions for insight, both useful in their own way.

1) "The capacity to gain an accurate and intuitive understanding of something". What makes this interesting is that it has to be intuitive and accurate. Giving somebody a formula they can use is not insight, because they don't intuitively understand it (you may recall that the volume of a sphere is given by ⁴/₃πr³, but do you intuitively understand it?) Giving somebody an unreliable answer is not insight, because it is not accurate (for example, saying to somebody that if you are allowed to carry a weapon you will be safer). So, an insight has to be something that they did not know, that provides a deep understanding, that is accurate, and which having been given the information, is understood intuitively (not just remembered as facts, in the way that children in the UK used to be taught the kings and queens of England).

2) the second definition is drawn from psychiatry, "awareness by a mentally ill person that their mental experiences are not based on in external reality". So, perhaps finding a way to convince your client that everybody does not in fact love them, that objectively their product is not the best, or that despite what their advertising agency is saying, this new idea is not 'a paradigm breaking, outside of the box, disruptive revolution' counts as insight too?

Bicycles?

With a hat tip to Robert Pirsig, here is an example of insight in context of bicycles. Please note, I have just made this up this evening (the example, not the facts), so I hope it works!

Have you ever wondered why the front wheel on a penny-farthing bicycle was so big? If you already know this then it probably won't be an insight for you. Think about riding a bike, to go faster you peddle faster. But, there is a limit to how fast most of us can peddle, in many cases this will be about 60 turns per minute (Lance Armstrong does about 120 turns per minute). If you do not have gears or cogs then each turn of the peddle is equal to one turn of the drive wheel (on modern bikes the back wheel, but on the penny-farthing the front wheel). The distance you travel, per turn of the peddle, is set by the size of the wheel. If you have a small wheel, then one turn is not very far, if you have a big wheel, then one turn is further.

With a 24 inch wheel, peddling at 60 turns a minute will generate a very modest 4.3 miles per hour (about 6.9 kilometres per hour). However, if the drive wheel were expanded to 53 inches (about 1.4m) then 60 turns a minute would generate about 9.5 miles per hour (15.1 kph). This is why over the course of a few years the wheels on bikes became so big and the penny-farthing was developed.

So, the insight is that penny-farthings had bigger wheels so that they could achieve speeds faster than a quick walking pace, and they disappeared when bike makers started to use a chain, which by using a different sized cog on the front wheel and back wheel, allowed the drive wheel to rotate at a different speed to the peddles. If the cog next to the peddles has twice as many teeth as the cog by the wheel, the back wheel will turn twice every time the peddles go round once, and wheels can now be the same size. 

Adding gears to a bike means the cyclist can choose how many times the back wheel will go round, by changing which front and back cog are in use. To go down a hill the rider might want one turn of the peddles to turn the back wheel several times, because down hill is easy, but going uphill the rider might want 1 turn of the peddle to only turn the back wheel once, making the effort required much less.

Hopefully, if you have stayed with me, you now have an intuitive idea of why the penny-farthing had a big wheel, to go faster, and an insight into a major reason why it died out, a solution was found that was safer and easier to ride (the penny-farthing was very prone to headers - a fall forward over the front wheel). If you have gone from not knowing this to now 'feeling' like you understand it, then that is an insight into how bikes work, as opposed to just a piece of knowledge.

If this example worked for you, let me know, if it didn't, please let me know

Persuasion: The Third-Person Effect

August 3, 2010

Why people think they are less influenced than others by adverts and persuasive messages.

One of the most intriguing things about the psychology of persuasion is how many people say that persuasion attempts have little or no effect on them. Other people, oh sure, adverts, work on them. But not you and I, we're too clever for that.

Attractive woman holding a bottle of beer? Hah! How stupid do they think we are? We know what they're doing and we wouldn't fall for such cheap tactics.

Would we?

Persuasive experiments

So pervasive is this feeling that only 'other' people are influenced by things like adverts that many studies have explored the idea, with an initial surge in the 1980s and 90s. Psychologists wanted to see how much people thought they were influenced by persuasive messages like adverts and compare it with actual attitude changes, if any.

Typically these studies first got participants to watch an advert, read a newspaper article or other medium containing a persuasive message. Then they were asked how much it had influenced them and how much it might influence other people. Since the experimenters measured actual persuasion and knew from previous research how influential the messages were, they could compare people's guesses with reality.

What they found, in study after study, was that participants thought others would be influenced by the message, but that they themselves would remain unaffected. When psychologists looked at the results, though, it was clear that participants were just as influenced as other people. This was dubbed the 'third-person effect'.

Third-person effect

Reviewing the research in this area, Perloff (1993) found that studies on political adverts, defamatory news stories, public service announcements and many more all showed a robust third-person effect. Similar conclusions were reached by Paul et al. (2000), who looked at 32 separate studies.

Perloff also found that when people don't agree with the message or judge its source as negative, the third-person effect became even stronger. The effect is also stronger when messages aren't directly relevant to people.

In other words people are likely to be influenced more than they think on subjects that are currently of little or no interest to them. An everyday example would be seeing an advert for a car, when you're not in the market for a new car. We'd probably guess it has little or no influence on us, but this research suggests we'd be wrong.

Take back control

The third-person effect is unusual because it goes against the general finding that we overestimate other people's similarity to ourselves.

This is what psychologists call the false consensus effect: we tend to assume that others hold more similar opinions and have more similar attributes and personalities to ourselves than they really do.

The third-person effect, though, goes in the other direction. When it comes to influence, instead of thinking other people are similar to us, we think they're different. There are two facets of human nature that support this exception:

• Illusion of invulnerability. People prefer to believe that they are, on average, less vulnerable than others to negative influences, like unwanted persuasion attempts. We all want to protect our sense of control over our lives. One way we do that is to assume that ads only work on other people.
• Poor self-knowledge. Although it's an unpalatable idea, we often don't know what's really going on in our own minds (see the hidden workings of the mind). Not only does this make scientific psychology a tricky enterprise, it also means that many of our intuitions about the way our own minds work are scrambled and subject to biases like the illusion of invulnerability. The effect of persuasive messages is a good example of this.

People often react to this sort of research by saying it's disheartening, which it is. It's not a happy thought that we don't know how easily we are influenced because we don't really know what's going on in our own minds.

However, sticking our heads in the sand and pretending influence attempts don't work is likely to increase our vulnerability. On the other hand, if we acknowledging our lack of insight into our own thought processes, we can raise our defences against the power of advertising and messages of influence, and take back control for ourselves.

Eye Tracking Basics

The scientific study of eye movement began in the 19th century; Alfred Yurbus developed today's modern tracking techniques in the 1950s. Yurbus' often-quoted 1967 book, Eye Movements and Vision, explains how a person's purpose and motivation affects how they move their eyes. Below, Yarbus shows how eye movement over a picture is different for each task assigned to the viewer.

The most recent utilization of this type of research comes via Tobii's Eye Tracking Glasses, which, according to Internet Retailer, sell for $45,000 a pair. Large corporations such as Procter and Gamble haven't shied away from the price of the glasses because it allows for highly precise testing of how branding, product placement and marketing work for target demographics both in the store and on the Web.

A Web-Connected Contact Lens?

Last summer we wrote about Babak Amir Parviz and his University of Washington students embedding LEDs into contact lenses. Says Parviz, "We already see a future in which the humble contact lens becomes a real platform, like the iPhone is today, with lots of developers contributing their ideas and inventions. As far as we're concerned, the possibilities extend as far as the eye can see." Parviz's in-depth article about this subject can be found at IEEE Spectrum. .

Smart Words Watch Your Eyes As You Read

Text 2.0 is software that discerns and reacts to reader eye movement via an infrared light and camera. Eventually, according to EyeTrackingUpdate, "If a reader takes a little more time on a certain word or phrase, eye tracking could trigger a translation, pronunciation or sound effect, a biography on a name, a definition, or an image/animation to supplement and provide meaning." Created by the German Research Center for Artificial Intelligence, this software works with Tobii Eye Tracking technology and will soon be able to recognize when we are skimming and respond by slightly fading non-essential words. Or if your eyes react strongly to a significant passage on a page it could be highlighted and shared with colleagues.

In the same way the Internet thrives on links, eye tracking technology can help us expand the ability of those links to not just more information, but to also control real-life objects - also known as the Internet of Things.

eyeDriver Allows You to Steer A Car With Your Eyes

The research vehicle Spirit of Berlin is a fully automated driverless car. Software called eyeDriver allows a remote driver to steer this car with eye movements. Using similar tools as Text 2.0 eye movements are converted to signals that control the steering wheel. If the remote driver averts their gaze, the car will begin to brake until the driver has put their eyes back on the road. The project, which is funded by the German Ministry of Education and Research, goes by the name AutoNOMOS and intends to make cars safer by eliminating user error, as well as by developing semi-autonomous and entirely autonomous driving.

Affordable Eye Tracking

In a OpenEyes, which promises open-source eye tracking for the masses. Then there is the Ycombinator-funded GazeHawk, which has dropped eye-tracking research costs for websites under the $50 per test subject barrier. And because websites as big as Google get accurate results using only a half dozen test subjects, eye tracking analytics is now affordable for most businesses. Here's a sample of a GazeHawk eye tracking heat map:

Helping the Disabled

Tony Quan is a legendary Los Angeles graffiti writer, publisher and activist. His tag name is Tempt One. Quan was diagnosed with ALS in 2003 and other than the movement of his eyes, he's completely paralyzed. Free Art and Technology (FAT), OpenFrameworks and the Graffiti Research Lab got in touch with the Not Impossible Foundation to bring together a group of open-source hackers to create Eye Writer. Now Quan has virtually returned to the streets to again tag walls, this time with a hi-power video projector. Thought his artist days are over, eye tracking has freed Tempt One to express again. With it he says, "Art is a tool of empowerment and social change, and I consider myself blessed to be able to create and use my work to promote health reform, bring awareness about ALS and help others."

The Eyewriter from Evan Roth on Vimeo.

If you know of other innovations in eye-tracking software and hardware that have not yet been mentioned please add your links and comments below.

First photo from Paul Sapiano.
Second photo from WikiCommons.

See Also

• 6 Brain Sensors You'll Be Using Soon
• Every Machine Can Be Used To Watch You. Interested in Privacy?
• Half Past a Freckle: The Software That Could Make You Wear a Watch Again
• 6 Geeky Devices Connecting the Internet to Your Things
• Personal Sensors: Measure Your Sleep with Zeo

I have experienced a simple version of web monitoring myself: since simply checking out some office chairs at some obscure shopping website, almost every website I visit (including my own blog) is plastered with multiple Google Ads of that specific shop. After several weeks, the office chair issue is long vanished, but the ads still persist. Where can I opt out please?

What They Know [wsj.com] is an interactive graph by the Wall Street Journal, illustrating the degree to which the 50 most popular websites exposes visitors to monitoring (think cookies, beacons, Flash cookies and the like, which together reveal large parts of your past browsing patterns).

Each red slice at the top represents a unique website, like msn.com or photobucket.com. Each website tends to send hundreds of tracker files to third-party websites, such as Google, AOL, Yahoo, or Microsoft. Figure out yourself why Google (26 trackers) seems like an angel next to privacy-grabbing websites such as merriam-webster.com (131 trackers) or careerbuilder.com (118 trackers).