In a February 2007 report called “Measuring Rich Internet Applications,” Forrester Research Analyst Megan Burns found that Web site owners were rushing to add rich Internet applications and Web 2.0 technologies to their Web sites. With these more advanced applications, however, respondents faced a problem: They were gathering only basic metrics because tracking and understanding data from rich Internet applications is trickier. Such applications have more trackable actions, the report said, and change the meaning of metrics such as page views and engagement.

A year later, rich Internet applications continue to stymie users, Burns said. “The key here is that people are still getting caught up in the complexity and actually end up making the process more complicated for themselves,” she said. “So while analytics can be extremely helpful, they are still tripping a lot of people up.”

A study by the Web Analytics Association came to a similar conclusion. In the “WAA 2008 Industry Outlook” survey, published in January, 500 people were asked in which areas they would most like to improve their knowledge this year; 64% of respondents said they would most like to improve their knowledge of measuring Web 2.0 technologies.

Measuring Web 2.0 technologies is more difficult for a number of reasons. The first is technology-related. In the past, Web analytics measured simple page views; site operators would place an HTML tag on a page and, when someone visited the page, that tag would register a page view. When Web 2.0 applications first started gaining in popularity, site operators would simply use the same tag they used for their Web pages to tag their Web 2.0 elements. But there's a problem with this, said Eric Hansen, president of Web site optimization provider SiteSpect.

“With Web 2.0 technologies, you're not actually loading a page,” he said. “You're loading a micro-event. So you're measuring things like how many times people click, hover or mouse over it. You're launching applications and you need to measure behavior.”

Most analytics providers, however, quickly realized this and started introducing special tags that are specific to each application. So for example, if you have a Flash product demonstration, you can tag it with a special tag that logs events within the application—not just the single event of someone launching the application, said Brian Tomz, senior director of product strategy at Web analytics and optimization provider Coremetrics.

“These are totally different than the traditional page view,” he said. “They are taking sub-page measurements and let clients get more granular. People know not just that someone came to the page but how long they looked at something, when a new pop-up is chosen [and] in what sequence choices are selected.”

This is important, he said, because marketers are trying to understand what site visitors are interested in beyond what they are buying. “So while it's interesting to know that someone bought a red chair, it's also interesting to know that before they bought it they also looked at red and blue,” he said. “So maybe when I cross-promote, I should promote those colors to convert additional transactions.”

The second challenge that people face is one that's a carry-over from Web 1.0—determining exactly what they should track and what those metrics actually mean, said Brent Dykes, principal consultant in Web analytics provider Omniture's consulting group.

“We need to go back and define success,” he said. “What are we trying to drive here? Leads? Are we trying to close sales? Whatever the success events are still apply, which is difficult when you move from a world of Lego blocks—a page view approach—to a world of Play-Doh when you can mix applications like colors on a single page.”

Indeed, how applications are placed together on a page can affect how long a user stays and whether or not they are interested in what you're offering in terms of site content, Dykes said. “You really need to understand not just the individual elements but the interaction,” he said.

This is where Web analytics experts are particularly valuable, said Brian Induni, president of consulting firm Induni NorthWest. However, he said, there are too few such experts. “We're facing a shortage of people who are trained and know enough about analytics to be helpful with this aspect—figuring out what to measure—as well as with the integration of the data [into] the business. Many companies are going to do best by working with consultants and firms until there's enough people who know what they are doing.”

No matter how people solve these challenges, most experts agreed that for most companies, analytics isn't going to be enough to figure out whether or not a Web application works and how usable it actually is.

“Focus groups and surveys are a good way to get information and monitor whether the audience is responding well to a particular Web 2.0 element,” Tomz said. “You can start slowly and go element by element so you can be ready to make changes.”

You can use analytics to figure out where to start, though, choosing the elements or pages that get the most traffic. You can also use these data to select where you should put the newest elements and compare your site usage with different audience segments, said Dan Robbins, director of marketing at Lyris, a marketing software company, because certain referring sites may have better informed users or those who are more likely to prefer next-generation applications.

“This is going to be important when you're looking for conversions,” Robbins said. “You can see—if someone is coming in from YouTube—what exactly they did on your site. How much time did they spend there? Did they convert? They might like 2.0 technology more than someone who finds you from a search engine.” M