Managing Analytics

There is no lack of talking about analytical framework. Each of them has its pros and cons. The one we created is called “A.M.M.O”. Let me explain this to you and see if you like it or not.

First, let me explain what the acronym really means:

“A” stands for “Analyze”; First “M”, stands for “Monetize”; Second “M”, stands for “Mobilize”; “O”, stands for “Optimize” .

See, I managed to find four words all end with “ze”.

“Analyze” is straight forward. That’s what we, as web analysts do every day, so no need to explain, right?

One thing I want to caution is though, we need make sure that the data we are looking at is correct. If the raw data is not even correct, then that’s “garbage in, garbage out”, regardless how sound your analytical framework is. The analysis results might be misleading and hurting the business, rather than helping the business.

In the web analytics world, we usually are dealing with two sets of data. The first set of data is either the overall metrics at the site level or the most granular data at page level. For this type of data, it will be hard to make mistake. And if you ever made a mistake, even your HIPPO will quickly point it out for you!

The second set of data is at “segment” level. At this point, all web analysts should know that “segmentation” is at the core of any web analytics project. If you don’t know how to segment your visitors or traffic and analyze visitor behavior accordingly, then you should really go back to the training camp.

But the risk is also with “segmentation”. Anybody has experience with Omniture Discover on Demand or Omniture Insights (previously Discover on Premise and prior to that Visual Sciences) or similar tools, probably understand what I mean. There are two major factors could screw the data. First, the exact way to build the segments: in Discover or Insights, you have many different ways to build a segment and not all of them give you the same data. Second, the sampling issue: due to huge amount of data, a lot of web analytics applications can only give you a sample rather than the complete data set, depends on how much data you are asking for.

With this understanding in mind, it is imperative for any web analyst to have a healthy amount of suspicion of the data accuracy. Not only the data pulled by somebody else, but also the data we pull. Yes, we make mistakes too!

There are ways to mitigate the issue of course. (we have hope-it’s not the end of world!). What I usually do is to make sure that I can get the data from two different data sources and compare them side by side. If you are using Omniture, you should always try to compare the data you get from Discover vs. the data from SiteCatalyst. For example, if you are building a segment for a group of pages, you should at least compare the total visits for that group of pages from Discover with the sum of all visits of those pages. These two numbers will never match but they shouldn’t be way off. If I see a huge difference between them, that’s a red flag to me. [Bonus tip: if I have to bet $10 on either one, I will definitely bet on SiteCatalyst and go back to review the data I got from Discover]

Ok, that’s all I want to say about “Analyze”, assuming you’ve got the rest. In my next few posts, I’ll touch upon the other letters one by one and round up with a final summary. Stay tuned!