Most modern analytics tools today provide internal site search analysis functions. In today’s post, I will discuss how to use Coremetrics to track the on site search performance.

Understand Site Search Usage: Monitoring Site Search Effectiveness

The first step is to set a baseline for search usage and impact for ongoing comparison to future time periods. To do so, you might want to create some key performance indicators specific to your business which allows for consistent internal benchmarking of site search performance.

In Coremetrics, you can get internal search related metrics in the on-site search report. Also, you can apply an internal search segment to the top line metric report. The following table shows the output of a typical on-site search effectiveness analysis.

Search Terms Analysis: Reduce Unsuccessful Queries

Knowing what people search for on your site is very important. This information can not only let you understand visitors’ intent, since internal or external key phrases convey intent, it also could help you identify popular search terms that return no search results to users. By identifying theses terms and tuning your site search engine to return results, you can drive incremental revenue and customer satisfaction.

In Coremetrics, you can get the internal search terms in an on site search report. By sorting ascending, you can get the following output of a typical zero result return analysis.

Measure internal site search quality: improve site search design

Bounce rate is most analytics’ favourite because it is the most useful metric to measure your external search and landing page performance. For measuring the internal search quality, we use a similar metrics called “exit rate”. In Coremetrics, you can use a Clickstream report to monitor where people went after typing the search terms.

If you classify traffic by site departure, second search, viewed additional results page, and product details page, you will get some useful metrics including search exits rate.

Once you get this report and discover the problem, you can take the following actions:

• High Rates of second searches: This means too many results are returned and search ranking system doesn’t show what the visitor wants. You can consider adding filtering query refinement technology to allow users to refine their searches without needing to enter a new query.
• High rates of site departure (Site Exits): This indicates that unsuccessful search results were confusing, causing visitors to depart. You need to investigate the search results page design and ensure that this page provides clear instructions of refining queries in the case that a visitor did not receive results.
• High rate of abandonment to the other page (Path Exits): This indicates that visitors did not find the information they were seeking via search. You should analyze results’ relevance to understand and improve search engine effectiveness.

Summary

The discussion above only shows how to measure internal search by using Coremetrics’ default reports, you could get more in depth if you segment the traffic by keyword terms or many other interesting things. Also, you could easily use the concepts above to do internal search analytics by using Google Analytics.

On-site search is very useful to enhance the revenue performance for many websites, especially for the ecommerce site. If internal search is important to your site, make sure you are tracking it adequately so you can improve it and increase your overall website conversion.

Internal Site Search Analytics Tips is a post from WMpS, your one stop digital agency.

Why Integrate Web analytics and CRM?

Those who are using web analytics as the main decision making tool can easily know a great deal about what happens on the website, like when your website sells something, but that means we are losing the most important part: off-line data. And the true success event often takes place off the website. More importantly, for all the great information we have from web analytics, it’s all anonymous. We don’t really know who the visitors are, so we can’t easily connect their website behaviour to other interactions.

In contrast, CRM is the system that stores all the information you have about your prospects or customers. It normally includes all contacts with customers while they were prospects, all customer service touches, what products they use and how much they pay for each. The main thing to understand is that CRM systems contain pretty much all data about customers that takes place after you know who they are.

What if we could take all of that anonymous website behaviour and somehow connect it with the known prospects or customers’ behaviour stored in the CRM system? Imagine if every time a customer filled out a form on your website, the sales person could see what that person had viewed on the website, what products they had looked at. That would get you more meaningful insight.

How to connect Web Analytics to CRM?

How to connect the session ID and Cookie Id in your web analytics platform with the customer ID in the CRM? Many enterprise web analytics vendors have provided an interface to connect your web analytics platform to a CRM system. The Omniture SiteCatalyst provides a transaction ID that allows you to connect online and offline data by establishing a “key”. By passing same ID in CRM to SiteCatalyst, you could upload offline data related to this ID, and it will be associated with all web metrics.  For instance by passing the return information to your web analytics platform by the transaction ID, you would know the website behaviours for the visitors who returned their items, which is information you normally can’t get from a web analytics platform.

Also, you can pass your web analytics data to your CRM.  The major web analytics suppliers all provide data export functions, and some enterprise vendors such as Coremetrics provide tags to track registers’ information. Therefore, you could pass the web metrics to your CRM systems by using the same registration ID.  For example, a visitor came to your website and filled in the lead form.  Coremetrics and your CRM system would both generate a record using the same registration ID. If you have exported the registration ID and other useful metrics to your CRM system, when this customer come to you or call your sales team, you will already have discovered what products he is interested in.

Summary

The history of web analytics tracks that of the web itself. We went from logs to tags, from IT-centricity to marketing–centricity. Today’s web analytics could be viewed as a smaller, narrower and more agile little brother of business intelligence. Eventually, as web analytics grows and integrates with business data, it may become part of business intelligence, and maybe an important part.

Deeper Insight: Integrating Web Analytics and CRM is a post from WMpS, your one stop digital agency.

What does it do exactly?

It allows you to view website clicks on your website through visual heat maps. This is important for a number of reasons such as monitoring landing page performance… and ultimately improving it. Is my call to action as effective as it should be? Are visitors overwhelmed with the options available? It gives webmasters the important information they need in an easy to understand format. The data can be broken down to a ‘Confetti’ mode which breaks down visitor behavior by search engine search term, browser or referring source as examples.

Visitor Click Heat Maps

Impressively the data can be viewed live which is a big plus point compared to something like Google Analytics which normally has a couple of hours delay.

It requires a JavaScript section of code to be added to the desired tracked page on your website in order to work.

Prices start from around £5 per month which we feel is very cost effective for the data which it provides.

Crazy Egg – Visualize Visitor Behaviour is a post from WMpS, your one stop digital agency.

Data Collection Challenges

The current web analytics data collection is based on JavaScript. The JavaScript initiates a 1px image request to an analytics service provider and the relevant information desired for an understanding of the visitor behaviour is sent along with the image request. Among the information gathered is both a persistent and a session cookie. However, although some smart phones like iPhone and Android do support JavaScript, there still are hundreds of different kinds of devices in the markets and most of them don’t support JavaScript yet.

Another issue is cookie support. If you use different analytics tools to measure your mobile site at the same time, you will be surprised by the results.  Two different tools can return totally different unique visitor numbers. These differences are because many mobile devices don’t support cookies and analytics vendors have different ways to identify mobile unique visitors.

Bypassing the issues

In order to combat the issue that some low-end mobile devices don’t currently support JavaScript execution, we have to send the image request manually instead of calling preset JavaScript methods. Fortunately, Google Analytics provides a smart solution by calling the server side snippet solution instead of sending ugly image request query string parameters. Google has pre-defined the image request server script in the most popular languages such as PHP, JSP and .Net. Users just need to download the preset script file and put a tracking snippet on the mobile site to call the pre-defined file. You can download the pre-defined file from Google Code, and then put the following snippet (PHP Version) at the end of your mobile site:

<?php
  $googleAnalyticsImageUrl = googleAnalyticsGetImageUrl();
  echo '<img src="' . $googleAnalyticsImageUrl . '" />';
?>

Unfortunately, many analytics vendors, including Google Analytics, haven’t provided any solution to bypass the cookie support issue. As a result, Google Analytics always counts a mobile visit as a unique visitor and displays inflated figures. In order to get correct visitor information, we need the persistent visitor ID. Some enterprise analytics vendors, such as Coremetrics, allow the users to send extra query string values with each tag to associate it with both a session and a cookie ID.  The cookie ID source value will be persistent and based on a value defined within the device and accessible via API or OS layer to the code generating the image request. And the session ID could be randomly generated with each device session or generated using a device session value. As a result, Coremetrics could be able to identify new visitors or repeat visitors.

Pitfalls of Current Mobile Analytics Tracking

Again, like standard web analytics, the data collected from analytics vendors is not 100% accurate. The cookie IDs generated by the device API are not always stable, and manually sending image requests do not support some high level analytics functionality such as event tracking in Google Analytics and Marketing tracking in Coremetrics.   With increasing popularity of mobile usage and high demands of accurate data, I think more improvements will be made in the near future.

Mobile Analytics Tracking: Challenges and Solutions is a post from WMpS, your one stop digital agency.

Now we have established that Hypothesis testing is a way of systematically quantifying how certain you are of the result of a statistical experiment. It starts by forming a null hypothesis such as “Design A performs better”, and then converts it to a mathematical statement. Finally, we need to put in a probability distribution to test it using a specific confidence level. In today’s article, we will apply this to our real world analytics application: A/B Testing.

What is A/B Testing?

A/B testing is one of primary tools in any data-driven environment.  It’s a way of conducting experiments where you compare a baseline control sample to one or more test samples by assigning each sample a specific single variable change.

For example, you might have a landing page that shows the latest products list. You’ll want to test various layouts to try and maximize the sales made by this page. Normally, we use “conversion rate” to measure the page’s performance. By assigning the control sample and each test sampling similar traffic, we can make a decision by observing the conversion rate.

The Fake Data

You might ask: “Why do we need hypothesis testing if we already have conversion rate to measure the performance?”

Assume you are running an email campaign to show the latest offers. You have 3 versions with different layouts: control sample, test sample A, and test sample B.  You run an A/B test before formally running the campaign. Here are the results you might get:

In terms of the results, could we make a judgement that A is best now? When the sample size is large, the results might turn into this:

Don’t be surprised if you get these results, because the sample size matters. The problem is when you run an email campaign you wouldn’t be able to get a large test sample size. If you make a judgement only based on the comparison of the conversion rate, you might just make a wrong decision.

Hypothesis Testing

In order to avoid the wrong decision, you might need to use hypothesis testing to justify the results you get, especially for the small sample size or similar results.

Remember the statistics principle we mentioned in the last post: if a small probability event happens in your test sample, you could reject the null hypothesis. In this case, the null hypothesis could be that the conversion rate of the control treatment is no less than the conversion rate of our experimental treatment. So mathematically

H₀ = P – P_c ≤ 0

Where P_c is the conversion rate of the control and P is the conversion rate of one of our experiments.

Therefore if the probability of H₀ is low enough, we could reject it and go for the alternative hypothesis, that is “the experimental email campaign has a higher conversion rate”. That is what we want to see and quantify.

In order to measure the probability of H₀, let’s say P(H₀), we need to know its probability distribution. The sampled conversion rates are all normally distributed random variables just like the coin flipping. Instead of seeing whether it deviates too far from a fixed probability we want to measure whether it deviates too far from the control treatment. There is another statistic rule: the sum or difference of two normally distributed variables is itself normally distributed. With this rule, we could do Z-Test and calculate the 95% confidence interval like we did in the coin flipping example.

Z-Test

Mathematically, the calculation of Z-Score for the probability of H₀ is:

Where N is the size of the experiment sample and N_c is the size of the control sample.

Do you remember that we used the z-score of 1.96 to correspond to the 95% confidence interval? This time it’s a little different. We will use 1.65 instead of 1.96. Why? In the coin flip example, the null hypothesis is P = 0.5. Therefore we could reject it if the probability is too high or too low, but this time we only care one way.

If the z-score falls in the blue part, we assume it is a small probability and reject the null hypothesis. In the coin flip example, the blue part distributes on both sides of the normal distribution. It is called a two tailed test.

In this A/B Testing example, we’ll only reject the null hypothesis if the experimental conversion rate is significantly higher than the control conversion rate. The blue part is only on the right trail of the normal distribution. This is called a one tailed test.

Using the formula above, we could get the following results:

We could find that none of the z-scores is large than 1.65, which means the results would likely change if the sample size goes larger. In this case, we can’t decide which one is best, and we’d need a larger sample size to get a right decision.

Hypothesis Testing in A/B Testing is a post from WMpS, your one stop digital agency.

Hypothesis Testing

Hypothesis testing is something that you can use to make decisions based on experimental data. It can systematically quantify how certain you are of the result of a statistical experiment.  For example, you might want to test if coin flipping would give a fair result. So you make an experiment where you flip the coin 100 times and you get results of 52 times for head side and 48 times for tails side. Would that be fair enough from the statistical view? That’s why you want to do a hypothesis testing.

Most hypothesis testing uses null-hypothesis. The null-hypothesis, denoted H₀, typically proposes a default position, such as the coin flipping is fair. And it is typically paired with an alternative hypothesis, such as the coin is biased, denoted H₁. Normally, we put the hypothesis we care or we want to be true as the null-hypothesis. And the main goal of hypothesis testing is to tell us whether we have enough evidence to reject the null-hypothesis.

Turning to statistics

After stating the relevant null and alternative hypotheses, we’ll need to think in a statistical way. In the above coin example, if we say the coin is fair, that means the probability of getting a head side should be 50%. But the results of the experiment we got were 52%.  But is this normal because we just flipped it 100 times as the experiment?

Statistically, if the probability of something is very low, we consider it as impossible. In this case if probability of having the variance between the experiment results and hypothesis is small enough, we could reject the hypothesis. That’s because if something is considered to be impossible and it happens in the small set of observations, there must be something wrong with the hypothesis.

Mathematically

Now we have to do it completely mathematically.  The null hypothesis in the coin example could be expressed as

H₀:p₀ = 0.5

A 95% confidence level means we reject the null hypothesis if p falls outside 95% the area of the normal curve given above. We can see this corresponds to approximately 1.98 standard deviations.

Then we use “Z-Test” to get the z-score which tells us how many standard deviations away from the mean our sample is, and it’s calculated as

The p is the sample mean, and the P₀ is the expected mean, and N is the sample size.  For the coin example, P₀ is 0.5 and N = 100.

In our experiment we flipped the coin 100 times and got heads 52 times, so the sample mean is 0.52%. After calculating the z-score by the formula above, we get the z-score as 0.4 and make the conclusion that the coin is fair.  If we use coin 2 and get heads 60 times, we can reject the hypothesis and say that coin 2 is biased.

What’s next

Of course, this could be much more complex than coin flipping in real world applications. Fortunately, hypothesis testing for the A/B testing is usually quite similar to coin flipping. In my next post, I will show you how to apply our hypothesis testing knowledge to A/B testing to determine whether new features actually affect user behaviour.

Statistics and Web Analytics – Hypothesis Testing is a post from WMpS, your one stop digital agency.

About Predictive Analytics

Basically, the main idea of predictive analysis is to use past data to predict future events. The statistics models is used to capture relationships among many factors to allow assessment of risk, determine market patterns, or predict potential opportunities for growth.

Why Is It Difficult to Include in Web Analytics?

One of the most important factors responsible for reliable predictive analysis is data quality. Because all predictive events are based on the data we have, the information can only be as effective as the abundance and accuracy of data available. Due to the mechanism of data collection, unfortunately, web data for the most part is completely anonymous, usually incomplete and unstructured.  For example, cookies and JavaScript could be blocked by some proportion of users or it could be loaded and activated improperly due to slow network speed or other technical reasons.  And it is important to know that these are quite normal for data collection in web analytics today. So it is really hard to do predictive analysis when the core things you are relying to capture data are anonymous cookies and sensitive JavaScript tags.

Available Predictive Web Analytics

Many web analytics solutions providers have started to research to offer predictive analytics solutions along with web analytics. One example is The Intelligent Offer, which is provided by Coremetrics and generates personalized cross-sell and up-sell recommendations based on web data. It uses cookies to build individual visitor profiles based on historical and current session data. The profile is then fed into the personalisation algorithm, which determines if and how recommendations are tailored to the individual.

Traffic prediction also is a feasible possibility for implementing predictive analysis in web analytics. Because visitor traffic is time series data and relatively cleaner than the other metrics we get from the web analytics tool, many statistical models could be used to analyze it and forecast future traffic based on past data.  Here is a mobile traffic prediction example we did for a client. We employed a polynomial regression model to forecast future mobile traffic.

Web analytics Needed for Multichannel CRM

Many companies now have realized that basic click stream analytics don’t tell the whole story. They are looking for a more intelligent way to find hidden customer behaviors. In order to implement predictive data analysis for web data, they need to expand their web analytics strategies and integrate web analytics with CMS (customer relationship management) systems.

With multichannel CRM, aggregated web data could be integrated into offline channels and provide great insight, even from simple analysis. It would enable online analytical processing (OLAP) to produce historical perspectives on customer acquisition cost, cost per conversion, etc. And combining web analytics with predictive analytics, such as data mining, could provide both historical and predictive insights.

Predictive Analytics: Do They Work? is a post from WMpS, your one stop digital agency.

Why Do Businesses Need to Measure Facebook Analytics?

Since Facebook is a good place for marketing, there must be some demand for analytics.

• Observing visitors’ interactions with your fan page will give you basic insights into the performance of your business’ Facebook fan page and other apps.
• Facebook analytics can provide much more valuable information than normal analytics, such as demographics, gender, and education level. It is important to know who your potential customers are before you run a campaign.

Why Is Measuring So Difficult?

As I mentioned before, Facebook requires that organizations use their own markup language, Facebook Markup Language (FBML), to build a custom fan page instead of standard HTML. Furthermore, Facebook doesn’t allow standard JavaScript to run on a page on load. This means that when a visitor opens a Facebook fan page, JavaScript can’t be activated if the visitor doesn’t do anything. Instead, the developers have to use Facebook’s own solution FBJS (Facebook JavaScript) if they want to include any JavaScript in their custom fan page. Whether Facebook wants to protect users’ privacy or to build their own empire, this requirement makes tracking using traditional web analytics JavaScript tags impossible.

Available Facebook Analytics Tools

The main analytics vendors have already announced their capabilities to measure social networking. The Facebook analytics war began when WebTrends first announced their Facebook analytics capability. A few days later, Omniture and Coremetrics unveiled their own. Meanwhile, Facebook provide their own free analytics service, Facebook Insights.

• WebTrends was the first company that announced their Facebook analytics capability from the three main vendors.  They use an innovative method which uses a data call to pass parameters from the data collection API to capture all of the typical data as well as tracking flash, pop ups, email signups and other custom fields.
• The partnership between Adobe-owned Omniture and Facebook has enabled marketers to buy the capability to measure Facebook media through Omniture’s search campaign management platform. Ominiture has also developed a FBJS measurement library that allows them to track behavior data natively within Facebook.
• Social network analytics as well as mobile analytics are highlighted as one of the key new features by Coremetrics whenever they promote their new product ‘Analytics 2010’. This allows users to see how interaction with specific tabs leads to web site engagement and conversion. However, Coremetrics hasn’t dedicated any focus to reporting user interactions within Facebook.
• Facebook Insights is a free analytics tool provided by Facebook. It offers an aggregate view of fans’ interactions and demographic information across the custom fan pages and apps. Earlier this month, Facebook announced a new enhanced version which will show data for both fully-integrated sites and sites that use Facebook’s social plug-ins. In other words, now you can view the specific story that people ‘liked’ on your site or how many people commented on posts on your Facebook fan page.

Can I Just Use Google Analytics?

Finally, it is time to turn to our favourite analytics program – Google Analytics. Google hasn’t officially announced their capability to measure Facebook analytics. However, many intelligent GA fans are working on it and trying to figure out how to enable Google Analytics on Facebook. There is an open source tool called FBGAT (Facebook Google Analytics Tracker) hosted by Webdigi which allows users to generate a Google Analytics tracking code for Facebook fan pages and paste it into Facebook custom tabs. The online version of the tracking code generator can be found here. After pasting the code on the FBML page, your results will appear in the Google Analytics content report in the next 24 hours. You can find more information about the Google Analytics for Facebook project at Facebook’s Developer Wiki Page.

Facebook Fan Page Marketing part 2: Facebook Analytics is a post from WMpS, your one stop digital agency.

The travel industry today is facing severe challenges like price wars, safety and security issues (and that’s not even going into the volcano problems of late). Now the internet is providing a medium that  increases customer awareness of these issues. Essentially being a tour operator online is as difficult as ever. Only adding to the difficulty is the issue of measuring a tour operators website performance, which can be hard at the best of times in comparison to a retail site due to the company selling invisible products like ‘experience’. Website traffic and conversions can easily be tracked, measured and compared but how do you define a conversion for example on website where you can’t directly book? This is a problem a lot of luxury tour operators face. A way out is to find out the Key Performance Indicator (KPI) to measure performance.

What is the difference between a metric and a KPI?

Before I start to go through the travel site examples, it probably makes the most sense to talk about the difference between metrics and KPIs. A metric just is a number, it can be viewed as a count (number of visitors) or a ratio (conversion rate). All of the data we get from analytics tools are metrics.

KPIs are metrics, but not normal metrics. A definition of a KPI is a metric that helps you understand how you are doing against your objectives. In other words, KPIs are a bridge between business objectives and web analytics data. Because different companies have different objectives, the KPIs tend to be unique to each company.

For example, on an ecommerce site like www.24studio.co.uk, the objective is to sell as much product as possible. So the KPI could be based on the number of orders, and average size of orders. For the a luxury travel site www.turquoiseholidays.co.uk, the business objective can be sending out so many brochures to encourage a holiday purchase, so one KPI could be the amount of brochures sent out that lead to conversions.

Web analytics measurement framework

Here is a framework proposed by Avinash Kaushik, which is very useful to prevent us from getting lost amongst the data and metrics.

The framework is split into 4 parts:

• Business objectives:  These are the answers to questions like “Why do I need this website?” or “What do I want from this website?”.
• Goals are specific strategies that will help you to accomplish the objectives.  We can define some high level goals to help identify the specific activities we should spend our valuable time on.
• KPIs: as I stated above, KPIs are the metrics attached to our goals.
• Segments: Without segmenting the data, overall totals can often be quite useless, so the best way to find insight is to segment the data. For example, we could choose visitors who spend more than 2 minutes on the site and look at their activity specifically. These are the useful metrics we want to look for.

KPI examples for an online travel website

Now we could follow the framework and start to identify the KPIs for a Web Travel Site.

After we define the Key Performance Indicators, we should start to think about what segmentations we should make in order to measure and improve the KPIs. For example, a KPI above is Revenue/Booking. So the segments we need might be:  what are the most popular destinations or hotels, where are is the traffic coming from or who looks at these hotels?

What else do we need to do with KPIs?

There actually is a one more thing we need to do with KPIs and that is to set KPI Targets.  They are the numerical values which are pre-determined as an indicator for success or failure. It is crucial to create the targets for every KPI. In order to set the number, we need to look over the historic performance. Without having a KPI target, you can collect all the data you want, but you won’t know if you are hitting your objectives or not.

That’s all for this week, check back next week for another look into the world of analytics.

Metrics vs KPIs: All you need to know! is a post from WMpS, your one stop digital agency.

“Over the past year, we have been exploring ways to offer users more choice on how their data is collected by Google Analytics. We concluded that the best approach would be to develop a global browser based plug-in to allow users to opt out of being tracked by Google Analytics. Our engineers are now hard at work finalizing and testing this opt-out functionality. We look forward to make it globally available to our users in the coming weeks.”

Why Google is offering an opt-out

When I first saw this news, I naturally recoiled at the idea of voluntarily allowing measurable data to slip through our hands. Rationalizing web analytics data is already hard enough, and now it will be even harder.

But I was not surprised. There are increasing concerns about privacy issues and Google’s products. This week, a open letter co-signed by officials in privacy commissioner roles in France, Germany, the Netherlands, the UK and 10 other countries was sent to Google’s CEO Eric Schmidt, raising concerns about privacy issues in Google Buzz and Google Street View.

What does this opt-out mean for Google Analytics Users?

In my opinion, it is highly doubtful that this will do any substantial harm to Google Analytics and its users. Here is my point of view:

First of all, not all users will know about the opt-out plug in, and only a few of them will actually download and install it. And most of those people will be privacy fanatics who might have already blocked the cookies or enabled javascript to make themselves invisible on the tracking list.

Second, even if we assume that there are some people who will undoubtedly adopt the opt-out plug in, it’s still unlikely to strike a deadly blow to web analytics. As far as I know, web analytics is more concerned about data trending, not the data numbers themselves. So even if the numbers drop a little, the trending data will remain valuable for web analytics.

And Google are not foolish enough to kill their own business. If the new plug in did have a negative impact on Google Analytics in the future, Google would find a way around it.

3 Ways to Prepare

Even the flutter of a butterfly’s wing can affect a tornado, so the opt-out plug in will definitely have an impact. There are some things we should start thinking about before it goes live.

1. Benchmark our traffic and known visitors before and after the release of the Opt-Out so we can evaluate the impact on our traffic measurement and whether the traffic drop-off is correlated to the visitors we need to meet our site’s goals.
2. Be ready to accommodate the bias that Google Analytics opt out incurs on our metrics.
3. Be ready to have another analytics tool as a backup.

An Opt-out for Google Analytics On the Way? is a post from WMpS, your one stop digital agency.