How to monitor user behavior in webpage

Sometimes there is a need for website owners to monitor user behavior on the site so that they can know what pages are mostly visited and which parts are more popular so that they can provide better service to their users. These behavior usually contain user clicks, user mouse over events etc. These data can be sent back to server when triggered with some meta data.

In this post, we will cover a few ways to monitor user behavior on a web portal and send data back to backend sever.

1. Synchronous AJAX

A common way to send data back to backend server is using AJAX to send the data in the page unload event. However, asynchronous AJAX may not succeed in unload event callback as the page is being unloaded. The browser may send the data, it may also not. Hence need to make the AJAX call as synchronous.

window.addEventListener('unload', function (event) {
  let xhr = new XMLHttpRequest();
  xhr.open('post', '/log', false);
  xhr.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
  xhr.send('foo=bar');
});

The third parameter in the xhr.open() call indicates this is a synchronous request. The major drawback of this method is that browsers are moving away from allowing synchronous AJAX call on main thread which means above code may not work in the future.

2. Asynchronous AJAX

It is also possible to use asynchronous AJAX call in unload event callback, but the precondition is that there must be a synchronous operation in the event callback which takes enough time so that the asynchronous AJAX gets the chance to send out the data.

function log() {
  let xhr = new XMLHttpRequest();
  xhr.open('post', '/log', true);
  xhr.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
  xhr.send('foo=bar');
}

window.addEventListener('unload', function(event) {
  log();

  // a time-consuming operation
  for (let i = 1; i < 10000; i++) {
    for (let m = 1; m < 10000; m++) { continue; }
  }
});

3. Track user click

setTimeout can also delay page unload so that the asynchronous call can be successful. Below is one example of tracking user click.

const clickTime = 350;
const theLink = document.getElementById('target');

function log() {
  let xhr = new XMLHttpRequest();
  xhr.open('post', '/log', true);
  xhr.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
  xhr.send('foo=bar');
}

theLink.addEventListener('click', function (event) {
  event.preventDefault();
  log();

  setTimeout(function () {
    window.location.href = theLink.getAttribute('href');
  }, clickTime);
});

Above setTimeout call waits for 350 milliseconds and then the page is redirected. And at the same time, some user behavior data has been sent to backend.

4. Bounce tracking

Another way to track user click is using bounce tracking. Bounce tracking refers to the process of redirecting the page to an intermediate tracking page and thereafter redirect to the target page.

const theLink = document.getElementById('target');

theLink.addEventListener('click', function (event) {
  event.preventDefault();
  window.location.href = '/jump?url=' + 
    encodeURIComponent(theLink.getAttribute('href'));
});

In above code snippet, when user clicks the link, it will first be redirected to the /jump page and that page will record the user behavior and then redirect to the target link. 

Search engines like Google and Baidu are adopting this mechanism when user clicks the links on the search result page.

5. Beacon API

All above methods mentioned so far will delay the page unload which impacts user experience. To solve this issue, modern browsers implements a special API called Beacon API. It allows the call to be separated from the main thread and will be put into the browser process which ensures the request will be sent out all the time.

window.addEventListener('unload', function (event) {
  navigator.sendBeacon('/log', 'foo=bar');
});

The navigator.sendBeacon() method can guarantee the asynchronous request sent out. The first parameter is the URL and the second one is the data to be sent. The request sent by Beacon API is a POST call.

6. ping attribute

The a tag in HTML has an attribute ping, it will send a POST request to the URL specified as long as user click it.

<a href="https://baidu.com" ping="/log?foo=bar">
  click
</a>

When user clicks the link, a POST request will be sent to /log. It seems ping can only accept data as query parameters.

Reference: http://www.ruanyifeng.com/blog/2019/04/user-tracking.html