published on December 3, 2009

This is the brief version of my actual master thesis proposal, which is attached in PDF format .

Introduction

My bachelor thesis was about making Drupal web sites load faster. 80 to 90% of the response time (as observed by the end user) is spent on downloading the components of a web page. Therefor this is also the part where optimizations have the largest effect.

To be able to prove the positive impact of optimizing the loading of the components of a web site — thereby proving that the work I was going to have done had a positive impact — I researched existing page loading profiling tools. Episodes (which refers to the various episodes in the page loading sequence) came out as a clear winner.

Also as part of my bachelor thesis, I wrote a simple Drupal module 1 that could create simple charts to compare the average page loading time per day per geographic region.

Despite its obvious (intended) lack of optimizations, it was sufficient to prove that File Conveyor 2 when integrated with a Drupal web site and thus providing CDN integration for that web site, had a positive impact: the test web site consistently loaded about twice as fast. especially for visitors with slower internet connections, such as visitors from Brazil. Without this proof-of-concept implementation, I would never have been able to prove the positive impact on performance.

Context

More and more companies are paying attention to page loading performance. Notable recent proposals include SPDY (a proposed new version of HTTP. with much better performance characteristics), Resource Packages (zipping several resource files together into one package, to reduce the number of requests), Web Timing (a proposed specification to integrate parts of Episodes’ functionality directly into the browser, to allow for more accurate and more complete measurements).

To top it off, Google is almost certainly going to include page loading performance (“page speed”) as a ranking factor (they’ve already included it in Webmaster Tools and are providing a faster DNS service, Google Public DNS ).

Problem

Simply implementing all known tricks is not enough, because using a CDN might speed up your web site for half your visitors and slow it down for the other half — although that’s an extremely unlikely scenario. That’s why you need to be able to do Continuous Profiling (cfr. Continuous Integration ).

Continuous Profiling means that you are continuously monitoring your real- world page loading performance: you must track the page loading characteristics of each loaded page! That by itself is easy: all it requires is to integrate Episodes with your web site. The actual problem lies in analyzing the collected data. To be able to draw meaningful conclusions from the collected data, we need to apply data mining techniques as well as visualizing the conclusions that are found.

So what I think is needed, is something like Google Analytics. but for page loading performance instead of just page loads .

Proposal

So that is exactly what my proposal is: an analytics suite for tracking page loading performance. An application that can automatically extract conclusions out of Episodes logs and visualize them.

Update — accepted!

Great news, my master thesis proposal has been accepted! My promotor will be professor Jan Van den Bussche. whom is a well-known researcher and an excellent speaker. He has many, many publications on query optimization, data mining and related fields in theoretical computer science.

After this semester’s exams (which will be in January), we will discuss the details. If you’re For more details, see the full version in PDF format . (The LyX file is also available.)

File Conveyor is the daemon that I wrote to automatically sync files to any CDN. regardlesss of the file transfer protocol used. ↩