HTML5 - Video and Encoding

I've recently thought I'd dive into the world of showing video online and being an up to date web developer, I don't want to be using no flash stuff.... I want to use the latest and greatest HTML5 video tags after all, it's meant to be easy right?

Wrong. Well, kind of.

If you have a video that is in the right format and encoded with the correct codec (take a look at w3schools for a list of them), then it is actually very simple, you can use the HTML5 Video tag like so:

HTML5 - The Future

---

<video width="320" height="240" controls>
  <source src="movie.mp4" type="video/mp4">
  <source src="movie.ogg" type="video/ogg">
Your browser does not support the video tag.
</video>

---

The multiple sources allow you to define different formats of the same video. The browser will go down the list until it finds a format it can play. If it finds a playable format then it'll do just that.

However, what if you don't have a video in the correct format? What if you're trying to generate your own content, on the fly using a simple web cam on your laptop? Surely saving a video in the format you want is pretty straight forward?

Wrong.

Let's take you through the dark and nasty world of video's in managed code but first, let's give you some idea of what I'm trying to achieve.
I've just been given a Raspberry Pi with a camera module (a great Christmas present by the way) so I thought I'd go about and set up a little home CCTV system. To go a step further, I want the system to be able to detect movement and at that point, start uploading a live feed to a website, where I can then log on and view this live feed. I've also got a couple of laptops around the house equipped with web cameras so my plan is to use them as extra cameras to the system, when they're turned on.

That's the simple brief. I say simple, when you scratch beneath the surface, it gets complicated. The laptops are on various versions of Windows (Windows 7 and Windows Vista) with various versions of the .NET framework installed. The Pi runs on Raspbian which is a port of Debian wheezy which is of course, a version of Linux. So we've got different OS versions with different architectures. Because of these complexities, I want to make this little system with managed code using the .NET Framework. There are quite a few challenges to over come here and I don't want the fundamentals of a language I don't really know to be getting in the way, so I'm going to play it safe and stick with what I know.

Now at this point, I should say this is a work in progress, this project isn't completed by a long shot but I thought I'd blog about the problems I encounter as and when I encounter them.

So, for the time being at least, I'm going to ignore the Raspberry Pi camera module, I'll come back to that later. I haven't done the necessary research but I suspect Mono (the cross platform, open source .NET development framework) won't support the necessary libraries I need to use to be able to capture video feeds but I have a cunning plan for that... that, however, is for a separate blog post. For now I just want to be able to capture a video feed from one of my laptops.

So, where to start?

I said this system should detect movement. To do that I need to compare a frame from one moment in time to a frame in another and if there's a difference then something has moved. Fortunately, there's some great blog posts around movement detection algorithms and I implemented one that's shown here: http://www.codeproject.com/Articles/10248/Motion-Detection-Algorithms

As you go through the above post  you'll notice it has the option of writing to file. Great!
You'll then notice it's writes it as an AVI file. Bad!

AVI uses the Windows Media Video 9 VCM codec. The word "Windows" in there should give you a pretty good indication that browser vendors like Google aren't going to support it and you'd be right. It's not a supported codec for HTML5 Videos and browsers like Chrome and Safari won't play it.

So how we go about saving this thing in a format that is supported by most browsers? In particular, how do we save this thing in mp4 format encoded with H.264?

Well, the motion detection algorithm uses a framework called the AForge.NET Framework. This is a very powerful framework and as their website states, it's a "C# framework designed for developers and researchers in the fields of Computer Vision and Artificial Intelligence - image processing, neural networks, genetic algorithms, machine learning, robotics, etc.". I'm particularly interested in the "image processing" part of that.

As it turns out, AForge has a library called AForge.Video.FFMPEG. This is a managed code wrapper around the FFMPEG library. This library has a class called "VideoFileWriter" and it seems like we're on to something here. It has an Open method with the following specification:

---

public void Open(string fileName, int width, int height, int frameRate, VideoCodec codec);

---

That last parameter allows you to define a VideoCodec to encode it with. Great! Now we're getting somewhere. Surely all we need to do is set that to H264 and we're there! VideoCodec is an enum so let's check out it's definition.

---

public enum VideoCodec {
    Default = -1,
    MPEG4 = 0,
    WMV1 = 1,
    WMV2 = 2,
    MSMPEG4v2 = 3,
    MSMPEG4v3 = 4,
    H263P = 5,
    FLV1 = 6,
    MPEG2 = 7,
    Raw = 8
}

---

What?! No H264? To make matters worse, none of those codecs are supported by the major browser vendors. You've got be kidding right? I'm so close!

Surely the FFMPEG library has an encoder for H.264? It's meant to be the "future of the web" after all...

Let's check the FFMPEG documentation. After a bit of searching you'll come across that yes, it does. Why on god's green earth can we not use it then?! Unfortunately, that's not a question I can answer. However, with AForge being open source, we have access to the source code and with us being software developers, we can solve such problems! After all we know the the AForge.Video.FFMPEG library is just a wrapper around FFMPEG. Come on, we can do this!

If you open up the AForge.Video.FFMPEG solution after downloading the source code of AForge, the first thing that will hit you is this isn't C# we're looking at... this is Visual C++. Now I haven't touched C++ since University but not to worry, we're only making a few modifications and I'm sure it'll all coming flooding back when we start getting stuck into it.

Now where on earth do we start? We've got a library written in an unfamiliar language which is wrapped around another library that we have absolutely no knowledge of. I could download the source code for FFMPEG but let's cross that bridge if and only if I have to.

First off, we know we need an H264 option under the VideoCodecs enum, so let's add that. Open up VideoCodec.h and you'll see the enum definition. Add H264 to the bottom so it looks something like this:

---

public enum class VideoCodec {
    Default = -1,
    MPEG4 = 0,
    WMV1 = 1,
    WMV2 = 2,
    MSMPEG4v2 = 3,
    MSMPEG4v3 = 4,
    H263P = 5,
    FLV1 = 6,
    MPEG2 = 7,
    Raw = 8,
    H264 = 9
}

---

Unsurprisingly, we can't just add an extra option and expect it to work. At some point that enum will be used to actually do something. The first thing it does is to select the actual codec and pixel format to use for the encoding of your video. It does that by looking up the codec and the format from two arrays using the enum value as the position of the item in the array.
These arrays are stored under VideoCodec.cpp. Open that up and you'll see the definition of the video_codecs and pixel_formats array. We just need to add our options in here like so:

---

int video_codecs[] = 
{
    libffmpeg::CODEC_ID_MPEG4,
    libffmpeg::CODEC_ID_WMV1,
    libffmpeg::CODEC_ID_WMV2,
    libffmpeg::CODEC_ID_MSMPEG4V2,
    libffmpeg::CODEC_ID_MSMPEG4V3,
    libffmpeg::CODEC_ID_H263P,
    libffmpeg::CODEC_ID_FLV1,
    libffmpeg::CODEC_ID_MPEG2VIDEO,
    libffmpeg::CODEC_ID_RAWVIDEO,
    libffmpeg::CODEC_ID_H264
}

int pixel_formats[] =
{
    libffmpeg::PIX_FMT_YUV420P,
    libffmpeg::PIX_FMT_YUV420P,
    libffmpeg::PIX_FMT_YUV420P,
    libffmpeg::PIX_FMT_YUV420P,
    libffmpeg::PIX_FMT_YUV420P,
    libffmpeg::PIX_FMT_YUV420P,
    libffmpeg::PIX_FMT_YUV420P,
    libffmpeg::PIX_FMT_YUV420P,
    libffmpeg::PIX_FMT_BGR24,
    libffmpeg::PIX_FMT_YUV420P
}

---

Now we're getting somewhere. Now when we compile this and add it to our project, when we open up a VideoFileWriter using VideoCodec.H264 as the final parameter, the system finds our codec and tries to encode the video using it. Yes! We're there.

Wrong.

What's the red error appearing in our console window?
"broken ffmpeg default settings detected"

Damn. So close. What's going wrong now? As it turns out, the default settings that FFMPEG set for the H264 codec are a load of rubbish. Nothing is ever easy eh?

More on that in the next blog post...

HTML5 - Prefetching

Once upon a time I blogged about the new features included in the HTML5 spec and I was slowly making my way through the big new additions.

That pretty much died out due to a lack of time but I recently attended WebPerfDays and a new feature mentioned there jumped out at me. This feature is prefetch and it has some fantastic implications for web performance.

What is Prefetch?

Prefetching is the ability to request a page even though you're not on it, in the background. Sounds odd right? Why would you want to do that? Well, requesting a page means the browser can pretty much download all the content of a particular page before the user has requested to see it so, when the user does click on a link to go to that page, the content is immediately shown. There's no download time required, it's already been done.

To enable this, all you have to do is add a link tag like so.

<link rel="prefetch" href="http://clementscode.blogspot.com/somepage.html" />

And that's it. When the browser comes across that tag, it'll initiate a web request in the background to go and grab that page. It will not affect the load time of your original page.

The implications of this for web performance is obvious. Having the content of the page available before it's even requested by the user can only speed up your website but it has to be used properly. Adding prefetching to every web link on your website will cause unnecessary load on your web server so this functionality needs to be thought about before being used. A good example of this is Google. If you search for a term on Google, the first link brought back by Google will be prefetched (feel free to check the source to prove that I'm not lying!). The other links brought back are not prefetched. That's because Google know that in the vast majority of cases, the user clicks on the first link brought back and this functionality allows Google to provide you with that page as quickly as possible.

Are There Any Other Benefits?

That depends on your point of view... I primarily work on ASP.NET WebForms applications most of which are not pre-compiled... not ideal but we have our reasons. Using prefetching enables us to request pages before they're hit which, if it's the first time that page has been hit, forces it to be compiled. So we're improving performance two-fold. That initial compilation time has now been taken away from the user and we're getting the usual benefit of prefetching so users are presented with a page almost instantly after clicking.

That Sounds Awesome But What Are The Downsides?

Well, you're requesting additional pages, as long as the user actually goes to that page then that's great but, if they're not, you're placing an additional load on your server that serves no purpose.

Also, if you're gathering website statistics such as number of page hits and such then this will throw those stats off as technically, the user may not actually view that page even though it's been requested.

Finally, this obviously uses client resources, where as this may not be a problem on a nice big powerful desktop, it may be a problem on small mobile device.

And that's about it. Another great addition to the HTML5 spec. As with most things in our world, you need to think about its use rather than just blindly prefetching everything without any thought of the disadvantages of doing so.

Enjoy!

HTML5 - Geolocation

After my last blog post, I thought I'd take a look and see what this geolocation melarky is all about.

As it goes the API is extremely simple and easy to use which makes blogging about it pretty straightforward.

The aim of the API is to get the longitude and latitude of the device that is accessing your website. This information can then be fed into another application to make it more user friendly. For example, a common use for this information is to put it into Google Maps to show exactly where your user is.

So, how do we use this API?

First off we need to make sure that the browser supports geolocation. To do that, we simply need to check that the navigator.geolocation object exists. That can be achieved with the following:

---

if(navigator.geolocation){
   // Do Geolocation Stuff
}

---

All straightforward so far. So now it's time to grab the longitude and lattitude. There are two methods available within the API to accomplish this: getCurrentPosition and watchPosition. getCurrentPosition runs once and gives you the users location. watchPosition will continually run, allowing you to track the user as they move. Both run asynchronously. There is one final method included in the API, clearWatch. This method allows you to stop tracking the users position after you've called watchPosition.

Now let's have a look at these methods in a bit more detail.

getCurrentPosition

The function definition:

---

getCurrentPosition(successCallback, errorCallback, options)

---

The success and error callback parameters explain themselves. The options parameter allows the developer to specify the timeout value, the maximum age of a position call (basically, how long it lives in a cache) and how accurate the position call will be.

To determine how accurate the position call will be you need to understand how geolocation works. There are a variety of ways in which a users location can be worked out, as listed below.

• WiFi - Accurate to 20 metres. The MAC address of any and all WiFi access points detected by the users device is sent across the web and mapped against a database of wifi access points and locations. From what I can find out, different browsers use different databases. For example, Chrome and Firefox use a database created by Google. Safari however uses Skyhook's Wireless service.
• IP - Accuracy varies greatly and can give false information. In this instance, an IP address, or a range of IP addresses are mapped to locations within a database. Unfortunately, due to the nature of IP, addresses and locations change and occasionally IP addresses can get mapped to a completely wrong location.
• GPS - Accurate to 10 metres. This method only works outside so is only ever an option with a mobile device. GPS is quite an advanced system, if you're interested in the gritty details of how it works, have a read of this. This method unfortunately can take a bit of time and can use more power which may be important if you're target device is a mobile phone/tablet. It is however, the most accurate out of all of the geolocation methods.
• GSM/CDMA IDs - Accurate to 1km. This will take unique ID of your mobile device and will use it against the local mobile phone towers to triangulate your position. Accuracy will significantly improve where there are more towers, so, if you're in an urban area, accuracy will be far greater than if you're in a rural area.

Ok, so, here's an example of using the getCurrentPosition method:

---

var success = function(position){ 

    alert("Latitude: " + position.coords.latitude + ", Longitude: " + position.coords.longitude); 

};

var error = function(error){

    alert("Error. Cannot find your position");

};

navigator.geolocation.getCurrentPosition(success, error, { enableHighAccuracy: true });

---

The enableHighAccuracy option essentially means that the method of determining the users position that provides the highest accuracy will be used. This only really matters when your user is using a mobile device with GPS. GPS uses more battery power than the other methods and can take a couple of minutes to return so may be something you want to avoid where possible.

watchPosition
The function definition:

---

watchPosition(successCallback, errorCallback, options)

---

Look familiar? It should do. It's exactly the same as the getCurrentPosition method, with one small difference. This method will return a watch ID. The watch id is then passed into the clearWatch method to stop tracking the users position, but more on that in a bit.

So, like I said, the purpose of this method is to track the users position. When they move, assuming that no errors are thrown, your successCallback will be called. This callback takes the exact same parameters as the getCurrentPosition successCallback function did so, you can grab the new longitude and latitude co-ordinates and use them for whatever you need.

Just as a very quick example of the use of this:

---

var success = function(position){ 

    alert("Latitude: " + position.coords.latitude + ", Longitude: " + position.coords.longitude); 

};

var error = function(error){

    alert("Error. Cannot find your position");

};

watchId = navigator.geolocation.watchPosition(success, error, { enableHighAccuracy: true });

---

clearWatch
The function definition:

---

clearWatch(watchId)

---

Pretty simple eh? This effectively cancels your watchPosition call made earlier. So, as an example, you could have something like this:

---

var success = function(position){ 

    alert("Latitude: " + position.coords.latitude + ", Longitude: " + position.coords.longitude); 

};

var error = function(error){

    alert("Error. Cannot find your position");

};

watchId = navigator.geolocation.watchPosition(success, error, { enableHighAccuracy: true });


function stopTracking(){
  if(watchId != null){
    navigator.geolocation.stopWatch(watchId);
  }
}

---

You can then call the stopTracking function from wherever you want.

And that's about it for the API. If I'm honest, its simplicity surprised me. I personally think that location based services will really take off in the next year (if they haven't already) as HTML5 becomes the standard on more and more browsers. Location information opens up a world of possibilities, from helping you to find your friends in a crowd to providing more relevant information to you when you're performing web searches. Now that the technology is there, it's up to us developers to use it to give our users the best possible experience.

Before I sign off though, I should mention one thing regarding location services and it involves one of the 'keywords' being spoken about all the time at the moment.... privacy.

Google Chrome prompting the user.

Obviously, tracking a user's current position does come with a few privacy concerns. Web browsers have done what they can to address this by ensuring that when you request a users position, either via the getCurrentPosition or the watchPosition method, that the user is notified and they are then given the option of denying you access to that data. (If this occurs, your error callback will fire rather than your success callback.) However, this in my eyes isn't enough. I think websites should take a bit of responsibility too!

Ok, here comes a bit of a rant so feel free to stop reading here but a pet peeve of mine is sites that do request my location information and then use it for something that isn't stated on the tin. Take for example Facebook: Now and again I may want to publish where I am in the world so my friends can see for whatever reason. So, when I go to publish my location the website prompts me that Facebook wants to access my location data. Not a problem, how else are they going to publish my current location? What I do not then expect is for my location to be posted on everything afterwards. Especially when I've gone to the pub instead of heading home! Facebook doesn't just do it after you've published your location - it updates your location whenever you make a comment or a status update and as far as I'm aware, there's no easy way stopping that from occurring.

Now I'm sure Facebook has some setting somewhere that lets me turn it off for posts but I shouldn't have to. I agreed to let Facebook access my data to do a specific task that could not be accomplished without that data. I did not agree to let Facebook plaster it over every post/comment I subsequently make. I'm sure Facebook isn't the only culprit here and I understand that if you throw up a configuration option for each and every action then it's going to become pretty tedious. But, if I had been notified of their intentions at the start then I may not be ranting.

Ok, rant over. In conclusion to that little out-burst... just make sure your users know why you need their location data and exactly what you're doing with it. If not for your users benefit, then do it for your own as I'm sure someone will be ranting about your website if you don't.

HTML5 - Web Workers

It's been a while since my last blog but, here's the next chapter of my HTML5 overview, Web Workers.

So, what are Web Workers? Let's start off with some background information about JavaScript. JavaScript was originally developed by Netscape back in 1995. Its primary use was to allow developers to manipulate web pages, which, as you can imagine, were very basic back in 1995. In order to do this JavaScript was designed as a single-threaded language. Unlike its namesake Java (which is a completely unrelated language by the way) and many other languages, JavaScript does not support threads. The reason for this, I imagine, was very simple. How would you go about designing a multi-threaded language whose primary aim was to modify something (the Document Object Model (DOM)) that was shared between threads, without incurring deadlock problems? This problem remains unsolved. And so JavaScript runs on one single thread.

One Single Thread -  A one-trick pony?

Is it really a bad thing? I suppose it can be argued that in itself it's not. The design decision not to support threading in JavaScript was a good one. It makes it simpler to learn; it avoids some potentially horrific problems and; as the web has thrived in the past two decades, so has JavaScript. It can't be that bad right? Well, yes and no. When everything runs on one thread  it can lead to a very poor user experience. The User Interface (UI) can become non-responsive if not programmed correctly. In order to address this problem two functions were built into JavaScript: setTimeout and setInterval. These allow a piece of code to run after a pre-defined amount of time. The idea being that you could schedule long running code to run when the UI wasn't busy and the thread was free, essentially "hiding" the fact that JavaScript all runs on a single thread. These little hacks have allowed developers to get pretty inventive and have allowed JavaScript to flourish.

Ok, all is good then. What's the problem?

As I said, these are basically "hacks". What happens when the user starts clicking but you've already started to execute a long-running piece of code? You have a problem! The system will not be able to respond to the user's action until the code has completed its execution. And after all, some code, especially data centric code, just takes a long time to run. When this occurs, you'll see an error similar to this:

There's not a whole lot you can do about that. If you do have code that'll take a long time to run then you're a little stuck.

So, where do Web Workers come in?

Simple. Web workers bring multi-threading to JavaScript. They come with a few restrictions though and one is quite a biggy. Web workers cannot access the DOM. Allowing multiple threads access to a non-thread safe resource (the DOM) would cause all sorts of problems so the same design decision was made as in 1995. What they do allow you to do is to process and return data in a separate thread to the UI, so the time of seeing those pesky "unresponsive script" errors should now be gone forever!

Multi-threading eh? Woo! Where do I start?

Well, first you need to make sure you're using a web browser that actually supports web workers. To find that out you can simply visit caniuse.com and look it up. I should mention here that if you're using Chrome and the JavaScript file you're testing is stored locally and isn't running on a web server such as IIS, then you need to enable a flag on Chrome for everything to work. Simply start up Chrome with this command: chrome.exe --allow-file-access-from-files. This problem does not exist with Firefox. For more information, check out this Stack Overflow post.

Now that you are using a web worker enabled browser, you need to define your web worker. As the worker is in an entirely different thread, it has no access to loaded scripts, so you need to tell the worker which script to load. To do this we can use the following line:

---

var workerOne = new Worker('worker.js');

---

where worker.js is the name of your script.

Web workers communicate with the main UI thread in the form of messages. When a message is sent to a web worker, it causes the message event to fire within the thread. To hook in to this, your worker.js file needs to have the following content:

---

self.addEventListener('message', function(e) {
   var message = e.data;
  // Do something with the message
  self.postMessage(message.sort());
}, false);

---

To give you a quick overview of what's happening here, when the web worker is sent a message, the message event will be fired and the inner function defined above will run. It'll get the sent message by fetching it from the event object. Then, in a useful scenario some action would be performed based on that message. You'd then post a message back to the caller (usually the main UI thread). This could just be to notify the thread that it's completed or if you've done some data manipulation, you could post back the modified data. In the above example, the message is sorted and sent straight back.

So, that's the web worker defined. How do you now post messages to that worker thread so you can use it effectively? Well, you've defined your worker object earlier, you just need to:
a) Define what happens when the UI thread receives a message from the web worker and;
b) Send a message to the web worker which will start the whole process.

In much the same way that you need to hook into the message event within the web worker, you also need to hook into the message event on the web worker object itself, within the UI thread. Something like the following should do the job:

---

workerOne.addEventListener('message', function(e) {
        var numbersOne = e.data;         // Do something with this data
     }, false);

---

This will fire when a message is posted from the web worker to the UI thread. In the previous example, e.data will now contain your sorted data!

Ok, now all you need to do is send your original data to the web worker for processing. You use the same method as when you posted the message from the web worker to the UI thread but this time you perform it on the worker object within the UI thread, so you'll have something like this:

---

workerOne.postMessage([1,4,2,7,9,2,4,7,6,9,4]);

---

Now you have something that's a working demo, the array of integers (1,4,2,7,9,2,4,7,6,9,4) is sent to the web worker. The web worker starts up in it's own thread; picks that message up; sorts it and then sends the data back to the UI thread. The UI thread now has a sorted array of data but it hasn't actually done any processing to get that information. It has left the UI thread free, so to the user the system seems responsive. Ok, in this particular example with 10 or so integers there isn't going to be much of a difference, but when you're playing with millions of objects, this can have a significant impact.

Performance

While I was looking at this, I wondered if I could make use of Web Workers so that it would give some significant performance gains, especially in terms of data processing. If web workers work like standard threads then this should be fairly straightforward to test.

Here's my very simple test case:
How quickly can I sort three arrays containing two million integers each?

I'm going to test in three ways:
                1. Use standard javascript. Sort each array, one after the other and time how long it takes.
                2. Use a single web worker. The sorting of all  of the arrays will occurr in one web worker.
                3. Use a web worker for each array sort.

With what I knew about threads and web workers, I thought I'd find the following...
- The first and second test case would be comparatively similar in terms of time taken.
- The first test case would freeze the web browser until all data had been sorted. The other methods would not.
- The third test case would be the fastest, with all three sorting algorithms occurring in parallel. In theory, the time it takes for the third test case should be roughly 66% quicker than that of the first test case.

Each test case was repeated 10 times and an average time was taken, here are the results:

Test Case One: 11.24 seconds
Test Case Two: 13.75 seconds
Test Case Three: 7.21 seconds
(If you wish to actually repeat the demo yourself, you can pick up the files from here)

Interesting! Ok, I wasn't quite right about Test Case Three being 66% quicker, but it is around 33% quicker which isn't too bad. What is interesting is that test case two is almost 2.5 seconds slower than test case one. Just to open up a new web worker and to send/receive the massive arrays adds an extra 2.5 seconds to the processing time, that's almost a 22% time increase. That seems rather high to me but, it's good to know at least.

It's around about this time that I should mention just how the UI thread and worker threads post messages to each other as it can have an impact upon performance. You're transferring data across threads so you can’t just pass a variable by reference. Instead, you need to do a full copy of the variable. How this occurs depends on what you’re doing and how you’re doing it. If you’re passing across a string then the data will be serialized into JSON and sent to the worker thread. It’ll then be de-serialized at the other end. If however, you’re using a complex data type, File or Blob for example, then an algorithm called structured cloning will occur. This will effectively copy the contents of the variable, which for a variable containing megabytes worth of data, can be slow. There is however, another way! Google have come up with a concept of “transferable objects". This allows you to transfer the owner of an object from one thread to another using a zero-copy which is significantly faster. There is one down side to this: once you’ve transferred the object, you can’t then use it in the thread you transferred it from. It can only be accessed by the thread that has ownership. For more information on this, check out this page on HTML5 Rocks.

Ok, now I’ve got that covered, just out of interest, I thought I'd run the same tests as before but this time instead of using unsorted data I'd sort the data on already sorted data, making the sort function significantly faster (as it won't do anything meaningful). I was expecting to find the same sort of patterns as above, just with smaller numbers. Here's the actual results:

Test Case One: 1.91 seconds
Test Case Two: 4.10 seconds
Test Case Three: 3.22 seconds

Two interesting things are highlighted here:

1. Test Case Two is slower than Test Case Three. Why? I haven't managed to find an answer to that yet. I can only assume that the overhead of sending all three arrays at once, which I wrap up into one object, performs badly when using the structured cloning algorithm to post messages to the worker thread.
2. Test Case One is the fastest. This case doesn't use any fancy web workers, it's just plain old JavaScript executing each sort function one after another. So, by adding web workers, we've actually slowed down the data processing process, which is the exact opposite of what we were trying to achieve. The reason for this... the overhead of creating a web worker and communicating with it out-weighs the benefit we get by using a web worker and running data processing in parallel.

Eh? This makes things slower, not faster! What a waste of time!

Well, no. First, slower or not, the UI thread is always responsive when using web workers so, to your user, the system will seem faster than taking the traditional method. Secondly, although using web workers performed worse than the traditional approach in the last test, that won't be the case in all scenarios, as shown by the first experiment. If the overhead of creating a web worker and passing messages to and from it outweigh the amount of time saved by performing calculations in parallel on different web workers, then, yes, the overall performance will be worse, but, if you're performing a vast array of data manipulation on a great many records, then you should see a big performance gain. Like always though, it's best to see how it would perform with your actual data (or something similar). Only then will you be able to gauge just how much quicker Web Workers will make your web application, they are however a tool that you should definitely be aware of as we approach the on-coming HTML5 world!

Finally, if you want to follow this blog post up with further reading about HTML5 Web Workers, the best tutorial I found was posted on the Mozilla website, here.

Enjoy!

HTML5 - Offline Web Applications

As I'm sure some of you are aware, one of the more highly anticipated features of the HTML5 spec is the ability to make websites available offline. This is becoming more and more useful with the explosion of the mobile/tablet market where internet connectivity may just not be available.

I've now got a bit of experience in dealing with this part of the spec so, I thought I'd share a few things with you. For the most part, making your site available offline is pretty simple, but before we start, let me make one thing clear, mainly because this caught me out a bit...

HTML5 offline web applications only truly work with static content.

When you think about it, this makes perfect sense. Usually dynamic content will require some sort of connection to a server and if you're offline then this isn't possible but, what caught me out is that even if there is a connection to a server (i.e. you do have your internet connection), then it's still not possible to update your content, well, not easily anyway.

So, why is this? Essentially, offline support works by the developer specifying what files should be loaded into a cache (the application cache, more about this later). Your users will then hit the site the first time, download all the files asked of them and the files specified by the web developer will be put into the browsers application cache. From this point on, every time the user visits that website, their web browser will check it's application cache for each and every file required by the website, if it finds that file within the application cache then it'll load it from there, if not, it'll go fetch it from the web server. So, if you're offline and the files required are in the browsers application cache then they'll be loaded from there, the web server will never be hit and there you have it, your website is available offline. However, this process happens regardless of whether you're offline or not. This causes problems for dynamic content, take this situation for example:

1. User A goes to a website, and the file that contains the latest news story is put into the users application cache. 
2. User A re-visits that site a few minutes later, the latest news story is loaded from the application cache but as the latest news story hasn't changed, everything looks fine.
3. User A visits the site a week later. The latest news story is loaded from the application cache, the web server still isn't hit. Now, the latest news story is thoroughly out of date, your user is effectively looking at a snapshot of your website which was taken the first time they visited. This obviously isn't what you wanted.

There are ways to force the application cache to refresh (again, more about that in a bit) but, it's not straightforward and requires the user to visit the website twice so is less than ideal so, only use this for content that will very rarely change.

Ok, now I've got that warning out of the way, let's go into detail about how to actually implement this.

The whole HTML5 offline support revolves around getting files into the browsers application cache. To do this, you need to create a manifest file. What's a manifest file? Essentially, it's just a normal text file that has a specific format which will define which files to go in the application cache and which should be fetched from the web server (if available). A few details about the manifest file:

• This file is defined within the <html> tag of your web page, so, for example:

---

<html manifest="/cache.manifest">
<head>
...
</head>
<body>
...
</body>
</html>

---

• The file must be served with a content type of text/cache-manifest. How you do this depends on what web server you're running. Personally, when using ASP.NET, I set up a new HTTP Handler to handle .manifest files and set the ContentType on the Response object to be text/cache-manifest.
• The first line of a manifest file must be CACHE MANIFEST
• There are three different sections to manifest file:
• CACHE - This section defines files that will be added to the browsers application cache and therefore, will be available offline.
• NETWORK - This section defines files that will ALWAYS be loaded from the web server. If no network connection is available then these will error.
• FALLBACK - If a resource can't be cached for whatever reason then this specifies the resource to use instead.

Let's see an example of a valid manifest file now:

---

CACHE MANIFEST

CACHE:

/picture.jpg

/mystyle.css

NETWORK:

*

---

So, what's going on here? Well, the files, picture.jpg and mystyle.css are both added to the application cache (note, that the HTML page you're currently viewing is by default, added to the cache). Under the network section there's a * symbol. This is a special wildcard symbol which effectively says "whatever isn't cached, go and fetch from the web server".
And that's it, you've now got an offline web application.

But.... when are things ever that simple to develop? There's a few more things you should know about developing offline web applications. I'm going to put to you a couple of scenarios and offer a solution to each:

Scenario 1: You've added a new file to your website and need it to be added to the application cache. How do you go about doing this?

Well, logic suggests you'd update your manifest file to include your new file and hey presto, it should be added. Well, you're half right. The problem is, with all HTTP requests, browsers will try and cache the files they retrieve, this is no different for manifest files. So, you'll update your manifest file but, the user won't ever retrieve the new manifest file due to the fact that the browser has cached the old version.

To solve this, I made sure that the manifest file is never cached by the browser and as I use an HTTP Handler to deliver the manifest file, that's easily accomplished by using something like this:

context.Response.Cache.SetCacheability(HttpCacheability.Public); 
context.Response.Cache.SetExpires(DateTime.MinValue);

Scenario 2: The content of one of the cached files has changed. How do I force the user to re-download the new file?

A web browser will only re-fetch cached files when it detects a change with the manifest file. In this particular case, there is no change with the manifest file so how do you get around this? I simply use comments within the manifest file. So, taking our previous example:

---

CACHE MANIFEST
#Version 1

CACHE:

/picture.jpg

/mystyle.css

NETWORK:

*

---

You'll see I've added a version comment. Now, when the content of one of the cached files changes, I increment the version comment and hey presto. The browser will detect the change and will re-fetch all the files to be cached. Be warned, you'll still have the problem of scenario 1 though!

And finally...
Just a few more things to bare in mind while you're developing:

1. If for some reason, one of the files you wish to cache cannot be downloaded then the whole caching process fails. This can be a bit of a pain when you're trying to track down problems.
2. There are JavaScript events you can hook in to, to see what's going on. There's an actual applicationCache object on the window object that exposes useful methods and events. (see here for more details and examples).
3. To maximize the benefits of offline support, you could use local data storage to store data that could then be used offline and/or uploaded to a server when an internet connection is available. See the following for more information: Dive Into HTML5 - Storage for more information.
4. While developing, I suggest you use Google Chrome as your browser. It provides some very useful tools that a developer can utilize for offline web application development, here's a couple I found particularly useful:
1. If you hit F12 to bring up the developer tools then, go to the Resources tab, at the bottom there's an Application Cache option. This will list all the files currently stored in the application cache for the site you're currently viewing. It should help you track down problems when downloading particular files for the application cache. (If they're not listed then something's gone wrong!).
2. Within the address bar, if you type: chrome://appcache-internals then Chrome will list all the applications it has stored within it's application cache. It then gives you the very handy option of deleting it meaning you can be assured that the next time you visit the site, new content will be fetched from the web server.

I've covered a fair amount here, but, if you want further resources, I've found that the Dive Into HTML5 website to be a great resource for all things HTML5-esque. For their article on Offline Web Applications, try here.

And that's it from me for the time being.

Good luck!