A traditional problem with websites is that they are very much about the moment. A page exists in a single moment and once a user is done with it it’s gone. Session state, cookies, query strings and so on have all been attempts to address the inherent stateless (or in the moment) nature of the web. More modern web frameworks provide built in functionality for handling the age old problem of form data being lost once the form is submitted which developers previously addressed through many lines of code.

As websites become web applications and are increasingly more complex this problem becomes more acute in another area that is less thought about. I remember many years ago a client asking if they could receive an email each night of all the orders placed on their site during that day. This would seem like a simple task but in fact highlighted a major problem. How would this email be triggered? Dynamic websites, like their traditional static forebears, are made up of pages which only exist when a user views them. For any code on a website to run it must be part of a page that runs when a user visits it. The idea of some code running without user interaction fell outside of the website framework.

The early answer was to have a script file (such as a page) that was triggered by a timer running on the web server.  Windows Server for example offers scheduled tasks which could be used for this purpose but you needed a hosting company who offered the service (unless you had a dedicated server). Today we can solve the problem in other ways, for example it is very easy to write a windows service using .NET which runs independent of any website and carry out tasks at any time. Again, the necessary level of control over the hosting server is required though.

For many people the dedicated servers that make this type of functionality available are too expensive. This is now changing with the introduction of Virtualisation within the server environment. A lot has been made of VMWare lately and now Microsoft has just released Hyper-V for Windows Server 2008. Virtualisation brings the benefits of a dedicated server environment without the cost of dedicated hardware. This should greatly reduce the cost of dedicated solutions.

The result is that we can start developing outside of the traditional web technologies. We can introduce Windows Services or even desktop apps for handling many administrative tasks. Cleaning up data such as clearing unfinished orders out of a database can now be done away from the website and handled in the background. We can start to use workflow technologies such as Microsoft’s Windows Workflow Foundation (WWF) to create far more enterprise level applications. Finally we can start to leave the days of the stateless web behind and build richer, more efficient and more integrated applications.

Yes, I know for many this is going to be obvious but it is always worth covering. There are two types of website, dynamic and static.

Static websites are the more traditional type in which every page is its own individual and unique file usually with the extension .html or .htm, for example MyPage.html. Dynamic websites contain pages that change according to the individual user or the moment in time. An example is a homepage that says ‘Hello, John’ because it knows that the user’s name is John. If Jane were to visit that page it would say ‘Hello, Jane’, the rest of the page may stay the same but that one part changes based on the user. Dynamic pages are usually more complex whereby the entire contents of the page change.

An ecommerce site such as Amazon will have a product page (a single script file) that shows different data depending on the product being viewed by the user. The benefit is that the website developer only has to create a single page to show any number of products as opposed to a static website where they would have to create an individual page for every product. Just imagine having to build a page for every product on Amazon? Well, there was a time when that was the only option.

As a developer I feel that I should be concerned about errors in any application that I work on. I know it’s tempting to ignore errors if possible as then you don’t need to fix them until someone else spots them. It is also true that the latest development tools and software make it easier to write error free code and therefore perhaps we can become complacent.

Some errors are unavoidable, it you spin up a page and get a 500 screen then you’d better fix it but some errors are more subtle. JavaScript often fits into the latter category. JavaScript can have errors that are not immediately noticeable in terms of the functioning of a page. For example, many ads or widgets use JavaScript that should it not work does not affect the page in a way that a user would not be able to use it (sometimes they may even not notice).

But here’s the thing, errors are bad, they are a sign of something bad, errors mean something is not working. Something needs fixing!

As a developer I want to catch these errors as much as possible and therefore I have my browser set to pop-up error messages rather than hide them as an icon in the corner. By being so disruptive I can not fail to notice when there are problems. Unfortunately this also means that any JS errors on any website I visit also pops-up a dialogue. This happens a lot, and I mean a lot. I’m not going to name and shame here but some major tech sites throw up half a dozen errors on every page at every visit and they never appear to get fixed. Any site that is littered with Ads is usually a primary offender.

We recently started working with Flex and part of the FlexBuilder install is the Flash plug-in for developers which throw ActionScript errors. This has been interesting because now as I surf the web I am getting AS error dialogues to compliment the JS ones.

Using the internet is frequently turned into an extremely irritating experience by the onslaught of error messages. Now, for most uses with the regular Flash plug-in and JS errors turned off this will not be an issue. However, what it does mean is that there is an enormous amount of code out there that is NOT working. Whatever it is supposed to be doing it isn’t and somebody somewhere should be worrying about that, shouldn’t they?

If not for the sake of your applications then for my surfing pleasure please developers turn on your error messages and fix those bugs.

I took part in a recent panel discussion on whether designers should learn to code. I was on the side that the designers should have a good understanding of code and what makes up the web, but refrain from touching the stuff. I have always thought it’s important for a designer not limit they creativity too early on by believing that something isn’t possible or difficult to do. If a designer thinks that he will have to code the layouts himself he maybe reluctant to do anything very creative or out of the box that may cause problems. Sorry to the developers reading this! It’s just got to be your problem

This Computer Arts August 08′ issue is now in the shops.

This post marks the first of what we intend to be a series of short items that hopefully debunk some of the most commonly used terms in application design and development.

Our first post is about Relational Databases (RDBMS) which refers to a type of data store. Most applications will need to store data that they collect such as user profile information. There are different data storage options available; Excel, Microsoft Access and FileMaker are examples of data stores that many people are familiar with. For web applications we need something that can store very large amounts of often complex data that can be accessed by many users at any one time. Today most people choose to use a Relational Database from one of the major vendors such as Oracle, Microsoft, IBM or Sun (MySql). The term relational refers to how data is stored. For example, customers in an ecommerce system may be grouped together in a table called ‘Customers’. Their orders may then be grouped into another table called ‘Orders’. To know which order belongs to which customer the two tables will be joined by a relationship.

Alternatives to Relational Databases are Document-orientated Databases, for example Lotus Notes, or Object Databases that utilise XML.

In a recent post I wrote about the importance of the database in a web application. Continuing the techie theme I thought I should also mention the next most important issue concerning web apps: the architecture.

I still think the database is more important but architecture is certainly a very close second. I remember creating my first websites and everything was in one folder. All of my HTML pages and images sitting together made life simple when coding image paths and links to other pages. As the sites grew however this set-up soon became hard to manage. More pages, more images, and then CSS files and external JavaScript files and the one folder structure soon became a chaotic mess.

Now we have complex web apps consisting of databases, script files, images, Flash movies, audio/video files and more. Storage of all of these files can become expensive so we off load them on to dedicated storage or media networks. Suddenly our apps are distributed across multiple servers and networks and we have to manage scaling issues as storage requirements grow and user numbers soar.

Running today’s web apps with everything in one folder would be insane. So, the physical structure of our app is important if we are to be forward thinking (which we should be). Therefore we need to consider from day 1 how our apps will handle growth both in terms of the amount of data and files and with respect to increasing user numbers.

We have done some work recently with Flickr’s API and it is interesting to note that as part of the data relating to a photo there is information for the server and farm IDs. This indicates how Flickr handle the vast numbers of photos they have and how they are distributed throughout their server network. Imagine if they had tried to build Flickr with all photos stored in one folder, or even on one machine? As soon as that hard drive ran out of space their business would have been over.

One of our clients had a similar challenge relating to storage of files and we addressed this from the beginning. Rather than just storing a file name in the database we made sure we could store other information about the location. This allowed files to be distributed across multiple servers and even across the internet. Had we not considered this initially then we would have had some major changes to make to the file structure, code and database later on which would have been very painful.

Both this and the previous post on databases also highlight one other important consideration when building a web app: planning. Many folks often forgo much planning in order to start coding and get their apps out there as fast as possible. Fast is great but without good planning you could be on the road to nowhere.

The other day I was asked by an industry analyst how our clients are addressing Application Lifecycle Management (ALM). This was a great question for which I had a simple answer: they don’t. Now it would be easy for me to point the finger at our start-up clients but the truth is that no one is talking about ALM. The next question of course was why?

The short answer is that web has always seemed removed from other types of application development. The web is seen as transitory and agile, in Web 2.0 it’s about the iterative approach. If something does not work on a website we will just fix it, no problem. Therefore it is not just ALM that gets ignored but just about any development methodology or even basic questions like “will this scale?” do not get asked. Web just isn’t seen as serious development. Many businesses will throw away their website every few years, so why worry about ALM when the lifecycle is so short.

This is okay if the website in question is so called brochure-ware but today that is rarely the case. More and more business websites are integrating with backend systems to form end-to-end solutions that often go beyond the ‘in the moment’ nature of traditional sites. For example workflow processes are now part of the website lifecycle. Throwing these types of complex systems out every few years is simply not viable from a financial perspective. So surely any company starting down this path should be thinking about ALM at the very least?

With start-ups the picture is more complicated but in many ways more relevant. These are companies with applications at their very core. Unlike a traditional retailer’s website offering ecommerce for a start-up the technology solution is their entire business. The decisions they make on day 1 can have major repercussions on the long term health if not success of their business. Surely under these circumstances methodology and ALM are vital?

The problem is that these companies often lack the resources, time, experience and finances to implement solutions in this way. Start-ups tend to take a far more ‘fly by the seat of your pants’ approach to development. If something does not seem to be working this week we will change it next week and then change something else the week after that. They function on small teams of often dispersed developers sometimes with little long term vested interest who simply deliver in the moment with little regard for the future.

The reasons are understandable as time, money and resource are in short supply and the goal is to get a working application with as many users as possible in as short a time as possible in order to secure funding before their initial budget expires. In this world no one cares much for tomorrow. How many registered users have been added this hour is far more relevant than how the app will hold up 2 years from now?

Unfortunately time catches up and decisions made in the heat of battle often return to haunt such companies. There are some well known names that one could mention. Instead if you look at the companies that have done well such as Google you see examples of people who thought ahead from day 1. They planned well for issues such as scaling and reliability and addressed running costs and TCO from an early stage. Boring stuff in the world of iterative development and mashups but you do not see Google running into the problems Twitter has today.

The truth is that you do not need to be a blue chip tech company to think about many of these issues or even to incorporate ALM into your development workflow. Some of the key considerations I have touched on in other blogs such as good database design and scalable architecture. This adds a little time to the initial build but nothing significant and the benefits can be enormous later on when the user numbers start ramping up.

Addressing ALM could simply mean formalising the processes and roles that you already have in place. I know numerous start-ups who operate modelling, development, testing and deployment workflows. Taking some time to sure these up and apply some form or metrics to manage the movement between stages could reap significant benefits.

Whatever the size or nature of the business it is time for web application development to grow up. We are building more advanced and complex web apps involving numerous technologies and often distributed dependencies that are serving more people. They are becoming more integrated and mission critical and therefore it is time to start applying more traditional development approaches in order to secure a more robust future.

It would be great to hear from any start-ups who have implemented some form of ALM and how you feel that has benefited or not your application and business.

YouTube has been playing around with their UI recently and on the most part seem to be making some well thought through and subtle changes to both the web interface and the video player. With the latest update the rounded corners have been banished and replaced with a more compact and all together squarer interface, which seems to make a lot of sense given that videos can crop up anywhere on the web and square corners are going to make fitting video in different sized interfaces simpler. The UI now also degrades gracefully, dropping lesser used interface elements as the embedded video is reduced in size

However, the most recent changes bring a rather confusing menu system to the bottom right of the video, in which as a user I am required to click a button to open a popup menu with only one button in it, which just seems crazy? Surely just replace the menu button with the one in the menu, they are after all the same size!

There are some really smart people over a google / youtube, so I cant help thinking, is YouTube being stupid or have I missed something rather more fundamental?

I stumbled upon this lovely site today. At first I wasn’t quite sure what I was looking at and It took a good 10 min before I started to suspect that the video wasn’t going to end and I’m ashamed to say a further 10 min were wasted trying to workout if I was stuck in loop or if the events happening where leading somewhere. Normally this would lead to quite a lengthy rant about usability to an unsuspecting colleague. However on this rare occasion, I was more than happy to be subjected to a lengthy UNI QLO advert.

Best viewed on a large screen with the sound turned up and 20 min to waste.

Last week we attended the first Fuel conference in London hosted by Carsonified the folks behind FOWA and FOWD. This was a great event that seemed to reach out beyond the usual Web 2.0 community crowd. We were fortunate to have a short speaking slot thanks to our relationship with Microsoft. We knew that we were going to promote Microsoft but wanted to convey a message that we felt was important. We decided to dedicate our 8 minutes to ‘how to choose a technology’. The talk focused on three areas that we believe are most important in terms of making this decision and then how Microsoft addresses these.

Our three primary objectives when choosing a technology were: reducing time to market, minimising cost and the ability to integrate with other products and services. These were born out of our experience of working with a cross section of clients from start-ups to traditional businesses. The first two cross-over and also raise some other important considerations when looking at different technology options.

Reducing time to market naturally means shorter dev times and therefore leads to reduced cost. At the same time we do not want to sacrifice reliability, scalability and user experience in order to cut the initial dev time. Getting a product to market that has poor user experience or doesn’t work is a waste of time. Minimising cost often leads to a focus on the start-up costs of technology (e.g. how much do the tools and software cost) but I also raised the issue of over time costs such as the price of scaling. The final objective concerning integration may seem to be born of Web 2.0 and the mashup generation but to be honest it has had around a long time with many websites integrating with existing business backend systems for some time.

Talking about technology is always tricky as while it is important it’s never exactly entertaining stuff, despite the introduction of baked beans. I thank those who made complimentary remarks afterwards and to Microsoft for asking us to speak in their slot.

We now look forward to October when Jeremy will be speaking at Future of Web Apps about building desktop apps on your web app.