At work it’s been a bad week for our hosting partner. We found out on Monday that they had been serving Webalizer on our server without telling us for a whole year. So that’s one year of confidential marketing reports freely available to anyone who stumbles on the URL (which was not hard to guess). After ignoring my first request (marked urgent) they replied with a snotty email telling me it stats are installed by default, but these days they don’t do that. Of course, I was overjoyed to hear that their new customers don’t suffer this embarrassing lapse in security.

No apology.

On Tuesday I chased a new account request with them that wasn’t done from the previous day. An equally unapologetic response.

On Wednesday we realised that after repeated requests to have somebody who has moved to another team in my company to be removed as a contact, it still wasn’t done. They told us to go through every individual domain registered with them and to change the contact name listed. We responded with a firm ‘hell no,’ as we’re the paying customer and will not spend hours going through our domains because of their inability to create an efficient management tool. They responded back with what was effectively flipping us the bird, saying they’re not willing to make the changes.

Today they told me that the Senior Developer at Host wasn’t an authorised contact onour account. I sent them email proof that I requested Werner has the same authorisation as I do back in January. Their response was to tell me that we’ve got two accounts, and Werner’s only authorised on one. No apology.

I understand how it can be hard to say sorry. When something goes wrong, putting a ’sorry’ in an email takes guts. Although when you don’t do it, it makes you look cowardly and a little pathetic. We don’t doubt they’re good at what they do, and an apology wouldn’t change that view but actually help us in that we’d know they understood something went wrong and will make sure it doesn’t happen again.

‘Sorry’ for the rant. See?

Just as a side note, I am aware that browsers released in the future will continue to support HTML 4 and other previous versions, so I’m not trying to encourage any kind of panic. I’m simply imagining using HTML 5 (based on the linked article, above) for the things my team and I do every day.

Tracking
There’s a new attribute for the a and area elements called ‘ping’. This is used for tracking, so when a link is clicked the URI is pinged at the same time. This is designed as a simpler alternative than the current method that involves sending the user to a page that tracks the click and redirects to the content page. This also gives the user the ability to switch tracking off. I know that absolutely none of my clients would appreciate this as a feature (why would they opt for profiling a fraction of their audience opposed to all of it?). Another aspect of this would mean that users would be made more aware of website user tracking. This is not necessarily a bad thing, but I could imagine plenty of confusion for a lot of users in determining the difference between regular audience trackers and snooping on an individual. How would you explain tracking to your aunt without her feeling as if her privacy was being invaded?

Forms
HTML forms are getting some fresh air. New input types support inserting time-related data, URLs and email addresses. Added to that is the attribute ‘required’ that’s been thrown into the mix. This is a step in the right direction given the number of bad JavaScript form validators in circulation, but the helpfulness of such an addition is limited to how browsers implement the changes.

If I were to create a simple form asking a user for a name, email and serial number I may run into some problems. For the ‘name’ field I would just add the required attribute on a normal text field as a name is going to be text: there’s nothing else to validate. The email field would use the new input type of ‘email’, and also the required attribute to ensure the data was indeed an email address. But what of the ’serialNumber’ field? There is no input type for my serial number, so in order to properly validate the data as being a valid serial number I’d have to use a regular expression within a JavaScript function to check that the field value follows the format I expect. I’d be mixing native browser data validation with supporting JavaScript functionality, which I think is potentially very messy!

I think the addition of the time and date input types are really useful. It means users can select dates from a familiar interface. I think the email and URL field types are an attempt to bring data validation to the browser, and given the complexity and the range of data that often goes into forms I think that this could prove difficult.

The good bits
There are some really nice additions to this version of HTML. Making the embed tag a w3c recommendation will mean easier Flash implementation in the future.

Header and footer as elements will possibly confuse new web developers with both ‘head’ and ‘header’ being used in the same document but for different uses. They will probably be more useful for SEO reasons, as I don’t see why developers would stop using div tags with a ‘header’ or ‘footer’ id attribute.

The nav tag will be great for improving a web page’s accessibility offering, since it enables the developer to indicate where the site navigation sits.

I’m interested in the canvas element, and what that will mean. It is meant to provide a space for rendering media on the fly and spawning on-page games and graphs.

There aren’t many new features that I saw that are geared-up towards AJAX fans, however the ‘event-source’ and ‘output’ elements might prove to be useful.

Back with forms, there is now a ‘form’ attribute to be used within fields that means a single form field can belong to more than one form.

No more frames!
Framesets and frame elements have been shown the door in this version of HTML. The Iframe element still exists, but the w3c found that the standard frame elements were decremental to accessibility so therefore scrapped them. I haven’t seen frames used in quite a while, but I do remember them being an important part of my first few websites in the 90s. I’ve also seen framesets used by SEO companies (the SEO content would go within the ‘noframes’ tag). I think scrapping frames is a good move, but it could really help accessibility.

Problems with HTML emails and CMS content
One thing my agency do quite a lot is build HTML emails for subscribers to our clients’ brands. With the end of the width and height, background, and most importantly the style attributes it’s going to be hard to build HTML emails for applications that have poor HTML parsers. This includes Hotmail, Gmail and Yahoo! online email readers that all parse HTML email content on the server before it’s displayed in the browser to prevent the content impacting the application interface. I agree that in principle, all style information should be kept out of the HTML template – this is the job of CSS. Although building emails is a big part of my team’s responsibilities and the abandonment of the inline style information is going to either force the email applications to be more compliant or is going to make life a living hell for the guy in my office who builds these emails!

Also I’m writing this in Wordpress: one of many CMS’s available for free or commercial license and one of millions when counting the bespoke CMS solutions created all the time. To allow authors using these CMS’s to control how their information is displayed (such as alignment, colour etc.) the style attribute would have to be used. I cannot see how a CMS could be efficient in managing style information as a separate entity to the content being written given that the style attribute is to be pulled as an HTML standard.

My thoughts
I think the w3c have been careful in analysing popular use of HTML and how they can improve it. Although I think the developments of HTML in this version may be trying to become too involved directly with the user that might create a lot of poorly implemented web pages. I also think that dynamic content created with CMS applications and such have not been considered properly.

I definitely agree with attempts to improve accessibility, and the inclusion of better data descriptors will no doubt help that effort. However, I doubt that the ping attribute will prove popular with developers unless users come to expect to be able to choose whether they’re tracked or not; I question the need for browser-based form data validation; I’m also unsure about the decision to rely more heavily on CSS over inline style information.

Now I’m in an agency where I’m the only Web Developer, I’m on the interview panel when it comes to finding other developers. This process has meant I’ve realised how hard it is to find people suitable for the job. There seems to be a mix of designers wanting to try their hand at developing and semi-skilled developers trying to go for a senior position before their time. The creative industry doesn’t seem to attract the talented developers it requires.

I’ve met a few other developers who are skilled but wouldn’t like working in a creative agency. They’re application developers, not web developers. When I asked, they said they actually make an effort to stay clear of jobs like mine. I was not going to be able to convince them to send me their CV. Finding Web Developers for our agency isn’t going to be so easy.

There are characteristics I think developers in creative agencies need to be born with or quickly acquire in order to survive:

1. Social skills. Geeks notoriously don’t have the best people skills, and when it comes to working in a creative agency where sitting in the corner keeping to yourself isn’t going to work, you’re going to have to face up to the fact there’s no getting out of going to lunch with the chatty project manager who thinks as a web developer you should be able to fix his X-Box. I’ve also had to feign an interest in sports a number of times, just to keep up.
2. Ability to work with computer illiterates. Being able to resist self-harm when I’m politely asked to download files and burn them onto a DVD as that is, apparently, what I do for a living. Oh, and I can also offer lots of advice about PowerPoint transitions.
3. Put a project down, picking it up again; repeat. This is more about developing in a way that allows the project to be continued after being put on hold for x amount of time. For example, I’d stop working on Project A because a more urgent Project B needs kicking off, then a number of weeks later I’d be asked to carry on with Project A from where I left off for a couple of days until Project C gets signed off. And no, I can’t get more time for Project A as it’s already over-budget and we don’t want the client back in for another meeting about timings as he eats all the best cookies and hits on the receptionist.
4. Explain yourself in small short words. Bearing in mind that some people in the office still think that a hyperlink is a new type of Japanese train, they’re not going to understand DNS propegation is the reason we can’t launch the website right now, after lunch on a Friday afternoon. Explaining technical concepts to non-technical people take a long patience and short words.
5. Fight for testing. Or test post-launch. Testing is something expected by everyone to be in the project plan, it even makes it into the Gantt charts the producer writes up, but it will be the first thing to go if the deadline looms a little too close for comfort. Or there may be a testing phase, but this is soon followed by changes to the creative that nullifies all testing done so far. Or there will be a testing phase, but only after the site has launched. I can’t point the finger of blame at anyone in particular, as it’s the way things work when you’ve got a small project with a tough deadline. The client asks for changes but the deadline won’t move. Proper testing seems to be a luxury reserved for those clients lacking extensive change requests and a clear understanding of what ’signing off’ really means.
6. See the design as the designer sees it. I’m anal about my work, so quite rightfully the designer will be anal about hers. A bad designer will never say when I’ve messed up a design because she’s scared of me, a good designer will tell me about a mistake I made, a great designer will tell me that I messed up and then tell me why it needs to be fixed. If I can understand the thinking behind the design, I can build it better.

I don’t think that working for a creative agency makes me a better developer, or a worse one. I do think it means I have to approach projects differently. I think working for an agency means you So there are quite specific skills that I think are needed to work in a creative agency as a developer. A lot of these skills are a little different from the stereotypical developer skillset. However, a lot of very talented developers do prefer the creative environment (I just need to find them). I like the small and exciting projects I do in a creative agency. I don’t have the attention span to stay put on one application for months on end. I’d rather download files and burn them obediently to a DVD for the TV producer who doesn’t know what a codec is, whilst performing self-harm here at my desk with a soggy cocktail stick.

Things I’ll now see as hints I shouldn’t hire a developer:

1. Boasting that he can touch-type as one of the leading items in his skillset
2. Claiming his previous employer ‘didn’t understand him’ and spends 10 minutes ranting about them during the interview
3. He has a law degree and claims that he’ll rip any contract apart
4. He can’t talk plain English to plain English speaking people. Types on an imaginary keyboard as he speaks.
5. He’s 42 and lives in the mountains with his mother. Probably
6. Says something ridiculous and possibly inappropriate in a meeting, then claims it was a ‘brain fart’.
7. Tells my junior behind my back that he’ll learn nothing from me. Tells him that indenting code is a waste of time.
8. Keeps saying ‘crunch crunch crunch’ really loudly when explaining how his code parses data
9. Seems to think a brief is below him, but doesn’t seem to have fully understood it 4 days later
10. Won’t give a time estimate after reading the brief; technical spec; and after several meetings. Has to wait until he’s spent several days working on a script for which you have no use, and then finally saying that he’ll need 8 weeks for a 3 week project without offering any formal breakdown of timings.

This is someone who claims to have 10 years of experience, which is twice what I have. Of course, I’m taking responsibility for hiring the muppet and I thought that with 10 years under his rather long belt he’d find this project easy. Lesson learnt. Next time I’m definitely checking references, asking for sample code and disregarding the claims of grandiose experience and perfect development skills.

If anybody knows of a good PHP developer in Sydney with agency experience please get in touch.

My problem is that on my Facebook (this idea could apply to other sites but in this post I’m just going to use Facebook) I have lots of people listed as friends. I have a few close friends, then I have friends who I know but who aren’t close – workmates, friends of friends and so on. Then the bulk is made up of the people who I haven’t seen in years, but for some reason are still my friend. The result is too much information for me to manage that isn’t effectively organised. Sure, Facebook lets me define who I want to hear more about and who I want to hear less about, but this is a manual control and I think it should be able to work this out by itself.

How can a system like Facebook work out which is more important to me: wedding photos from someone I went to school with; knowing that my flatmate ‘is bored at work’; or photos of my workmate’s new dog?

Step 1: How much do I care about the person?
Facebook could work this out from the amount of correspondence between my friends and myself on the site. Then there’s the number of photos we appear in together and the groups or networks we’re both members of. Although granted, my close friends and I don’t speak exclusively through Facebook, so some manual intervention may be necessary. But for the most-part my degree of ‘closeness’ with the friends I have on Facebook could be calculated.

Step 2: How important is the news to others?
First of all the relative importance of the news for the author could be calculated by how often the author posts news. By taking the ‘cry wolf’ concept; if someone posts news every day it is possibly less important than someone else who has posted an item for the first time in three months.

Step 3: How popular is the news to other people who know this person as well as I do?
If someone else has clicked on a news item for a mutual friend we both went to school with, it’s possibly also interesting to me. If I post an item that only my close friends have shown an interest in, it probably won’t be of interest to someone I worked with years ago and don’t speak to any longer.

Existing Features
Facebook already allows you to select the kind of news you want to see more of. So for example, if you happen to like seeing new photos from your friends, it will offer these up. If you don’t like seeing people’s status changes you can see less of them. You can also select the people you’re more interested in, as well the people you’re not interested in – although this is done manually: Facebook can’t determine how ‘interested’ you are in each of your friends by your dealings with them through the site.

Opening It Up
Of course, a lot of people are on more than one social network. A decent aggregator would sit outside of Facebook and MySpace, watching and analysing everyone’s reactions to news.

Potentially, the system could even take on StumbleUpon to monitor more than social network activity. If several people you know visit a certain website, or a news story or video, you could be alerted to the same content based on how close you are to the friends who also visited that page.

It’s a matter of managing mass-information. As more people become proactive in creating web content, there should be a way of managing the publicising of this content to the right people. Monitoring social interactions does wreak of an Owellian telescreen; but there should be a way of creating a system that would benefit its users rather than its administrators.

I tried to find some code I could adapt for this function, but came up short. So I wrote my own. I’m putting it here in the hope that someone else might find it useful. Sorry it’s not so elegant, but I hope the method is easy enough to follow:

/**
* This function accepts a truecolor image object and analyzes the colors.
* It works by dividing the image into x*y blocks and identifying the dominant color in each
* block. It returns these colors in an array in the format array[xPos][yPos]
* @author Ben Hindmarch
* @copyright 2007
* @version 0.2 (2007-06-26)
* @abstract Grabs the main colors from an image
* @param binary $im The image object
* @param integer $xCount The number of blocks along the x-axis (default 3)
* @param integer $yCount The number of blocks along the y-axis (default 3)
* @return array containing x and y position of color, individual decimal red, green and blue
* values as well as a hex value
*/
function analyzeImageColors($im, $xCount =3, $yCount =3) {
//get dimensions for image
$imWidth =imagesx($im);
$imHeight =imagesy($im);
//find out the dimensions of the blocks we're going to make
$blockWidth =round($imWidth/$xCount);
$blockHeight =round($imHeight/$yCount);
//now get the image colors...
for($x =0; $x<$xCount; $x++) { //cycle through the x-axis
for ($y =0; $y<$yCount; $y++) { //cycle through the y-axis
//this is the start x and y points to make the block from
$blockStartX =($x*$blockWidth);
$blockStartY =($y*$blockHeight);
//create the image we'll use for the block
$block =imagecreatetruecolor(1, 1);
//We'll put the section of the image we want to get a color for into the block
imagecopyresampled($block, $im, 0, 0, $blockStartX, $blockStartY, 1, 1, $blockWidth, $blockHeight );
//the palette is where I'll get my color from for this block
imagetruecolortopalette($block, true, 1);
//I create a variable called eyeDropper to get the color information
$eyeDropper =imagecolorat($block, 1, 1);
$palette =imagecolorsforindex($block, $eyeDropper);
$colorArray[$x][$y]['r'] =$palette['red'];
$colorArray[$x][$y]['g'] =$palette['green'];
$colorArray[$x][$y]['b'] =$palette['blue'];
//get the rgb value too
$hex =sprintf("%02X%02X%02X", $colorArray[$x][$y]['r'], $colorArray[$x][$y]['g'], $colorArray[$x][$y]['b']);
$colorArray[$x][$y]['rgbHex'] =$hex;
//destroy the block
imagedestroy($block);
}
}
//destroy the source image
imagedestroy($im);
return $colorArray;
}
?>
The function works by marking out several (3×3 as default) blocks for the image; shrinking each block down to 1×1 pixels; converts the pixel image to a colour value and finally adds the value to an array which is mapped out according to the x and y position of the block before being returned.

Examples of the code working below. The image on the left was created using CSS to display the colours in the relative location they were found on the image:

Overview
What if someone wanted to log into your website using someone else’s account? What if they managed to get a copy of your database?

If you have any information stored on a site that contains powerful functionality or information or that is valuable (or sensitive) to you or someone else, it’s worth making an extra effort to stop anyone else getting access. The first step is to make sure your users don’t use ridiculous passwords (because they probably will). How you can do that is up to you and isn’t within the scope of this article.

After the usual form submission and validation you’ll be at the point where you need to store that username and password in the database. You could do something like this:

"INSERT INTO users (username, passwd) VALUES ('$username', '$password')"

That would enter everything as-is and not help you at all if someone could see the contents of the ‘users’ table.

MD5 The Password
One of the first things I learnt about attempting to store information was the md5 (Message-Digest algorithm 5). This is a mathematical formula I don’t understand and never really want to. All I know is that it encrypts text strings with a 128-bit hash value and that’s good enough for me. On the sample table below, you’ll see how passwords are stored using the md5() function.

To log in as user I would take the password they enter, pass that password through the md5 function and use the result to check for matches in the database.

The weakness here is that if someone really wanted in on your system they could use a word list of passwords against the table to crack the password. There’s actually online services (called rainbow tables) that offer to crack an md5 encrypted password in no time at all. These services will usually check the md5 hash against their databases of words and their md5 equivalents to find the password.

That’s a scary thought for anyone who felt comfortable with their md5 hashed passwords. There’s a method called salting that makes decryption far harder:

Salting
This is a really simple idea. You prefix passwords with a string constant made up of random characters (that you keep secret). So you’d set the constant in your php code like below:
define('mySalt', 'dvxcbTgh1245X');

You should also consider using extended characters to make your salt even more indecipherable.

And the passwords stored are prefixed with the constant:

$saltedPassword =md5(mySalt . $unencryptedPassword);

This will mean that the services to look-up passwords are going to have a much harder time, as they won’t recognise the md5 hashes they’re given:

So we’ve done pretty well, although there’s a big hole in the plan. If our hacker was really determined, he’d notice that the passwords for Jo and Pam are the same. In practise, this happens a lot – especially on systems that are used by people working for the same organisation. The chances are that they’ll both have used their company name, street name, city or post code as a password. Finding two passwords that are the same will be a hint to me that the password will be a word common to both, so potentially guessable for any intrepid hacker.

Using the username
In order to make the identical plain-text passwords distinct from one-another when they’re encrypted we could consider making use of the username when storing the password.

We won’t use the username as the salt, as this could be a step backwards for us (think of someone using ‘green’ as a username and ‘fields’ as a password, which is probably not too hard to crack). So we’ll just add the username to the method:

$saltedPasswordWithUsername =md5(mySalt. $usernameLowerCase . $password);

I’ve converted the username to lowercase, as I don’t want my username to be case-sensitive, and different casing would affect the final md5 hash.

You can see that Jo and Pam’s password hashes are now different, so there is no way to tell that unencrypted they are the same password.

Further Work
If you encrypt the passwords, you won’t be able to send password reminders to your users for those occasions when they’re forgotten. Instead you’ll need to write in functionality to reset passwords. Advocates of storing passwords as plain-text usually use the ’send me a password reminder’ functionality as their main defence in using this strategy. Sure, password reminders are easier to develop and the user probably prefers them to password resets, but having to apologise to your users for allowing a malicious hacker to gain access to their personal login information is a post I’d never want to have to write.

Possible Additions
You could possibly salt and md5 the username too, so to take away the one piece of information the hacker would use to get started. As mentioned above, it would be worth converting usernames to lowercase before they’re put through the md5 function as users aren’t used to case-sensitive usernames.

Because I’m a web developer, a lot of the time I get to these pages which offer tips about what to do and what not to do when it comes to web development and design.

It’s annoying, because a lot of the time the tips are written by an arrogant upstart with a couple of year’s experience under his/her belt. They’re written with such authority the tips are given more gravitas than they deserve.

After Stumbling, I read a tip by a Danish guy (who isn’t Jakob Neilson) that suggested users are bored of drop-down menus and users should never be given the option to view a site in Flash or HTML. If you offer a site in Flash, you really should offer an alternative HTML version too.

In a young industry, when do you become an authority on web design and development? It sure as hell isn’t when you install Wordpress on your webspace and start writing. Should you ever say to others what they should never and shouldn’t ever do?

I’ve had enough of these tips. Every time I read through these lists it makes me realise the value in going to the bookshop and buying books written by people with credentials that make their advice worthy of being in print. You shouldn’t learn everything about the Internet on the Internet, as quite often the instructions they give aren’t worth the HTML they’re written in.

To be honest I don’t have much user experience with RSS. I’ve created RSS XML pages that are dynamically created through a CMS and I’ve got the Happiness and Cyanide comic feeding through to my iGoogle page. So it was also an education for me too. I don’t know much about FeedBurner or Aggregators. I admit, I’m a bit behind.

Ben explained everything very well, and actually got me on-side: I went into the talk with the idea that email is the standard way a website reaches out to its audience, and will continue to be the standard for a long time. Back in the UK I received my Amazon newsletter that was completely customised to the things Amazon told me I needed to buy because I’d like them. In a nutshell, I felt that you can customise an email right down to the individual preferences of the recipient, and then push it out to your users. What I found out is that as a user, once you’re familiar with RSS you educate yourself to control the information you read.

I’m a developer, so I’ve had to send out newsletters before to thousands of subscribers using various tools (at my previous agency we used de-mailer, a mass mailing application built by my then-colleague Matt Knight). Sending emails is far cheaper than sending out pamphlets and catalogues through the letterbox, but it still costs something. RSS feeds can be created and subscribed to at absolutely zero cost.

Emails highlight a point in time when the information contained within them is relevant. You can’t update the words in an email after it’s been sent, and if there’s something incorrect or something missing from the original email you have to weigh up the pros and cons of inundating your subscribers’ inboxes with a follow-up email, that might tempt them to unsubscribe from your newsletter mail-outs entirely. You can correct or add to an RSS feed at any point, and all your subscribers will see the updates.

There are drawbacks to RSS. Well, two of them that stand out to me.

First of all the penetration of the technology is still mostly geek-central. RSS features exist on eBay and the BBC news website, but that doesn’t mean my mum knows what RSS is. My mum uses email, MSN Messenger and will book her holidays online, but hasn’t heard of Really Simple Syndication, and the words ‘really simple syndication’ don’t exactly make it any clearer. IE7 and of course FireFox support RSS (although IE7 doesn’t support secure RSS – so can’t deliver private content, which means things like having an RSS feed of your new Gmail emails isn’t possible). The support of mainstream software is a step in the right direction, but doesn’t mean the general Internet population is aware of it.

Secondly you need to have content to support the subscription. An email newsletter might be sent out once a quarter, if you didn’t update your RSS feed for 3 straight months you’d have people unsubscribing after seeing the same content on their screens for so long. The advantage email has is that subscribers to a quarterly newsletter will see their email when it’s relevant, and then it will sink to the bottom of their inboxes, forgotten until the next newsletter. If your website can’t support regular content updates, there would be very little point in providing an RSS feed.

In the end, the user has a system where all the information of interest to her is available in one place. There are no spam messages popping up, and in the same vein your messages don’t get sent to the junk folder. Someone might subscribe to 60 or so websites and use their RSS reader to check the updates from all of them. What then happens is interesting. Instead of being overwhelmed by the amount of information available, it’s skimmed, in much the same way as a newspaper is read: you read the headline and possibly the first few words and from that decide if you want to read more.

The user has the power to review content before it’s read. This means the user is driven straight to the content she wants to see: targeted traffic for your site, and a more efficient user experience for your subscribers.

Going back to my opinion before Ben’s talk, if I were in the UK I would still have my Amazon newsletter because it’s a brand I like and a service that’s useful to me. Although if I were to consider the newsletters from Threadless, eBuyer, Busted Tees and eBay all as important as each other; the weight of the information from each wouldn’t get my attention as well as it would if it were formatted in an easy-to-read list such as RSS.

Of course, there is the argument that your website isn’t suitable for an RSS feed since your content isn’t updated frequently enough. Although to counter, one might say that if you don’t update your content you don’t really have much of a website anyway.