The mighty sysinternals site is no more .....

Not sure if this is a good thing or not - only time will tell. Sysinternals, provider of excellent utilities such as regmon, filemon and the pstools have been bought by Microsoft.

First of all the good:

• In a short space of time, regmon, filemon and process explorer have been merged into one tool, called Process Monitor 1.0. It is really excellent, and allows you to get a complete view on everything going on in an application, as well as improving on the reporting capabilities of its predecessors. Hopefully there will more improvements to come in this and other tools.
• Support for Vista in both i386 and x64 code.

The bad:

• Source code has been dropped for the reason 'The number of source code downloads didn’t justify the migration, support, and possible integration problems it might cause with other Windows components down the road. They also mention the fact that it has been used in malware, and some of the programs use undocumented APIs'.
• Removal of linux versions of applications such as Filemon (no surprise there!)

A lot of people are getting very upset about the source code issue. To be honest, a lot of the decent tools didn't have the source code online, and I'm not sure how many people really looked at it anyway. It's one of those things it's nice to have, and a lot of people will complain if it is not there, but not many actually look at it.

I think all the negative comments on the technet sysinternals discussion are especially unfair, particularly the 'sold out' comments directed at sysinternals founder Mark Russinovich. Here is a guy that has provided some of the best tools out there free of charge (as well as discovering the Sony BMG rootkit, and the NT4 workstation to server registry hack!), and if after a long time he 'sells out' to Microsoft, then good luck to him. Microsoft are continuing to support and develop the tools, so it should be for the benefit of the community in the long run.

As for the purists out there who like to see source code and check it over by hand to ensure it is safe for the rest of us (do they really exist?), why don't they just band together and come up with a true set of open source sysad tools? I'll host the site if anyone out there is interested!

Good luck Mark, I hope Microsoft are good to you, and thanks for all you have done for the industry.

Moore's law for development

It's been a long time coming, but developers will no longer get to continue with Moore's law 'for free'.

For anyone that doesn't know about Moore's law (named after Gordon Moore, one of Intel's founders), it states that computers will double in performance* per dollar spent roughly every 18 months. This actually equates to a hundred fold improvement every ten years, or around a million times faster every thirty years.

There's an interesting image on Wikipedia which shows that this 'law' has held since 1900 when electromechanical devices somehow managed to achieve an amazing 1 calculation per second for every 10 million dollars spent. Nowdays we have Pentium 4 chips that carry out over 25 Billion operations every second for a few hundred dollars.

Developers have had it pretty easy during the last 100 years or so - chips get faster, and so does their software (although they seem to keep finding ways of slowing it down with new 'features'!). But not any more.

Intel and AMD are struggling to make a single core chip run at higher speeds. There seems to be a limit to how fast you can run a chip, and this limit is caused by the heat generated. So they are looking for other ways to deliver performance.

The biggest trend is now in multi-core chips. Most chips from Intel and AMD now run two core chips - the equivilent of having two processors instead of just one. This trend looks set to continue, with Intel having just released a four core chip, and the Cell Chip (which will be the PS3's CPU) having nine cores in total. Intel are talking about chips with 80 cores or more in future.

The problem at the moment is most developers write programs that only run on a single core, so when you have a program churning away on your 100 core desktop in several years time, it will only have access to 1% of the computers computing resource.

This has got to change, and developers need to start thinking about multi-threaded applications that will take advantage of whatever the computer has to offer. This isn't as easy as flicking a switch - sometimes a multi-threaded application needs to be built from the ground up, as the logic can be quite different.

For (a trivial) example, a program to add a thousand numbers together looks something like this :

for loop=1 to 1000
{
x=x+y
}

with a multi-threaded application, it is more like :

work out how many threads you need (perhaps query the abilities of the computer)
split the job into small chunks (say 10 sums at once)
while there is work to do
{
start lots of threads to add 10 numbers at a time. Each thread would have logic similar to the job above.
}
The first time you go round this loop you will end up with 100 'results'. You then need to go back round again to add these together to get 10 results, and one more time to get the final answer.

As you can see, the second example is an order of magnitude more complex, and that's not even taking into account areas such as thread locking and concurrant access to resources, which make it far more complex still. I'd urge anyone writing real programs for a living to start at least playing with multi-threaded apps, before you get left behind.

* Moore's law actually refers to transistor count, but most people use it as a mark of performance.

Free Tools for IT Departments

It's amazing how many free tools there are available on the Web these days, that are as good as, or often better than their commercial rivals. Some of the most useful ones I've found to date are as follows :

Development

Not everyone needs the full version of Visual Studio professional, particularly if you are a sole developer or part of a small team. Microsoft make a Visual Studio Express version, which has everything in Visual Studio Professional minus a few enterprise features. Available languages are vb.net, c#, c++ and j# as well as a web development edition.

If you dont like the Microsoft way of the world, or fancy developing for multiple platforms, have a look at Eclipse, which is a free development environment which has been created by giants such as Borland, IBM and Red Hat to challenge Microsofts dominance. It's completely free, and supports a huge list of languages including Java, C, C++ and PHP.

Finally, for hobbyists there is the excellent phrogram, which is one of the quickest languages to develop in on the market, as well as being great for beginners to learn. Soon to be updated and rebranded as phrogram.

Database tools

Microsoft now make a free edition of it's popular database available, called SQL Server Express edition, and it's not badly restricted unlike previous freebies. It is limited to 1Gb of RAM and a 4Gb database size, which is plenty for small developers. Reporting Services, which is an excellent report generator is included.

MySQL has made great progress over the last few years, and is even producing the database for many SAP implementations, so is becoming pretty mainstream. With good SQL support, and being totally free its as good as it gets!

Troubleshooting

Got programs that wont work, and you cant figure out what the problem is? Turn to the excellent sysinternals site and download regmon and filemon. When you run them, they record everything that is happening in the registry and filesystem in real time, which is really useful to find out what is really happening.

Other great tools on sysinternals include PSTOOLS, which allow you to do all sorts of cool things like shut computers down remotely, or manage individual processes.

Graphics Tools

The GIMP is a free graphical tool that runs on Windows and Linux that is a bit like Photoshop, but totally free. Also worth looking at in the Windows world is Paint.NET which is a powerful image editor based on the dotnet framework being developed by Washington State University. Finally for simple image viewing and simple photo touchups (like redeye removal), IrfanView cannot be beaten.

Security

Last, but certainly not least, there are a huge number of security tools out there. For keeping multiple Windows computers up to date with the latest patches I recommend looking at WSUS from Microsoft, which is a central management console for patching.

Nessus is a security scanner, often used by hackers. Run it yourself to check your own network for vunrabilities, and I guarentee you will get some surprises. Now available for Windows, as well as the original Linux version.

For intrusion detection, take a look at Snort and probably run Base as well to report on the Snort logs. This keeps a record of suspicious activity on your network which you can use to track down intruders. Also available on Windows and Linux.

These are just a few of my favorites, there are many 1000's more out there. Let me know if you have any favorites you think should be on the list.

May the force be with you

Slightly off topic, but the US Government's Star Wars defense programme makes Microsoft's lack of progress in the last 5 years look pretty good. I came across an interesting article about the latest development in the programme.

Now, call me old fashioned, but $100 billion and 23 years work is quite a lot of time and money to spend on something that has only worked once in testing, and they are not sure if it will work in practice!

If you are wondering what happened to all that money, and where the 'space laser defence' system went to, have a look at the Wikipedia Strategic Defense Initiative article.


Do or do not... there is no try

Geospatial Information Systems

I was recently involved in a project where the contractor was to fly over an entire region taking aerial photos and then stitch them together, taking contours into effect, in a process known as ortho-rectified photography. Due to adverse weather conditions they were unable to complete the work during the summer, so suggested finishing the work off in the year following.

This concerned us on a number of fronts, and we have asked for a complete re-fly. Somewhere in Canada however, someone made a different call, and we ended up with something quite bizarre.



I'm presuming that the Northwest section of the photo is the more expensive part of town, and few people want to live in the cold 'South Side'. There's even a different climate on the East side of the city, so perhaps something for everyone in Edmonton!

For those of you running Google Earth, you can have a look yourself by downloading the location from here.

If you are not yet using Google Earth, I'd strongly advise you to check it out. Its basically an online globe, containing aerial photography of most of the planet, together with a community inputting to local content, so is a fascinating toy to play with. Best of all, its completely free. Get it here

Thanks to the excellent online publication The Register for pointing me in Edmunton's direction.

Is it getting hot in here?

Heat is probably the number one problem facing IT departments these days. Modern chips run at 3 Gigahertz or above, which means that electrical signals are being sent 3 billion times per second. That's a lot of signals, and although they are all tiny, when you put them together they generate a lot of heat.

In the old days, PC's had a heatsink (essentially a small radiator) on the CPU and a fan at the back to extract the hot air. These days, there are high speed fans on the CPU, Graphics Card, system case and even some high end RAM chips. For a home PC that adds up to a lot of noise, but imagine it in a room full of high end servers.

In a standard 42U cabinet (which is around the size of a single bedroom wardrobe) you can fit 250 blade servers, each with 2-4 high end processors, several hard drives and tens of gigabytes of RAM each. Open up a blade, and it looks like it has been designed by an aerodynamic engineer, with arrays of fans drawing the air through the entire server and radiator fans lined up in parallel with the airflow from all the major components. Fire up one of these beasts and its like a plane taking off - imagine the noise of a full rack, or an entire datacentre.


A blade enclosure with 10 nodes, each supporting 2 hard drives and 2 cpus.

I spent some time this week in a server room where we needed to turn the air conditioning off for a couple of hours. We monitored the temperature and shut down more and more servers as the heat kept rising. In the end, we had over 80% of our servers off, and were only just keeping the heat under control.

Many server rooms have no failsafe systems, and no monitoring of the temperature other than a thermometer in the room itself. If the air conditioning fails at night or over the weekend, then your servers are going to be toast when you come in on Monday morning. If you are lucky then they will keep running for a couple of weeks before they start failing. If you are unlucky then you've just wiped out your entire network.

If you havn't done so already, install environmental alerts that will let you know if the heat rises (as well as monitor humidity / smoke etc). Link them up to a pager or SMS gateway so you get alerted. Also have a system for rapidly shutting systems down when the alerts are reached. Keep your phone nearby, so you can respond rapidly if you need to. Or even better, link the environmental monitor to the shutdown system and the pager. A lot of modern servers have built in thermal shutdown features - just make sure you enable them, as they are often not enabled by default. Do all that, and you might just sleep that little bit better.

Server watch has an interesting article about environmental monitoring and Minicom make a remote power switch with environmental controls.

For fast manual shutdowns, you can download PsTools from the excellent Sysinternals site, and best of all it's completely free. Use PSSHUTDOWN in a batch file to shut down a whole bunch of servers at once from a remote location.

A change is not always for the best

Continuing from my last post, all this lack of change has not necessarily been a bad thing. 75% of the Internet browsing population are using XP, and in business XP is pretty much standard across the worlds corporate desktops.

In terms of standardisation, training, ease of support etc, this has been a great thing compared to the turmoil in previous years which saw IT departments supporting a mix of 95 / 98 / NT4 and 2000. Every machine sold in the last 4 years is easily capable of running basic Windows XP at a good speed (although games are another story), with plenty of room to store documents, music and photos.

However, times are changing in this space as well. The rapid rise of Linux as a viable desktop operating system, combined with decent free office software (such as openoffice) have closed the gap significantly with Microsoft in the last 5 years. The new Apple operating systems, which are also Unix based, are ahead of Windows in many respects and the upcoming Vista Operating System from Microsoft is only just starting to bridge the gap.

Corporate customers, who quite often pay a yearly upgrade fee to Microsoft (called Software Assurance) are asking themselves why they keep paying, when Microsoft has hardly released any upgrades in the last 5 years. Microsoft is now playing catch up, and has a massive number of products on the horizon over the next year or so, including new versions of Windows Desktop, Office, Windows Server, Exchange, Sharepoint, dotnet framework, Live Communications server, SMS, MOM ... the list goes on and on. They are also shortening the upgrade timescale to two years between new versions to encourage corporate customers that change is worthwhile.

So we are moving from a long period of stability to a period of rapid change. On the one hand, we have stable, reliable, uniform software. On the other, vast spend in research and development on 'improvements'. Which is going to be better for business, only time can tell.

Happy Birthday Internet Explorer 6

A milestone in Internet history passed last weekend without so much as a mention from the IT press. In August 2001, Internet Explorer 6 was released, alongside Windows XP, and this weekend they both celebrated their 5th birthday.

Now, in computing terms, 5 years is a long time. For example, if we look back to the previous 5 years from 1996 - 2001 the amount of change is incredible.

On the operating system front, we see the move from Windows 95 - Windows NT4, Windows 98, Windows ME, Windows 2000 and ending up with where we are today at Windows XP. All in all, 6 possible changes to the desktop in the previous 5 years. In the last five years, no change.

The slowdown on the browser front has been just as marked. August 1996 marked the release of IE3, taking over from IE2 (which was a year old at the time) along with Netscape 2.01. Between 1996 and 2001 we had 8 major releases of Netscape and 4 major releases of IE, which eventually won the 'browser wars'. Between 2001 and 2006, IE6 has had 2 bug fix updates to improve security, but no new features have been added.

If you want to see what web browsers looked like in 1996, download NCSA Mosaic (on which IE was based) here

The difference between browsers in 1996 and 2001 is staggering. The difference between 2001 and 2006 is not. There is no doubt that the difference between the two periods is the level of competition. For most of the 'naughties' IE has had nothing to compete with, so has stagnated. Those in the industry also suspect an ulterior motive. Every improvement to the browser technology gives companies like Google a bigger stick to hit Microsoft with. Microsoft is not winning the battle for the Internet itself - why create tools to give its competitors an edge?

However, there is a light on the horizon. After years of teething problems from its separation from Netscape, Firefox (originally named Firebird, to indicate the phoenix rising from the ashes) is gaining momentum, and now has 27% share of the browser market. Amongst those in the know, Firefox is the browser of choice, being faster, smaller, supporting web standards better, supporting more features (tabbed browsing for example), being more extensible and generally better in most respects than IE6. This has led to Firefox grabbing nearly 30% of the browser market, as can be seen in the following graph.



Microsoft are working hard on IE7 to stop FF gaining too much market share, and this will be released as a 'critical update', meaning it will automatically spread to a high percentage of the PC population, which some regard as a slightly underhand move.

What happens in the next 5 years is up to us all. With no competition, we can look forward to a few security updates to IE7, and perhaps not much more. However, if we all keep an open mind, and consider browsers such as Firefox, Opera and Safari as realistic (and often better) alternatives to Internet Explorer, then perhaps the browser wars II will develop and lead to another massive shift in how we use the Internet.

Your browser needs you.


Firefox - Opera - Safari