The Great Untruths

There is a difference between a lie and an untruth. Neither is true but a lie is intentional. So, when I say that something is an untruth instead of a lie I am making a value judgment. The assumption is that someone did not tell the truth but didn’t know that it was not true. Instead, they were ignorant, got too excited/upset/scared or are parroting back what might be another untruth or lie.

A lot of products fail (and some fail dramatically) because of untruths. If an untruth slips in during a crucial phase of development a very bad decision will be made and bad product will emerge.

Here’s some of my favorites.

Everyone Wants This!

You hear this from sales and marketing people a lot. Someone comes back from a trade show or customer visits and hears about a feature that “everyone” wants. There are a couple of untruths rolled into one here.

First off, who is everyone? I can’t tell you how many times it turns out that “everyone” is one person. After hearing it one time, the sales and marketing team feels compelled to ask others about it. They usually get a “Sure. That sounds good” type response. It’s really a sample size of one with all other responses coming from a poisoned well.

More importantly, people often want a great many things that they aren’t willing to pay for. There is a difference between wanting something and needing it enough to turn over money out of a limited budget. This is why there are a lot more Ford Focuses on the road than Lamborghinis. Everyone wants a Lamborghinis, just not enough to pay for it.

Finally, customers don’t often know what they want or even need. Instead, they have problems that need solving. Asking a customer “Do you need this feature?” will get you to the untruth. Asking “What do you need to accomplish your goals” will get you to what they really need to do and might be willing to spend money on.

We Can Do That Too (Version 1)

Someone (Sales, Marketing, a customer perhaps) comes to the development team with a request in the middle of a project. This is commonly called scope creep. We’ve all seen it because it happens all the time. Sometimes, it’s no big deal. Other times it kills products. You can’t just add things without ramifications.

Scope creep kills products in two ways. One, it loads products up with a ton of features that make it bloated. In the end, by trying to please everyone you please no one. You also trash your margins.

Often when faced with these requests Engineering will say “Yes! We can add that.” This is an untruth. Technical folks are, by nature, builders, makers, and fixers. It’s in the DNA. Pride comes from the ability to create something that people will say “Awesome!” to. The last thing they want to say is “We can’t do that.” So they say yes but neglect to mention that they lack the resources or even the expertise to accomplish it. Look out for the techie that says “No problem. We can do that” without first doing some analysis. You will not find out until much later that they are trying to figure out how to do something that they simply cannot do.

We Can Do That Too (Version 2)

Another flavor of “We Can Do That Too” untruth is timeline reboot. Like the new Star Trek movie. If the technical team knows that they have the expertise and resources they will instantly say yes to a new request. Wonderful! However, what they don’t tell you is that they don’t have the time. You find that out when you sit down to review a project and realize that the technical team just assumed that the timelines were extended. The real request is “Can we add this without changing the timeline?” The untruth is saying yes to that knowing that is what is meant.

I can already hear the response – good project management will head this off at the pass. That assumes that the technical folks ever communicated the new timelines to the project manager. I can’t tell you how many times I’ve seen technical teams develop a new timeline in their heads and never mention it to anyone. Their intention was always to stick to the old timelines but when it becomes apparent they couldn’t they extended it and added a touch of collective amnesia.

Who hasn’t heard “The trade show is in two weeks?! How come no one told me that?” Of course you did but they need to save face. It’s about pride again.

We Can Fix This

About the worst thing that can happen in the middle of product development is the realization that something is terribly wrong. You hit a technical snag that was completely unexpected. Unfortunately, some problems are too big or costly to fix. But that’s not the bad part.

The great untruth is believing you can fix the unfixable. There are just some problems that are insurmountable and this is when you shut down the project. A lot of technical folks don’t know when something is unfixable. The assume that with enough time they will come up with a solution. Before you know it every last drop of resource in the company is engaged in trying to find a solution to an intractable problem. The idea of cutting one’s losses is rejected, more out of pride than anything else. This is when you have to swallow your pride and stop fooling yourself. Then you can find a real solution like buying the technology from someone who has figured it out. Or perhaps dropping the feature that wasn’t going to generate much revenue anyway.

I’ve seen this happen with businesses too. Some businesses face a problem they can’t handle no matter how much time, effort, or money they put into it. They destroy all the value in the business in a quixotic quest to find a way out. Sometimes, the only way out is by the door. What’s especially bad about this situation is that opportunities to scrape something out of the business by changing direction, selling the business or divesting of some it its assets are lost. Here’s to tilting at windmills.

Everyone Else Already Has This

I always leave my favorite to the last. This is by far my favorite untruth. To justify more resources, a new feature, or an accelerated timeline someone will declare that you are behind the curve and this is a feature that everyone already has.

The truth is more likely that they have only made announcements. They might even be working on it. It is even possible that one company has an early version of it. Rarely is there a critical feature that no one anticipated that everyone but you has. Everyone does not have this feature yet. Freaking out over it will only cause you pain.

This type of hysteria can derail a development effort. It results in a lot of time and money going into an early feature that no one is ready to deal with yet. Not you and not your customers. The trade off is almost always dropping something that your customers really do need for something that is new. Listening to competitor’s press releases can be dangerous to your product health.

I want to stress again that no one does these things on purpose. Pride and fear are at the root of these untruths. They are powerful emotions. When the emotions are genuine they can make nothing seem like something. It can even be hard to ask questions about the validity of untruths. The people uttering them believe them and will feel like they are being attacked if you question their perceptions.

If you suspect an untruth here’s a tip: ask for time to look into it. Don’t make a decision right away. Ask sales or marketing to generate a revenue estimate for the new feature or a competitive analysis so that you “can be prepared for launch.” Ask the technical team for a revised project plan so that you can better plan your launch. Let them discover their own untruth and they will be less likely to react emotionally.

Whatever it takes, don’t let the untruth guide you.

Tom's Plain Language License for Mere Mortals

For the past few months I've been working sporadically on a piece of software. It's not a great looking bit of code but interesting none-the-less. It's a simple document management system called Document Locker that allows you store files in a repository (so you know where they are) and wrap some metadata around them. That's not the interesting part. What's neat is that you can define relationships between documents and see which is connected to which. This approach will be most helpful when organizing multi-part documents, scientific papers, and a software project.

Finally, the day is near that I am to release it into the wild, including source code. Roughly two weeks hence, Document Locker 0.5 (the first iteration) will be made available through www.techalignment.com. Part of preparing for the release is developing a license. All software needs a license to keep it from being misused and to protect the creator. This includes open source software. It never ceases to amaze me how little people realize that open source has a license.

I looked over all the major licenses such as Apache and GPLv3. All of the open source licenses I examined had the same problem – they are a mass of legal gobbly gook. I'm used to reading contracts and license agreements (which are a kind of contract) and they were still a tough slog for me. And long? As Sarah Palin would say “You betcha!” No sane person would subject themselves to reading these documents unless driven by necessity. It's like eating bugs. You would do it if you had to but not if there was an alternative.

The reason that even open source licenses are like this is because they are written by lawyers. Lawyers, like engineers, have their own technical language. They have their own concerns and worries and they think in a certain manner. The documents reflect this. I'm not saying it's a bad thing but for a great many uses this type of language obscures more than it illuminates.

When you get down to it, a license exists to create an agreement between people about rights. As the creator, I hold all the rights. You can do certain things with my creation but only those things that I allow. If you can't understand what I'm allowing you to do, how can you be expected to uphold your end of the agreement? You can't.

So, I set out to develop my own license. I'm pretty sure that a large number of lawyers would think I'm nuts just I would if lawyers wanted to write software. My goals were:

1. To make obvious what I was allowing a recipient to do.

2. To make obvious what they can't do.

3. To say so in plain language that anyone could understand.

4. Keep it short. Life's too short for long licenses.

5. I wanted to make a point about licenses.

The result was Tom's Plain Language License for Mere Mortals. It is as plain language as I could get. Unfortunately there is no getting away from referencing other, big, heavy, legalese licenses. Since I have to reference the Java and Neo licenses the reader is still struck with reviewing those licenses. Too bad. Otherwise, it's pretty straightforward and just a bit irreverent. Irreverent? That helps me to achieve goal number five and make the point that software licenses, even benign ones, are so complicated, so full of legal jargon, that they are useless as the basis for a relationship.

With Tom's Plain Language License for Mere Mortals you know where you stand. If you can't understand what you are agreeing to then you shouldn't be mucking about with software. Really.

If you are interested in a preview, you can check it out at www.techalignment.com/TomsPlainLanguageLicenseforMereMortals.pdf. Twitter comments (direct message please) or email them to me if you know my email.

What's On My Mind

A little career downtime can sometimes be a good thing. Whether your sabbatical is planned or, as in my case unexpected, it represents a rare opportunity to delve into new areas of interest. Academics and clergy do this regularly as a way of expanding their skills, working on projects that they can never get to, or simply as a way of recharging their psychological batteries. This is not an extended vacation or time to simply relax. Career downtime has to be used to expand your horizons.

Consistent with my beliefs about downtime, I've been using my current “sabbatical” to embark on areas of discovery that I previously hadn't time for. As my Twitter followers know, some of my time was used to explore the biotech industry. On the more geeky side I've been going back to my software roots to look at things that have fascinated me for a long time. Namely, how to manage unusual or difficult data stores in different ways.

More precisely I've been looking into:

• Managing large unstructured data sets, a constant problem in certain industries;

• Applications driven by relationships between entities more than their structure and;

• How to make smaller applications by embedding data management into them.

This journey of discovery has led me to a number of software technologies that I find very interesting. Let me share with you what's on my mind these days.

Managing Metadata

One of the major bugaboos of the last decade or so has been dealing with the explosion of unstructured data. The simple solution is to wrap metadata around the real data. This descriptive information adds the machine manipulable context that unstructured information lacks. However, managing the metadata itself has become a big problem. Individual metadata is generally small and many data management tools, such as relational databases, are overkill. Implementing a full blown SQL database to manage metadata is like hunting deer with a tank. Expensive and more than you need to get the job done.

XML has helped but managing XML text files is difficult when there is a lot of them. You might need to open and close lots of small files constantly, straining a lot of file systems. The other option is to process one giant XML file which can be processor intensive and slow. Worse yet, these are decisions you have to live with and are hard to change once an application is underway. XML is not ideal for dealing with relationships between entities either.

Social Media Does Change Everything

Okay, I don't believe social media changes everything. What it does do is address the fact that humans are social creatures. We view the world as a series of relationships. Relationships between ourselves and the world around us, between each other, and between everything that makes up the world. The natural schema for a human is relationship based.

Computers don't always reflect that. They tend to be concerned with structures more than the quality of a relationship. This is one of the problems with SQL. It is damn hard to code reciprocal relationships, the strength of a relationship, and the ways entities may be interact. A good DBA can tell you how to do it but it gets complicated quickly especially when modeling real human type relationships.

Small Applications That Can Grow

Enterprise applications have gotten huge. Worse yet, they require boat loads of infrastructure. This is why enterprise applications developers always talk in stacks. LAMP stack, WAMP stack, .Net stack. Developers can declare that their apps aren't really as big as they are because they assume that a stack is in place.

There are a lot of negatives to relying on these stacks. For one, you are at the mercy of whoever is designing the pieces of the stack. Applications also use different versions of the programs that make up these stacks, leading to compatibility problems. Not to mention finding an application that you love on a stack you don't support.

The biggest problem with stack-based enterprise applications is that they are not compact and simple. They don't port to small systems and devices easily. Try implementing a single user version of most enterprise apps. Who is going to install and maintain an Apache web server for one or two people? Cloud businesses like this since they provide an alternative but not all applications lend themselves to the cloud.

What Am I Looking At?

In thinking about these issue, I have come across technologies that address some or all of these problems. I especially like embedded data management tools. I especially like Derby and SQLite, Lucene, and Neo. Derby and SQLite are open source or public domain RDBMS' that allow developers to embed a SQL database in an application. Derby has a server version as well, allowing applications to be small and compact or to scale up to large enterprise size. Derby is from the Apache Foundation and Java-based. This allows it to nicely integrate with Java applications and object mapping frameworks like Hibernate. SQLite is C++ based making it excellent for embedded applications and is extensively used by the Mozilla Foundation. Being a Java geek, I'm planing on spending more time with Derby.

Another Apache project, Lucene, embeds a search engine in an application. With Lucene, a developer is able to manage large amounts of unstructured text using methods familiar to everyone. Lucene also works well with other types of data management tools to add search functionality to all kinds kinds of data.

One the other technologies that Lucene works well with is Neo. Neo is a graphing or network database (there is debate as to what the difference is between the two). Graphing databases view data a bit differently than an RDBM. Data is stored as key-value pairs called properties in an interconnected network of nodes. Finding data is through it's relationships with other nodes. With Neo, information is stored and retrieved in a way that humans organize information, by it's relationship to other data. This fits in well when modeling people or other entities that rely on interactions with others. Some examples are biological ontologies, proteins, and documents. At the moment I'm experimenting with Neo and Document and Content Management.

While there are a lot of things that stink about career downtime, if used effectively it can be a transformative experience. Discovery of this type almost always leads to something good. If nothing else, it helps us to grow as professionals and people. It's also better than sitting around watching television.

Vista: The Good, the Bad, and the Ugly

Something like a million bloggers, journalists, and pundits have written, talked, and pontificated about Microsoft Vista. So why should I bother. Because I am that unlikely combination of techie and tidewad. While I love the geek stuff, I'm usually too cheap to buy things when they come out or just to be the first one on my block with the latest doo-dad. That gives me “value investor” approach. I don't swoon over the latest technical innovation nor am I the Luddite who dismisses them out of hand. Instead, I look for good technology that actually solves some serious problems.

What's nice is that I see that in Vista. By and large, despite the backward compatibility problems, it offers real value over XP. A quantum leap? Nope. That's just fine with me. I want good incremental value. That's not to say that Microsoft is hitting on all cylinders here. There is plenty to dislike about Vista, though little to outright hate. And isn't there enough hate in the world to be hating an operating system? I think so.

The Good

Search

Search is everywhere. It is integrated into the operating system like no add-on can. You can search for files from anywhere. If you open a folder, you have a search bar at the top and search that folder for whatever you want. Sweet! You can search everything from the Start menu (is it still called that? It's just a little circle now...). You can do simple searches and more advanced ones. Best of all, you can search the program menu. In XP, once you acquired a significant number of programs, you ended up paging through a billion menus to find and launch them. Microsoft has fixed this problem. Not only have they redesigned the program menu system but you can search for your program. This has eliminated dozens of Desktop folders that only contained short cuts.

Wireless Networking

With wireless access becoming ubiquitous, Microsoft rightly decided to make it easier to connect to wireless networks. It's pretty much automatic with Vista. The only decision I usually have to make is to determine what type of network I'm connecting to, which alters the security options.

Security

Speaking of security, that too has improved. I'll take the other writers word for it that the underpinnings of the OS are more secure. The biggest, most visible security changes are the inclusion of features that you used to need additional programs for. These including a personal firewall that is finally useful and an anti-spyware program. Oddly, Microsoft still does not include anti-virus in the system. I'm sure there's a good reason, such as a court order or threat of a lawsuit.

Visual Elements

Vista has a lot of eye candy. It's not useful but it makes the day more pleasant. For example, there is this new feature whereby the you see a miniature picture of each open window and you can scroll through them to get to the one you want. Useful? No more so than then XP's task switching (which still works) but much prettier. The same goes for the opaque window dressing. It's pleasant even if it's not particularly useful.

Tons of Useful Goodies

Microsoft continues its tradition of adding small but useful programs to the OS. Most of these are available somewhere else, often for free. Microsoft is not known for originality in this respect. The just turn it into a feature. The Gadgets are just another desktop widget like Yahoo! Widgets or the Google version. What's different is that they are there out of the box and don't require you to download something special.

The included calendar program is also decent. It's not as full featured as the one that comes with Outlook but it is serviceable and stable. The same is true for the photo organizer (a Picasa clone) and the snipping tool.

The Bad

TCP/IP Printers

It is still a monumental pain in the tookus to set up a printer that does not support Windows naming conventions. I know it is in part the printers fault (how hard is it to put SMB support in a printer anyway) but that doesn't takeaway from the fact that this comes up a lot. There are boatloads of networked printers and print servers that only hand out a TCP address. Vista, like its predecessors, makes you jump through a dozen hoops to create a port, load a driver, etc. Given the geniuses at Microsoft, you would think someone would have figured out a way to detect and configure these printers in a more user friendly manner.

Living in a Microsoft World

The only drawback with Vista's search engine is that it cannot index all types of files. As an OpenOffice.org user, my documents do not get indexed which limits what search can do. All the search engines have this problem but it seems about time to include what has become a major office suite. It makes you wonder if it's on purpose. This is true about any program in Vista. They don't seem to understand that theirs is not the only file formats.

No Program Launch Bar

This one is so simple. Microsoft has that lousy little program launch toolbar at the top. That forces you to have small icons. Why not have a big fat tool bar like the Mac OS X Dashboard? Clearly someone at Microsoft was thinking of that. They have the sidebar for the gadgets that kinda works that way. I can download ObjectDock but why have another program floating around when it's obvious people want to do this. It's just obvious.

The Ugly

User Access Controls

One of the best features is also the most annoying. Vista makes it difficult to do certain little tricks that viruses like to use such as changing registry options or spawning other programs. Vista also looks to see if a program has been certified as genuine too. The same is true for programs that could do a lot of harm such as those that change system settings. Since there are lots of legitimate reasons a program may break these rules or not provide certification (such as being an older program), you get a dialog box that asks you if you want to run this program. The reason why this is so annoying is that it happens a hundred times a day. It would have been better if the program asked this once and then remembered that it was okay to do so.

Now you can shut this behavior off but you lose the protection. If you try to fine tune this behavior you quickly find out that there is a security policy program (secpol.msc) that can do this and that it doesn't come with Vista Home Premium, the most ubiquitous of the various versions. So, for most people the only options is to run it at full throttle or not at all.

Backward Compatibility

Like I said in a previous post (To The Pain), backward compatibility is a real problem. An awful lot of applications - fairly recent ones - simply don't work or won't work right. If there is one reason not to use Vista, this is it. If you can't run a critical applications under Vista, then it's of no use to you.

Vista is a worthy update. The additional security, search, and wireless networking support are serious improvements. While it's not enough to spend money on an upgrade, if it comes with a new machine, you will be pleased. That is, if you don't mind half your programs not working.

To The Pain

In the truly excellent movie The Princess Bride, the hero tells the evil prince that he will torture him “to the pain”. He describes “to the pain” in this way “It means I leave you in anguish, wallowing in freakish misery, forever. “ That's pretty much how I feel about my recent, unexpected move to Microsoft Windows Vista. I can deal with the death of my computer but not the pain of moving to Vista.

The story begins, like so many of these stories, with an otherwise usable machine dying an untimely death. My trusty computer of these past five or six years, simply refused to wake up one day. It's probably the power supply. The problem is that it costs a lot to find out if that's true and and then replace it. It's simply not worth trying to revive it. A new computer was in order and that meant Vista.

Now, Vista is a fine OS. It is clearly more secure than XP and has some neat user interface tricks. It's very nice to look at and easier to use than XP was. The search features alone are almost worth the price of upgrading. So far, this looks to have been a lucky move. That's until the “to the pain” part started.

The key problem is application backward compatibility. Several key applications simply do not work and many more don't work right. What really is tortuous is that we are not talking about ancient applications from the Regan era. No! Instead, many of the applications that do not work under Vista are only one or two years old.

For example, Quickbooks 2006 doesn't work. It won't even install. Intuit doesn't seem in the least bit interested in distributing a shim or patch either. Their advice – upgrade to Quickbooks 2007. That's insane. It's not like they didn't know that Vista was coming. It's been delayed longer than Quickbooks 2006 has been released. According to the support forums, this isn't Microsoft's fault either. Intuit apparently did some not so nice things to the Windows registry, exploiting a hole in Windows security that Vista plugged up. If true, that's not right. Actually, it sounds like a scam to me.

I had the same thing happen with NHL 2006. The graphics go wacky when you try and play with a 3D card, which is recommended by the way. The response from EA Sports is “Sorry we released that before Vista”. Nice. Haven't these folks ever heard about developer programs. At least EA Sports gave me some hints on how to get around whatever the problem is. I don't think they will help but at least they're trying.

Interestingly enough, all of my open source applications work flawlessly. Firefox, Thunderbird, Sunbird, and OpenOffice.org all installed and worked perfectly fine.. Even Truecrypt worked and creating encrypted volumes can be tricky. Maybe it's because developing cross platform applications means you can't use goofy operating system tricks.

Anyway, Vista's multimedia hooks are great and the interface much improved. Overall, I'm pleased with the OS. Not that I had a choice really. This time around, I think Microsoft got it right and the application developers hosed us.

All You Need is Love!

It's true. All you need is love. So, do we really need another scripting language? Ever since Perl and Javascript showed us that we could crank out code quickly by using a simpler, non-compiled language, scripting languages have proliferated like nuclear weapons in the 1960s. Besides the old languages such as C Shell, AWK, TCL, or SED, and the middle period ones like Javascript, VBScript, Flash Actionscript, and Perl, we nowmust contend with a whole host of new ones.

In the last few years we have seen the emergence of Ruby, Python, and PHP. All have their adherents who defend them with nearly religious fervor. PHP has become one of the most deployed server side scripting languages while Ruby is often used for interfaces that provide rich user experience (in other words Web 2.0) web sites. We can also add to the mix Java Server Pages (JSP), a scripting language for Java Servlets (popular amongst the Tomcat crowd) also for user interfaces. On top of that, Javascript has been extended and integrated into the AJAX model to provide for pretty much the same purpose.

Now, we can add a couple of new languages or environments to the mix. Some have been pushing Groovy, another servlet-oriented language. Despite the obvious Java-oriented scripting language proliferation, Sun has seen fit to launch yet another scripting language, JavaFX. Microsoft has also announced its scripting effort called Silverlight, despite have ASP.Net already.

Why do we need all of these languages? My take on it is that software companies need to have an entry in this space. Most scripting languages have come from the user community, with Javascript being a notable exception. Unlike compiled languages, which typically have high learning curves and need extensive tools, scripting languages are easy to learn and implement. By creating new scripting languages, many of which are similar to proprietary compiled languages, they extend the franchise in the part of the business that makes them money and neutralizing the user scripting languages which don't. This explains Sun and Microsoft's latest announcements.

I also think the open source and hobbyist communites feel compelled to develop new scripting languages because they simply don't like something about existing ones. It usually starts out with the need to have a language that solves a very narrow problem and expands into a general scripting language. Python and Perl comes to mind here.

The result is too many languages that offer few benefits over existing ones. Most are similar to another language but not exactly the same. This leads to a large number of mistakes that actually slow down coding. Anyone who writes both PHP and Javascript will know exactly what I mean. It also makes for a confusing landscape when trying to choose the what languages to use for a project. What I want is a language that can be implemented as compiled code, client side scripting, or server side scripting. It should start with a standard computer language syntac like C++ or Java. That way I only have to learn one language and can use it for everything. I can also start junior programmers on the more stripped down client side scripting version and work them up to the full, middleware enabled, component-oriented version used in my high volume server applications. This is something for Sun to consider since they, more than anyone else, have the capable to deliver this breadth of options based on the same core language, specifically Java. Microsoft might be able to accomplish this if they standardize on C# syntax.

Scripting languages are a very good idea. They allow for rapid, iterative, development, are exceptionally useful for user interface programming in browser-enabled applications, and make it much easier to deploy small applications quickly and with few resources. They also helped recreate programming as a hobby, something that was on the verge of disappearing as professional programming became more complex. We just have too much of a good thing and need to pare it down a bit. That way we can get on with writing code and not learning new computer languages.

My Top Five Reasons Why The Open Source Community Should Stop Whining

I like open source products. I use them, write about them, and recommend them. What I can't stand is the constant whining and sniping about Microsoft from vocal members of the open source community. They position Microsoft as the Evil Empire. The popular wisdom amongst the anti-Microsoft crowd is that the company owes its success to being technology buccaneers. They steal other people's good ideas and then crush those people under their mighty fist.

I see this every day. Every press announcement from Darth Ballmer or Gates is seen as a pronouncement from the depths of Hades. Vista was lambasted even before it was released and not for the reasons it should be lambasted. The loudest and most persistent whining was over the mutual patent agreement with Novell. It's not a sign of the apocalypse. Companies make these kind of arrangements every day. It's called covering your behind.

What these folks fail to see is that Microsoft is good for open source. In the spirit of giving back to the open source community, I have compiled my top five reasons why Microsoft is good for the open source community. Take heed and maybe you will have something to think about.

5. Jealousy Makes Open Source People Look Like Babies. Oh grow up! When all you can do is cry about Microsoft you look silly. Wah! Wah! Bad Microsoft took all the corporations away from me. Stop telling us why they are lousy and tell me why open source is good. “They stink more” is not a an effective communication strategy. Nor is “Because Microsoft is a bunch of meanies.” If you can't present a compelling reason why your solution is better then maybe it's not. What's sad is that there are a lot of compelling reasons for open source. They just get lost in the self pity.

4. They Have Figured Out the Desktop. Unlike everyone else, they figured out that mix of features and cost that works on the desktop computer. In fact, I think pushing the “Wow” factor of Vista is misguided. People who want eye candy pay the extra it takes for a Mac. People who want really cheap but more geek factor use a Linux desktop. Microsoft has the mix right and is an example for future desktop operating systems. Good enough and cheap enough.

3. Microsoft Actually Makes Some Good Software Sometimes. Sorry but it's true. Let's face it, OpenOffice.org is a copy of Microsoft Office. And every word processor touts MS Word file compatibility, even Google Docs. In this instance, Microsoft became the leaders by making good software such as PowerPoint (which created an entirely new category of software). You can't act like everything they do stinks. It's simply not true or fair.

2. The Eye of Sauron Effect. Without Microsoft to draw their attention, open source software would attract the black hats. For proof, look at the growth in the number of Firefox attacks since it started taking market share from Internet Explorer. Also keep in mind that the short comings of open source software are often overlooked because people are busy complaining about Microsoft.

1. Competition is good. If Microsoft did not exist, then open source would stagnate. Without something pushing the development, without a sense of oneupmanship, it would be hard to justify the effort that goes into open source development. The open source community needs a big monolithic company as its foil. Otherwise, they have no reason to exist.

The unkindest cut of all is that Microsoft actually helps promote open source products. Check out the MSN Tech site. Right on the front page is a link for Firefox and other open source products. I'm no Microsoft apologist but fair is fair. Criticism is fine but constant attacks are unproductive.

Stop the whining and crying and get on with making software. If you make it better than Microsoft then you deserve attention. Otherwise, you end up looking like a crybaby. And here's the world's smallest violin playing just for you...

Why PRDs Screw Up Product Development

Anybody who has ever been involved in product development, especially in the computer industry, knows that it is a process that rarely proceeds smoothly. At some point in the development process it is becomes obvious that the product will either not be what was hoped for or will be late. If you are lucky, you will get a product that is just about good enough and close to on time.

Sometimes, you get what you want (or more accurately, what the customers want) when you want it. Rarely. Products like the iPod are the exception to the rule. They hit the right market at the right time with the right features. It's rare because it takes an extraordinarily visionary with the clout in the company to make it happen. Not every company has a Steve Jobs.

Instead, we all get to that point in the product development cycle where Engineering and Marketing/Sales are at each others throats and things are heading south quickly. The culprit? The Product Requirements Document a.k.a. the PRD. It might be called almost anything (Specifications, Requirements, Concept) but it's always the same - the document that defines what the product is and who it's meant for.

"Now wait a minute!" you might be saying "That's the very situation the PRD is supposed to avoid!" By outlining the features in sufficient detail, engineering knows what to build and marketing has the commitment that it will be built when they need it. How could that be the problem.

The problem isn't the document per se. It is what it really means to the people involved. To most of the folks involved in product development, the PRD is a social contract. An agreement between those who build and those who sell to produce something that the company can make money with. There is a lot of emotional and social capital invested in it. For marketing people, their ability to do their job properly as well as their reputations are on the line. For the engineers, their sense of self - as builders and innovators - is at risk.

Inevitably, there is some important feature that turns out to be much harder to produce than was originally thought. That is the nature of innovative products. When the PRD - the contract - cannot be met all kinds of social problems start to raise their ugly heads. To some, failure to meet the PRD is a failure of the people involved, a kind of dishonesty. A breaking of the contract. There is a tendency to try and find fault with "the other side". The feature wasn't well defined. Sure. You guys in the back have been goofing around with unimportant features. Right!

This inherent conflict cannot be resolved easily either. Too much is at stake for all parties for that to happen. Two major drivers inhibit resolution. The first is humiliation. To the engineers especially, this is personal. They are the makers and they can't make. It's like a chef messing up burning your steak. Few organizations are able to deal with this effectively. Even if most of the team acts well, there is almost always some who do not. There is always a few folks who will act dismissively. It is typical for the marketing fool to say something like "I don't understand why this is so hard. It's just code (or plastic or whatever). This is a non-too-subtle way of saying "You are a loser who can't do your job". Not that the marketing dweeb in question has any idea how hard it can be do to code or plastic or toilet seats.

By the same token, there is always some geeky engineers who, in trying to save face, say things like "Why don't you just sell the features we have finished". Again, a dismissive attitude about something they know nothing about. Sure. No problem. Of course, the fact that the competition has what we have plus the feature you can't make doesn't matter. Sometimes that's ignorance talking (most engineers have never had to sell anything - and we are thankful for that). Most of the time it is a clumsy attempt to deflect criticism. In the end, acting this way toward each other makes everyone furious and shuts down the ability to solve the problem.

As an aside, one of my favorite behaviors that you see in these circumstances is The Pointing Out of Flaws. Similar to the Airing of Grievances at Festivus (Watch Seinfeld reruns if you don't get that) it consists of one arrogant jerk pointing out all the problems with a plan while never offering up a solution. I can't stand that. It accomplishes nothing while making everyone want to slap the pompous ass doing it. These people must be put on the spot with "What do you suggest then?" or "Thanks for the analysis. I'm making you responsible for coming up with solutions to these problems."

The only thing that short circuits this behavior is a lack of tolerance for it. Team members have got to be able to admit they are struggling with something without fear of being publicly humiliated by others. Not only shouldn't the dismissive behavior be tolerated but an environment has to exist where small failures can be admitted so that the team can come together and form a solution.

The other reason for product failure is, oddly enough, lack of accountability. In too many cases, either no one or the wrong person is held is held accountable for failure. That happens all the time. In some cases, we are afraid to take people to task because they will become unmotivated (a highly questionable assumption). In other cases there is no way to do so since metrics are vague or nonexistent. Or worse, the metrics are stupid such as lines of code per day. Since no one is accountable for failure, no one is responsible for solutions. So, you don't get any solutions.

The whole team suffers from the slackers or incompetents. Either everyone has to work doubly hard to make up for those who fall behind or everyone gets blamed. That's when the humiliation factor kicks in. If you feel like you are being tarred with the same brush as the guy who isn't keeping up, you will get pretty defensive. It's simple mental self-protection..

The way out of this dilemma is also good management style. Those who bear the most responsibility, the team leaders, need to identify ways to measure outcomes along the way and deal with those who are not making the grade. That can be frightening when it means you may end up shorthanded on the project. Shorthanded is still better than behind held back. If you can't rely on someone, you're better off without them. Jettison them immediately before they do too much harm.

And perhaps, most of all, change the PRD into something that sets the initial course of the project then, toss it in the back of the file cabinet. Let the managers manage to the people and the needs of the market and not a slip of paper. Make decisions together rather than pointing at a paragraph in a document and say "See! It's right there and you signed it. Loser!"

PRDs are fine for describing what you want to do. The team is what makes it happen. Free your team from the contract, refuse to tolerate dismissive behavior and hold everyone (including yourself) accountable for success and failure. You will get better products that way and on-time.

In The Wiki Wiki Wiki Wiki Wiki Room

I don't think that many people would dispute the success or value of the Wiki. For those of you who have been living in a cave for the past two years, a Wiki is a collaborative document system, environment, or philosophy, depending on who you talk to. While the most famous is Wikipedia, the online, user written encyclopedia, there are zillions of Wikis on the Internet. Most are information repositories of some sort, especially on-line documentation.

The advantage of the Wiki is that it is very easy to write and edit entries. Formatting those entries, while a bit unconventional, is also pretty easy. This yields highly attractive, easy to navigate online documents.

Deploying a Wiki application, such as MediaWiki, the software behind Wikipedia, is also incredibly easy assuming you have a working L/WAMP stack operating. I've worked with all types of collaborative systems including Lotus Notes and a number of knowledgebases, and few are as easy to use as a MediaWIki and none are as easy to install.

Some time ago, I installed and started using a Wiki for tracking storage companies, my bread and butter. That in many ways is a conventional use of a Wiki. Think of it as building an encyclopedia of storage companies. Nothing Earth shattering there.

I also started using my Wiki for development projects. I build my own applications from time to time to provide me something useful, try out technology that people are buzzing about, and simply to amuse myself. On my latest project I used a Wiki to help me to design the application and capture my decisions along the way. What I found was a product managers dream application.

Anyone who has had to build product specs, PRDs, etc. knows what a pain they can be. They are dynamic, require input from lots of often unwilling people, and whole sections usually need to be written by specialists. In other words, they require collaboration, the stock in trade of a Wiki. The same would go with design documents that engineers use.

As time has gone on, I have been able to add and expand sections, eliminate others, and make changes as needed. All without losing the thoughts behind them. As is typical with web applications, you can link to other documents or sections of documents with easy including outside ones. If there were engineers writing the technical design elements, they could simply write them in. If a change occurred to a feature requirement, everyone would know immediately.

Contrast that with the way this is usually done. Dozens of Word documents tossed at the product manager who has to somehow integrate then all without getting something terribly wrong. Playing cut and paste with e-mails and other documents, trying to pull them together into a unified whole but never having a sense of the entire project. Gack! Makes me choke just thinking about it.

Heck, by the end of this, I'll almost have a design manual on how to write this type of application, useful to anyone who would want to do the same in the future. Development and knowledge capture all at once.

This is the killer app for product managers. Using a Wiki, you could pull together requirements documents in half the time and tie them back to the design documents and anything else associated with the product. tech support would love what comes out of this if QA used it. And it cost practically nothing to deploy. The software is free as are most of the infrastructure.

Free and transformative. What could be better.

Same Engine, Different Body

For various reasons I have been writing software again. It's mostly hobbyist stuff or small, useful applications. My TagCloud software has been incredibly useful and was fun to work with. I used to write and design software for a living and it was not nearly as much fun (that's why it's called work folks!).

So, my latest project is a rewrite of an an old travel planning tool that I built in Access years ago. Keeping with my current fascination with modern web-based development, I decided to implement this application as a Java servlet within a Tomcat container. I've been writing Java code since it was new and like many aspects of the language and virtual machine architecture.

One of the big advantages to OO programming is the ability to hack together pre-written chunks of code to build applications quickly. Java makes it very easy to dump in different frameworks for important functions. Need to interface with a database? JDBC makes this easy. Make mine a web server application please. It's easy with the servlet interface.

This advantage also leads to Java's biggest problem: framework and component proliferation. This is most evident when you are building web interfaces. There are all types of competing frameworks and it's easy to get confused. Should I generate the interface as pure HTML/JavaScript from within the servlet? How about using Ruby on Rails, though that means learning yet another language. JavaServer Pages (JSP) might handle some of the static portions of the interface. Is that enough? Should I use a web UI framework? Okay, which one? Struts or JavaServer Faces (JSF)?

It makes my head hurt.

This is the dark side of component and object technology. The things you need to know become so great that you need armies of specialists to write even a simple application. If you don't think this is out of control consider this: Nine years ago, ever thing I needed to know about Java could be found in the two O'Reilly books "Java in a Nutshell, 2nd Edition" and "Java Examples in a Nutshell". At that time they had 11 books altogether about Java, but many rehashed the same subjects. The "Java in a Nutshell" book was only 610 pages long.

Contrast that with what is available today. They O'Reilly web site lists over 125 books on the subject (I got tired of counting) and the 5th edition of "Java in a Nutshell" is listed as having 1252 pages, more than twice the 2nd edition. In 1997, there was one UI choice, the AWT. Now there is still AWT but also Swing, Spring (whcih includes a bunch of other components), the aforementioned JSP, JSF, and Struts, and the Eclipse SWT. Or maybe I could use Ruby on Rails to generate the interface or even JavaScript on Rails. I don't even want to get into all the other extensions, component libraries and frameworks available (J2EE, JavaBeans, AspectJ, JXTA god know what else).

Now if you think this is simply the ravings of a weekend programmer who can't make a decision, consider the poor system architect who has to design a major enterprise application. Not only do they have to evaluate and choose what parts to use but those choices will drive specific costs. Training, books, and time will cost money. Just keeping up with versions and updates could occupy someone full-time. It's job security I suppose.

To be blunt, this is too much of a good thing. Granted, no one wants to have to write all of these parts from scratch. Talk about serious time issues. However, we could do without so much duplication. Do we need Struts and JSF? I'm sure that someone out there is yelling "Yes! By gum we do." and will have a dozen reasons why. Most of those reasons will turn on some minor technical or philosophical issues that matter to only tho a handful of people. I'm sure that some will also argue that it's about choice and choice is good. Maybe, but it's also confusing.

I don't offer a solution here. This probably won't happen with .Net. Microsoft keeps too tight a control on its technology to have this happen. It might also help if the Java community would line up at least the competing technology and pick just one in each group.

In the meantime, I will, like everyone else, make my choices and hope they are the best ones.

First, Do No Harm Mozilla

ARRRGGGHHH! No, that is not the sound of me practicing to be a pirate for Halloween. Instead, it is a strangled cry of anguish brought on by my attempts to run the new Firefox browser. As my gentle readers are aware of, I am usually a Mozilla fan. I have tossed away Microsoft Outlook in favor of Thunderbird and use Firefox, not Internet Explorer. The new version of Firefox, version 2.0, is giving me pause however.

You would think that these folks never heard of backward compatibility. I'm not talking about the extensions that no longer work. I kind of expect that and usually they catch up over the next couple of weeks. I mean things that are supposed to work that no longer do. Worse, features that work inconsistently.

For instance, I had my browser configured to open most links in a new tab with the exception of links within a page. Bookmarks, search results, new windows all come in a tab. It is the single most useful feature of Firefox, in my opinion. The only thing that I wanted to open in the current window, is a link from within the current window. This is typical application behavior. I loaded up Firefox 2 and, behold, everything works exactly the opposite. Worse yet, changing the tab settings seems to have absolutely no effect on the behavior of the browser. Tell it to open external links in the current tab, it still opens them in a new one. No matter what I tell it to do, the silly browser does what it wants to do. Frustrating as all get out.

My TagCloud system no longer works. It somehow generates a 403 error from the web server. To put it plainly, some of my AJAX applications no longer function. Perhaps it is some new security feature that is ignoring the security profile that is in my user preferences. Perhaps it's ignoring my user preferences all together. Maybe it's just acting randomly.

The point of this particular rant is that this is what kills software. Backward compatibility is a key component of upgrading software. If business enterprises are to be expected to implement these product, if these products are to become something other than a hobbyist's toy, then upgrades have to respect current configurations. New features don't matter if the old, useful ones, no longer work.

In case anyone is thinking that this is just me, look around the message boards. You will find all kinds of similar problems. This is also not the first time this has happened. Several times in the past when Mozilla came out with upgrades a lot of things broke and for good.

The extension situation also exposes the soft white underbelly of open source. Over time, certain extensions, plug-ins, or whatever you want to call them, become so widely used as to become a feature. When that happens you can no longer rely on a weekend programmer to maintain it and keep it current. It is incumbent on the main development team to make sure these critical features are delivered.

New features are nice, often important. You can't deliver those by breaking other existing features. For me, it means a return to the older version that I have, v1.5. That presents it's own problems since the update mechanism won't update to my previous version, 1.7 and a lot of extensions will no longer function below that level. All this aggravation is making me take a good look at IE 7. Exactly what the Mozilla team does not want to happen.

Sorry Mozilla, this is a lousy roll out. You won't win corporate hearts and minds this way.

The Evil truth of Open Source Code Dependancies

Lurking... just beyond the shadows... lies an evil so hideous that...

Okay too dramatic. Still, there is an big problem that is cropping up more and more with many open source projects. It seems that in an effort to leverage existing open source code (which is good) we are creating a series of dependencies that make implementation daunting. Just look at the dependency list for the Apache Software Foundation's Ant. Ant is build tool, something programmers use to help compile and link a big program that has a lot of interdependent components. For all of you UNIX/Linux fans out there, think make on steroids. In any event, one look at the dependency list is enough to make a strong stomached open source supporter turn green. There are over 30 libraries and other related components, from nearly as many different sources, that are required to use Ant. Use, not compile.

The core problems with this type of system are:

• Complexity - The obvious problem. It's so difficult to get things installed and configured right that you go nuts


• Version Control - You now have to worry about what version of each dependant component you are dealing with. A change in a small library can break the whole system. Woe be to the programmer who uses an updated version of a component in his new application.


• Bloat - Open source used to have the advantage of being fairly lean. Not so anymore. This is not to say it's any more bloated than proprietary systems like Windows Server 2003. It's just not very different anymore in that respect


• Conflicts - So, you have applications that use different versions of some core component. Have fun working that out.

This is a good example of why people go with closed frameworks like .NET. Even though you are at the mercy of Microsoft, they at least do the heavy lifting for you. Dealing with all this complexity costs money. It costs in terms of system management and development time, as well as errors that decrease productivity.

Ultimately, these factors need to be worked into the open source cost structure. It's one thing when open source is used by hobbyists. They can get a charge out of monkeying around with code elements like that. For professionals, it's another story. They don't have time for this. What's the solution? One solution has been installers that put the whole stack plus applications languages on your system. Another option is to pull these components into a coherent framework like .NET . Then you can install just one item and get the whole package. Complexity and conflicts can be managed by a central project with proper version control for the entire framework. There are commercial frameworks that do this but we need an open source framework that ties together all open source components. Otherwise, open source development will be cease to penetrate the large scale enterprise software market.

Eating My Own Cooking

Like so many analysts, I pontificate on a number of topics. It's one of the perks of the job. You get to shoot your mouth off without actually having to do the things you write or speak about. Every once in awhile though, I get the urge to do the things that I tell other people they should be doing. To eat my own cooking you might say.

Over the past few months I've been doing a fair bit of writing about open source software and new information interfaces such as tag clouds and spouting to friends and colleagues about Web 2.0 and AJAX. All this gabbing on my part inspired me to actually write an application, something I haven't done in a long while. I was intrigued with the idea of a tag cloud program that would help me catalog and categorize (tag) my most important files.

Now, you might ask, "Why bother?" With all the desktop search programs out there you can find almost anything, right? Sort of. Many desktop search products do not support OpenOffice, my office suite of choice, or don't support it well. Search engines also assume that you know the something of the content. If I'm not sure what I'm looking for, the search engine is limited in it's usefulness. You either get nothing back or too much. Like any search engine, desktop search can only return files based on your keyword input. I might be looking for a marketing piece I wrote but not have appropriate keywords in my head.

A tag cloud, in contrast, classifies information by a category, usually called a tag. Most tagging systems allow for multidimensional tagging wherein one piece of information is classified by multiple tags. With a tag cloud I can classify a marketing brochure as "marketing", "brochure" and "sales literature". With these tags in place, I can find my brochure no matter how I'm thinking about it today.

Tag clouds are common on Web sites like Flickr and MySpace. It seemed reasonable that an open source system for files would exist. Despite extensive searching, I've not found one yet that runs on Windows XP. I ran across a couple of commercial ones but they were really extensions to search engines. They stick you with the keywords that the search engine gleans from file content but you can't assign your own tags. Some are extensions of file systems but who wants to install an entirely different file system just to tag a bunch of files?

All this is to say that I ended up building one. It's pretty primitive (this was a hobby project after all) but still useful. It also gave me a good sense of the good, the bad, and the ugly of AJAX architectures. That alone was worth it. There's a lot of rah-rah going on about AJAX, most it well deserved, but there are some drawbacks. Still, it is the only way to go for web applications. With AJAX you can now achieve something close to a standard application interface with a web-based system. You also get a lot of services without coding, making mutli-tier architectures easy. This also makes web-based applications more attractive as a replacement for standard enterprise appliacations, not just Internet services. Sweet!

The downsides - the infrastructure is complex and you need to write code in multiple languages. The latter creates an error prone process. Most web scripting languages have a syntax that is similar in most ways but not all ways. They share the C legacy, as does C++, C#, and Java, but each implements the semantics in their own way. This carried forward to two of the most common languages in the web scripting world, PHP and JavaScript. In this environment, it is easy to make small mistakes in coding that slow down the programming process.

Installing a WAMP stack also turned out to be a bit of a chore. WAMP stands for Windows/Apache/MySQL/PHP (or Perl), and provides an application server environment. This is the same as the LAMP stack but with Windows as the OS instead of Linux. The good part of the WAMP or LAMP stack is that once in place, you don't have to worry about basic Internet services. No need to write a process to listen for a TCP/IP connection or interpret HTTP. The Apache Web Server does it for you. It also provides for portability. Theoretically, one should be able to take the same server code and put it on an any other box and have it run. I say theoretically because I discovered there are small differences in component implementations. I started on a LAMP stack and had to make changes to my PHP code for it to run under Windows XP. Still, the changes were quite small.

The big hassle was getting the WAMP stack configured. Configuration is the Achilles heel of open source. It is a pain in the neck! Despite configuration scripts, books,a nd decent documentation, I had no choice but to hand edit several different configuration files and download updated libraries for several components. That was just to get the basic infrastructure up and running. No application code, just a web server capable of running PHP which, in turn, could access the MySQL database. I can see now why O'Reilly and other technical book publishers can have dozens of titles on how to set up and configure these open source parts. It also makes evident how Microsoft can still make money in this space. Once the environment was properly configured and operational, writing the code was swift and pretty easy. In no time at all I had my Tag Cloud program.

The Tag Cloud program is implemented as a typical three tier system. There is a SQL database, implemented with MySQL, for persistent storage. The second tier is the application server code written in PHP and hosted on the Apache web server. This tier provides an indirect (read: more secure) interface to the database, does parameter checking, and formats the information heading back to the client.

As an aside, I originally thought to send XML to the client and wrote the server code that way. What I discovered was that it was quite cumbersome. Instead of simply displaying information returned from the server, I had to process XML trees and reformat them for display. This turned out to be quite slow given the amount of information returned and just tough to code right. Instead, I had the server return fragments of XHTML which were integrated into the client XHTML. The effect was the same but coding was much easier. In truth, PHP excels at text formating and JavaScript (the client coding language in AJAX) does not.

While returning pure XML makes it easier to integrate the server responses into other client applications, such as a Yahoo Widget, it also requires double the text processing. With pure XML output you need to generate the XML on the server and then interpret and format the XML into XHTML on the client. It is possible to do that with fairly easily with XSLT and XPath statements but in the interactive AJAX environment, this adds a lot of complexity. I've also discovered that XSLT doesn't always work the same way in different browsers and I was hell-bent on this being cross-browser.

The JavaScript client was an exercise in easy programming once the basic AJAX framework was in place. All that was required was two pieces of code. One was Nicholas Zakas' excellent cross-browser AJAX library, zXml. Unfortunately, I discovered too late that it also included cross-browser implementations of XSLT and XPath as well. Oh well. Maybe next time.

The second element was the HTTPRequest object wrapper class. HTTPRequest is the JavaScript object used to make requests of HTTP servers. It is implemented differently in different browsers and client application frameworks. zXml makes it much easier to have HTTPRequest work correctly in different browsers. managing multiple connections to the web server though was difficult. Since I wanted the AJAX code to be asychronous, I kept running into concurrency problems. The solution was wrapper for the HTTPRequest object to assist in managing connections to the web server and encapsulate some of the more redundant code that popped up along the way. Easy enough to do in JavaScript and it made the code less error prone too! After that it was all SMOP (a Simple Matter of Programming). Adding new functions is also easy as pie. I have a dozen ideas for improvements but all the core functions are working well.

The basic architecture is simple. A web page provides basic structure and acts as a container for the interactive elements. It's pretty simple XHTML. In fact, if you look at the source it would look like nothing. There are three DIV sections with named identifiers. These represent the three interactive panels. Depending on user interaction, the HTTPRequest helper objects are instantiated and make a request of the server. The server runs the request PHP code which returns XHTML fragments that are for display (such as the TagCloud itself) or represent errors. The wrapper objects place them in the appropriate display panels. Keep in mind, it is possible to write a completely different web page with small JavaScript coding changes or even just changes to the static XHTML.

The system has all the advantages of web applications with an interactive interface. No page refreshes, no long waits, no interface acrobatics. It's easy to see why folks like Google are embracing this methodology. There's a lot I could do with this if I had more time to devote to programming but Hey! it's only a hobby.

At the very least, I have a very useful information management tool. Finding important files has become much easier. One of the nice aspects of this is that I only bother to tag important files, not everything. It's more efficent to bake bread when you have already seperated the wheat from the chafe. It's also good to eat my own cooking and find that it's pretty good.

Service-oriented Architectures: Taking large scale software to the next level

Distributed, dynamically bound systems are not new. The Internet is an example of a highly distributed system where resources are made available over open protocols. Back in the 90's there were an awful lot of service bus architectures as well. These usually involved a big piece of infrastructure such as an object broker. Both approaches have shortcomings. The Internet doesn't have enough programming heft. You have to augment it with all types of other objects that are loaded into clients. Object brokers place restrictions on programming languages and are very complex and expensive to implement.

A service-oriented architecture, on the other hand, combines concepts from both. You have loosely coupled resources, protocol simplicity, language independence, and easy of implementation. This affords an opportunity to virtualize an enterprise application. In turn, this virtualization makes development easier and more robust. Scaling up applications becomes much simpler when the components are not tightly tied together. Need a new functions? Add a new callable service and publish it. Need to change something? Change one service without compiling the whole application.

.NET does this too but on too small a scale. When you consider systems with thousands of functions, working over the Internet or other long distance networks, accessing components becomes a serious problem. In an SOA, you don't need to request the processing object, just that it perform the function that you need. This is also much less bandwidth intensive.

The hang up with SOA is the underlying infrastructure. While SOA makes the application layer easier to manage, the components of the SOA are too often tightly bound to physical infrastructure such as servers and storage devices. To make SOA truly efficient, we need to virtualize and abstract this layer too. Servers, network resources, storage, and system level applications also need to become services. This way, applications can request system resources as they need them and only recieve what they need. That's very efficienct. If a service is running under a light load, it would be good to have it only ask for lower processing and network resources. That frees resources for other, heavy users. Rather than have administrators analyze and make these changes manually, dynamically allocating only the resources that a componet needs leads to a better use of available resources. By negotiating with those resources, every component gets mostly what they need rather then an all or nothing scenario.

SOA's are an evolution not revolution, but it's a good evolution. It give system architects and administrators an opportunity to have a more efficient and more easily manageable system. Let's get the hardware under the same control. Hardware, software, it's all just resources that need to be reallocated as needs change.

Linux Breaks Apart Under the Open Source Model

I first worked with Linux back in the mid-90s. At the time there was only one real distribution, the Slackware distro, and it was a serious chore to install. It required manual configuration that was much more difficult than your average UNIX box let alone Windows or DOS.

Back then, as we experimented with Linux, one thing became abundantly clear - it was a one size fits all approach and that didn't work. The truth is, operating systems variations proliferate because different missions have different needs. A server OS is different than a desktop OS. A secure server is a totally different animal altogether. The requirements of multimedia development, programming, and games all vary widely. Linux in the old days really wasn't designed for that.

To be fair, Linux was (and technically still is) just the kernel. Diversity tended to be across operating systems then instead of within them. Desktop applications? Windows. Server applications? UNIX and maybe Windows NT. File server? Windows NT and Novell for the die hards. Multimedia and graphics? No question about it, go Mac.

This is in sharp contrast to the OS world of today. Starting with fragmentation of UNIX in the 1980's, we now experience so many variants of operating systems, that the mind boggles. The latest list of Vista variations shows six different types and that doesn't include the Microsoft server operating systems or versions of Windows for embedded and mobile applications. Windows Server 2003 also has a bunch of variations, such as the Storage Server, and Longhorn is expected to be completely modular. Just trying to figure out which future Windows to use makes my head hurt.

It should then come as no surprise that the once unified Linux now ships in more distributions (a fancy word for versions) then there are mints in a Altoids tin. The Distrowatch website lists a top 100 distributions. That implies that there are more than 100 hundred distributions! I can believe it. There actually are hundreds of distributions. While many are simply different packaging, most are specialized distributions aimed at increasingly narrow markets. Talk about slicing the baloney thin. There are special multimedia distros like Gen Too, desktop-oriented ones like Ubuntu, server versions like Red Hat Enterprise, and many more. Some are designed for lower end machines, such as Damn Small Linux, and a few, like Slackware, seem to be heirloom distributions. More to the point, there are dozens of entries in each category, even relatively narrow ones like embedded real-time operating systems.

Ultimately, this is a ying-yang situation. While being able to find an operating system variant that suits a very narrow need is attractive, supporting that many operating systems drains resources away from the core system. It is also pretty confusing. Just trying to compare this many versions of an OS can be daunting. It's no wonder that only a small number of distributions make up the majority of installations with a few additional ones staking our majority claims only in special niches.

Altogether, this is what happens with open source. Everyone wants their own flavor or feels like tinkering with it. Before you know it, variations proliferate like zebra mussels. Even fairly obscure new technology like VoIP fractures quickly under the open source model. There are lots of SIP servers out there and a bunch of variations on one implementation called Asterix. Great stuff but there's no control.

It is clear that there should be variations on the Linux theme but a limited number. One for servers, one for desktop, one for embedded, etc. The very nature of the Linux and open source world makes this unlikely. Unfortunately, the proliferation of Linux distros will weight down on Linux, hurting it in the long term.

The fracturing of the Linux world into hundreds of variations is a side effect of the open source movement. In the end one of two things will happen. Either most of these distributions will fade away as the programmers get bored with it or Linux will eventually fail altogether, paving the way for more years of the Windows hegemony. I wonder if Bill Gates and Steve Ballmer are sitting down with a glass of champagne right now toasting Linux.