The problem with all of these is that often they start off with some good ideas and then become rigid and formalized and complicated and often become not much better than whatever people were using before them. When I start seeing companies spring up to "help" you understand how to do them better or provide tools (usually expensive) to make them easier I start tuning out. It's such a growth industry you can't throw a rock without hitting someone who is more than willing to take your money and tell you how to write better software. Most of them probably know less about writing real software than your grandmother. I can remember enough presentations from companies who promised Nirvana where I wanted to run away and join a circus; most of the time they vanish into oblivion when they run out of suckers to sell to.

When I think of agile I think of a mountain goat. Amazing animals that never seem to lose their balance, bounce from rock to rock, and hardly notice whatever apparent peril they might appear to be in. I never think of a hippo on land who though they can run fast aren't very nimble at all and can't turn or run very far. Can you turn a hippo into a mountain goat? No, but feed a goat enough and they can bloat until they resemble the hippo (with tragic consequences on a mountainside).

agile (little a) means the ability to work quickly, adapt easily, deliver what is needed when it is needed and repeat as needed. But no matter what methodology you subscribe to or how much you want it not every project or organization can ever really be agile. Virtually all of the named methodologies promise that you can make anything agile if you just do this or follow that but I am convinced after 3 decades of doing this that sometimes it's just not possible.

The larger company I work for does try to follow a formal Agile process but it really doesn't succeed and I don't think it ever will. Our industry is too complicated, our core technology is too ancient yet too irreplaceable to ever be anything other than a hippo in sneakers. Many companies with long histories in industries such as banking, insurance and healthcare have so much invested in heavy old technology and software that no amount of modern methodology will ever make more than a modicum of improvement. But it doesn't keep people from spending money on those who would convince them otherwise.

Of course just because a large organization can't improve their hippo-ness doesn't mean that smaller groups inside can't rev up some nimble goat-ness. The small team I work with has lots of overlapping products and services which we deliver continuously and rapidly shift as the market changes directions. I would say we don't really do any formal anything process but remain very goatlike in how we manage things. Our group operates somewhat independently of the larger organization and we've learned over time how to take advantage of that independence to continuously increase our share of the company's business. I like to think most companies realize the advantage of finding ways to better their success by giving small in-house groups more ability to operate on their own. Think of it as instead of trying only to shrink the hippo, make yourself more nimble by breeding more goats.

From another industry I remember how GM tried to improve its business by creating the Saturn brand and company and letting it operate independently. For a while it worked great until too many in the parent company grew jealous and forced it to fail. Unfortunately a nimble goat can make a fat hippo look bad.

I look back on the opportunities I have had to participate, develop and deliver products and every single success came without any named formal software process at all. Virtually all of them were iterative in nature, continuously adaptive to change, built in small teams, with constant access to the customer or market representative and always featuring a great team willing to try something different. All the projects that were dismal failures or were cancelled had virtually none of those things. Not one of those successful projects involved paying anyone to tell us how to succeed even when we had never done anything like it before. Sometimes I think people who lead projects (or more likely those higher up the food chain) are so under-confident that only by paying huge money to some external entity or subscribing to some heavy formal process will guarantee success.

The old waterfall process (which seems to have been described originally as what not to do) still exists today because it gives upper management the illusion of control. I think the "Agile" type processes can do the same if applied to a hippo organization as a rigid system; you can say you are Agile and assume that things will go much smoother in the future, and if things don't seem to get better you can blame it on not being Agile enough.

But agility can never be formal, can never be rigid, can never be a detailed set of steps or processes or a magic formula. You can't create a goat by making a hippo take ballet. Being agile is always a loose process and only works in small teams so all you can do is find ways to split off bits and pieces from your hippo butt. Once you have a small team it's not what the process is called but finding flexible people willing to organize themselves into a process that succeeds. To paraphrase a great movie line "agile is what does agile".

So try to be a goat, realize the limitations of trying to create a svelte hippo with a cooly-named process, and always remember that the best agile should be spelled with a small 'a'.

In a nutshell the author of the article seemed to think you should hire a programmer on the spot if they went way overboard on the coding tests and other challenges, like the guy who at the end of a two hour coding test was seen feverishly adding HTML to his Javadoc comments.

I wouldn't hire this person under any circumstances. No, really. I've hired and interviewed enough people in my life that I want nothing to do with someone who fails to comprehend reality. Heck I've never made anyone write code in an interview and probably wouldn't hire someone who demanded to do so.

I know some of you might think I'm nuts and offer the old argument "but you wouldn't hire a carpenter without asking him to hit a nail". My point is what are you hiring them for? If you want someone to build a house, skill at hitting a nail is almost immaterial. Sure it's nice to know they can smash one into a two by four with one blow but that's a far cry from building a house. The skill set needed to build a quality dwelling far exceeds what you can tell from a tiny sample of a tiny portion of someone's ability. Unless you are hiring people to write short pieces of code, what's the point of testing them on that?

You can ask me, but how do I tell a good programmer from a terrible one without a coding test? Shouldn't I see how they work on a problem or if they are sharp enough to see a piece of code that has a bug? Sure, go ahead if it makes you feel more comfortable, but it won't tell you squat about how good they are in a year long project with 10 other people, writing code in multiple languages or environments, dealing with adversity, taking charge of difficult problems and working together with others. Your careful interview testing will tell you almost nothing about that. So why spend an inordinate amount of time on relatively minor parts of a programmer's skill set?

I've said it before in previous posts but give me an hour one on one and I can tell more about how a (senior level) programmer will be in the long run than all the coding tests in the world. Why? Good programmers talk like they work when you engage them in details of past projects they spent months on. You can't fake something you invested a good chunk of your life on. A good programmer learns from errors they did or others did; they remember how they dealt with some nasty design problem or fixed a complicated bug. Good programmers who learn and grow over time have good programming in their DNA, and love to talk about them with a peer. Crappy programmers evade and toss out bullshit or forgot what they did. They don't remember what other people on the team did or the environment around the project. Good programmers see everything; bad programmers forget everything.

Even after 30 years in this business I can still remember details of projects going back decades, especially those that I spent a lot of time on. Everything I've done is engrained in my mental DNA. Lessons I learned from the very first program I ever wrote (in a high school computer class on a teletype machine with a paper tape) are still with me. With all the good programmers I've known over the years you can hear the same thing if you listen to what they say and how they talk. The mediocre and terrible ones all sounded the same to me.

When I hear about 2 days of interviews at some big companies I have to laugh. Have they done any testing on effectiveness? I wish someone at Google, for example, would take a few positions and interview two ways (or more, heck they have plenty of time and money); one the normal massive overkill method, and another, meeting candidates with a couple peers at a coffee shop for an hour. Then make decisions about both positions and track the quality of the actually job performance and contributions for the next year. I bet there is little difference. But the normal way costs a lot of money and time, and the second costs spare change and almost everyone has a good time.

Now I am not saying you should not have some pre-qualifications like you do today, I'm talking about the actual interviews. Today for sure you get a lot of random people for every job posted.

My favorite idea is still contract to hire everyone after your (hopefully reasonable) interview; that way you can see how they do real work on real projects before you commit. Since you can't spend a few weeks at their former job position seeing what they did (like you could do with the home builder) this is still the best way to get a real picture. Programming is not about writing lines of code, it's about completing projects or shipping applications or whatever you do and generally working in a team. None of that is evident in any kind of interview test or whiteboard session.

Not every programmer has to write perfect code or invent Einsteinian algorithms. You're not trying to recreate the Avengers where everyone is a superhero. You want a team that makes your business better, your customers happier and makes more money or whatever is important.

I remember a long time ago going to the company gym to play a little basketball and there was a team trying to find some people to play a practice game against. They were all clearly amazing athletes. Me and 4 other random guys agreed and we played a full court game. We killed them despite being rather average in ability, because somehow we functioned as a team on both ends of the court. The better team lost because despite their individual prowess they really didn't play together at all and actually got in each others way. I've always remembered the feeling of what a team really is. Yet if you'd interviewed the 10 of us and made us perform basketball skills neither me nor my random teammates would have been considered. Sometimes what you want in the end isn't evident in a single moment of time and you should never think you can somehow milk that one moment to get the perfect programmer.

Now they complain in emails, tweets and comments about how crappy your service was. So you tell your CEO when he/she asks why: "Our servers are too slow; our caching systems aren't aggressive enough; the mainframe can't keep up with traffic; our database needs tuning; our software is too old".

You know what, the users could care less! They want service and you offer great (and maybe even truthful) excuses. Your competitors take advantage and you look like a chump.

Face it, no matter what the reason is someone who wants to do business with you isn't the slightest bit interested in what is making your service slow, or lack functionality, be unreliable or maybe even be unavailable. Even if you (as in whoever supports/creates/runs the show) have no control over anything, someone somewhere in your organization does. They had better get with it and make things work or find someone who can or customers will simply go elsewhere.

No matter how great you think you are it doesn't matter a bit if your customers (or likely ex-customers) disagree.

Now it is possible your competitors are all in the same boat so you think you are safe. Forget it, technological businesses that suck are ripe targets for new competitors who don't make excuses and give your customers what they want. Even one of your competitors might see the light and make major improvements or try better approaches. If all you have is excuses you will get run over. It's guaranteed.

Today everyone is so used to going to Google, typing a few words and getting immediate results. The results even proudly mention the search took "0.17 seconds". How well would Google have done if they started off taking 10 seconds and didn't get any faster? Nowhere. Yet I see lots of places where results come back in 10, 15 or even 20 seconds. On the internet if I as a user push a button and it takes longer than 1 second (assuming the network is fast) I start to get antsy. 10 seconds and I go to Google and find somewhere else. 20 seconds and I grow cobwebs.

Imagine if Amazon took 20 seconds to return a search? Would they be so big today? What if an Amazon search only returned results 90% of the time? Would you shop there?

Sure there are excuses for your systems and some of them are valid like you have no control over anything and all your data comes from elsewhere. Still, someone somewhere in your organization needs to care. I know it's easy in large companies (especially banks and insurance companies) where there are giant disconnects between those who face the public and those who control the data or systems or whatever backs what you are providing. If whoever runs the place doesn't care then someday you will be out of a job.

Steve Jobs never allowed anyone to make excuses and Apple's success is a direct result. The buck stopped with him and if he felt the end result (customer satisfaction and loyalty) was compromised the buck likely wound up upside your head. He made people think creatively to solve seemingly intractable problems. I'm sure people hated being yelled at for offering what seemed like a reasonable excuse for why something wasn't fast enough or smooth enough or even doable. He knew perfectly well that anti-gravity feature wasn't possible but he wanted people to find solutions that they would otherwise have never considered.

Sadly most companies don't have leadership that sees the bad, hears why but encourages or demands every possible solution to make things work better. What usually happens is lethargy, politics, ego or even fear keeps slow, unreliable or unusable services or products available to the public. If you are lucky no competitor is any better. But that never lasts for long.

Look at the phone industry circa 2006. Crappy phones cheaply made and ineptly marketed. Enter one iPhone. All of the market leaders from 2006 are now dead, dying or sold for pennies. Huge well known companies who assumed the competition was never going to change and no one would be able to do anything different. Good night, ladies.

For me, anything I do on the internet that isn't as fast as Google is a problem that has to be fixed. People expect internet time and don't care why they can't get it. Nothing you tell them will ever change that.

Anything I use I want to function reliably. I don't care that you back up your database on Friday night because that's when the admin has time and all the web apps start throwing random exceptions because they don't want to be bothered shutting them down (I worked there). I don't care your searches are terribly slow because some moronic security guy insists that your Oracle database servers should run anti-virus software (worked there, same place!). I pay you to process my data today, not tomorrow because your apps leak like a sieve and you have to restart them many times a day so it takes 26 hours to process it every 24 hours (I worked there and pointed out the leak but you wouldn't fix it because it might make you look bad). I want to give you money for your service but I can't because your system is so old it won't let me order the new features (I worked there briefly and many years later you still haven't figured out how to do it). I could go on.

I might know what's wrong, I might understand your reasons, I might even sympathize with the pain, but most users don't understand, don't care and will voice their displeasure or more likely go elsewhere. It's your business and someone there needs to realize that if you don't fix what's wrong, the user's business and money and attention will go elsewhere. Technology waits for no one. Ask Kodak. Ask Nortel. Without Steve Jobs you could have asked Apple in 1998.

Take a look at your internet, software or other product offerings and see if they are insanely great compared to every competitor you have. Imagine a weakness and a new competitor who will kick your butt over it and go there first. Fight excuses with imagination. Fight politics with whatever you can muster. Remember customers and users have money and you want some of it but know they won't give you any if someone else pleases them more. Losing money to smarter and more nimble competitors can be a mighty big club. Apple didn't become the world's most valuable company by playing it safe and being just as good as everyone else.

So the next time you read a customer complaint about how slow you service is, how unreliable things are or why things don't work, send them a detailed set of excuses. I can guarantee they will still think you suck.

Oh wait, that's a movie plot. I meant to say that things get so complex that we can't figure out how to keep them secure any more. That's not as interesting a plot but far more likely to happen, if it hasn't already.

In the old days before credit card numbers and the internet existed you had to actually go to a bank and point a gun at a teller and generally all you could get was a bag full of money. Security generally involved a gun-toting guard a donut shy of a hundred dozen. Today there are millions of people on the internet looking for any tiny crack in your systems hoping to snag millions of credit cards, personal identity data or company or government secrets. This isn't your great great grandfather's old west anymore.

Today it's not just about your web site but all your client applications, your back office systems, desktop and portable devices, flash drives and cloud environments. Companies often have connections not just to customers but to suppliers, services, analytics and cloud servers. Your data comes from everywhere and goes everywhere, while existing everywhere at once. Somehow someone needs to understand where everything is and what it touches and who has access to it even in a wildly dynamic world.

It must really suck to be a security chief: no one knows who you are and what you do until 5000 web authors point out how stupid you were. Even with all the security breaches and hacked systems you see read about every day I imagine far more are never reported.

The problem with securing a complex connected company is that there are so many touch points to keep track of that it is easy to miss a single one. A recent theft of a large number of SS numbers and other personal information from a government Medicaid office happened because a single server was not configured correctly. One mistake and the masses that search the internet for a hole pounced and all the possibly excellent security (I don't know but they did sound like they had a plan) may as well have been a donut munching stereotype. That's what makes security so challenging: one error is enough to screw up everything.

An enduring but common metaphor, that a chain is only as strong as its weakest link, remains a good description of why security is so hard. A tired programmer, a manager trying to impress, a poor QA person, a bug in a script or even a wrong button push can kill even the best designed security plan. Now throw more and more complexity in a highly heterogeneous environment and it becomes impossible to actually understand how every works sufficiently to keep out the bad guys or even protect against accidental problems. Add in enormous cost for security and pressure to build the business fast in this high speed world and now proper security has to compete with sales and profits.

What we have now and will only get more of in the future is too many features backed by too little security.

The group I work for has several clients and a suite of web services; the parent entity has web services and caches, databases and data suppliers; its parent entity has more of the same plus mainframe systems which have many layers of services and systems, data suppliers and consumers. When I make a web service call I have no clue many layers of systems are getting called and updated and where the data even lives. Travel is a mind-numbingly complicated mess. Does anyone know where everything is and how it all works together? I don't think it's even possible. Now consider something like the US Defense Department which is orders of magnitude more complicated and protects nation secrets and nuclear weapons instead of seats on a plane. Everyone wants some of that, and it's protected by sub-sub-sub-contractors.

One of my favorite quotes is "If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization". If a single program can be hard to harden, imagine how hard an entire company set of systems can be to protect, or even an entire country set of company and government systems. The Stuxnet attack on Iranian nuclear systems continues to be a great research problem and likely harbinger of what the ever more complex future is likely to bring.

In the early days of interconnections between systems passwords were even required. Today you have to consider everything as being a possible attack vector. It's not enough to protect against "hackers" broaching your website. Threats can come from anywhere and often come from inside.

I worked for a healthcare company where I found that the production server and database passwords were stored in text file in the code repository, and half the company knew what they were. When I brought this up to the CTO how easy it would be for someone who worked there to steal data, he remarked that "we trust our employees". I wouldn't trust myself with valuable data much less hundreds of random employees and contractors. The bank robber of yesteryear is now sitting in front of a computer attacking your systems, getting a job at your company or even paying someone to saw a hole in your floor. Trying to keep everything functioning and in control is hard expensive work.

If it's hard today imagine 10 years in the future. As our systems get more and more interconnected and data becomes more and more valuable stealing, breaking and denying access will become easier to do and more and more people will find it tempting to get involved. I really do wonder if we can keep things secure anymore. I have no doubt that more and more breaches will occur, whether we hear about them or not. Even today you hear how desperately companies try to hide security lapses and how a single successful attack can slow down or even destroy a business. How imagine this happening every day in public and you wonder at what point people will become wary of technological progress.

There's a woodpecker out there waiting for our rapidly advancing technology and I think we might soon reach a point where we might really wonder if it will come crashing down. There aren't, after all, an infinite number of credit card numbers, an endless amount of money, or lots of spare nuclear missile launch codes that we can afford to lose.

Security really should be job one: if it isn't now it better soon be. You can pay for it now or lose it all later.

Two major changes over the years make me wonder if the next generation of programmers will be one trick wonders.

Division Of Labor

In bigger IT organizations the division of labor in building applications keeps dividing faster than a hydra hit with an ax. When I started at General Dynamics we had programmers, a manager and often not much else. Once I started Data Tailor in 1985 three of us built Trapeze and we all functioned as programmers, designed the features and the UI together. Persuasion was fairly much the same, with the primary author and the 4 of us working together although Aldus provided some real QA and a product manager. For Deltagraph our team consisted of a product manager, 4 and eventually 5 programmers and a couple QA people. Again we designed the UI, corroborated on features and did all the programming.

I didn't even hear the term architect until I started doing web for a little consulting firm in 1998; we mostly did our web programming in WebObjects at the start and during the next 4 years I often functioned as architect, analyst (figuring out with the customer what was needed), project manager, programmer and UI designer. We had an artist who rarely did design, and a DBA who mostly did system stuff. Otherwise each team generally worked the same way on projects. Sometimes the customer paid for a separate project manager but that was still rare. We did have people who specialized in hardware and systems, but programmers generally did everything software or UI related. The cool thing was being able to understand how everything could work together and gain experience in doing all aspects of creating software.

As the dotcom era collapsed and I worked at a few other places I started to see and be forced into more specialized roles. Suddenly architects were more about high level stuff. Web designers did UI. Project managers made decisions. Coders wrote code. I remember an interview where the group I met with had an architect who never wrote code but created architectures, lead programmers who designed classes but never wrote code and programmers who filled in the classes. No one was allowed to overlap at all and they were proud of how organized they were. I couldn't run out of that interview fast enough. Yet it was a harbinger of what was to come.

As an architect at a financial services company I made sure I still got to program. I was the first person to use Ajax (soon after it was named). I still got to build actual applications in addition to functioning as an architect.

My foray into working at a healthcare company showed me that my era of doing it all was coming to an end. Here they had project managers who only coordinated, business analysts who only talked with customers, another group that only wrote process documents, architects who created architectures, project leads who dictated how software would be written, UI designers who provided all UI designs, database designers who controlled databases and others who managed database infrastructure. Oh and a few programmers who purely wrote code. It sure sounded organized. Yet what it really was was inefficient. Whereas earlier everyone involved in a project knew everything now we had eternal coordination meetings, incredibly detailed documents that took months to produce, and very little was actually accomplished. When I left there was one project I had estimated at two weeks worth of coding where we had had six months of weekly in-person and phone meetings and for which the requirements document was nearly 120 pages which had occupied a single analyst the entire half a year to produce. I'm pretty sure the project could have been written a dozen times over during this time.

Narrower and Narrower Programmer Skills

The other specialization irritation I see is where programmers are hired based on a continuously narrowing set of skills. Whereas in the early days you were hired as a programmer, based on experience in programming, today it's more likely that you know version X of programming technology Y. People want a square programmer who fits in a square hole and has done nothing but be a square. We want an iOS+Android programmer, we are looking for someone with Java on WebSphere, you need 8 years of .NET, or you must be a Spring guru.

Sure, programming today is far more complex than it ever was and it's tough to be good at a broad set of technologies since there are so many. But the underlying issue is that you rarely get the chance to do something different than what you have been doing. You are typecast today based on your last couple of jobs. Even within a single organization the opportunity to master multiple technologies, must less disciplines, is a thing of the past. Can I be a programmer and UI designer? Sorry. Can I talk with the customer but still write the implementation? Nope. Can I architect a solution and deliver it? Are you nuts?

Sure that's what people want today but for me I find it sad. Today I work as an iOS programmer but designing the UX architecture, the UI, deciding on features, working on the back end and working with customers or upper management is no longer part of my job, and that makes me sad sometimes.

Of all the projects I have done in the second half of my career the most fun I ever had was a huge project back in 2000 which was done by exactly two people. We met with the customer, educated them on what was possible and understood their business, I built the whole UI and the other guy built the database and together we wrote virtually all the code. We also managed the systems during development and provided a continuous running version for the customer to work with. The project was our first in Java and had both an external web application for the external customer and internal workflow for the internal team with full auditing and management features.

It was a great success and saved the customer millions of dollars per year and ultimately was automated (which sadly cost the entire internal team 120 jobs). We did most of the work in six months before the customer's IT decided to get involved (they then took six months to implement the login page!). It was fun, rewarding, successful and cost almost nothing compared to the financial benefit. That was the last time I ever got to do everything like that.

I guess nothing can ever be the same but what I find sad is that in specialization you wind up with easy obsolescence (I know one thing, if it goes out of style I'm toast) or cement technology (we know this one thing and damned if we will ever change). The more things you have experience in as a programmer the easy it is to learn something new, and the more likely you are to be willing to try something new (but still be able to separate new gold from new turds). The more organizations specialize the less efficient they become and the more coordination they require. Despite a desire to be "Agile" the less agile they become as a ponderous division of labor makes it difficult to move quickly. The more people have invested in protecting their narrow piece of the pie the less likely the pie will ever be baked.

No, I'm not some old guy yearning for the days where all I needed to know was K&R C, 3 binders of Inside Macintosh and a 68K assembly manual. But I do wonder if the modern silvers of programming knowledge will lead to less willingness to try something new, create something better or just get'r'done.