You may be correct that this makes for a better programmer, or you may just be rationalizing the obsessive compulsiveness of programmers like you--and me! ;)

It was all going so well until this:

"It's why good programmers are always looking for open source alternatives"

Using or not using open source software is irrelevant when determing if someone is a "good" programmer.

I don't think that's a good programmer. I think that's a good anything. Take a good sysadmin. He/she should want to know why they make the ndd changes on solaris as well as what is being changed as opposed to "I dunno. All the other machines are setup like this." Also, look at the example of a good chef. He/she wants to understand why a hollandaise can break and how to prevent it as opposed to just re-emulsifying with some cold water or a bit of heavy cream.

Mike #1: On the contrary, Open Source is critical to understanding. If you don't have the source code, you are greatly hindered in this regard. Using a closed-source library is basically "faith-based programming". (Or voodoo, as I like to think of it.)

I have to agree with Mike #1. I think open source software can be one way to be a good programmer, perhaps depending on the project, but I don't think it's required.

I'll bridge the Mikes here. I think that understanding the technologies you're using can be an important part of development, and it's certainly true that Open Source gives you another avenue to that understanding, and that other avenue can be important for the very detailed comprehension.

That said, sometimes, Open Source (and, to a lesser extent, commercial products) lack the clear documentation that make getting an initial understanding easy. This can be just as damaging.

great post.

I have the drive to understand, so I know what you mean, but my question is, where is the reasonable end-line of understanding and the beginning of big picture abstractions?

Is it the programmer's layer where they could have solved their issues by writing automated unit tests and following a Test Driven Design approach?

Is it the object itself where a competent framework programmer should have written automated unit tests and overriden hashCode() if they've overriden equals()? The underlying Java compiler and byte code? operating system calls? computer architecture? logic design? transistors? VLSI design? Understanding of p substrates and n substrates? Understanding of electrons? quantum theory?

While I have some conceptual understanding of all of these layers, there is too much specialization for me to be an expert in all but maybe 2-3 of them. Especially with all of the layers above the "physics" layer changing so frequently, and our knowledge of the physics layer constantly changing.

I think you should be a chef, but follow superior recipes. Winging all your dishes will get you fired pretty quickly.

I wholeheartedly agree with your article, however your inclusion of the Tao confuses me. Taoism is all about accepting things as they are and not trying to define things (as once you've defined something you've limited it) and that you don't need to understand how it works in order to appreciate it. Confucianism, on the other hand, focuses on the understanding of everything.

One example where you need to understand a system clear down to the metal is when you're pushing performance. "This API needs to be structured such-and-such a way because the Flash memory we're using at the bottom has specific timing and bandwidth issues." Or modern video game consoles: You have to understand the caching architecture and the details of the GPU or you're not going to get the frame-rate you need.

I disagree with "You have to have open source." Mostly that's a nice-to-have. The down-side of having a completely exposed system is that you might -- conciously or unconciously -- couple to stuff beneath abstractions, making it much more difficult for the system to change in the future. This was rife on the first versions of the Macintosh, where programs depended on low-memory system globals that made compatibility a nightmare for the Apple folks (and for the companies whose software kept breaking) for the next decade. Sometimes "open" is bad for you.

RE: Mike's comment -

While preference for open-source may not make or break a 'good' programmer, I have met very few good programmers who use strictly closed-source tools. As the post argues, an effective programmer is inquisitive and constantly looking to improve his or her craft. The visible nature of the open-source development ecosystem fosters these attributes whereas closed-source tools have a tendency to spoon-feed and obscure technical merit with marketing.

I absolutely agree with you because I'm that kind of programmer. Unfortunately, it slows me down at times because I have to stop and do research. It's a good day if I write only two lines of code but know everything there is to know about those two lines.

The open-source question is very interesting in this context. I've been developing for quite a while now, and while I probably wouldn't argue that ALL 'good programmers are always looking for open-source alternatives', I would argue that most good programmers gravitate towards open-source tools and software. They may not use them in their job all of the time, but they usually end up using them when they can. This isn't because you cannot be a a good programmer using only closed-source software, because of course you can. It's because of curiosity. If you're a person who is extra-curious, then you'll probably become a better programmer, and you'll probably also enjoy open source (you can satisfy your curiosity down to bare metal). The two don't necessarily cause each other, but they often go hand-in-hand. The worst programmers I know are not curious at all, never try to understand anything beyond what is required to do a particular job 'well enough', and don't get this 'open-source' thing at all.

As a programmer, I choose open source more often for cost, platform support and long term stability purposes. For instance, I'll probably never look at Ruby, Apache or MySQL source code, but I do have some faith that those projects will be around for the long haul, and will continue to pursue their stated goals rather than being shelved for marketing/profitability concerns. If I build something on top of these tools I feel fairly confident that it will be able to run and be supported as long as it needs to be.

The open source tools I use for the purpose of actually looking at the source tend to be higher level stuff. Web frameworks for instance. These are more likely to contain bugs and maybe even need a little coercion beneath the abstraction in some cases as a result of greater specificity and fewer eyeballs.

Then there's a whole slew of tools for which I don't care at all. Some programmers prefer Linux because they love the ability to tinker. But personally I prefer OS X because I just want the OS to work and let me focus on what I'm interested in. I use Photoshop instead of the GIMP because I just want to push pixels as efficiently as possible.

I agree the need to understand is a prerequisite for being a good programmer. However to be a great programmer I think you need to know when you've understood enough and how to accomplish your goals balanced against future maintenance needs. To be a truly legendary programmer I think you need a vision and knack for the architecture of systems... eg. what's most important? how can the problem best be characterized? what's the simplest architecture that meets this set of needs robustly? how might the system grow in the future?

"Of course anyone can write buggy code, unless you are NASA and can afford to spend tens of millions on a rigid programming methodology"

Unfortunately, NASA writes buggy code as well. For example:

http://www.ksc.nasa.gov/facts/faq08.html

-Buford Twain

I'm not sure just how useful Open Source is to an inquisitive programmer. I don't consider myself an idiot in terms of programming, but trying to find the code that affects what I'm doing in some of the Open Source frameworks and libraries I use can be close to maddening. If anyone here has tried to figure out how the Ruby on Rails association functionality works you might have an idea of what I'm talking about.

I think there is also an issue of understanding the business being automated. I am not sure that the kind of deep, technical understanding you (rightly) value is compatible with also understanding the business and being able to talk to the business people who run it. And if programmers must pick between the two, how can we make sure the business logic we code into the sysem matches the business needs?

JT

http://www.edmblog.com

The need to understand is one trait of a good programmer. It needs to be balanced by an instinct for when digging deeper becomes detrimental to solving the problem at hand, and when to trust that the slightly higher level apis available work the way they claim. Or attempt to work around their peculiarities. For example, If You were tasked with building a simple, intranet frontend to a database in the 90's, using then available open source tools instead of 'shitty, bad programmer' Visual Basic would rarely be a sign of programming genius. Also, good and plentiful documentation, and an active and literate user community can make up for lack of source in many cases. Good programmers can be distinguished by the quality of their programs, not how they went about writing them.

@Stuki - I was involved with a couple of projects that requested just that sometime in the (late) 90's. And had a few others of the sort in the organization I worked for. And you could see a lot of difference between the applications of people who wanted to just get it done, using data-bound controls in VB or tools like Power Builder, and between the people who had some clue how databases work, how to write SQL, or how to create UI that fitted the user's needs better than what came bundled with the controls.

The point wasn't "open source - good, closed source - bad". It was that a tendency to check open-source solutions is good, because it shows interest in knowing how things work internally, so that you'd know what happens behind the scenes, and have a clue what to do when it doesn't work quite as it should.

@Student - Confucius didn't try to understand, and certainly didn't try to get anyone else to understand. Just to follow a set of rules. He defined how things are, how people should act, sometimes being quite vague of course (So he did expect understanding from his student, but only of the points he was trying to make, not anything else) and that is that. As a philosophy, it's practically the opposite of trying to understand. "These are the APIs, these are the docs, all you need to do is use the APIs according to what the docs say" would be how Confucius would teach programming. Though of course he'd write docs which are not well defined. ;)

Have you seen Alistair Cockburn's abstract titled "Unknowable and Incommunicable"? http://alistair.cockburn.us/index.php/ASDbookextract:"Unknowableand_incommunicable" It's more about the nature of communicating knowledge, which is another crucial part of good programming, unless you're willing to rely on your programmer outliving his/her code.

I agree with the statement that good programmers look for open-source projects. In the event of a problem the programmer can "do something about the problem". Possibly even apply a patch, rebuild and give it back to the community. In this case everyone benefits. The good programmer doesn't have to fight through the red-tape (to get a patch) and other people benefit from his/her willingness to debug someone else's code.

So the "good programmer/open source" comment is about taking ownership of a problem and diving into it as deep you can go.

I agree with mike #1. If the commercial alternative is better than the open source equivalent, then use the commercial alternative. Not all software should be open source.