Separate the Conceptual Work from the Contextual Work

by Tan Yew-Wei on September 3, 2010

(Apologies for the developer-centricness of this post)

“Go out and get your hands dirty!”, “The best way to learn programming is to program!”, “Just build something!”.

This advice is often directed towards people who have yet to ship a product or service. To some, it’s perfect advice. To others, it may be a mistake.

Concept vs Execution, and that Big Gap in between

I think the main factor is where you are on the concept-execution-continuum (CEC). Specifically, the advice above is gold for those right at the gap.

This is because conceptual work is inherently foreign, and is learnt through the process of gradual osmosis. On the other hand, contextual work means applying what you know to a current problem; it assumes that you have the knowledge to do so.

Fortunately, conceptual understanding is oftentimes accelerated by actually having an example/problem to work with.

To put things in more concrete terms, someone who is starting out with Git will have to understand how Git branches and merges work instead of trying to endlessly commit to the master repo.

The issue is that a lot of the time, no one is going to tell you that you’re doing it wrong. I’m just as guilty as anyone else of picking up idiosyncratic bad habits that fail to go away. Taking the time to consult with someone more experienced usually trumps running in headlong yourself.

However, there are some areas which demand a thorough conceptual understanding before even beginning to delve further.

One such example is writing concise Mathematical proofs. The first introduction to an epsilon-delta proof often leads to mass confusion. Even if you can graphically represent the epsilon-delta definition of the limit, there’s this huge gap between seeing that image in your head and writing down the proof to a new problem.

When it comes to complex software design, we are faced with exactly this scenario. Like it or not, you can’t just dive into shell scripting, you have to conceptually “get it”, which only comes after many months of acquiring the necessary technical background.

Bradford Cross described it best in his post ‘Learning About Machine Learning – 2nd Edition’ , where he says,

“The caveat is that you need to be honest with yourself about when you encounter something that you do not understand and you are unable to work through despite trying your best. At this point you’ve found your way to some material that you do not have the foundation for. Put the book down, insert a bookmark. Close the book. Walk away. Go find a resource to build the foundation. Come back later. Repeat.”

Why Bother WIth all this rambling anyway?

If you intend to make a mark with your software at a large scale, you’re bound to run into complex issues that require deep technical understanding of your domain.

Some people have gotten there simply by trying and failing, but more often than not, these people are outliers. There are some people who just “get it” when they see a squiggly mathematical equation. And more often than not, this affinity for the task leads to a propensity to put the hours in.

In short, these people are smarter than most of us and have put in way more hours to their work than most of us.

The rest of us need a strategy. Specifically, we need the patience to let ideas sink in, while constantly striving to understand the fundamentals. Like it or not, the slowest way initially (grokking the fundamentals) is the fastest way eventually.

The Brain Keeps on Working

The surprising thing is, you may be further along the conceptual line than you think.

For those who have struggled with math, I’m sure you’re familiar with the feeling of frustration, when you’re just unable to see why and how these squiggly formulas work together to form a solution. Countless numbers of examples and clarification don’t help either.

That’s were most people (and sadly, teachers) give up. But some people continue reading and digesting the material. They may not spend a lot of time with it, but it’s certainly not being relegated to the bottom of their thinking stack.

Then a couple of days later, they see another example of the formulas in action, and suddenly, everything clicks.

The brain works 24/7 on problems higher up on the thought heirachy. The trick is then to constantly ensure that the important concepts retain that high-up position.

It usually can’ be Rushed

Trying to accelerate this process seems to suck in general. Rushing into epsilon-delta proofs yields sloppy assumptions and procedural memorisation; you know what you’re doing, but you don’t know why.

That’s not too big of an issue for passing math exams, but when it comes to building complex software, you need to start off on the right track if you’re intent on staying sane through the inevitable debugging sessions and server failures.

So what should I do now?

I still think that we should jump on opportunities when they present themselves. But I also think that we should have the discipline to see how complex a problem truly is, and have the professionalism to back away when we know something’s temporarily out of our reach.

So start on what you know. But always bear in mind the conceptual gap and slowly attempt to close it.

Final Words

Those who succeed in executing a successful product or service are either close to, or beyond that concept-execution gap. They have a good grasp of their tools, and oftentimes a good dose of domain expertise as well.

Developers often take for granted how long it took them to master their toolset. Everything from mastering emacs (or vim, but not textmate), to understanding the flow of git, to recognising weaknesses and integration in APIs, is necessary to write good code and ship good products.

That said, we’ve seen many people write sloppy code while still being able to please their customers. They usually pay for it with a high level of stress, but they suck it up, and arrive on the other side wiser. For some people, this may be the preferable, and maybe even necessary path (especially if money is really tight).

And then there are people like Marcus Frind, whose understanding of his server architecture has led to some astounding scalability miracles at Plenty of Fish.

As a personal preference, I’d like to be in that latter crop of developers, and so, I’m biding my time to learn the ropes while tinkering with what I can. Though I’ve got nothing yet to show, I certain have my covert ops waiting to be unleashed!

Get the PDF or the Print to view the concepts more clearly!

{ 0 comments }

Diamond on the Horizon

by Tan Yew-Wei on September 2, 2010

Although I love the digital format of most of my scribbles, I still do many of them with pen and paper. Most don’t get published, but all are loved.

There’s something about pen and paper that we still haven’t managed to mimic with a graphic tablet. Wacom makes some pretty dam good ones, complete with changeable tips, etc. However, I’m still attracted to the simplicity of pen and paper.

It’s a different kind of simplicity though. Not the shallow type of human-invented simplicity. Rather, it’s the kind of simplicity derived from an underlying complexity, as all natural things are. I think this leads to a subtle acceptance that things are just the way they are.

In any case, scribbling down drawings with pen and paper makes ideas flow. For some reason, I’ve never be able to mimic the complexity of my random patterns on the graphic tablet.

This leads to a different kind of art, one that was inherently purposeless, at least to the conscious mind. In my digital drawings, I usually have a purpose in mind before the first stroke is drawn. Not so with pen and paper, and evidently, I have as much clue as to what I’m drawing as my viewers. (ie: none)

So that’s a long way of saying that the picture above was formed spontaneously. I don’t know what I was drawing, and I don’t know why I drew it. But it sure was fun.

More importantly, I’ve now got the ‘permission’ to give meaning to my work just like we irrational humans always do. So until next time, I’ll be searching for that diamond on the horizon.

{ 0 comments }

Why The Amazon Kindle Rocks For Programmers

by Tan Yew-Wei on September 1, 2010

Although I work with code and stay abreast of all the latest tech news, I don’t believe in spending on gadgets. So no iPhone, no fancy computers, etc. For me, it’s always been a laptop (MacBook Fall 2008), a Sony Ericsson w880, and good old pen and paper.

However, just 2 days ago, I added a new item to that inventory: An Amazon Kindle 3.

Honestly, I didn’t know what to expect, and hence I placed my expectations low. I’d read prior reviews about how great the screen was – no eye strain, better than iPad, etc, etc. However, I paid much more attention to the reviews from fellow programmers, and the message was clear: The Kindle sucks for Technical books.

However, the low price point really got to me, and so I thought I’d give it a legitimate shot.

Initial Reaction

First, I’ve got to praise Amazon for their quick delivery. I’m in Melbourne Australia, and was given an expected shipping date of September 7th. On Friday, I got the email confirming that the package had been shipped. I got it on my doorstep on Monday afternoon.

The screen is beautiful, and I really thought that the initial welcome screen was a sticker. Despite that, some eye strain was present when I first booted it up for reading. I attribute that to getting used to reading from a computer screen, which I speculate encourages the use of too much peripheral vision as compared to the small Kindle screen.

With the small screen of the Kindle, I felt like my eyes were constantly darting around looking for various other signals, and finding none, they gave up in protest.

Fortunately, I adjusted really quickly, and with adequate light, it’s all great for reading now. (Which is why I’ve got that theory, but it’s all speculation nonetheless)

Trying to Read Technical Books

Some fiddling around with different file formats, and the consensus was still clear – you can’t read SICP on this device.

In short, too much scrolling, tiny text, and the lack of easy navigation doesn’t suit the non-linear reading fashion of technical texts.

Why it’s useful

First, at the time of this writing, I’m a college freshman. Reading lecture notes and slides on this thing is wonderful. Basically, any short-form summary or narrative information is presented very well on the device. You must say that having all your lecture notes handy is a great plus.

Next, the 2 column layout of most scientific papers fits just right for zooming in and reading. (click to enlarge)

But most surprising to me, is that all of the sudden, programming cheat sheets became A LOT more useful.

Holding a Program in One’s Head

Once a project gets up to speed, a programmer knows how the structure of the program flows. The problem is that during the act of programming, one achieves a state of flow, that is too easily interrupted by grokking through an unwieldy stack of poorly-written documentation.

That’s where cheat sheets come in. Cheat sheets are great because like ideas a notebook, each reference item is surrounded by a whole set of related material (assuming the cheat sheet is good). Oftentimes, I’d be looking at a cheat sheet item, and then seeming serendipitously catching another related item in the corner of my eye. Everything clicks, and my program flows again.

The great thing about having it on a Kindle of course, is that it’s portable. That means that you can conduct code reviews with people having their own cheat sheet which they are familiar with.

It also helps with picking up some fickle languages (R), getting to know some libraries, as well as getting up to speed with feature-rich text editors (VIM image).

So why the Kindle versus some other portable reader?

I can see myself doing the exact same thing with an iPhone or iPad. That’s surely a viable solution, and if you own one of those devices, then exploit it.

But of course, the Kindle also doubles down as a great reader of the subset of all books \ {technical books}, so that could be a plus too.

But probably more importantly, the Kindle looks and feels like a book, and thus you treat it like one. Ergo, you treat the content on it having a more everlasting quality compared to that on an iPhone. Hence, it’s easier to give the content the due respect, and utilise it fully. In more succinct terms: Placebo effect FTW!

{ 2 comments }

Travel is most fun when…

by Tan Yew-Wei on August 31, 2010

Travel is fun when you’ve got a destination, have no idea how to get there, but know that you’re going to get there anyway.

It’s equally fun when you’ve got a map, and simply decide to wander.

It’s less fun when you’ve got both a destination and a map, because then you’ve got an artificial experience. And by definition, such an experience takes away (or severely diminishes) the best feature of good travel - Serendipity.

{ 0 comments }

Decide Where You Call Home

by Tan Yew-Wei on August 30, 2010

Home is where the heart is, and you better hope your heart is free.

Many have died for the love of their country, and we honour them for that deed. But love is a two way street, and one should always remember that’s it’s always going to be harder for your country to love you.

We should look to history for important lessons, none more poignant than the 1930s Europe. 1933 saw Hitler win German hearts. By 1935, the parading troops should have spelled a warning. Some people took it seriously, and fled while they could. Those who were trapped in the hell that followed were those who either had not the means to leave, had not the foresight to leave, or were too stubborn to leave.

Don’t be one of those people.

Get the PDF or the Print

{ 0 comments }

Slave To His Heroes

by Tan Yew-Wei on August 27, 2010

We all have heroes. These are people whom we admire for a certain skill set, or for a particular personality, or for no explicable reason at all.

The problem is when admiration devolves to worship, and one puts one’s heroes onto a shiny podium atop a mental high ground.

I’ve seen people defend their heroes harder than they’d defend their parents; the line between their heroes and themselves has become blurred.

The sad part is that in doing so, such people lose sight of themselves, and are stuck trying to become their heroes instead of becoming themselves.

I think it pays to keep a certain distance from your heroes. Know that from a distance, your heroes look perfect, but face-to-face, their just as human as you.

Work hard, and keep looking up to your heroes from a distance. Soon enough, I’m sure we’ll all get to the same level as our heroes, and when we finally meet them, we can talk like humans do.

Get the PDF or the print to see his heroes (and maybe your’s too!)

{ 0 comments }

Stupidity Reigns, but Rationality Prevails

by Tan Yew-Wei on August 26, 2010

Yesterday, I talked about how education in most parts of the world is screwed. At the end of that, I said that we should do as Mark Twain says [1], and take charge of our education.

Of course, the education system is just a subset of a system in general, characterised by a large number of inter-dependencies.

Hence, what we know is that when something stupid gets into the system, it’s going to take a long time to get out.

The bad part, is that it’s some people’s job to screw up the system. For example, University administrators have both a moral responsibility to bring higher education to the masses, as well as the professional responsibility to keep the gears greased and running smoothly. ie: School need to be profitable, and as a professional, you’re there to get the job done, not screw around with morals.

So through a combination of bad luck, bad design, and sometimes, bad people, we’re stuck with many bad systems. The good news is that rationality prevails, and the system will correct itself eventually. As part of the cycles of natural selection, the education system will have to be updated to the times, with some inefficiencies removed.

Heck, the transition to digital everything (including textbooks) is just beginning. And yeah, such systems move slowly, but it was only 6 years ago that schools had never heard of anything like an online portal, which is now basically ubiquitous.

Two key arguments to be drawn from this:

  1. Once something becomes profitable, we’ll do it whether it’s stupid or smart
  2. When profitability is aligned with actually getting things right, the system is improved, but only for the average

What this is saying is that those who are already ahead in the system will continue to be ahead, with the converse sadly true. Mark Twain was lamenting about it since his time, and we’re still lamenting about it now.

I’m betting on education fixing itself, in the same way I’m betting on a technological Singularity – change will come shockingly fast, and you want to be in a position where you can change easily, while continuing to live and plan for the present. Key word: Robustness

Until then, I guess we’ll just have to do our own thing and succeed by our means. Which is a very great thing indeed.

Notes:

[1] “I never let my schooling interfere with my education.” – Mark Twain (click to get back to text)

{ 0 comments }