Portrait of a Learning Curve

There are a number of things that don't come naturally to me, but let me tell you — being introverted is not one of them. I've faced a battery of Meyers-Briggs personality tests over the years, and am proud to say I consistently demolish my competition on the "happier-alone-than-at-party" questions. Whatever my errant phases, I inevitably return to being the kind of fellow who finds camping alone in a desert to be a great use of vacation time.

Well, here’s one thing about how us shy folks behave: we always choose DIY approaches over dealing with other people, no matter how much it needlessly bolsters our workload, whenever and wherever we can.

For years I had no idea why this was, until it hit me that I kept score of professional life exclusively by tallying skills learned, instead of relationships created. When I spent time interacting with people, I viewed it as time I could be spending alone “getting ahead” by learning new things. Extroverts (as far as I could tell) saw it the other way around; in their mind’s eye they are building a network of human connections upon which they can then traverse to reach previously unreachable places.

Because of this, I regularly spent time on activities that could be learned and practiced without the aid of others (i.e. playing guitar > playing racketball). Alone-time learning always felt more productive, regardless of the realities behind it.

Wikipedia + Me: A Tragic Love Story

Needless to say, this is how and why I learned to code. Around 2008, I was helping to manage a recording studio and — shortly after — was running the online division of a music magazine. Within these roles, the need to implement some new digital thing (website, email marketing, etc) would sometimes arise. I would uncomfortably survey my options. I knew the simplest answer was to hire someone, yet blanched at the prospect of interacting w some strange outside expert. Besides, personality tests declared my need to avoid such a thing was Supported By Science. Much better to embrace its opposite: burn up each day’s potential leisure hours tunneling through wikipedia computer science links until I achieved existential crisis!

Me: “How do I programmatically make text display on a screen?” [click]
Wiki: “Text is composed of strings.”
Me: “What is a string?” [click]
Wiki: “A string is often a type of literal.”
Me: “Okay… what is a literal?” [click]
Wiki: “A literal is notation to represent a fixed value.”
Me: “Value? Like money? …” [click]
Wiki: “The members of a type are the values of that type.”
Me: “That… type…?” [click]
Wiki: “Data types are used within type systems, which offer various ways of defining, implementing and using them.”
Me: “All things are devoid of meaning.”

Despite the inefficacy of these dialogues, a roughly effective approach to handling technical tasks was beginning to emerge for me: if I locked myself in a room with the problem at hand for a sufficiently lengthy and vociferous period of time, I would emerge miraculously with a thing that worked. Even if I couldn’t say precisely how it worked. This latter fact did not alarm me. I had previously worked as a music producer, and in the arts, it didn’t matter how something worked; just that it did. If a song made you feel something, the case was closed. I had no reason to believe coding to be any different.

And, I got by on this methodology for quite a while. I built blogs and e-commerce web apps and photographer portfolios and restaurant sites. I egregiously undercharged. I did not know what a “best practice” was. I put all my code in one file, top to bottom. I didn’t use versioning. I used unnecessarily verbose variable names. I worked directly on the server without a local copy. I always worked alone. But, the products always had just the right tone and vibe, and my clients felt understood.

This part came easily enough: my decade in music production had been built upon successful interpretation of the visions of others, and was a well-oiled habit. But were there other pieces to this puzzle? If there were, I assumed they weren’t important.

“Real” Engineers

Somewhere around the end of 2014, I decided I didn’t want to build websites anymore.

I wanted to build things that, like, really did things. No more cute, meticulous little blogs for newly retired folks with pent-up fantasies of becoming famous thought leaders who end up only writing one and a half posts. Platform wasn’t important, but I wanted to build apps. You know, like a real programmer.

So, I set my intention and put out the word. No more front end thingies for individuals. I would work exclusively for companies. I would work exclusively on serious stuff. I started turning down smaller things, and took up hot yoga.

This — for whatever reason (the California hippie in me openly suspects setting my intention) — worked, and worked rapidly. Within a few weeks I was contacted and scooped up by Michael Shores — a real swiss army knife of a person, with a background in pure math, philosophy, design, and deep programming — who set me to work helping to automate back end workflows for centerforjobs.org, a data-viz app that lets users rapidly compare economic indicators across California geographic divisions (note: they do this for all 50 states now and it is pretty badass).

This all went shockingly smoothly, mainly because Michael kept my tasks manageable and perfectly bite-sized. Ten hours of estimated work was ten hours of work. Hell yeah. I was engineering things. Emboldened, I asked the universe for something bigger and harder. I upped my yoga frequency to six times a week. Code could solve anything. Time to save the world.

The Everything Machine

Whatever allegiances I now have toward rationality and skeptical inquiry, I have to begrudgingly admit one thing as I write this — whenever I “asked the universe” for something, it damn well seemed to deliver. And deliver fast.

In early 2015, pretty much right after I submitted my give me something bigger and harder request via what I was told was my crown chakra, I was approached with an impressively ambitious project. The project had two arms. First, a conventional one: build a modern online publishing platform that was pitch-perfect for the client’s brand. Second, the ambitious one: build an app that pulled in every piece of public data in California that touched state politics, and connect all of it together canonically. Then, build an interface for users to ask complex questions and get back concise, relevant tabular results.

The idea was that stories published on the first could be driven by data from the second. For example, a one could ask something like:

Show me campaign contributions from the beverage industry in 2015 to senate candidates that previously voted against bill SB 203.

It was this second one that was presented for me to spearhead and, to be honest, I salivated at it. Was such a thing feasible? Of course it was! Anything was possible, with grit and belief. One put oneself in impossible situations, which forced one to shatter perceived internal barriers and rise to the challenge. After all, the seed must split open in order for the plant to grow.

Yes, these are the kind of spontaneous thoughts churned out by brains doused in 90 minutes of daily yoga.

Loosely Vedic aphorisms aside, my operating principle at the time was that you got hired by saying yes, then worried about mechanics later. In fact, I would go further: it was my job to say yes. Producers produce things. Developers develop things. This meant making things happen, not shooting things down. Plus, I didn’t see what was so insane about this idea — it just seemed… big. Involved. Extensive. But not impossible. Not ill-advised.

My operating principle at the time was that you got hired by saying yes, then worried about mechanics later. In fact, it was my job to say yes.

If my mind had such a thing as a feasibility analysis engine, it was trained on art and athletics. Mulling over this idea of ramping up to tackling something huge and new, I thought about my recent first half-marathon. The race was three times as long as earlier 5K runs I had done, but despite concerns over my super-rickety knees, I had completed it. During training, the slope of difficulty was sort of a gentle log(n) — the more miles I added, the less each new one seemed to matter. The race itself had been sublime and exhilarating.

From that vantage point, it seemed to me everything could be achieved with proper amounts of pacing and passion. The notions of runaway complexity, of combinatorial explosions of test cases, of the fact that jogging in a straight line from point A to B was the exact opposite of what I was about to attempt, were all summarily ignored.

Suffice to say, this would be the last time in my life I neglected such things.

Rubber Hits the Road

Over the next four months, my teammates and I gradually, doggedly worked up a version of this thing. Also, as best as we were able, we committed absolutely and fully to doing everything right. And, we felt like we were doing everything right.

We chose modern, sensible technologies: python and pandas for the data science, PostgreSQL to hold it all, Django to orchestrate and deploy it on the web. We were in communication constantly. We traded volleys of technical emails about the smartest architectures, the savviest paths forward on all the particulars. We dug into the latest in natural language processing. I personally unraveled, sussed out, and carefully reassembled a dozen different California public data sources into clearer, more complete versions of themselves. We worked perpetually, and really, really hard.

We wanted this thing to happen. We wanted to manifest our client’s vision in the world. We were genuinely driven toward it, beyond all distraction. Yet despite all of that, despite all the grit, passion, and above-and-beyond-ness I could personally squeeze out of myself, it kept falling short.

Every two weeks or so, we would meet with the client to illustrate our progress and run a demo. These were supposed to be happy meetings. Demos were supposed to be rituals of victory. In previous projects, I’d be stressed and panicked beforehand, but then I’d manage to push a little harder and suddenly I’d burst through. The demo then came and things worked as promised and everyone was happy. I’d breathe a sigh of relief. I’d feel relaxed, satisfied, filled with that movie-ending, just-by-the-skin-of-our-teeth feeling. I looooved that feeling.

These demos were not like that.

The successful instances of these demos involved the app limping its way slowly up to the lowest settable performance bar, and displaying some underwhelming and occasionally confusing result. Success meant that nothing overtly exploded in front of already tense stakeholders. It meant whatever elephants might be evident to everyone in the room, we could maintain a veneer of client-agency normalcy.

The failed instances, on the other hand, involved the app freezing or crashing while asking it to do pretty much anything. Failure meant our poor project manager — pretty much as deft and sharp and diplomatic as they come — had to scramble on the fly for narratives that framed these events as completely normal on the Path to Assured Victory. Oy.

It bears mentioning that the success version here was the decided outlier.

The Finest Teacher

In the end, the publishing platform bit launched on time, and actually launched beautifully. And, upon delivery date, the app did operate in the manner we had all agreed upon. However, it was clear the client had lost faith — gradually but profoundly — along the way, and had internally pivoted in a more traditional direction. As it played out, this direction worked, and they flourished, but with the Everything Machine pretty much abandoned. Despite the arguably happy ending, it was my first visceral encounter with a true technical failure. What the hell happened? We worked so hard. We proceeded so meticulously. How did we do everything right but end up here?

Well, the answer was that we didn’t do everything right. We left no decision within the project unexamined, while ignoring whether the project itself was the correct one. We doted on every micro while ignoring the macro.

Failure hurts, but is the finest of teachers. The experience taught me three crucial things:

1. Part of “making things happen” is sometimes asserting that things can’t happen. This means tread carefully, even at the highest levels. Don’t necessarily take the foundational tenets of the project for granted. We only have so many professional hours on this earth — the most crucial decisions aren’t how to make things happen in the thick of things, but rather which thick-of-things are we most likely to make meaningful things happen in.
2. Adaptability is just as crucial to success as smarts, tenacity, and effort. It was pretty much a complete news flash for me that you can’t just wad up a fat ball of IQ and perseverance, hurl it in an arbitrary direction, and expect reliable results. Especially with any ambitious project, important new essentials about the nature and limits of what you are trying to do will spring up as you go. Sometimes, these new essentials will be complete non-starters. It is up to you to heed these for what they actually are, and adjust accordingly despite the upset it may create.
3. Adhering to an image of perfection for clients will often limit your ability to be adaptable. This is a tough one. A big chunk of what we hire professionals for is to put all that unpleasant stuff like uncertainties and missteps in a black box so we don’t have to deal with it. Yet when things clearly aren’t working behind the scenes, the ability for client and agency to dialogue and re-configure things candidly in crisis moments seems to me the best option for averting future disaster.

Also, #4: Don’t try to build Everything Machines. Remember when you learned that you legitimately can’t please all the people all the time? It’s the same thing, and breaks down in the same way.

These principles inform my professional life to this day. That said, there was a meta insight that I willfully ignored. It would be another couple years of getting the technological snot beat out of me before I finally broke down and accepted the larger truth to which they all pointed.

Autodidacticism: the Sketchy Parts

Recently, I recognized the ghostly outline of these lessons in The Mythical Man-month, possibly the most beautiful book written about the true nature of software development, and required reading for many a CS curriculum. At the time of the Everything Machine, I had not read it. Truth was, I hadn’t yet read a single book about my craft and how to do it properly.

Why? Because like many others choosing the freelance programmer route, I was self-taught, a relentless autodidact, a free-roving iconoclast. Learning to code was a means to live on one’s own island, a castaway life free from society, where one didn’t have to answer to anybody.

So, back to introversion. The fact is that for many of us, writing code first presents itself as a beautiful, dense, endlessly engrossing mechanism to create a professional life free from the need to interact with other people. This is its underlying allure: we cross that enticing bridge of technical knowledge, and then get to live quietly and safely inside our esotericism, without the gut-constricting uncertainties of human relationships.

For me, therein lay the Big Gotcha: learning to code is actually about learning to accept your need for the minds of others. It is stepping off your protected island and admitting: doing this right is extremely hard, I know practically nothing, and the wisdom of other coders is the most powerful, precious commodity available.

It took me several years of coding to recognize this. Yet once embraced, it has made me better at building software than the entirety of my time trudging away in defiant seclusion.

Don’t get me wrong. I love being an introvert. And I love my island. I love my desert. I guard and cherish my solitude like a treasure. But sometimes, I must remind myself that the bustle from all those raucous, roving crowds of extraverts harbors deep wisdom, and — no matter how insular I consider my craft — it is crucial to listen.