Friday, May 28, 2010

Chefs, Curators, & Developers

Besides that these people all eat, sleep, and lineup for the latest iStuff, developers ought to have another thing in common with chefs and curators. I would say software development is part engineering/science and part creativity, even as high 50/50 or maybe a little more.


The term "custodian of a collection" doesn't describe what curators really do. It almost makes it sound as if they just take care of a collection, but not necessarily care for it. A museum without a curator simply turns into a warehouse. It is the curators job to prevent that from happening. To care for the collection and keep an eye at the big picture.
Every piece, big or small, serves a purpose and delivers an essential part of the experience. The opposite of a curator is a hoarder.


Similarly the chef does not just cook the food, they keep an eye on how the items on their menu works together, how the dishes enhance the experience, and how every ingredient in a dish works with the other ingredients. Every detail counts, and great head chefs will look after it all. They are food curators.
Musuems, food, and software is made better by what has been left out - purposely - and not what's included.

How do they do it?

Great museums don't happen overnight, they happen over time, time spent doing the same thing - improving the museum. The task is never done. It's iterative, and perpetual. Curators aren't afraid of tossing things out because they don't work, or something better came along. Curators are careful not to hoard shit, even if it's all good shit. They don't go about it alone, they get feedback from other curators, from their clients, and finally their gut & others' guts. Your unconscious brain has a lot to do with this task, sometimes it is hard to explain, but often your gut is on target.

Lessons from Chef Ramsay

The number one thing Gordon does to fix a failing restaurant is "cut the crap" - after he is done ripping through the owners of course. If you watch the show, you probably have seen him throw out half the menu - if not all of it in some cases. Take the episode of the "Curry Lounge", tons
of curry based dishes. None of the waiters guessed what any dish was during the blind fold taste test - except for the french fries; at an authentic Indian restaurant. Chef Ramsay tweaks and iterates over the food, how it's made and served over the course of the week. That's his formula. That is why he is a great head chef.

Don't be scared of "No"

There are many good reasons for saying "No". There are also wrong times to say "No". Say you are at a restaurant, and you ask the chef to skip the pistachio on your baked chicken because of allergies. That is a valid request you cannot say "No" to. On the other hand, don't be surprised if the chef says "No" while at an authentic Italian pizzeria and ask for extra cheese on your Pizza Terreno - he knows better than you. A good way to say "No" is to always provide reasons and alternatives. Most clients will consider alternatives, after all they are paying you to provide the best possible service you can afford them. Sadly, you don't always win, or you are just wrong and you lose the battle and have to provide exactly what you have been asked.

"But that is not how we do things around here"

There is always room for improvement, and there is always areas and times where you can play the role of the curator at what you do. Always start with the things you control. What if you work at a hypothetical restaurant that followed the Waterfall model? retarded right? but at every point in the process a curator could help, and a curator can make the experience better - even a little. This Waterfall restaurant basically takes orders at 5pm, by the time they are validated and cooking starts at 8pm. Then the dishes get checked, fixed and reheated. Food starts to come out at midnight. And into the wee hours of the morning the cooks have gone mad trying to
figure out what was ordered eight hours ago. Every single role in this kitchen can play the curator in what they are doing. As badly suited the experience is, it's much worse if everybody minds their own business and blindly follow the blind.

Kitchens get dirty one dish at a time

"Too many chefs spoil the soup" True, and applies everywhere. Why else do the Marines and special commandos operate in small, tightly knit groups? Do you seriously believe that anything will need an army of develop
ers yet the army elites can perform heroic missions with a team of 15?
Don't let your kitchen get dirty, it's easy to say you will do it later, but know that your effort to clean it does not have a linear relationship with how long you left it dirty.

Everybody can be a curator

Whatever business you are in, you are involved in curating and improving the product/service you provide. Whether you design buildings, write software, an author, or a sales guy, a president, or an army general. We all do it. We all know that the first draft of an essay is always the worst. When you walk into a board room to make a pitch, how many times do you revise and tweak that slide deck? how many times do you do it the night before? or even 30 minutes prior? We all do it.

So here is the big question, when did we say that code should be written once and done with? What tends to happen with development teams is this task is not performed as often as it should - specially as the team size grows. It gets worse when dates start to slip, as things like continuous improvement and cleaning up the kitchen are ditched. Here is the other problem, managers, executives, planners, etc. don't like to see tasks that don't have end dates on the plan. They want to know things like effort spent, time elapsed and percentages for each. An iterative task only has elapsed time, but nothing to identify how much is left because it's never complete. The affects of this curating task are phenomenal. Almost always you will be amazed by how little you changed or even better removed and how substantial the improvement was.

Making something better is never easy

...but the harder the decision, the better shape that thing is. To be successful at being a curator you really need to understand what your customer wants. Not just a little, you need to get in their head and figure shit out. You need to figure out how you can add value, or if you just can't. Value add is highly dependent on the context. If I am in a rush, fast food will be the best value for me at that time. I'm hungry, and need a quick solution. On the other hand, you wouldn't take your wife there on your 10th anniversary. The best way to bring yourself to a point where you can understand how you can add value, is by aiming for the simplicity on the other side of complexity. If you are looking for a good healthy relationship, you need to invest the time up front, you need to climb that complexity hill, sometimes it's steep, sometimes it's long, but rest assured it will flatten out after the peak. Thats where you want to be. Not many people make it to the other side, most will be hanging around at the base or on the way up. That region is highly competitive. The elite don't hang out there. Take Apple for example. We can consider Apple to be one of the elites of the tech industry - almost like they have the Midas touch. People have tried to replicate the iPod with little success. This week, slowly but surely Apple wrestled the 2nd highest market cap from Microsoft - without even dominating the market. They know their customers, and they don't even want everyone to be their customer.

Everything looks great on paper, it's not until it comes to life you start spotting glitches and things to improve. Good restaurants will shutdown for a day occasionally and just cook the whole menu and their staff will critique everything and make adjustments. Iteration is key. In school you are taught to review your essays, read them out loud, tweak and adjust the paragraphs, shuffle things around, all of this to find the essence and flow of what it is you are trying to accomplish.

There are only two kinds of developers.

Those that just get shit done - literally -, and those that will get it done very well. From a time line point of view, you won't be able to tell the difference. They both finish on time, and their stuff will work. However, the first, don't care about what it is they are building, don't care about who they are building it with, they put in their hours, get paid and that is it. Excellent developers don't have to be geniuses or come with Masters & Phds. These are the ones that want to be proud of what it is they are building. They are the ones that think several steps ahead, and design for change. They will embrace change if it makes what they are building better. They will try to resist change that adds no value. They are the ones that keep an eye on the big picture. Those are the ones that will invest the extra time to re-factor the redundant classes they spotted. They are the ones that experiment and try stuff out because they are always on the look out for a better option. They are the ones that will tell you "No, you shouldn't do that". That is how you separate the two. An excellent developer will wave his hands and yell when something is just not right. The first kind will shrug their shoulders and pile on the dishes.

Monday, May 24, 2010

The Mythical Man-Month

I think it was in first year software engineering that we had to read this book, and nine years later I really, really understand the underlying purpose of this book. It may just be yet another book back then, but the lessons that are hopefully learned from it will last a life-time - not just for software projects, but any project.

The Mythical Man-Month

Unfortunately on software project plans, developers, designers, testers, business analysts, product managers, etc. etc. are considered "just another resource" that are added and removed off of tasks. The assumption is that all are equally effective and skilled in all the required domains, and all will produce the same volume and quality. So in a perfect world it makes sense to scale the team to meet deadlines, although we don't live in a perfect and linear world, this is still the method of choice. Even though the biggest effect of this method; drastically increased non-linear communication time is widely known but mostly ignored.

Sadly this is the state of this industry, project plans that are too often disconnected from reality. I think part of the problem is driven by dividing tasks into a unit of time, after all that is how budgets are built, teams are put together, and progress is tracked. However on the other hand, this unit of time does not measure the real size of the task. Its just an illusion. Its like a building, we don't measure building size in number of months it took to build, we measure it by number of floors or in meters i.e. something relevant and real. If a 40 meter building was estimated to take 12 months, and in 6 months we are at 10 meters, then we are 25% done, and not 50%. However if this building were a software project its assumed we are 50% complete. Then at 10 months we realize we won't meet the deadline, forget about the Mythical Man Month and scale up. Why did this happen? Because software does not have a realistic metric, software is abstract.

Some say you get better at estimating over time, but that too assumes we live in a linear world. Ex. Project X took us 3 months, so we will estimate that project Y will take 9 months. We don't live in a linear world, and humans aren't good at estimating non-linear stuff. We may think that the second project is 3x as long as the first, but it could be x^2. However the hope is that over time you can make a non-linear project more linear by improving the processes for the non linear components. That effort is also non linear.

You can try to measure by team size, effort, or lines of code, etc. but all are just an illusion of measurement, none are real. Whether you measure buildings by floors or meters, you can translate between both. On the other hand, you can't translate lines of code into time, effort or team size.

I don't know what a better alternative is, but surely it is not this. Perhaps the problem is just trying to estimate that far into the future with too many unknown variables. Feel free to comment.

I recently read the book "ReWork" by the guys at 37Signals and one paragraph I absolutely loved has to do with project estimation.

The book is a must read.