Organic software development

When Firefox 3 first came out this year, one of their selling points was "100% organic software" - although they have some justifications to that claim, but to the average person's eyes, it still looks like a shameless attempt to borrow from organic food's brand name.

But really, they have good reasons to use the word "organic" there, because it is deeply embedded into their development process. It's not just a brand name.

Why is it organic?
Associations between "open source" and "organic" can be found back in 2001 - at Horde project's presentation in OSCON that year. The term "organic" back then was used in a much more literal sense than in Mozilla's case - Horde project's code didn't come into existence because the project founder planned it. The whole project - the code, the architecture, the features included and to be included, was grown, like a plant. If you look at academic references, it goes back even further. Back in 1986, Frederick Brook's paper, "No Silver Bullet: Essence and Accidents of Software Engineering" mentioned, "Incremental development--grow, don't build, software."

Horde started with rudimentary features and an architectural mess, but ended up with a large feature set and a well-designed architecture. A significant share of the project's features and designs were implemented or suggested by the external user and developer community, not just the project manager and his team alone. Outside of managing his own team, the project manager is very much just a filter - even in the absence of the project manager, community developers would still write new features and bugfixes - the project manager gets to decide what goes into the trunk and what doesn't. But he cannot decide with total certainty what features to develop and what features to include - the community comes up with new code independently; and rejecting too many of the suggested features (remember, a lot of those suggested features already come with working code patches) will typically cause a fork and rapid loss of market share.

So in the project manager's view, once an open source project grows to a certain scale, it begins to take on a life of itself - thus, organic. While the project team can significantly influence the project's direction, it's the community who sustains it. Even without detailed documentation, development does not stop when the a certain important team member leaves - there's usually more than one person who's been looking at any important part of the code base. And if the core team does a bad enough job - the good part of the code base will be forked to another project where progress can be made (e.g. X.org's fork from XFree86).

How is "organic" implemented?
All that is high level, wishful, stuff, it remains to be explained how organic software development actually works.

At the beginning of a typical organic software project, the project manager, and probably his team (if he has one), implements something simple that "just works". The initial version is a very simple piece of software that may even be very buggy. Why don't they take some more time to come up with a more polished initial version, you may ask? For a few good reasons:

1. Since the project wasn't started with a clearly defined requirement from some customer (often there's no customer at all!), it's not certain what kind of feature the market wants. More features in the initial version would most probably mean more abandoned features later.
   
   
2. As a result of 1, trying to polish the initial version will result in the code base being overengineered - because of the need to accommodate more useless features, the code base would be more abstracted, complicated and harder to understand than necessary. Scaring off community developers and even developers in the core team, and lengthening development time.
   
   
3. Having a working code base (even a simple one) to develop on and test for is easier for developers than coding everything out of abstract UML diagrams and hoping everything would integrate well later.
   
   

You can get the first three advantages by taking an incremental approach, what separates organic apart is point 4:

4. You can attract an external developer community as fast as possible by releasing an early prototype for them to try out and fiddle with.
   
   

Once the software project has a working prototype and a small community of skillful developers and users around it, the rest will happen just like any other open source project - the project team will get a clearer direction from the community's inputs; patches, features and bug requests will be accepted, modified or rejected according to the direction that the project team is taking. Incremental improvements will be made, and when complexity of the code base gets out of hand (in whole or just in parts), it will be refactored or even rewritten. If the project team is incompetent yet the software is highly demanded, community developers will fork to project so progress continues.

Relevancy to today's software startups
So that's how many of today's open source software projects work - some of them are commercial and quite profitable, mind you. But what if your startup isn't doing an open source project?

Obviously, most software startup companies aren't going to release their source code like open source projects do. But when you consider SaaS companies, you can draw many similiarities:

1. Because you're in total control of the software (it runs on your servers), you can change and update it as often as you like - just like an open source project. Suddenly, the open source mantra - "Release early, release often", makes sense. Release early means your market and your developers have something concrete on their hands as early as possible; release often means any discovered bugs are closed as soon as possible and you react to market signals as fast as possible.
   
   
2. Although you can't let external developers look at your source code, you can give them an API so they can build upon your code base. This is the approach that Salesforce takes, and last time I checked, they're a highly profitable company. They're doing well even though they're competing against an open source counterpart - SugerCRM.
   
   
3. While users of non-open source software may be less eager to give you feedback and advices, you can still get statistics of what they like or don't like by tracking their activities (e.g. mouse clicks) within your applications. It is possible, again, because you have total control over your servers and thus your software. Of course, if you can somehow form a user community as well, that's even better.
   

Tethering your iPhone 3G with your notebook

The iPhone 3G doesn't come with any way to tether with a computer by default - a gaping huge missing feature since one of the best uses for a mobile 3G connection is using that to go online with a notebook computer. Fortunately, you can add a tethering function to your iPhone yourself by jailbreaking it, here's how:

1. Jailbreak your iPhone 3G.
   Jailbreaking may sound dangerous and it actually WAS a dangerous thing to do before the iPhone 3G came out - but it's no longer a dangerous and complicated process. Just update your iPhone 3G to the newest firmware (v2.2 as of today) and use QuickPwn to jailbreak it. Jailbreaking with QuickPwn is a painless process. If you fail (typically by missing the button combo for entering DFU mode), you just try again. I haven't seen anyone bricking their iPhone 3G by using QuickPwn.
   
   
2. Install PDAnet 1.33 and MobileTerminal.
   You can find PDAnet from Cydia - one of those new icons installed after jailbreaking. Now, the newest version of PDAnet (v1.40 as of today) has turned into trial software and will become restricted after a certain number of days so don't download that. Instead, you should first add a source repository with PDAnet v1.33 inside - go to Manage->Sources->Edit and add the repository http://www.iphone.org.hk/apt/. Then, search for PDAnet v1.33 and install that.
   
   PDAnet 1.33 has a bug though, which will occasionally (often?) require you to use a UNIX shell to fix. But iPhone doesn't come with a way to let you access its UNIX shell by default... so you'll need to download another app to do it. Search for "MobileTerminal" in Cydia, and install that as well.
   
   
3. Create an Ad-hoc Wifi network in your notebook.
   PDAnet works by acting as a software router (or more precisely, NAT gateway) between the 3G network and your notebook, so obviously your notebook will need a way to connect to the iPhone via a private IP network. How can that be done? You do that by creating an Ad-hoc Wifi network from your notebook, and then instructing your iPhone to join your notebook's Wifi network.
   
   It can sometimes happen that both your notebook and your iPhone are seemingly in the same Ad-hoc network but they're actually not. To confirm your iPhone and your notebook are really connected, you can either try to ping your notebook from your iPhone's Terminal or do it the other way round. You should be getting healthy responses for your pings. Otherwise, disconnect both devices from the ad-hoc network and try again.
   
   
4. Fire up PDAnet.
   If your notebook and your iPhone 3G are connected via an ad-hoc Wifi network, and your iPhone 3G has 3G connectivity, firing up PDAnet should give you the screen above and your notebook should be able to go on the Internet.
   
   A common problem here is getting a "PDAnet is all setup and ready to go" message instead but no Internet connectivity at your notebook. It happens because your notebook's IP and MAC addresses, for some reason (e.g. you just pinged your iPhone? Bonjour on your MacBook?), is already on your iPhone's ARP table, which screws up PDAnet's detection logic. You'll need to use the mobile terminal to fix that in your iPhone. Quit PDAnet and fire up mobile terminal, and type the following commands:
   

   $ su -
   <Enter your root password here, default is 'alpine' if you haven't changed that.>
   # arp -ad
   

   Go back to PDAnet after entering the two commands above, and you should see the router screen pronto. If it still doesn't work, reboot your iPhone and try again.
   
   

Notes:
If you have an unlimited 3G plan then you may be thinking you now have unlimited mobile Internet access. But there are some caveats that prevent you from doing so:

1. Battery life. Even with your iPhone 3G plugged into a power source, PDAnet will still manage to drain the battery slowly. It takes quite a few hours but your iPhone will turn itself off eventually. So the Internet access you can get from tethering isn't really unlimited.
   
   
2. Latency and bandwidth. Latency on a 3G connection, in my experience, is much worse than a real Wifi connection. Ping times to local sites can easily go up to 400ms or more depending on location and connection quality. Also, most 3G carriers won't give you full HSDPA speeds. The best download speed I've seen from my 3HK carrier was barely over 100KB/s - that's less than 1Mbps, a far cry from the advertised 3.6Mbps and the theoretical maximum of 14.4Mbps for HSDPA.
   
   
3. No incoming connections. PDAnet doesn't seem to have any port mapping, DMZ or uPnP features. Even if it has - your 3G carrier is most probably giving you NAT-ed IP addresses anyway. So even if you can chat with other people via MSN Messenger in your notebook, you'll find great difficulty in sending and receiving files. And if you're thinking about running a mobile web server, you'll have to find some way of getting around that limitation (e.g. by using a VPN and doing port mapping over the VPN).
   

Webplay at PlugNPlay

Since thanksgiving holiday is here, I decided to go to this Webplay conference at PlugNPlay to see what the panelists have to say. I had an OK experience last time at PlugNPlay so I was hoping that it would be better this time.

Like last time, the food was quite good and the programme itself is decent. Unfortunately, once again the panels are a little stale and answers are somewhat generic. Thankfully, as it goes on the quality of speakers go up and I've typed out some interesting comments here.

The notes are a little disorganized as there aren't really a structure during these panels and I didn't think there are too many connections between thoughts so I just copy down some interesting quotes:

Eyal Herzog (Founder of Metacafe)
Different pattern of video consumption
Not easy to switch channels
Related videos will be 80% (Recommendation Algorithm)
Producing Media is going to be easier

Jeremy Riney (Founder of Playlist)
Music has always been social
Selling lots of ringtone, merchandise

Brad Jefferson (Founder and CEO of Animoto)
Animoto is about storytelling
Algorithmic logic of how to create this video
Analyzing the beat, energy of the song with motion design
Right amount to tease the user

Lance Tokuda (Founder and CEO of Rockyou)
Self expression as a viral loop
Color actually affects the demographic
Users contact closest friends with relevant information
Build one’s analytic’s platform
Checking in and reengaging with users
Social advertising to reengage users
Facebook app 99% dies
Play Fish- Check that out
Message or photo- receiver wants to get these

Alex Mehr (CEO of Zoosk)
Instrument everything; measure, do AB test and repeat
Should provide value immediately once users hit the site
In Europe people happily spam each other

Jack Jia, Founder and CEO of Baynote

Listened to Jack talking on his experience starting companies, growing it and IPO. He's also the president of HYSTA, a non profit Chinese community promoting entrepreneurship.

Some interesting comments:
1. Web is not a computational problem, it's like a real society
2. Guiding principle is social science not computer science
3. Startup is a process; it's multiple lightbulbs lighting up not just one of them
4. Market research is asking people whether they'd buy and use it not reading analysts' reports
5. Signal will emerge and noise will cancel each other out using the wisdom of the crowd
6. Startup must have barriers (See Freakanomics) and have lots of dead ends
7. If people say no to your idea, ask them why.
8. Humans are animals of contexts
9. Must be mentally prepared: Startup fundamentally violates human nature (Humans live with the pack, startup is alone, lost, fearful)
10. You must exercise if you're doing startup
11. Engineer must look at the big picture
12. Level 5 leadership: Humility and Professional Will
13. Most startups are killed by themselves
14. Don't do things when everyone is doing it

Arista Networks Take 2

Turns out that Andy Bechtolsheim was giving a talk on Arista here today. Luckily I got a last-minute notice and squeezed into the room. The talk was divided into two parts - cloud computing and cloud networking.

In the first part, Andy reviewed the remarkable market size and growth of cloud computing, and how customers and vendors view it. It is estimated that the cloud computing infrastructure size will reach a staggering $42-billion by 2012. There is certainly much incentive for them to be a big part of that market!

Customers would love cloud computing because services will be (expectedly) cheaper, faster to obtain, and simpler to use. There is no upfront investment to get an ugly box with CDs/DVDs, and no need to worry about losing the discs, since the service is accessible anywhere at anytime. From a vendor's perspective, cloud computing provides a cheap way of making services available to a large audience. There is no infrastructure cost apart from renting the hardware resources in the cloud. It is also possible to continuously add stuff to services without having to ship out patches / upgrades.

There are, however, certain reservations about cloud computing. For instance, customers might not want their data to be put on the potentially insecure cloud, and could have trouble adopting new applications in their daily routine. Vendors might also have inertia in moving services to the cloud since that may cost them their large user base. There is also the issue of agreements on responsibilities should hardware or software fail in the cloud. In essence, it may not be clear who should be responsible for what kinds of failure. Revenue is another concern, where vendors may have little profit in providing per-use services.

In the second part of the talk, Andy explained the necessity of cloud networking equipment. Basically, the cloud infrastructure is different from that of existing enterprise networks. There will be tens of thousands of servers providing a non-blocking, low-latency multi-terabit capacity and supporting dynamic on-demand application deployment. It must be reliable all-year-round and be of low cost.

Arista's short-term target is to achieve a $100 server with 1Gbps non-blocking capacity. But the real challenge is 10Gbps and up (40Gbps). They envision that 10 gigabit Ethernet would become mainstream in 2009, and almost all motherboards would have a built-in 10 gigabit Ethernet adapter by 2012. The increased capacity brings about more possibilities in designing cloud applications. A nice direction for tech companies is, then, to think of what kinds of applications would be feasible and popular in the next 5 years.

Tom Kelley, IDEO

Tom, who wrote the book The Art of Innovation and The Ten Faces of Innovation, spoke at Stanford today and here are some highlight notes:

1. Pablo Picasso: Every Child is An Artist
2. Gordon MacKenzie: What happened to all the artists? It's ok to be an artist
3. Start good habits- think like a traveler, treat life as an experiment, Nurture an Attitude of Wisdom, Use your whole brain (Daniel Pink) and Tortoise Mind and Follow your passion (Francis Coppola)
4. Marcel Proust: The real act of discovery consists not in finding new lands but in seeing with new eyes.
5. I haven't failed. I've just found ten thousand ways that do not work- Thomas Edison
6. James Dyson WD-40: Tolerated 5000+ failures
7. Mark Twain: It's not what you don't know that gets you into trouble. It's what you know for sure that ain't so.
8. Daydream and Find your own muse
9. Intersecting Circles (Jim Collins): Good at, Born to Do, Pay you to do
10. Stay young at heart
11. Be solution finders less problem solvers

Wokai

Lessons learnt from Wokai, the China microfinance startup by Courtney Mccolgan

Fundraising
Use personal contacts
Get skin in the game
Be direct; Be specific

Infrastructure
Develop a good set of PR materials and temporary website
Get sponsors

Operation
Increase information flow
Create feedback system

Technology
Find out what you like
Develop an outline and mock-up
Find a tech assessment team
Establish well-defined contract (milestones, goal-oriented payment schedule)
Break the site into pieces
Develop site openly
Don’t be afraid to pull the plug

LUNARR

This morning I went to a LUNARR event in the Graduate School of Business South building. They are amazing, in the sense that they can come up with unique ways of doing things. The meeting is supposed to be about design stuff, but I learned a lot more than design.

LUNARR is a new Web-based platform for team collaboration. When a group of people coordinate work on a particular item, they often find it difficult to keep track of changes made to the item by different people in the team. The (very) old model of so-called cooperation is simply email exchange. One would make changes to the item, attach it to an email, and mass-send it to others. The problem with this approach is three-fold. First and foremost, it's ultra-difficult to do version control on the item. Basically there can be multiple versions scattered around different computers, and there is no simple way to unify them. Second, it is unclear how a single person, say, the manager of that group, or a person struggling at night to make a presentation for that item, to identify the changes and, more importantly, how those changes came to be. For a word document, you could still do annotations, at the huge cost of having to manually combine the incremental changes in the final version. But for a general item, such as an image, a Powerpoint presentation, or a Web page, it is unclear why those changes have been made. Third, the ideas that went into the item might have been inspired by resources outside the team. It is very troublesome to incorporate those resources, such as Web links, pictures, documents, into a coherent view, if all we have are emails and minutes. This model is the so-called "shared model".

Something needs to be done about this. Well, instead of distributing all these nasty multiple versions of a file (maybe each under a different name!), let's just have one visible version, and have an automatic version control system help keep track of the changes. Then everyone in the team can simply make changes to this newest version, save it, and have those changes recorded by the system. This is the model used by Wikipedia. Obviously it solves the version crisis. Unfortunately, we are still unable to capture the communication between the team members in a Wiki page.

And hence, Lunarr introduces a third solution. An item under development can be thought of content written on a sheet of paper. What is wasted is the *back* of the paper. The ingenuity of Lunarr is that by the symbolic flip of the paper via a simple mouse click, virtually everyone can instantly understand the purpose of having a back page for an item - to record the collaboration and changes made to the item. Every email exchanged, logs for every chat session, changes to each version of the item, etc. are all recorded on this back page. Better still, the back page is tailor made for the individual user. So for any given item, the back page may be different for each team member, but any incarnation of it serves to give a clear, concise and complete view of what has been going on in developing that item.

The value of Lunarr can be tremendous, especially when a team consists of people who are not tech-savvy enough to edit Wiki pages or even annotate changes in documents. Its focus on adding value to the collaboration part of the innovation cycle is what makes the service stand out.

Apart from learning about it, I also got to talk to Lunarr's founders, Toru Takasuka and Hideshi Hamaguchi, two modest geniuses who have the vision and courage of going beyond what is comfortable and taken for granted. Toru left Cybozu, the number-one group software company in Japan he founded years ago, to pursue a dream that gives no easy guarantees on success. Hideshi quitted his long-time and high-ranked job at Panasonic to join Lunarr, a startup with untested ideas. This kind of vigor and enthusiasm is truly astounding.

Hideshi also told me something about their marketing and service provisioning strategies. Since they are a Portland-based company, there is little chance of tapping into the innovator pool in Silicon valley through normal means, such as advertising in newspapers, Web sites, etc. What they did was this. They bought billboards along the busy 101 freeway and put up hand-written ads that have nothing to do with Lunarr. They even occasionally made mistakes on some of the ads, and when people called in about it, they made changes by crossing off extra letters, punctuation, etc. This relaxed, friendly, funny and intriguing series of ads caught San Jose Mercury News' attention, which advertised for Lunarr freely by including an article on their service.

Lunarr chose to employ Google-like centralized architecture for its services, instead of leasing or selling the platform to companies. It appears that companies nowadays are less adamant about storing all data on their own servers, and are starting to see the benefits of using a cheaper centralized Web service. Hideshi noted that a US company divides the data into different security levels. Data at lower levels is allowed to be put onto public Web services like Google Docs and Facebook. Surprisingly, the company actually encourages employees to use free Web services for business purposes.

One would expect such a fast-growing technology company to be based in Silicon Valley. But it's not. Lunarr's headquarters is in Portland, Oregon. The reason, explained Toru and Hideshi, was that they wanted to get away from the distractions of the Valley, and concentrate on their products. They are, however, adamant about building the company in the US, since "it is so far the (only) best place to take your technology company global."

All in all, Lunarr is a fascinating company. They have a small but great team of engineers, a clear vision of what to work on, and a good start in the collaboration arena. I can only wonder what future products Toru and Hideshi were hinting at toward the end of the presentation.

Arista Networks

Alvin and I went to Arista Network's information session at the Gates building this evening. We were there to see what the next company from David Cheriton and Andy Bechtolsheim is like. It was a nearly pure-technical talk by Kenneth Duda, the VP in Software Engineering with a dazzling biography.

The company focuses on building network switches optimized for cloud computing in data centers. Right now, the situation is not good for innovators and companies who want to build services with their own cloud computing platform - they have to build the platform themselves. Ordinary data center equipment are tailor made for classical applications, such as massive databases. The rise of cloud computing has sparked the need for computing platforms in data centers. While one could use platforms like Google's App Engine and Amazon EC2, it may be desirable to have your own customized, potentially large, platform. But the problem is that it's too costly to make it scalable and reliable. Arista Networks come into play here by providing a network switch with a reliable OS architecture, low power consumption and small form factor.

Arista's Extensible Operating System (EOS) modularize the otherwise tight-bound network protocols in traditional switches. You could have SNMP running in a process, while the client interface and OSPF run in their separate threads. A central location called the sysdb stores the switch's state. Any changes to the state will be first reported to sysdb, which then forward the message to related modules. For instance, a link-down event could be propagated to the client interface, SNMP, spanning tree and routing modules. There is no complex inter-protocol communication, which is the root of many of the flexibility and reliability issues in today's switches.

With 45 software engineers and 25 hardware engineers, the company has managed to churn out several products already within 4 years of establishment. It shows how great a small but powerful team can perform. I hope our small team at Think Bulbs could also make spectacular products in the applications arena.

Potential

I have been thinking of what we might be able to achieve at Thinkbulbs. Here's one.

While taking a break from the grueling exam series, I came across a Dell ad for 1U servers - a whopping $862+ for a Celeron, 512MB RAM, 80G hard drive and 3-year basic enterprise support. As a DIYer, I simply cannot see the value in it. The basic enterprise support is not nice enough to justify the cost.

Well, one argument goes like this: Businesses don't have the manpower assembling and maintaining their own servers. By buying from Dell, you get a usable machine straight away with an all-in-one support package. But this does not include the cost of hosting and maintaining the services on the server. You still need to buy applications and server software to install on the machine, and you need someone to maintain it. Not cheap at all. So why not push the services out to a third party, like, say, Thinkbulbs? There is much potential for us to develop really great and *affordable* applications for small and medium businesses.

One aspect of businesses is particularly annoying - customer support. Even (maybe especially) for large companies. AT&T disappointed me so much yesterday when they told me I couldn't receive calls because of a "routing error" at some outsourced company, and that I would have to wait until they fix it. Uh... what kind of customer support is this? Today my friends told me that such incidents have occurred before, and apparently AT&T is taking no active steps to resolve the issue. I wonder what kind of customer support system they have. How can they not have found out this flaw already? Do they even review their service history at all? There is so much room for improvement.

Another aspect that strikes me as important is logistics. Last week I was at Bestbuy Milpitas looking for a particular LCD. The salesperson was absolutely sure there was no stock left, and I would have to wait for restocking. Undefeated, I tried my luck at another Bestbuy in Palo Alto, and guess what? They do have that LCD. Whether it is a logistics error or laziness on their staff's behalf, there is, again, much to improve for Bestbuy.

But then, these big companies are difficult to change. They often are too accustomed to their ways. Small and medium businesses are different. They can change. They have to, or they risk going out of business. We could help them out in maintaining their competitive edge.