Paul ‘chip’ Querna: Facebook & Open Source: Community is just as important as the Code

When you create an open source project, you gain almost nothing but a PR hit if there isn’t a community built around it. [..]

Just look at the massive community that has exploded around Apache Lucene and Apache Hadoop — Yahoo could of kept this infrastructure project internal, and sure, it might of fulfilled their original goals, but they wouldn’t of ever received the thousands of external contributions, which has turned the Lucene/Hadoop world into one of the most diverse and thriving open source communities of late, giving Yahoo a thousand times return on their investment in Hadoop.

Paul discusses the importance of letting company-created open source projects thrive by fostering a vivid community, and encouraging their use in competitors’ products against what might be considered common business sense.

Consider the contrast between two of Apple best-known open source projects: the earliest, Darwin, never took off the way Jobs had promised (and perhaps hoped for) back in 1999. Projects such as OpenDarwin (which eventually shut down), PureDarwin, GNU-Darwin have always lacked proper leadership and interest and suffered under opaque, unclear, apparently inconsistent policies on Apple’s part.

On the other hand, there’s WebKit: once given a proper website with a public blog, issue tracker and repository and allowing external reviewers, its success exploded — and in spite of the fact that many of Apple’s competitors now use it, Apple, too, reaps rewards. The sum of the contributions simply exceeds the initial loss of control and propriety by a wide margin.

Perhaps this kind of success is only possible with a limited amount of projects, but as a company, you have to be willing to take that risk.

We’ll start with the very end:

Daniel Eran Dilger is the author of “Snow Leopard Server (Developer Reference),” a new book from Wiley available now for pre-order at a special price from Amazon.

I don’t find it unreasonable to expect someone who writes such a book to strive for a certain level of accuracy. Having said that, let’s go:

Open source?

More importantly, Apple is providing its users with additional options that benefit both Mac users and the open source community.

What additional options for the open source community does Snow Leopard provide in the PIM area? A quote later on provides a clue what Dilger apparently thinks Apple has added:

Because Apple makes its money almost exclusively from selling hardware, it has opened up its own Snow Leopard Server applications, Address Book Server and iCal Server, as open source Darwin servers that can be compiled to run on Linux.

10.5 Leopard introduced the Apache-licensed Darwin Calendar Server, a subset of Leopard Server’s iCal Server, and this has been continuously updated (Snow Leopard Server ships with version 2). But while Snow Leopard Server ships with the new Address Book Server, there doesn’t appear to be any open source project for that. Marketing-wise, the iCal Server page mentions:

To further the widespread adoption and deployment of these standards, Apple has made the complete source code for iCal Server 2 available through the macosforge.org website.

No such thing for Address Book Server. Maybe they’re planning on it; maybe they’ve even said they are — but so far, this looks quite untrue. Dilger goes on:

That means Apple is essentially giving away both the client (to Mac users) and the servers (to the community) in order to encourage the use of open standards in messaging and collaboration.

No, the clients (Mail, Address Book and iCal) are most certainly commercial, closed-source software. Of the servers, all three are commercial and closed-source, although a subset of one is available in an open-source fashion. Which, by the way, is great on Apple’s part — but let’s not deny that a good configuration interface adds plenty of value, and Apple does not provide that for free (or otherwise openly).

Outlook not needed?

Next up, Dilger compares Apple’s trifecta of client apps to Outlook, with rather bold claims:

Integrated support for Exchange beginning with last year’s iPhone 2.0 means Apple’s mobile platform simply doesn’t need an Outlook client. Now Snow Leopard can also get by without Entourage/Outlook, thanks to new and improved baked-in support for Exchange in Mail, Address Book and iCal.

Microsoft has responded with the announcement that it will now be delivering a real (but still scaled back) version of Outlook for the Mac again

Now, unlike many, I’ve always been a fan of the separation into three apps. But even in 10.6, they are a far cry from Outlook being “simply not needed” or possibly to “get by without”. Public folders, anyone?

The Microsoft Bashing Tangent

With Snow Leopard and the iPhone each now providing their own client layer for accessing Exchange Server, Apple can now offer its users alternative access to other server products as well, from its own MobileMe and Snow Leopard Server offerings to web services from Google and Yahoo. This effectively turns Microsoft from a direct seller into a wholesaler that has to deal with Apple as a middleman retailer.

I’m sure this made some vague sense when it was written. It doesn’t when it’s read. The entire section goes on about Sears, CompUSA, Netscape, IE, IIS and off-shore wind energy. Actually, that last one was a lie. But a discussion of PIM client/server solutions this is not. He could have discussed Netscape’s brief ill-fated journey into the groupware market, but he didn’t. Instead, he’s talking about Microsoft’s evilness, implying a dominant position IIS has never had (”Microsoft first took control of the client with Internet Explorer and then began tying its IE client to its own IIS on the server side with features that gave companies reasons to buy all of their server software from Microsoft.”), and then switches over to everyone’s savior Apple with their open sourcing of Address Book Server, which hasn’t in fact happened. Finally, Snow Leopard Server apparently includes a “Push Notification Server”, which Apple knows so much about, nine out of the top ten results from Google are all articles of his, or links to them. So let’s skip this entire part.

Protocol confusion

Apple’s support for Exchange and its promotion of its own Exchange alternatives are two sides of the same coin, in the sense that they use the same technologies.

Well, this certainly is exciting news for Microsoft, who didn’t even know until this point that their very own Exchange Server has support for CalDAV and CardDAV built right in. (To be fair, it does for IMAP and LDAP, although it’s typically disabled.) Wouldn’t you love it if your developers don’t even have to build features, your marketing doesn’t even have to promote them, and yet you get to offer them?

Apple built its support for Exchange using WebDAV

No…

, the open specification that Microsoft supports on Exchange Server as a way to deliver messages to mobile clients.

…and no.

While Exchange Server has support for WebDAV, and WebDAV is very much an open specification, it’s such a broadly-specified protocol for file transfer and versioning over HTTP that it isn’t intended for mail, contacts, calendars, etc. in particular, so Microsoft has had to layer plenty of proprietary additions on top of it.¹ Yes, it uses WebDAV. No, that’s not all there is to it. It’s about as vague a claim as calling XML or CSV a format. For transferring files, WebDAV is a sufficient specification; for storing mails, contacts, calendar events, notes and more, including a ton of metadata, it’s incomplete — by design.

Apple did not license Microsoft’s Windows-only “Exchange Active Sync” software; it merely licensed the rights to implement a compatible EAS conduit with Exchange. Apple owns the Snow Leopard software that talks to Exchange.

This may be, but given that it’s in the same paragraph, I’m skeptical, and also question the relevance. It seems a poor and unnecessary attempt at making Apple look independent. Maybe they didn’t pay a licensing fee; instead, they had to pay their developers to develop client code of their own. So what?

The client applications Apple has upgraded in Snow Leopard to connect to Exchange, including Mail, Address Book, and iCal, also use WebDAV to talk to Apple’s own Snow Leopard Server applications.

The latter part is correct insofar as that CalDAV and CardDAV are extensions to WebDAV for calendars and contacts, respectively. They’re entirely incorrect for Mail (there is no WebDAV-based mail specification aside from Exchange Server’s proprietary method), as well as for Exchange.

This all leads to an entirely wrong conclusion:

This effort to support everything from integrated client software owned by Apple makes Snow Leopard’s support for Exchange of use to everyone, even if they don’t use Exchange. The client work Apple has invested in making Macs Exchange-friendly also improves the features available via MobileMe, Snow Leopard Server, and even some other third party services such as those from Google and Yahoo.

On top of incorrectly believing that Snow Leopard interfaces with Exchange Server through WebDAV, Dilger apparently goes even further that, since CalDAV, CardDAV and his imaginary MailDAV² are built on top of WebDAV and Exchange uses WebDAV as well, Apple is saving duplicate effort. That would be great. It’s also entirely off. First, Snow Leopard communicates with Exchange Server through the much newer, SOAP-based Exchange Web Services protocol. That’s why it requires 2007 Service Pack 1 Update Rollup 4; this entire interface is lacking in 2003 (and in the original 2007 release). Second, even if they were to use WebDAV, this wouldn’t help them much at all.

Imagine two CSV files, one with the columns Surname, Name and Birthday, and another with the columns Last name, First name and Phone number. Superficially to the human eye, they both clearly contain contacts. To the computer, they’re entirely different formats. One column is missing from each other’s format, and while two out of three columns have the same contents, they’re differently named. You’d have to write a converter to make them match. You have the same situation with different XML formats³, and with two different takes at implementing, say, calendars on top of WebDAV. And given that Microsoft is moving away from WebDAV, citing lack of efficiency, they probably won’t implement CalDAV any day now.

The App Store Tangent

App Store? Really? What does that have to do with anything?

The success of the iPhone App Store has benefited both developers and users by establishing a competitive market based on meritocracy. Snow Leopard’s support for Exchange, because it opens up equal access to alternative competition, similarly creates an iPhone-like market for desktop messaging services ranked by merit, not the vendor’s current market position.

Yes, vendors like, say, Microsoft. Also, software companies located in Redmond, Washington state. Can someone explain to me how an interface to a proprietary PIM protocol “creates a market ranked by merit”?

This will provide Snow Leopard users with not just the ability to talk to corporate Exchange Servers, but also the ability to access Apple’s own offerings and other third party services.

I’ll get right around to implementing Exchange Web Services in my own groupware. Surely the specification is somewhere on ietf.org. Oh, wait.

1. That’s not a criticism; it’s just a fact of life. ↩
2. MessageDAV? LetterDAV? Running out of corny specification names quickly. ↩
3. Consider Atom vs. RSS. ↩

For updates you don’t care to hear about again, Windows Update provides you with a “Hide update” function. One of those updates would be Silverlight, and I’d like to not have it installed; not because of some irrational hatred towards it¹, but because I currently have no need for it on what is a virtual machine purely for development purposes. But while the hide function works fine on other updates (language packs, for instance), it doesn’t appear to for Silverlight: seemingly, Silverlight re-appears a few minutes later. Try again, and it re-appears again.

I’ve finally taken the minute to figure out why this is. Don’t attribute this to malice on Microsoft’s part; Silverlight is not exempt from hiding. Instead, what’s really going on is that as you reject the newest version of Silverlight, you get the previous one. Reject that, and you get another older one. In total, you have to hide six updates to get the desired effect² — and you can’t do them all in one batch because the respective older one will only become available as the newer one has been hidden.

I’m glad they’re not trying to force Silverlight down our throats, but you’d think they’d have figured out a way, by now, to really hide an entire group of “updates” altogether.

1. Nor because it is very much a completely new component, rather than an update. ↩
2. You can easily verify this yourself by taking a closer look at the different knowledge base IDs after each hidden update. ↩

Thanks to Matt Gemmell for pointing towards a new TextMate Blog post:

TextMate development is going strong: TextMate 2 isn’t done yet, but progress is steady, it is starting to take shape, and the end is in sight.

[..]

It feels to me like most of the modules are getting close, say 90%. But as they say, on the horizon, mountains look small. While I use 2.0 for my own work, day-to-day, and the basic infrastructure is pretty solid, much of the front-end still needs work, and for now it’s all lacking the spit and polish of a finished app.

Terrific! And in other editor news, elastic tabstops, arguably one of the most important (and long overdue) innovations in text editing of the decade, are starting to gain traction. (Thanks Denis!)

It looks like I will be able to implement elastic tabstops in Visual Studio after all, since VS 2010 should allow me to set non-uniform tabstops on different lines.

For better or worse, I’ve been using Visual Studio a lot in the recent close to two years, so I’d love for this to happen.

CocoaHeads is a group all over the world¹ meeting up every month and discussing Cocoa (and Cocoa Touch), Apple’s primary development framework for the Mac and iPhone platforms. Alexander Repty (perhaps best known for his neat Lab Tick utility) took the initiative in launching a chapter for Bremen. We² met for the first time on Thursday. About a dozen people came (we were hoping for four, maybe five), and it turned out to be a great two and a half hours in a café.

Like In The Old Days

It struck me when explaining this event to someone else how oddly this must come across: as more and more social activities of our everyday life — both leisurely and professional — takes place over the Internet, with chat rooms, discussion forums, blogs, and other fast-paced media, here comes what amounts to a perfectly old-fashioned hanging-out over coffee and cake. I first met Alex in #macsb (for Macintosh Software Business), an IRC channel on FreeNode focused on running independent Mac development studios. It’s one of the stranger coincidences in life: despite being an international chatroom hosting only several dozen people, we actually grew up less than a mile from each other. And yet, we never met in person until Thursday.

So why, when you can use CocoaDev to look up API commentary, Stack Overflow to discuss problems that have you stumped and Twitter to follow what others are cooking up, would you really need to attend anything in real life³ any more? It is perhaps downright antithetical to the stereotype for a software developer to do.

Truth be told, the benefits are hard to describe exhaustively. As far as resources go, the Internet is absolutely unparalleled. And yet, because we aren’t forced to interact socially, we tend not to. Forums and even twitter are far from real-time anyway, and as for chatrooms, we tend to lurk for minutes or even hours, only sticking our heads in when we feel like it. A café doesn’t give us that option, and while it honestly isn’t something I’d want to experience every day, it is a refreshing contrast to the usual. So, immediacy places a role. Those who are there are actually… there.

There’s a quality to actually meeting that perhaps roughly matches what Rands calls The Pond; a shared, mutual breeding place for ideas that just cannot with our current technology be replicated or even closely imitated with telecommunication. I’ve witnessed this myself with the occasional work from home (or elsewhere) I do; sure, everyone’s reachable, but that’s a stark contrast to everyone being around. Got a problem and can’t figure it out immediately? In the office, you’ll ask your neighbor to take a look (and, typically, just the advantage of two additional eyes solves things fast). Elsewhere, you’ll hesitate to instant-message around, call anyone up or even write an e-mail, and will for no good reason be more inclined to solve things yourself.

Finally, perhaps the decidedly low-tech nature of this — though, to be far, some did show apps that they’ve written or are working on around — is simply a refreshing change.

I had never been to anything like this before — I’ve been to expositions, and I’ve done demos for current and potential customers, but conferences, not so much. One reason? I had regarded the very idea of meeting up in person as somewhat outdated and superfluous.

Now, not so much, because clearly, the benefits of socializing with others who share your profession go way beyond the obvious intoxication and “networking”.

Post Scriptum

I thank (again) Lexx for organizing, and everyone else for attending. For those in Bremen or nearby, we plan to meet the second Thursday of every month. If anyone wants to join in or perhaps even present something, please do!

1. Though Africa is feeling rather lonely right now, and South America even more so. ↩
2. To my own astonishment, that includes yours truly. ↩
3. It feels funny to stress this. ↩