Relax, Everything Is Deeply Intertwingled

Secrets and Time

Money doesn't solve everything. You can buy more disks and bandwidth, but you cannot buy more secrets and time.

      -- Rohit Khare in a group meeting today, talking about the key decentralization challenges of agency and latency, respectively, as described in his Ph.D. dissertation (and summarized in his SIGSOFT Distinguished Paper at ICSE-26 earlier this year)

Personal Servers and Information Clients, 2004

It's been about a year since I've thought about personal servers and Web Services Intermediaries and Mojito, and now that I'm spending a lot of time with Rohit I'm starting to think about them again. Mostly because he cannot stop talking about them, though this time he's calling the server TPd. (Ten years ago, he called it StarTPd or *TPd because such a server could handle ftp, http, and smtp messages in and out. The star got dropped and/or burned out sometime in the last decade, perhaps during the IMPP wars at the IETF in the late 1990s. Oh, the humanity! As for my own interest, I've been kicking around these ideas with Rohit -- Internet-Scale Event Notifications and munchkins for about a dozen years now. Wow, that's a long slog...)

I'm still inspired by Ben Hammersley's post in 2002 quoting Morten Frederiksen's description of Hep: "Software that transfers messages between different messaging systems. Eventually, it should transparently route messages between email, weblogs, and instant messaging. Currently Hep does one thing: fetches news feeds from Web sites and lets you view them in your email client."

That inspired Rohit to write,

Both Hep and ZOË are innovative new messaging engines that I categorize with the name Personal Servers. Personal Servers are a new kind of infrastructure that run on behalf of people to process the messages that come into their life through email, web, IM, news, ftp, RSS, SOAP, and Internet-scale event notifications. Mod-pubsub is peanut butter to a chocolatey Personal Server because

PubSub makes Personal Servers timely and actionable.

In the future, all decentralized applications will be developed using Personal Servers: bags of bits will routed hop-by-hop through whatever application-layer protocol is available, from source to destination. We call this insight Application Layer Internetworking, or ALIN, named by analogy to IP Layer Internetworking done by today's Layer 3 Routers (slides, notes). From an architecture standpoint, Personal Servers have just one rule of thumb:

The message is the medium.

Security, reliability, filtering/transformations, and so on all stem from this first principle. (Respectively, security is message-centric, reliability is an upper-layer concern, and filtering/transformations are rules at active proxies. More specifically, the edges are intelligent because the network is insecure, unreliable, and dumb.)

Personal Servers go hand in hand with Information Clients, for which Gary Burd did a nice roundup:

• Haystack: we aim to let users define whichever arrangements of, connections between, and views of information they find most effective.
• Chandler: users will be able to organize diverse kinds of information for their own convenience -- not the computer's convenience.
• ZOË: Think about it as a sort of librarian, tirelessly, continuously, processing, slicing, indexing, organizing, your messages. The end result is this intertwingled web of information.
• clevercactus: clevercactus is focused on information centric users who want a single solution to manage their core information needs.
• IdeaGraph: IdeaGraph is easy-dailyto-use software for creating visual maps of ideas, that can work with web pages, documents and images.
• Spring: It's a universal canvas where you interact naturally with singular, visual representations of the people, places, products, etc that define your life!
  
• Agent Frank: The goal of Agent Frank is to be a personal intelligent intermediary and companion to internet infovores during their daily hunter/gatherer excursions.
• Hep: Hep Message Server is software that transfers bits of information between different messaging sytems on the Internet. When it's done, you'll be able to use Hep to transparently route messages between e-mail, weblogs, and instant messaging.
• Dashboard: The dashboard is a piece of software which performs a continous, automatic search of your personal information space to show you things in your life that are related to whatever you happen to be doing with your computer at the time.

Note how Mojito fits into the Information Client vision, too:

• secure chat (like this, but in a browser)
• "live" RSS news aggregator (in a browser, without polling)
• "live" group blogging with multiple read and write permissions

Joyce hypothesized that these three applications all converge into one live, browser-based "super app" that allows secure, real-time collaboration and news feeds.

Not much has changed about the idea of Information Clients and Personal Servers in the last year; what has changed is that it sounds to me like Rohit is (finally) interested in working on them. (His PhD and wedding were distracting him for the better part of the last 18 months.) Let's see if we can get some momentum for this now that those distractions are off the table. Just in case, I'm keeping handy the Architect's Lament should Rohit need it...

"The first matrix I designed was quite naturally perfect; it was a work of art, flawless, sublime. A triumph equaled only by its monumental failure."

Groping For Facts

In American politics in 2004, there are very few facts. Instead are mostly-biased speculation and opinions of what's currently happening. Such is the theme of the editorial in Sunday's SJ Merc, Our Fractured Discourse, By Mike Antonucci, which says:

The box-office success of ``Fahrenheit 9/11'' appears to be heavily driven by one of Michael Moore's fundamental themes: that he's supplying vital information we don't get from the government or a gullible mainstream media.

In that context, the film is not as much about Moore's contempt for President Bush and his war on terrorism as it is about what the public can know and trust -- or simply wants to embrace as validation for what it already believes.

``Fahrenheit 9/11'' conveys a black-and-white clarity about staggeringly complex issues of war and social justice, provoking a hallelujah chorus from like-minded moviegoers. With equivalent passion, detractors have lambasted the film as transparent propaganda.

For everybody else, that leaves ``Fahrenheit 9/11'' as a maddening symbol of a nation groping for facts. We are largely united by our sense of national crisis, yet we are wildly divided by what we think we know about it.

``That is the question: How does anyone know anything today?'' said Michael X. Delli Carpini, dean of the Annenberg School for Communication at the University of Pennsylvania.

For Delli Carpini, ``Fahrenheit 9/11'' is just the latest flash point amid the confusion and controversy generated by a ``breakdown between news and entertainment, and between fact and opinion.''

...

If the discussion is expanded to high-profile reporting failures -- such as the New York Times' recent admission of serious lapses in its coverage leading up to the war in Iraq -- our information society seems to be flirting with a reliability meltdown.

Separating facts from semi-facts, speculation and loose assertions is a job that has become, perhaps ironically, much harder as we've been deluged with new sources of information. We're already worried about the weaknesses of the mainstream press, and now unsubstantiated ``news reports'' fly around the world by e-mail, gaining strength by virtue of their computer-accelerated repetition. And with Moore's instigation, we're sitting in theaters trying to sift through the allegations about distortions and falsehoods in his film.

...

``If you have the slightest familiarity with the World Wide Web,'' said [legal expert Jeffrey] Toobin, ``there's more diversity of information than our parents could conceive of, let alone see.''

With that much diversity, of course, comes the potential for more conflicting ``noise,'' rather than clearer news.

...

Consider Weblogs, the online journals that exemplify the ease with which anyone can use the Net to be his own publisher. Commentary by those writers, notes interactive media consultant Brian Reich, generally is unhesitant about declaring biases, such as political memberships or business interests.

By contrast, says Reich, the assertions by the established media that it's impartial or balanced in its coverage of news are received by cynical consumers with ``a wink and a nod.'' The concerns of those consumers are not only about the quality of news coverage, but the hidden and undeclared biases that determine what's selected for any coverage at all.

He added: ``The news is what the news tells us the news is, but that doesn't mean that's all the news that's out there.''

Interestingly, this is the point of Rohit's PhD thesis: that decentralized systems need to take into account the different opinions; that there is no single value for the truth of a statement. Instead, there is only the truth according to a source; software systems of the future must cope with the same lack of consensus that exists so openly in the difference-of-opinions in the politics of the 2004 United States.