Yesterday I started reading Paul Graham's Hackers and Painters, and it is wonderful. It literally is changing my perspective about how I think about the world we live in, and where we want to go from here.(Thank you for the suggestion, Aaron and Evan.)
So far the chapter that has really resonated with me most is The Other Road Ahead, in which Paul writes,
With web-based software, most users won't have to think about anything except the applications they use. All the messy, changing stuff will be sitting on a server somewhere, maintained by the kind of people who are good at that kind of thing... Desktop software forces users to become system administrators. Web-based software forces programmers to. There is less stress in total, but more for the programmers...
One thing that might deter you from writing web-based applications is the lameness of web pages as a UI. That is a problem, I admit. There were a few things we would have really liked to add to HTML and HTTP. What matters, though, is that web pages are just good enough...
Because web-based software assumes nothing about the client, it will work anywhere the Web works. That's a big advantage already, and the advantage will grow as new web devices proliferate. Users will like you because your software just works, and your life will be easier because you won't have to tweak it for every new client...
You don't have to ask anyone's permission to develop web-based applications. You don't have to do licensing deals, or get shelf space in retail stores, or grovel to have your application bundled with the OS. You can deliver software right to the browser, and no one can get between you and potential users without preventing them from browsing the Web.
Interestingly, I heard this line of reasoning using different words in the summer of 1998 when I was working at Microsoft. The source was an internal document called Weblications written by Adam Bosworth. To my delight, I have found a public document from May 1998 that contains a lot of Adam's discussions, titled Microsoft's vision for XML:
What have we learned from the Web?
First and foremost, we have learned that it isn't enough for something to be possible. It must be easy, open and flexible. The web predates HTML of course, but until the advent of HTTP and HTML, it didn't really explode. Why? The answer, succinctly, is empowerment. Once HTML and HTTP arrived, more people could play more easily. The solutions were not necessarily optimal from the point of view of performance or even robustness. They were optimal from the point of view of ease of getting started. In short, they were drop-dead simple. Many people point out the deficiencies of HTML, especially because of the sloppiness of its grammar...
The lesson is that simplicity and flexibility beat optimization and power in a world where connectivity is key. There is a second lesson which is key. Applications need to be constructed out of coarse-grained components that can be dynamically loaded rather than single large monolithic blocks. In the HTML world, these components are pages. In the applications world in general, however, this lesson applies. The reason for this is simple. The application starts more quickly, only consumes the resources it really needs, and most importantly can be dynamically loaded off of the net. Why is this so important? It is important because of deployment. Applications that can be dynamically loaded in from a central place don't require some massive, complex and difficult installation process onto clients' machines. Note that Java per se doesn't give one this. It is easy, as anyone who has built a large and complex Java application can testify, to build one, which requires literally hundreds of classes to run. That is monolithic. HTML had the serendipitous effect of forcing application designs to partition the application. To repeat, the lesson is that applications should be loaded in coarse-grained chunks.
It comes as no surprise then that Adam Bosworth recommends Hackers and Painters as well, for it echoes in his own philosophy:
For much (although certainly not all) of the work IT does, IT is like children building sand castles on the beach and watching the tide roll in. That tide is highly customizable web based solutions, Salesforce.com today, perhaps Talaris tomorrow. Ask the average Salesforce.com customer (meaning a salesrep) if he is happier with the solution he has now or the one he had back when IT was building a customer CRM for him. I think the answer will surprise you. Web Services have helped immensely here because it has made possible the integration of these solutions with internal logic for those things IT should still be working on. This is the promise and the future in my opinion.
To me, the passages from Paul Graham and Adam Bosworth articulate well The Web Way. Interestingly, Adam's journey has taken him from a startup called Analytica to Borland, to Microsoft, through a startup called Crossgain, to BEA, and now to Google; all of them were platform companies -- and it's notable that he hopped from a startup to a languages company, to an operating system platform company, through another startup to an application server platform company, and (now) to a web platform company.
I believe Adam's journey represents the evolution of the software industry over the last two decades: from desktop applications running on single-machines that helped individuals with productivity through word processing and spreadsheets and email, to enterprise applications in corporate data centers that helped workgroups and companies with productivity through automating business processes... and now to collaborative applications available to anyone from anywhere on the Internet, leveraging an increasingly-connected and ever-faster world. The web is the platform that subsumes the others.
The software platform of choice evolved from desktop operating systems, to enterprise operating systems, to what Tim O'Reilly calls the emergent Internet operating system. Which reminds me of Jason Kottke's description of the Google operating system:
Google's money won't be made with search... that's small peanuts compared to selling access to the world's biggest, best, and most cleverly-utilized map of the web...
Google isn't worried about Yahoo! or Microsoft's search efforts... although the media's focus on that is probably to their advantage. Their real target is Windows. Who needs Windows when anyone can have free unlimited access to the world's fastest computer running the smartest operating system? Mobile devices don't need big, bloated OSes... they'll be perfect platforms for accessing the GooOS. Using Gnome and Linux as a starting point, Google should design an OS for desktop computers that's modified to use the GooOS and sell it right alongside Windows ($200) at CompUSA for $10/apiece (available free online of course). Google Office (Goffice?) will be built in, with all your data stored locally, backed up remotely, and available to whomever it needs to be (SubEthaEdit-style collaboration on Word/Excel/PowerPoint-esque documents is only the beginning). Email, shopping, games, music, news, personal publishing, etc.; all the stuff that people use their computers for, it's all there.
By the way, Rick Skrenta pointed out (via the New York Times) in April 2004 that Google had at that time 100,000 servers. Eight months later, Charles Ferguson points out (via his MIT Tech Review article, What's Next For Google) in December 2004 that Google has 250,000 servers. If they double roughly every nine months, Google will have a million servers in summer 2006. Ferguson writes,
Today, the wholesale search market has significant barriers to entry. Economies of scale have asserted themselves, secondary competitors have folded, and the creation of new search engines by startups is becoming prohibitively expensive. Consider: to crawl, index, and search more than eight billion pages -- still only a fraction of the Web -- Google now operates a global infrastructure of more than 250,000 Linux-based servers of its own design, according to one Google executive I spoke with, and it is becoming a major consumer of electrical power, computer hardware, and telecommunications bandwidth.
I'm still stuck on the notion that in less than two years Google will have a million-node computer operating as a single, optimized operating system for web-based applications. Google gets it. Most professional developers look at web-based applications and all they see is "the lameness of web pages as a UI", as Paul Graham called it. They grumpily say in Eeyore fashion that web apps are the legacy apps of the future. They don't see that the power of Weblications is that "simplicity and flexibility beat optimization and power in a world where connectivity is key", as Adam Bosworth put it.
It is no coincidence that in 2004 Google hired Adam Bosworth. It is no coincidence that in 2004 Google hired among-the-best-and-brightest-in-the-world web developers such as Chris Wetherell and Aaron Boodman -- and they are searching for more. Because 2004 is the year that Google led the charge in making applications that showcase The Web Way -- using just the simple and flexible mix of dynamic HTML, JavaScript, style sheets, and a DOM-capable browser -- respectable and cool and (most importantly:) useful enough to be considered a viable alternative to The Three M's Of UI Lock-in (Macromedia, Microsoft, and Mozilla), all of which aim to fatten up the client and lock some users in and lock other would-be users out because they don't have Flash or XAML or XUL or any other doesn't-work-on-some-platforms complex soup like Java Server Faces and Struts taglibs. When it comes to simple, ubiquitous, usable user interfaces, Dynamic HTML is one ring to bing them all.
It took a long time -- upwards of a decade since Netscape released the first widely-used, widely-deployed browser -- for The Web Platform to become stable enough to build complete, beautiful applications upon. In 2004, Google's applications like Gmail and experiments like Google Suggest demonstrate that we can have robust, interactive, useful web-based applications. Joyce pointed this out over the weekend in her essay, Google is good for webdev, in which she wrote,
Google is quietly hiring -- they probably have between 5 and 10 of the top front-end devs, which doesn't sound like much until you realize there might be only a couple dozen out there with significant experience. That decreases the supply of the remaining ones, which as we all know means you're gonna have to show them some serious love to enjoy their scarce favors.
Most importantly, Google seems to be putting some chips down on the DHTML side of the table instead of the Flash or XAML or XUL or Laszlo sides. Given how important a few technology leaders are as role models for all developers -- I dunno how many arguments I've had where the magic words, "But Yahoo/Amazon/Google does it this way!" work their incantatory magic -- this is very much a Good Thing. So I've got to thank Google for making it viable to spend money building newer, faster, more responsive, standards-compliant, user-centric, cross-browser interfaces. Whatever the opposite of "collateral damage" is -- collateral benevolence? -- Google is doing it for webdev now.
What's so significant about the development of Gmail? Jon Udell looked under the hood, and was enthused by what he saw:
I’m ready to join the chorus singing the praises of GMail’s user-interface technology. Its combination of HTML, JavaScript, and the DOM makes the browser do some remarkable tricks.
One of my favorite acid tests is address completion. When you begin typing an e-mail address, your mail program should immediately show you the matching addresses and then dynamically constrain the list as you continue to type. Outlook does poorly on this test; you have to type CTRL-K to invoke the address book in a separate window. OS X’s Mail does address completion in situ, just as I expect. So does Gmail. And here’s the shocker: Gmail does it faster.
Gmail’s spell checker is another amazing hack. When you invoke it from the message composer, misspelled words turn red. Click one and a list of choices drops down, ending with an Edit choice. Click Edit and the suspect word converts, inline, to an input box. When you’re done correcting one or more words they merge back into the text.
As early adopters discovered long before I did, there’s an architecture behind this JavaScript/ DHTML wizardry. The best description I’ve found is from Johnvey Hwang, who deconstructed Gmail’s JavaScript code and created a .Net-based Gmail API. As Hwang described in his July 5 write-up, Gmail loads a JavaScript "UI engine" into your browser at the beginning of each session. Oddpost, he noted, was the first Web mail application to perfect this technique. That was a prophetic statement: Just four days later, on July 9, Yahoo acquired Oddpost.
Because Gmail’s behavior is embedded in the UI engine, all subsequent interaction between the browser and the Gmail service is just an exchange of data. What Hwang calls the DataPack format is not XML, though; it’s JavaScript. When you make a request to the Gmail service, whether to refresh your inbox or to modify the list of labels you can attach to messages, the response is a minimal set of JavaScript function calls and associated data objects that the engine uses to update the display.
This is very geeky stuff, I admit, but here are two important points to take away. First, as I’ve often said, intelligent use of browser-based technology can accomplish more than most people realize. You can’t do everything — not by a long shot — but for many of the things that information workers routinely do, even ordinary Web UI is good enough. And now Gmail is proving that we don’t have to settle for ordinary.
Second, Gmail’s architecture is not limited to Web UI. Because it is protocol-driven, developers can create new tools that speak to the DataPack format. Many have done so already.
In Gmail's case, the weblication was better than the non-web-based applications. Gmail raised the entire discussion of what is possible with web-based applications. It didn't go it alone -- it built on the knowledge of web-based applications learned by those who had previously written state-of-the-art weblications. What's significant is that it was able to break through expectations, and proved that "we don't have to settle for ordinary" with web-based applications. Koranteng Ofosu-Amaah clarifies this point in his essay On GMail and DHTML architecture again, in which he writes,
GMail's architecture is actually very generic for a DHTML app. Everyone with a clue should be trying to leverage the browser and that, in essence, is all they are doing.
It is riskier to do more on the rich browser client because it has been a more brittle platform over this past decade. Companies that do middleware and server-side tooling take a while to move from their core competency. Architects that thrive in that environment are essentially conservative and for good reason... Four years later, I now hear mutterings about drag-and-drop and richer clients in our corridors...
This only underscores the point that Jakob Nielsen's predictions about browser adoption cycles have turned out to be pretty accurate. Even though web application developers have been quietly spreading unobtrusive javascript usage in the interim, it is only now that there's a critical mass of clients that can leverage them; when Amazon and Yahoo move, something must be happening.
The developer tools and resources have gotten (slightly) better and there's more experience with the DOM. Increased adoption of broadband also helps reduce latency for the average client so you don't have to fight the inevitable arguments about performance and can couch your advocacy in terms of user interaction. In any case if and when you do have the performance discussion you can always argue that caching as close to the client as possible is a good thing and what better cache than the browser itself. It just so happens that applications like GMail, Bloglines and Oddpost are the state of the art in terms of browser leverage.
I recently wrote about this type of architecture in my recounting of the history of the DHTML spreadsheet and presentation components that are the genetic forebears of OddPost.
The idea is to fetch an HTML skeleton, decide what content you need, fetch that (as XML), and cache it wherever you get a chance. Render incrementally.
The pattern is simple:
Database <--> XML (Optional) <--> JavaScript Object Bindings <--> UI Bindings (HTML) + UI management code...Perhaps K-station was too bleeding edge trying to go for XML over HTTP, DHTML and extreme leverage of the browser client 5 years ago but that experience was a great testbed for me and I learned lots of lessons about building rich REST-ful applications, the importance of URIs etc.
Again the major missing feature for this rich web application platform is offline usage and synchronization without introducing new security holes in the browser. But then that's why Bosworth is at Google as the rumour goes, right? I suspect he's got other things in mind though...
I think that increased leverage of the browser and the DOM is a good thing. It's also a clear trend and for many applications, the browser is good enough. Good enough for Google, good enough for Yahoo, good enough for me.
And yes, the Beta release of Google Suggest does make me wonder why it wasn't until the end of 2004 that we saw such a web-based application speedily interacting with a web server to complete the form with what you're likely to be typing. We see this kind of interactivity regularly with our Oracle Financials or Peoplesoft HR or Siebel Customer Support system, where the forms are filled in interactively. Why did we wait till the end of 2004 to see such an interactive weblication?
Looking at the reverse engineering of Google Suggest and the Slashdot post where they describe the technique, we discover that
The source for the page is quite simple; most of the work happens in a condensed JavaScript library [google.com]. Not easy reading (note the word "condensed" above, meaning function and variable names are 1 or 2 chars, and all extra whitespace was removed...), but it's actually pretty straightforward.
It disables your browser's autocomplete on that textfield (for obvious reasons). Then it basically just defines a hidden div for that auto-complete dropdown (variations on this depending on browser... frickin' incompatibilities).
Each time you type a character, it populates that div body with the results of a quick, tiny query back to Google. It's NOT running the search for you; it's hitting (I assume) a simple, probably totally in-memory list of the most popular searches and number of results. That's how it can be so quick a response -- the lookup on their end is super-minimal, and the data to be transferred is probably less than 1k each time.
Cool. Nice concept, nice execution. And one of those nice "only obvious in hindsight" additions.
I have the feeling that we've turned a corner, and that more "only obvious in hindsight" web-based application tricks will be developed in the years to come -- thereby solidifying The Web As A Platform and continuing the spread of The Web Way as more users become True Believers who won't give up their web-based applications no matter how hard the "fat, rich client" camps try. As Joyce Park has said to me, "simplicity is its own revelation." It feels as if the world has had tremendous convergence on the thoughts in this post in 2004, and as a result the future looks very bright for The Web Way.
Jun writes about the Web of Applications:
This reminds me of what Rohit used to call application-layer internetworking: Jun has a nice chart describing the Web of Applications, and writes: Sounds compelling, I look forward to hearing more...Posted by: Adam | December 24, 2004 at 04:40 PM
Perhaps it will become possible for google to allow us to create applications that could be delivered using its platform?
For example, it will allow this type of application to use a specially designed UI markup that would be rendered by google. It will allow the app to store its context in user's gmail mailbox (and access using standard API). And allow special RPC calls for extended use of AI. And take care of supporting multiple browsers by offering a uniform widget interface such as laszlo .
The difference between traditional ASP applications, and the ones allowed to be served by google would be that the massive computational, UI and service infrastructure which a data center can't possibly provide.
Posted by: Vinod Kulkarni | December 28, 2004 at 01:14 AM
Adam you've written this a bit late in the year to garner any 'post of the year' accolades, but you've tied it all together very well into a cohesive piece. Nice summary!
I was once told that writing can be therapeutic for the author as well as informative for the reader. I certainly found reading the above very informative, i trust you found writing it therapeutic.
dHTML'ers please gather round for a warm fuzzy this New Year.
Posted by: Adam Bramwell | December 29, 2004 at 05:36 AM
"...The Three M's Of UI Lock-in (Macromedia, Microsoft, and Mozilla), all of which aim to fatten up the client and lock some users in and lock other would-be users out because they don't have Flash or XAML or XUL or any other doesn't-work-on-some-platforms complex soup...."
fwiw, one of the things I like about the Macromedia Flash Player is that it will work invisibly in a great range of operating systems and browsers... getting the engine is quick, simple, and doesn't change the viewing environment or user habits.
(DHTML is cool, and its range-of-use will increase year-by-year as browsers converge, but as this multi-engine approach gains ground, single-engine approaches stake out yet further ground. The development/testing costs come into play too. Could go either way... having complementary approaches helps.)
Regards,
John Dowdell
Macromedia Support
Posted by: John Dowdell | December 31, 2004 at 01:28 PM
You expect my wife to work the way she does with images and stuff using DHTML and stylesheets? Forget it.
She needs a full multimedia platform, Flash Studio.
Posted by: | January 06, 2005 at 02:09 PM
This is total crap. Yes, the web is okay for some applications that can get by with minimal interactivty. But, you'll never see a viable Microsoft Word HTML application (unless it uses browser or OS specific functionality, then why bother?) Same goes for PhotoShop and a bazillion other apps. For corporate applications, the only people who are better off with web apps are the people in the IT department. It's much easier for them, but end users suffer.
HTML is great for linking text to other text. Other than that, it's the worst application platform I've used in my 20 years of application development. It's time to acknowledge that hypertext does not a viable platform make.
Posted by: Peter Eddy | January 06, 2005 at 06:04 PM
The problem is that is very difficult to write real Web Applications because there isn't a real UI framework. Maybe XUL could be the answer. Take a look here for some real remote XUL application:
Mozilla Amazon Browser: http://www.faser.net/mab/
XUL File Manager: http://filemanager.mozdev.org/
Posted by: Gerald | January 07, 2005 at 09:24 AM
Thanks for the comments, folks; you've given me a lot of interesting food for thought.
To the point of real UI frameworks, I forgot to mention that OSAF's Application Project is doing some interesting work in that area as well.
On the lighter side, Star-Telegram listed what it thought were the most interesting innovations of 2004, and many of them are web-based:
While I'm posting, I'll note that Richard MacManus has a great weekly wrap-up about web-based applications, in which he cites the Mitch Kapor gem,
Nicely stated...Posted by: Adam | January 08, 2005 at 01:16 PM
John, Flash will never play here until Flash the proprietary server requirement goes away. I should be able to code a Flash app in a text editor that interacts with my web server running PHP.
One reason these richer dhtml/javascript apss aren't more prevalent is because it remains criminally unclear how to make post/get calls with JavaScript in the browser (using XMLHTTP or whatever). This should be trivially easy and yet no where does there exist a plain english explanation of how to do this.
Posted by: pb | January 09, 2005 at 11:13 PM
My views on web-applications
Posted by: Espen Antonsen | January 10, 2005 at 11:41 AM
"In 2004, Google's applications like Gmail and experiments like Google Suggest demonstrate that we can have robust, interactive, useful web-based applications." I really think that Gmail was a big leap ahead. It will let Google become even more popular, as it use to lose in comparison to Yahoo or MSN for the lack of additional services.
Jane, web designer
Posted by: Jane | January 18, 2005 at 02:07 AM
Now that data is shared via XML, it opens up all sorts of doors for data exchange, and service attachments. Specific processing of key data attributes could be a service provided by a 3rd party, that adds value through extra intelligence. I think we are starting to see Google move to the a company that offers services, that are extensible pieces or specific processing abilities that can be leveraged.
Also, I believe that we are going to see the advent of Neural SOA, the ability not only to attach services, but the ability to discover, configure and consume. Many of these moves are already underway through UDDI, and other emerging standards.
Posted by: Joe Peffer | February 14, 2005 at 07:18 AM
Nice article by Christopher Allen on Google Suggest Dissected and how this could be the way many web-based applications get built in the future:
It definitely inspires me. And by the way, using Google maps definitely inspires me, too. I'm getting the idea buzz...Posted by: Adam | February 15, 2005 at 09:28 AM
Via Matt Haughey I found this very cool article by Jesse James Garrett of Adaptive Path called Ajax: a New Approach to Web Applications. I got very excited when I read Jesse's article because I thought to myself, "Yes! Yes! He's saying what I was thinking but unable to articulate. This is it -- the thrill of the potential of next-generation web-based applications! Jesse wrote:
If people got excited about the Ajax vision -- and from talking with friends around Silicon Valley, it's safe to say that they are -- then I have the feeling Dojo Toolkit is going to blow a lot of peoples' minds about what is possible with web apps.
By the way, I love what Matt wrote in his post:
Companies are evanescent but movements have longer-lasting effects. The technology did change web application development forever, just not as quickly and not as directly as we had hoped. In showing those demos to everyone who would listen to us, we got many people to think differently about what was possible with web-based applications. Other grassroots efforts (including, but certainly not limited to, mod-pubsub) were having similar effects, and over the past four years, as the technologies matured, more people started to get it.This movement -- the movement of dynamic web-based applications -- started very slowly but is finally gaining significant momentum. I have a feeling that the best is yet to come.
By the way, the sooner companies stop asking webdev employee candidates the same questions they ask C++ or Java developers, the sooner those companies will have beautiful dynamic web-based applications to share with the world. Biz Stone and Bay Wei Chang get it. But not all companies do, so far...
Hey, why did this page suddenly get so many hits? (Flips through logs.) Oh. Got slashdotted:
Hmmm. Ajax: making the world more powerful and user friendly or just a household toilet cleaner? You be the judge...While I'm on the subject of weblications, let me take a moment to applaud IBM's endorsement of PHP. Peter Yared writes,
PHP saves developer time, which is far more valuable than CPU time. Jon Udell's "Myth: IT doesn't scale" article reminds us that architecture matters most when it comes to reliability and scalability -- not programming language choice.As a result, a project manager's main focuses -- functionality, cost, and time -- all depend on the productivity of the project's devs and ops people. I believe that history will show that LAMP/PLAD makes people more productive. We've passed the turning point, and IBM's announcement gives it momentum.
Posted by: Adam | February 22, 2005 at 07:45 AM
Let's hope that this "new approach to web applications" considers accessibility standards for disabled users.
Posted by: Christopher Vigliotti | February 28, 2005 at 07:47 AM
The people leading the pack in building the p2p and public domain version of what some people describe as GooOs are the espians, who call it the plex.
Someone should probably give them millions of dollars so that they can poach some their friends that are currently working for google and others who'd help out if it would pay their bills
Check out http://blog.espnow.net and join the fun at irc://irc.freenode.net/esp
Posted by: Josef Davies-Coates | March 01, 2005 at 04:58 PM
You folks might appreciate my Amazon Zuggest tool which uses some of the "Ajax" concepts you're talking about and works similarly to Google Suggest.
Check it out: http://www.FrancisShanahan.com/zuggest.aspx
Posted by: Francis Shanahan | March 04, 2005 at 08:25 AM
She needs a full multimedia platform, Flash Studio.
Posted by: danny | October 10, 2005 at 01:26 PM
This movement -- the movement of dynamic web-based applications -- started very slowly but is finally gaining significant momentum. I have a feeling that the best is yet to come.
Posted by: kostenlose foren | October 10, 2005 at 01:27 PM
This is an excellent article on [how the web as a platorm], and how moving to a de-centralized working environment is a good thing.
Posted by: wellness | October 13, 2005 at 08:02 AM
Let's hope that this "new approach to web applications" considers accessibility standards for disabled users.
Posted by: hangton | October 19, 2005 at 05:00 AM
Nothing is easier than Xin, when it comes to creating serverapplications, it's not even technically possible for anything to be simplier than that I think, so we'll soon find out if this reasoning holds true or not..
Posted by: mikael bergkvist | July 13, 2006 at 09:08 AM
you can look to action script 3 in adobe web site.i made those sites with php javascript and flash
haydibil
oyun
oyun sitesi
üniversite
Posted by: Account Deleted | March 27, 2010 at 10:05 AM
this is really a nice article. i hope this new approach towards web application would be better step.
Posted by: Account Deleted | July 05, 2010 at 08:00 PM
Excellent post and wonderful blog, I really like this type of interesting articles keep it up.
Nice job I really like it!
Posted by: Account Deleted | January 02, 2011 at 11:00 PM