Failed Screenwriter

Reading a recent post by Udi Manber (VP Engineering, Search Quality), Introduction to Google Search Quality, I was reminded just how Google's effort to improve search overlap with what we're trying to do with character bots and immersive story.

If we throw out canned dialoug as an option for immersive drama, we're left with the difficult task of natural language processing. Luckily others have been working to tackle this problem for years, especially when you think of chatterbots. One open source effort powering many of those chatterbots is AIML. AIML uses a set of heuristic pattern matching rules to find a predefined response.

If an artist is creating a character bot for an immersive story, they would start with a default set of AIML rules and responses. These would enable the character bot to respond appropriately to basic input. We would expect the artist to build on top of or modify these rules to enhance the default but more than that, create a true character and weave in the plot. If the immersive story involves a secret word; the set of AIML rules should identify when it is mentioned and respond appropriately.

But what about ambiguous meanings, slang, or players simply trying to trip up the character bot? The artist has two things working in their favor:

1. Suspension of disbelief - If a player is truly into the story, the experience, they will let minor things slide. As a child how many bad special effects did we let slide by?
2. Context - There are simply certain things you wouldn't say to a priest, or in the presence of a priest. If said the natural response may be a glare or some other default action.

Beyond that though, the artist and AIML may have trouble with slang or ambiguous meanings. But that doesn't sound that unlike what Udi Manber and Google wrestle with ever day.

Ranking is hard, much harder than most people realize. One reason for this is that languages are inherently ambiguous, and documents do not follow any set of rules. There are really no standards for how to convey information, so we need to be able to understand all web pages, written by anyone, for any reason. And that's just half of the problem. We also need to understand the queries people pose, which are on average fewer than three words, and map them to our understanding of all documents. Not to mention that different people have different needs. And we have to do all of that in a few milliseconds.

Seeing as Google is already trying to tackle this problem and generating billions of dollars doing so, it would seem futile to try and compete or reinvent the wheel here. Why not just query Google for ambiguous or slang terms. All the elements of "character" are already present in the character bot and AIML offers defaults to catch the majority of inputs, but the rare few that get through can be sent to the Google Natural Language Processing API. The Google API match could then be fed back into the AIML engine for the appropriate response.

Beyond that, this could be used for on the fly language translation - take that subtitles.

Nicholas Carr's latest book, The Big Switch: Rewiring the World, from Edison to Google is very insightful, however near the end he extends into fear mongering a bit. Some of that fear mongering springs from an interesting point, Larry Page and SergeyBrin's motivation for Google had more roots in AI (artificial intelligence) than being just a search engine.

They weren't just interested in perfecting their search engine, they said. What they really looked forward to was melding their technology with the human brain itself. "You want access to as much [information] as possible so you can discern what is most relevant and correct," explained Brin. "The solution isn't to limit the information you receive. Ultimately you want to have the entire world's knowledge connected directly to your mind."

Larry and Sergey are still pursuing their vision of AI.

"Every time I talk about Google's future with Larry Page," reports Steve Jurvetson, a prominent Silicon Valley venture capitalist, "he argues that it will become an artificial intelligence."

It is difficult to see exactly what scares Nicholas with this, other than general fear of omnipresent technology.

"Why not improve the brain?" Brin muses at one point. "Perhaps in the future, we can attach a little version of Google that you just plug into your brain."

Plugging AI resources into my brain doesn't scare me, if anything I'd be giddy about it as Larry and Sergey are. But what I'd like to see is the AI used not just to assist us, but to power our entertainment as well. And I don't mean in some pie in the sky, where AI constructs interesting stories for us on the fly. No I mean in a more practical way, say as to power language processing for virtual characters, enabling them to process slang and be topical with current events. That could be immersive drama's beach head as the search engine is for Larry and Sergey's AI ambitions, who knows where it could lead.

But alas I doubt Larry and Sergey's ambitions align much with entertainment. Entertainment is soft compared to their math and science backgrounds. Where as I see incredibly interesting data from immersive drama they may see too much ambiguity, but isn't Google's attempt to interpret queries fraught with as much ambiguity? Immersive drama represents a way to technically examine entertainment like never before. Just as we gather click trails for web sites, we'll be able to get the same for immersive dramas. We'll get metrics on engagement, time spent with a certain character, type of responses, etc. - practically everything needed to construct AI as Google has done by taping into people's mind through search. But if still Larry and Sergey can't wrap their heads around the entertainment aspect, maybe games will be the carrot they need. Games today has a little science built up around it, maybe that will capture their interest and eventually lead them to immersive drama.

One area I can see them moving quickly into is the metaverse, but not the one many of us think of when we hear the word. I see them expanding Google Earth, adding avatars, and integrating their AI. Think of this as an intermediate step to tapping directly into your brain with their AI - a digital interface to the AI before it goes direct.

Microsoft's $44.6 billion dollar offer for Yahoo! is a classic case of a company not getting it. Yahoo! is crap. I should know, I use to work there. I left in 2003 when it became obvious the company had no idea what it was, what's its objective should be, and had no gumption for innovating. I thought the press and analysts would soon see through it as well, however then came Google's AdSense. AdSense brought money back to the market and Yahoo! got swept up in the wave. But it's now obvious Yahoo! was just along for the ride. By the time this deal closes, if it closes, the combined search market for Microsoft and Yahoo! will be less than it is today. It is Google's wave, everyone else is just along for the ride.

This is a deal armchair strategists and Wall Street stock pumpers have been in love with for the last couple of years. Yahoo's been a weak stock and they'd love to make a profit on what's been a lousy investment.

- Microsoft-Yahoo the mother of all clusterbombs

It's Google's wave because they get it, just like Gates understood the market in the 80's and rode it to dominance in the 90's. Goolge's reign has just begun. So who if anyone is going to compete with it? Who will force Goolge, to buy a dieing company just to try and stay relevant? The social networks just went off a cliff.

The major challenge for MySpace, Facebook and every other social network: Figuring out how to turn their enormous online audiences into dollars. But they may have another problem: Figuring out how to keep their enormous audiences.

- MySpace and Facebook's Real Problem: Bored Users

Most likely the company, the model that will compete with Google has yet to materialize. And when it does it will be from where Google is blind - the metaverse. Google isn't completely blind to the metaverse today and will surely jump into the metaverse as it develops, but it's over confidence in AdSense will hinder its exploration. Their take aways from experiments will be clouded. They simply won't get it. At some point Google will leave an opening, an opening some industrious company to exploit. Seth Godin explains in his new book Meatball Sundae: Is Your Marketing out of Sync?, that the same thing happened to WordPerfect, the one time market leader for word processing.

When Microsoft introduced Windows, WordPerfect Corporation hesitated. They saw no reason to support Microsoft's effort to sell an entirely new operating system, and they (rightly) pointed out that version 1.0 of Windows was pretty lousy. So they reinvested in their DOS version.

And as they say, the rest is history.

I first met Udi Mander while we were both at Yahoo! Now he's at Google tackling the hard problems of search. How hard - well he recently outlined just what makes search such a hard problem. As interesting as that was, what I found most intriguing and use in the metaverse was what Google was doing to tackle the problems.

• Users and Queries - Ability to understand that 'GM' stands for 'General Motors
• Models - Ability to interpret questions, such as 'How much does it cost for an exhaust system' to ' cost of an exhaust system'
• Languages - Localize the above two techniques

Google is of course in an enviable position, collecting instant feedback from millions of a search every day. So it isn't too far fetched to believe that Udi and Google will overcome the hard problem of search. With that Google can look to further dominate the search market.

But what they're building can be applied to more then just search, and maybe spelling utilities. What if they turned that search engine power over to the Metaverse ? Not just to find stuff but to power natural language support for bots? Box all that in a black box and let bots ping it. Such a black box wouldn't supply responses but would greatly increase the interpretation of user inputs. Responses could be supplied by combining the black box with AIML chatterbot technology.

Community projects could sprout up a la Wikipedia style - letting the community supply appropriate AIML responses. Feedback to the black box could determine which response were found to be the most appropriate. This enhanced black box could be the starting point for bot creators, a default. Creators could create unique responses for their bot and rely on the enhanced black box for everything else, ensuring their bot would never be stumped.

This is just wishful thinking of course as Google has yet to announce their metaverse, despite the long running rumors.

"Release early, release often" is their motto and has allowed Google to make Yahoo! look like the reincarnation of AOL - great for your parents but not something you would use. They are the reigning powerhouse on the Internet, built on innovation. It's after the innovation that they get into trouble.

They simply have a talent of flubbing things. They launched Google Checkout with great fanfare but also a host of problems. But maybe that was due in part to their motto to release early and problems are expected. However even when they get it right they flub it. Take Gmail, it was light years ahead of the competition when it launched. In many ways it is still better then the competition, however they kept it invite only for a year or more. That gave Yahoo! and others time to not match their offering but do enough to nullify Gmail's advancements.

Now it looks as if they've flubbed the YouTube acquisition. Granted the acquisition was probably a bad idea in the first place, another Yahoo! buys Broadcast. But they failed to secure the necessary content partners and now the competition is gathering the wagons - heck they already have them in a tightly defended circle.

Google has the innovation part down and is willing to place big bets, it's the execution part that they flub.

Nice Business Week article on just who is responsible for identifying and removing videos that violate copyrights - The Online-Video Takedown Smackdown. Is it the responsibility YouTube's or the copyright holders? Either way it seems their fumbling methods are catching innocent video creators.