I never said she stole my money...

Natural Language Processing is hard. Make no mistake about it.





Here is a good example of the complexities of the English language that came from the Wikipedia entry for NLP:

"I never said she stole my money" - a simple sentence on the surface but is it?

Let's have a look at how this changes with the emphasis of the sentence.

"I never said she stole my money" - Someone else said it, but I didn't.
"I never said she stole my money" - I simply didn't ever say it.
"I never said she stole my money" - I might have implied it in some way, but I never explicitly said it.
"I never said she stole my money" - I said someone took it; I didn't say it was she.
"I never said she stole my money" - I just said she probably borrowed it.
"I never said she stole my money" - I said she stole someone else's money.
"I never said she stole my money" - I said she stole something, but not my money.

We have a hard time figuring out what this sentence means so how can we expect to automate that process. One of the key things that we are doing at Virsona is to try to understand sentences within the context of the conversation. This is how we process things in real life and is a vital component of being able to handle a conversation and understand as best as we can.

I just had a conversation with Babe Ruth who we currently have under development.

I asked him the question: "Who is your favorite teammate?" and he told me that it was Lou Gehrig. When I asked him "Who was your favorite team mate" and he gave me a great answer about how he loved playing for the Yankees. Good on ya Babe - have a bonza day.

The future is so bright....

We have been testing out some great applications for our Dialog Engine recently. Once you start to think about how this type of technology can be applied the uses are many.

However there is a big difference between what you think a technology might be used for and how it actually ends up gaining traction and becoming common place.

The promise of Artificial Intelligence has been around for a good 50 years now but it is still not main stream in anyway shape or form. We are starting to see some applications that are 'behind the curtains' but we are still in the early days of general acceptance of AI interaction.

One of the reasons behind this is simply the utility of dialog. For the most part conversations with chatbots have been stilted, narrow and cannot veer off into the types of conversations that we generally have in real life. At Virsona we are building an engine that will handle this broader type of conversation and hopefully provide more utility and a better experience for people using our engine as part of an application.

We are going to start rolling out applications next month and are looking forward to seeing how people will react to them. Hopefully we will find our killer app. quickly or maybe we just have to let our customers guide us in the right direction.

Can I stereotype Canadians...?

I spent this morning with some very nice Canadians.

The meeting was about the credits/grants that are available to business' doing Research and Development in Canada. The program, called Scientific Research & Experimental Development (SRED), is very beneficial to companies as it returns some significant percentage of expenditures back to the company performing the R&D.

The have been doing this for a long time now and clearly it works. The theory behind it is that R&D jobs are high paying jobs and once you have built up the infrastructure in a region the jobs stay in that region and continue to stimulate the local economy.

We have heard a lot over the last couple of months about stimulating the economy.

Here in the US I am one of those firm believers that small business' are the way we are going to build and grow our way out of this mess. Anything the US government can do to help encourage, promote and support small business is going to have a powerful impact on the future of the economy.

Its not enough to say it though...both Federal and State governments have to remove the red tape and bring these programs on line efficiently and in a timely manner.

Otherwise we might all be watching hockey, drinking Molsen and sipping coffee from Tim Hortons rather than Starbucks in a few years, eh!

A Flock of Robots...

I spent this weekend with Robots. Lots of them. I was at the First Lego League State Finals for Florida. Virsona was sponsoring a local middle school who had made it through several qualifiers to the State Finals – not bad!

There were 48 team each whose robot had to complete various missions. You can learn more about this program at First Lego League.

I watched these kids age 9-14 and clearly saw their understanding of how to make robots do complex activities. It really made me think about the age based digital divide and the attitudes kids have to technology. For people 30 and older I think we still have a general fear of technology – especially Artificial Intelligence. We grew up on a diet of machine intelligence as a bad thing – mad computers trying to take over the world (aka skynet) – modern day examples of the same Frankenstein nightmare of our creations gone horribly wrong.

These kids however were fully engaged. They don’t view technology as "that stuff", rather it is an integral part of who they are. Of course you can pause tv, of course you can carry around 10,000 songs with you, of course you can build a complex robot in your spare time, of course you can find out any fact immediately, chat with historical figures in real time – no problems. That’s just the way the new world works.

So while us pre-Digital Generationals tend to question the use of technology, agonize over the implications and moralize over the applications, kids just embrace it. The sooner we can learn to embrace it too the better we will be able to engage with kids. If we don’t, they will be describing 3 types of intelligence pretty soon – Digital Generation Human Intelligence, Artificial Intelligence and pre-Digital Generation Human Intelligence.

Anyone know where I left my Flock of Seagulls LP?

Ground floor .. going up?

A couple of weeks ago I got onto an elevator. There was a small boy, maybe 5 years old, who got on with his mom and a few other people. The buttons were all lit up, 3,7,8,12,15. The little kid looks at his mom and asked her:
"Which floor are we going to first?".

I think this is one of the best questions I have ever heard.

In makes no assumptions at all about how the elevator is going to function. We all assume that elevators go in order because we have that experience and knowledge. That is what they are supposed to do. The kid had neither of those pieces of background so he asked a reasonable question.

As we look at our future roadmap plans for our Ai technology we consider the addition of domain experience as a critical component of being able to enhance the conversational experience. In this respect we will be able to not just have a reasonable conversation based on linguistic rules but add a totally different dimension by really 'understanding' what the conversation is about as well.

In case you were wondering the mom did indeed explain the rules of elevator operation to the boy. He looked pretty disappointed.

I think therefore I might be...

I saw this great job posting this morning for a PhD student in Logic based at the University of Groningen in the Netherlands:

Job Posting for PhD candidate in Logic

They are looking for a candidate with... a strong interest in Logic (especially in areas such as Modal Logic, Epistemic Logic, Dynamic Logic, Belief Revision theory, Game Logic, Quantum Logic, Linear Logic, Conditionals or Game Semantics) and its applications to modelling information flow, learning, agency, interaction and rationality in Artificial Intelligence, Theoretical Philosophy, Computer Science, Quantum Physics (including Quantum Information and Quantum Computation) or Game Theory. Fluent English is a prerequisite.

Is logic really a strong component of AI learning,interaction and rationality? In looking at the projects we are working on it seems to me that the easy part is anything that is logical. The difficult part is making sense of things that are not logical...ie the way we humans think and respond. In many respects this is why pattern matching has run it's course in 'chatbots'. Pattern Matching is a logical approach to responding to a conversation. ie I take your input..find a key word and then match it to a response. In real life we do something far more complicated and take significant illogical paths through a conversation.

In our dialog engine we are working on numerous paths that try to mirror how a conversation will flow. This is based on the context, content, tone and pace of the conversation. Being able to make 'jumps' from one topic to another or even introduce nonsensical components into a conversation mimic what happens in real life. Rarely does a conversation follow a fully logical path.

BTW I am going to take a wild guess here that having no social life is probably also a pre-requisite for that job.

My friends fail the Turing Test...

From Wikipedia:

"The Turing test is a proposal for a test of a machine's ability to demonstrate intelligence. Described by Alan Turing in the 1950 paper "Computing Machinery and Intelligence", it proceeds as follows: a human judge engages in a natural language conversation with one human and one machine, each of which try to appear human. All participants are placed in isolated locations. If the judge cannot reliably tell the machine from the human, the machine is said to have passed the test. In order to test the machine's intelligence rather than its ability to render words into audio, the conversation is limited to a text-only channel such as a computer keyboard and screen."

When Turing devised this test he couldn't have seen a world in which tens of billions of instant messages and text messages are sent every single day. To use this test as a measure of intelligence seems to have become a bit outdated. Why? Well I have plenty of friends who I chat to over IM that simply would fail the Turing test. Does this mean they are not intelligent? No it simply means that they use IM poorly, their grammar is terrible and their use of IM shorthand is annoying.

Perhaps a better test, Turing2.0, if you will would involve paying attention, not using shorthand that I don't understand and perhaps knowing how to finish a conversation so I can get back to work. Now that would be intelligent.

All the Presidents Men...

Happy Presidents Day. I thought that it was only fitting that I had a chat with Abraham Lincoln this morning at www.chatwithabe.com. Here is part of what we discussed:

___________________________________________

Abraham Lincoln: Hello.
You: good morning
Abraham Lincoln: Good morning, Friend.
You: you were the president
Abraham Lincoln: A wise man knows what he doesn't know. I would not say I was the greatest but I tried to be honest to the American people.
You: what number president were you
Abraham Lincoln: I was the 16th President of the United States of America.
You: did you like being the president
Abraham Lincoln: Leader of the free world is not a bad job but sometimes the pressure of the responsibility was overwhelming at times, especially during the war.
You: they have a sale going on at Macy's in your honor
Abraham Lincoln: I recently learned that the U.S. mint plans to update the penny, which currently bears my profile on the front and memorial on the back, in honor of the upcoming bicentennial of my birth. I've always felt honored to have my likeness on the coin, but I agree the hundred- year-old one cent piece could use a new look.

____________________________

I would be more impressed with the penny thing too.

How do you spell Galapagoes..?

Yesterday, Feb 12, 2009 was the 200th birthday of two of the greatest influencers over the modern world. Both Charles Darwin and Abraham Lincoln were born February 12th 1809.

At Virsona we have been building a virtual Abraham Lincoln. You can chat with him at www.chatwithabe.com. In building this virsona I have come to learn a lot about Abe. As with other impressive statesmen he knew how to use words as a powerful force in their own right. It's not how much you say but rather what you say. His speech at Gettysburg was resoundingly criticized at the time as it was far to short. Longer, rambling speeches with lots of flowery words were the norm for that period. Getting Ai s to understand how to construct meaningful responses is a tough problem. Using such key tools as wordnet and concept net we can make good attempts but being able to create a Gettysburg Speech from scratch is a way off yet.

One of the interesting technologies we are playing with to help is the use of genetic algorithms. These are self replicating 'codelets' that live in a virtual eco-system and each generation follows the survival of the fittest rule. Each codelet measures how it performs in responding to input and then the fittest move on. At that stage they create slightly altered versions of themselves and start the cycle over. This takes time but we are seeing some interesting results.

So maybe if Abe and Charlie had met down in the Galapagos Islands and discussed how to create the next great conversational Ai they may actually have come up with something rather cool.

The machines are coming...

There are a fair number of news articles around at the moment about the 'rise of the Machines'. With the new Terminator Movie due in May it's not really that surprising is it? A standard literary theme (man vs machine) which has been around a long time.

http://www.nzherald.co.nz/technology/news/article.cfm?c_id=5&objectid=10556240

However if the machines are ever really going to have a chance they first have to figure out how to talk with us. English as a conversational language is tough. Even many native speakers aren't that good at it - just switch on day time tv for examples.

At Virsona we are working on a world class Dialog Engine. I see everyday the complexity that we have to deal with in terms of trying to understand not just what is being said but how to respond to it. There are some increadible advances in Natural Language Processing but we still have a long way to go. Integrating situational awareness, conversation context and domain knowledge are all complex problems that we (as humans) handle without thinking about it.

Perhaps at the end of the day all the machines really need to be able to say is 'I'm afraid I can't do that Dave'.

How difficult can that be?