(My) ideal society

Each individual is respected as such and has the freedom and the means to pursue its own interests without having to harm the others.

Don't know how it looks like. It's a pretty simple (non-constructive) definition, however.
I'm sure mathematicians like it!

Read more at my webpage:
http://hpenedones.googlepages.com/thoughtsonlife

Note: This essay will be in beta version, longer than any Google product.

Personal productivity, happiness and optimization algorithms

I spend lots of time wondering about the best ways to be both more productive and happy. Curiously, I'm coming to the conclusion that this is exactly what I should not do.

Being productive, like being happy, requires living the present moment, not thinking about it.

If you want to complete a task, the best strategy is just doing it! You might start by setting up a plan, a sequence of smaller actions that lead you to your goal, but once you have this, just do it. Spending too much energy re-planning and judging yourself along the way is just counter-productive.

Curiously, this is not easy! Our brain seems to have some bad habits hard-wired. Want it or not, we start thinking about the past or making predictions about the future. Worse, we start multi-tasking (as you read this blog, you might also be listening to music, doing some work, or chatting with your friends in facebook)
Perhaps the only solution is to re-train our neuron connections. One way to do it would be meditating or repeatedly performing a task that requires one to be focused on the present. Feeling, not thinking. After enough practicing, the brain should start rewiring.

I recently came across this famous Hemingway sentence:

“Happiness in intelligent people is the rarest thing I know.”

Perhaps intelligent people have the tendency to plan too much? Planning involves predicting the reward associated with a set of possible actions and choosing the best ones. What if the reward function is not easily predictable? Perhaps the best optimization algorithm in this case is a greedy one. Don't plan to be happy only next year or next month or even tomorrow. You are dealing with a real-time multi-agent system, you have only partial and noisy data about the world, the system is recursive, and finding the optimal reward is probably NP-hard-as-it-can-be!

Increasing the scope

In the past it happened that I didn't publish some potentially interesting thoughts in this blog, just because they didn't exactly fit the "about intelligence" topic.
I'm fed up of this self-imposed censorship. In the future the scope will be broader.

Machine Learning to AI

John Langford wrote a very interesting post on the failures of Artificial Intelligence research and why Machine Learning has been a safer bet. Read it here.

Google CADIE vs Wolfram Alpha

Google already has a tradition of April fool's jokes: this year they are introducing an Artificial Intelligence brain!

They describe the development process of their so called CADIE : Cognitive Autoheuristic Distributed-Intelligence Entity like this:

"For several years now a small research group has been working on some challenging problems in the areas of neural networking, natural language and autonomous problem-solving. Last fall this group achieved a significant breakthrough: a powerful new technique for solving reinforcement learning problems, resulting in the first functional global-scale neuro-evolutionary learning cluster."

Remember, this is an April fool's hoax. But now compare it with Wolfram's announcement of the new Wolfram Alpha:

"I wasn’t at all sure it was going to work. But I’m happy to say that with a mixture of many clever algorithms and heuristics, lots of linguistic discovery and linguistic curation, and what probably amount to some serious theoretical breakthroughs, we’re actually managing to make it work."

I find them quite similar! ;)

Now more seriously: I don't doubt Wolfram Alpha will have interesting features, but please don't try to sell it like the ultimate AI search engine. By the way, Daniel Tunkelang has a recent and well informed post on this topic.

Update: Indeed this sneak preview of Wolfram Alpha shows some cool features! In the meanwhile Google also gave some steps in the direction of better public data/statistics visualization.

Machine Learning artwork

Today I tried out a great site to generate tag clouds, it is called wordle.net. I rendered some images just by copy-pasting the text from wikipedia about machine learning.

The results were pretty cool and I guess one could print awesome t-shirts with them. What do you say?

This one became officially my computer wallpaper:

ACM Paris Kanellakis Theory and Practice Award 2008

The 2008 ACM Paris Kanellakis Theory and Practice Award was awarded to Corinna Cortes and Vladimir Vapnik "for the development of Support Vector Machines, a highly effective algorithm for classification and related machine learning problems".

It's not the first time this award is given to Machine Learning people. In 2004 it was awarded to Yoav Freund and Robert Schapire "for the development of the theory and practice of boosting and its applications to machine learning."

I found a bit weird that they left Bernhard Boser and Isabelle Guyon out of the prize, because they were Vapnik's co-authors in the 1992 paper "A training algorithm for optimal margin classifiers", which I guess is considered to be the first paper on Support Vector Machines...

Anyway, congratulation to the winners. These are indeed elegant algorithms with sound theoretical foundations and numerous sucessful applications to vision, speech, natural language and robotics, to name just a few.

---------------------------
Remarks:

Thanks to my cousin Rui for the link to this news.

---------------------------
Related post:

Vapnik's picture explained.

Social features on this blog

The readers of this blog can now:

1. Easily subscribe to the RSS feed with their reader of choice [left panel].
2. Decide to become a visible "follower" of this blog [left panel].
3. Rate each blog entry from 1 to 5 stars [end of each post].

I would be particularly happy to see people rating the posts. It's less informative than writing comments but still it's very good feedback for me.

Thanks!

Vapnik's picture explained

This is an extremely geek picture! :) Let's try to explain it:

First of all, as many of you know, the gentleman in the picture is Prof. Vladimir Vapnik. He is famous for his fundamental contributions to the field of Statistical Learning Theory, such as the Empirical Risk Minimization (ERM) principle, VC-dimension and Support Vector Machines.

Then we notice the sentence in the board: it resembles the famous "All your base are belong to us"! This is a piece of geek culture that emerged after a "broken English" translation of a Japanese video game for Sega Mega Drive .



Wait, but they replaced the word "Base" by "Bayes"!?
Yes, that Bayes, the British mathematician known for the Bayes' theorem.
Okay, seems fair enough, we are dealing with people from statistics...

By the moment we think things can not get more geeky, we realize there is scary inequality written on the top of the white board:

My goodness, what's this?! Okay, that's when things get really technical:
This is a probabilistic bound for the expected risk of a classifier under the ERM framework. In simple terms, it relates the classifier's expected test error with the training error on a dataset of size l and in which the cardinality of the set of loss functions is N.
If I'm not mistaken, the bound holds with probability (1 - eta) and applies only to loss functions bounded above by 1.

Sweet! Now that we got the parts, what's the big message?

Well, it's basically a statement about the superiority of Vapnik's learning theory over the Bayesian alternative. In a nutshell, the Bayesian perspective is that we start with some prior distribution over a set of hypothesis (our beliefs) and we update these according to the data that we see. We then look for an optimal decision rule based on the posterior distribution.
On the other hand, in Vapnik's framework there are no explicit priors neither we try to estimate the probability distribution of the data. This is motivated by the fact that density estimation is a ill-posed problem, and therefore we want to avoid this intermediate step. The goal is to directly minimize the probability of making bad decision in the future. If implemented through Support Vector Machines, this boils down to finding the decision boundary with maximal margin to separate the classes.

And that's it, folks! I hope you had fun decoding this image! :)

Computer Vision vs Computer Graphics

If I had to explain what computer vision is all about, in just one snapshot, I would show you this:




Computer Graphics algorithms go from the parameter space to the image space (rendering), computer vision algorithms do the opposite (inverse-rendering). Because of this, computer vision is basically a (very hard) problem of statistical inference.
The common approach nowadays is to build a classifier for each kind of object and then search over (part of) the parameter space explicitly, normally by scanning the image for all possible locations and scales. The remaining challenge is still huge: how can a classifier learn and generalize, from a finite set of examples, what are the fundamental characteristics of an object (shape, color) and what is irrelevant (changes in illumination, rotations, translations, occlusions, etc.).
This is what is keeping us busy! ;)

PS - Note that changes in illumination induce apparent changes in the color of the object and rotations induce apparent changes in shape!