tag:blogger.com,1999:blog-36376050101441316102024-02-21T19:41:43.762-08:00About IntelligencePosts on Artificial Intelligence, Vision, Machine Learning, Robotics, Neuroscience, Consciousness, Philosophy of Mind and more!Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.comBlogger44125tag:blogger.com,1999:blog-3637605010144131610.post-54446443449667581102013-07-14T07:44:00.004-07:002013-07-28T05:26:44.927-07:00Moved to wordpressThis blog will now be available at <a href="http://blog.hpenedones.org/">http://blog.hpenedones.org</a><br />
<br />
I don't have plans to continue posting here, so please update your bookmarks to the new address.<br />
<br />
In addition, my new homepage is now hosted at <a href="http://hpenedones.org/">http://hpenedones.org</a><br />
<br />
Thanks,<br />
Hugo Penedones<br />
<br />Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com0tag:blogger.com,1999:blog-3637605010144131610.post-27962368176091374152012-11-20T05:58:00.000-08:002012-11-20T05:58:59.700-08:00Machine Learning Workshop - Idiap EPFL 2012Yesterday I attended to this workshop at EPFL:<br />
<br />
<a href="http://www.idiap.ch/workshop/mlws/">http://www.idiap.ch/workshop/mlws/</a><br />
<br />
It was a good opportunity to see old friends and colleagues, and listen about their latest research. In general, the quality of the talks was quite good, ranging from very theoretical machine learning (sparse coding, optimization, etc.) to commercial applications of computer vision (<a href="http://www.faceshift.com/">www.faceshift.com</a>).<br />
Somewhere in the middle of that spectrum, I also quite liked the talk about learning image local descriptors (<a href="http://cvlab.epfl.ch/~trzcinsk/publications/brief_pami.pdf">BRIEF</a> and <a href="http://cvlab.epfl.ch/~trzcinsk/publications/nips12.pdf">LBGM</a>) as a compact and efficient alternative to SIFT or SURF, which are hand-designed, slower and use more bits. There were also applications to speech, face analysis and even remote sensing.<br />
<br />
Have a look at the program and keep an eye on it in the coming days, as the slides will probably become available. You will find several other interesting talks:<br />
<br />
<a href="http://www.idiap.ch/workshop/mlws/programme-2012">http://www.idiap.ch/workshop/mlws/programme-2012</a><br />
<br />
<br />Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com0tag:blogger.com,1999:blog-3637605010144131610.post-26936560000005979062012-11-12T03:24:00.001-08:002012-11-12T03:24:55.138-08:00Active Appearance Models<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.blogger.com/video.g?token=AD6v5dws7DbW9O-YXpjBxAXHtm62dXCoT7xD7xpogEsmc1L3fWgcqzkvKSG6sZS96Z-VqrPgW0ifJPfcl6Urf3Vo6A' class='b-hbp-video b-uploaded' frameborder='0'></iframe></div>
<br />
Lately, I have been working with Deformable Models and I am surprised by how well they can work.<br />
In the video above I am using an Inverse Compositional Active Appearance Model, which was trained with images of myself. It's specially tuned for my face, but I still find it quite impressive how well it can track my face in realtime!<br />
On the other hand, this model is quite sensitive to lighting conditions and partial occlusions. Training it, is also somehow of an art, because, as opposed to discriminative models, increasing the amount of training data might actually decrease performance. This happens because we use PCA to learn the linear models of shape and texture, which will degrade if data has too much variation or noise.<br />
Still, it's quite impressive what one can achieve by annotating a few images (about 50, in this case). In addition, as one annotates images, one can start training models that will help us landmark the next ones (in a process of "bootstrapping", similar to the one in compilers).Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com0tag:blogger.com,1999:blog-3637605010144131610.post-73123957113134346022010-11-12T06:52:00.000-08:002010-11-12T07:55:47.535-08:00The AI set of functionsI recently read an article from Y. Bengio and Y. LeCun named "Scaling Learning Algorithms to AI". You can also find it as a book chapter in "Large-Scale Kernel Machines"L. Bottou, O. Chapelle, D. DeCoste, J. Weston (eds) MIT Press, 2007.<br /><br />In some aspects it is an "opinion paper" where the authors advocate for deep learning architectures and their vision of the Machine Learning. However, I think the main message is extremely relevant. I was actually surprised to see how much it agrees with my own opinions.<br />Here is how I would summarize it:<br /><br />- no learning algorithm can be completely universal, due to the "No free lunch theorem"<br />- that's not such a big problem: we don't care about the set of all possible functions<br />- we care about the "AI set", which contains the functions useful for vision, language, reasoning, etc.<br />- we need to create learning algorithms with an inductive bias towards the AI set<br />- the models should "efficiently" represent the functions of interest, in terms of having low Kolmogorov complexity<br />- researchers have exploited the "smoothness" prior extensively with non-parametric methods. However many manifolds of interest have strong local variations.<br />- we need to explore other types of priors, more appropriate to the AI set.<br /><br />The authors then give examples of two "broad" priors, such as the sharing of weights in convolutional networks (inspired by translation invariance in vision) and the use of multi-layer architectures (which can be seen as levels of increasing abstraction).<br /><br />Of course here is where many alternatives are open! Many other useful inductive-bias could be found. That's where I think we should focus our research efforts! :)Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com0tag:blogger.com,1999:blog-3637605010144131610.post-906809782706754572010-11-08T07:53:00.000-08:002010-11-08T07:56:58.765-08:00Tutorial: handwritten digit recognition with convolutional neural networksI recently added to my webpage a <a href="https://sites.google.com/site/hpenedones2/sourcecode/usps_cnn">tutorial</a> on how to use <a href="http://torch5.sourceforge.net">torch5</a> library to train a convolutional neural network for the task of handwritten digit recognition.Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com2tag:blogger.com,1999:blog-3637605010144131610.post-40098463308103989602010-10-23T17:06:00.000-07:002010-10-23T18:59:02.766-07:00NYC Machine Learning Symposium 2010<div>The <a href="http://www.nyas.org/events/Detail.aspx?cid=1cdd40d6-fc64-44e8-b225-db49ac0d90f1">event</a> took place yesterday at the New York Academy of Sciences, a building right next to the World Trade Center. The views from the 40th floor were breathtaking:</div><div><br /></div><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEK3GjnYdQ5v_FNUevy9UB_fMJj60rW123Lgnoez-2LpU6ymaGGAUWkkQudvqQfylTE1vxwjX0QfocwGFrJ5sEttKsdb1V4S6UBQehU_ztaJHM3ALviJXN6AlZwWRYMTqmxSyFth4ESaE/s1600/photo.jpg"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 240px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEK3GjnYdQ5v_FNUevy9UB_fMJj60rW123Lgnoez-2LpU6ymaGGAUWkkQudvqQfylTE1vxwjX0QfocwGFrJ5sEttKsdb1V4S6UBQehU_ztaJHM3ALviJXN6AlZwWRYMTqmxSyFth4ESaE/s320/photo.jpg" border="0" alt="" id="BLOGGER_PHOTO_ID_5531400542634445986" /></a><br />The names of the participants in the room was no less impressive, (by no special order): Corinna Cortes (Google), Rob Schapire and David Blei (Princeton University), John Langford and Alex Smola (Yahoo), Yann LeCun (NYU), Sanjoy Dasgupta (Univ. California), Michael Collins (MIT), Patrick Haffner (AT&T), among many others.<br /><div>I particularly liked to see the latest developments in LeCun's group, including a demo by Benoit Corda and Clément Farabet on speeding-up Convolutional Neural Networks with GPUs and FPGAs. </div><div>Alex Kulezka and Ben Taskar had a nice work on "Structured Determinantal Point Processes", which can be seen as a probabilistic model with a bias towards diversity of the hidden structures. </div><div>Mathew Hoffman (with D. Blei and F. Bach) used stochastic gradient descent (widely used among neural network community) for online training of topic models. Sean Gerrish and D. Blei actually had a funny application of topic models to the prediction of votes by Senators!</div><div>I was also happy to see that there is some Machine Learning being applied to the problem of sustainability and the environment. Gregory Moore and Charles Bergeron had a poster on trash detection in lakes, rivers and oceans. </div><div>To conclude, the best student paper award went to a more theoretical paper by Kareem Amin, Michael Kearns and Umar Syed (U Penn) called "Parametric Bandits, Query Learning, and the Haystack Dimension", which defines a measure of complexity for multi-armed bandit problems in which the number of actions can be infinite (there is some analogy to the role of VC-dimension in other learning models).</div><div><br /></div><div>There were probably many other interesting posters worth being mentioned, but I didn't have the chance to check them all!</div><div><br /></div><div>On the personal side: my summer internship at NEC Labs with David Grangier is about to finish. It was an amazing learning experience and I am very grateful for it. </div><div>Next step: back to Idiap Research Institute, EPFL and all the Swiss lakes and mountains! :)</div><div><br /></div><div><br /></div>Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com0tag:blogger.com,1999:blog-3637605010144131610.post-33400249925657060132010-07-06T03:13:00.000-07:002010-07-06T03:33:17.364-07:00Machine Learning recent sitesIn the last few months (in which I haven't posted in this blog) there were a few interesting web platforms related to Machine Learning poping-up, most notably:<div><br /></div><div><a href="http://mlcomp.org">MLcomp.org</a> - you can upload your datasets and/or your algorithms, and experiments will run automatically. Then you can see statistics related to classifier performances and computation times. It is intended to help researchers and practitioners comparing different methods, and it works as a collaborative platform where code and data can be shared.</div><div><br /></div><div><a href="http://metaoptimize.com">MetaOptimize.com</a> - it contains a great QA about Machine Learning and related topics, using the same web platform <a href="http://www.stackoverflow.com">StackOverflow</a> has for programming topics. </div><div><br /></div><div>I find these two websites a great way to improve collaboration among the ML community. Highly recommended!</div><div><br /></div><div>The latest link is more market oriented, and it comes from Google:</div><div><br /></div><div><a href="http://code.google.com/apis/predict/">Google Predict</a> : it puts together well established ML algorithms in an API that developers can use to make predictions on their own datasets. </div><div><br /></div><div><br /></div>Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com0tag:blogger.com,1999:blog-3637605010144131610.post-32512478417650135362009-12-08T02:50:00.000-08:002009-12-08T03:01:14.254-08:00Optimism as Artificial Intelligence Pioneers Reunite<br><br />Just a short link to an <a href="http://www.nytimes.com/2009/12/08/science/08sail.html?_r=1">article of the New York Times about AI</a>.<br /><br />In 1978, Dr. McCarthy wrote, “human-level A.I. might require 1.7 Einsteins, 2 Maxwells, 5 Faradays and .3 Manhattan Projects.”<br /><br />I think we probably have the genius scientists around, but not so sure about the 0.3 Manhattan Projects!<br /><br />Update: You might also want to read <a href="http://www.vetta.org/2009/12/tick-tock-tick-tock-bing/">latest Shane Legg's predictions about human level artificial intelligence</a>.Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com0tag:blogger.com,1999:blog-3637605010144131610.post-13266148525140499502009-12-07T11:47:00.000-08:002009-12-07T12:07:11.562-08:00TEDx GenevaToday I assisted to the first edition of TEDx Geneva. This was a locally-organized event following the same spirit of the original TED talks: "ideas worth spreading".<br />I think the <a href="http://www.tedxgeneva.com/programme-2009/">program</a> was really good, because in this region there are some many incredible organizations. He could listen to people from CERN, EPFL, the United Nations, the Red Cross and some independent Swiss adventurers and entrepreneurs. We also had the opportunity to (re)watch some videos of the most popular TED talks recorded in the US. <div>All the speakers spoke in English, which in my opinion degraded the level of their presentations, simply because it's not their native language. Even if one is relatively fluent, it's always harder to make jokes and be entertaining. The event was also a bit too long, covering the full day.</div><div><br /></div><div>Still, I greatly appreciated the experience and recommend it to others!</div><div><br /></div>Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com1tag:blogger.com,1999:blog-3637605010144131610.post-54054765135290997512009-11-11T03:13:00.000-08:002012-11-27T08:36:10.863-08:00Choosing my tools<div>
I'm doing research in the fields of Machine Learning and Computer Vision, so each time we have an idea for a new algorithm, I have to write code, run experiments and compare results. I have realized that the experimental part is really the bottleneck, we have more ideas than we can test. For this reason, it's critical to chose a good set of tools you can work with. This is a list of my current choices, but I am continuously looking for more efficient tools. </div>
<div>
<br /></div>
<div>
<b>Operating system: </b></div>
<div>
<br /></div>
<div>
<a href="http://www.apple.com/macosx/">Snow Leopard</a> - In my opinion, Mac OS X has an excellent balance between control and usability. You have beautiful graphical interfaces, that just work, but still have a fully functional Unix shell. </div>
<div>
<br />
<b>Update:</b> Lately my preference is to use <a href="http://www.ubuntu.com/">Ubuntu Linux</a> because I have much fewer problems with apt-get than with macports. Sometimes, professionally I also use Windows. It seems that is hard to stick to one OS, when you change project, job, etc.<br />
<br /></div>
<div>
<b>Text Editor / Programming Environment:</b></div>
<div>
<br /></div>
<div>
<a href="http://macromates.com/">Textmate</a> - again, it's an excellent compromise between simplicity, usability and customizability. You can create your own code snippets (using shell commands, ruby, python and more), but to me it seems much easier to learn than vim or emacs.</div>
<div>
<br />
<b>Update:</b> Again, went back to the basics, and started using <a href="http://www.vim.org/">vim</a> and gvim. It is available in all the platforms, there is a much bigger user base and I really like the power of the command mode. In addition, recently I learnt how to write simple vi plugins using python, which literally means I can do whatever I want with my editor.<br />
<br /></div>
<div>
<b>Programming Language:</b></div>
<div>
<br /></div>
<div>
<a href="http://en.wikipedia.org/wiki/C%2B%2B">C++</a> - absolute power. So powerful that one must be very careful using it. <a href="http://en.wikipedia.org/wiki/Scott_Meyers">Some people say</a>, C++ is actually a federation of languages, which includes C, object oriented stuff, templates and standard libraries. Although I've been using it for while, I feel there is always more to learn about it. <br />
<br />
<b>Update: </b>In addition to C++ (and C which I really love), I also started using some scripting languages. First I learnt Lua, so that I could use the <a href="http://www.torch.ch/">Torch</a> Machine Learning Library. Then, I started using python, which I really love due to the wide availability of (easily installable) libraries. Ah, I look forward to learn the new C++11 standard, which seems to be quite neat.</div>
<div>
<br />
<b>Build System (new):</b><br />
<br />
<a href="http://www.cmake.org/">cmake</a> - it's cross platform and simple enough to start using it. I don't know the advanced features, but it's pretty easy to create a project that generates libraries and executables and links properly with other dependencies (like OpenCV).<br />
<br />
<br />
<div>
<b>Source control system:</b></div>
<div>
<br /></div>
<div>
<a href="http://git-scm.com/">git</a> - I was using <a href="http://subversion.tigris.org/">subversion</a> before, but I guess the idea of distributed repositories makes sense. You can work locally and still commit changes that you can synchronize later. So far, I use less than 2% of the commands!</div>
<div>
<br />
<b>Update:</b> git is definitely here to stay. Now I use private and public hosted repositories with Github or Bitbucket.</div>
<br />
<br />
<br />
<b>Cloud Computing (new):</b><br />
<br />
<a href="http://aws.amazon.com/">Amazon EC2</a> - I also used the IBM Smart Cloud, but Amazon has more features and better APIs. Recently, with the introduction of the spot instances, things also got a lot cheaper when you need to process large amounts of data.<br />
<br />
<br />
<br />
<b>NoSQL Databases (new):</b><br />
<br />
<a href="http://redis.io/">redis</a> - redis is what we can call a "data structure server" and it's probably the nicest piece of software I started using recently. It is just beautiful. Simple. Intuitive. Fast. I can not recommend it enough.<br />
<div>
<br /></div>
<br />
<b>Computer Vision Library:</b></div>
<div>
<br /></div>
<div>
<a href="http://opencv.willowgarage.com/wiki/">OpenCV</a> - it's quite useful for the low and intermediate level things (load and save images, convert color spaces, edge detection, SURF descriptors etc.). It also has higher level algorithms, but when you're doing research in the field, these are not so useful. It lacks some object-oriented design, but version 2.0 is starting to move in that direction.</div>
<div>
<br /></div>
<div>
<b>Machine Learning library:</b></div>
<div>
<br /></div>
<div>
None. Here I'm re-inventing the wheel, because I want to know everything about wheels. I do my own implementations of AdaBoost, EM algorithm, Kmeans and stuff like that. For a nice discussion of code re-use in the machine learning domain, read this <a href="http://mloss.org/community/blog/2009/oct/14/re-implement-or-reuse/">discussion at mloss.org</a> </div>
<div>
</div>
<div>
<br /></div>
<div>
<b>Object Serialization Library:</b></div>
<div>
<br /></div>
<div>
<a href="http://www.boost.org/doc/libs/release/libs/serialization/">boost-serialization</a> - I need to save the models to files in order to load them later. If I were using OpenCV for Machine Learning, I could also use the functions they provide for serialization, but I'm not. With boost I can serialize objects to xml or binary format. It's a bit tricky to use, because it uses C++ templates and when you have compile time errors it's really hard to understand why. I'm not specially happy with this choice, but once you get your code right, it works pretty well.</div>
<div>
<br /></div>
<div>
<b>Debugging:</b></div>
<div>
<br /></div>
<div>
<a href="http://www.gnu.org/software/gdb/">gdb</a> - pretty much of a standard. I haven't yet chosen an interface for it... Maybe I don't even need one. I find <a href="http://www.gnu.org/software/ddd/">ddd</a> look and feel really horrible! Maybe I will start using <a href="http://developer.apple.com/TOOLS/Xcode">xcode</a> interface to gdb for debugging. Not sure. Actually, 90% of the times I will identify the bug by making some prints and looking at the code, so I don't even run gdb.</div>
<div>
<br /></div>
<div>
<b>Static code analysis:</b></div>
<div>
<br /></div>
<div>
<a href="http://sourceforge.net/apps/mediawiki/cppcheck/index.php?title=Main_Page">cppcheck</a> - this is a recent choice, but it seems to give some useful alerts.</div>
<div>
<br /></div>
<div>
<b>Run-time code analysis:</b></div>
<div>
<br /></div>
<div>
<a href="http://www.blogger.com/valgrind.org/">valgrind</a> - I'm not using it regularly yet, but it's on top of my priorities. This should be the ultimate tool to help you find memory leaks in your code. I didn't manage to install it in snow leopard, which can actually lead me to downgrade to leopard. Have to think about it.</div>
<div>
<br /></div>
<div>
<b>Plotting:</b></div>
<div>
<br /></div>
<div>
<a href="http://www.gnuplot.info/">gnuplot</a> - really powerful and configurable. This one is a safe bet, although I heard there is nice python software as well.</div>
<div>
<br /></div>
<div>
<b>Image Processing:</b></div>
<div>
<br /></div>
<div>
<a href="http://www.imagemagick.org/">ImageMagick</a> (convert command) - good to resize pictures, convert colors, etc. I mean, from the shell, this is not to replace <a href="http://www.gimp.org/">gimp</a> or the like.</div>
<div>
<br /></div>
<div>
<b>Video Processing:</b></div>
<div>
<br /></div>
<div>
Here I should be using <a href="http://www.mplayerhq.hu/DOCS/man/en/mplayer.1.html">mplayer / mencoder</a> from the command line, but again I still have to solve some compatibility problems with snow leopard. <a href="http://ffmpeg.org/">ffmpeg</a> is also useful.</div>
<div>
<br /></div>
<div>
<b>Terminal multiplexer:</b></div>
<div>
<br /></div>
<div>
<a href="http://www.gnu.org/software/screen/">screen</a> - sometimes one needs to run experiments remotely, and you want your processes to continue running smoothly when you log off. Use screen for this.</div>
<div>
<br /></div>
<div>
<b>Screen sharing:</b></div>
<div>
<br /></div>
<div>
<b></b><a href="http://synergy2.sourceforge.net/">synergy</a> - I work directly on my macbook and I connect another screen to it. However, I also want to interact with my linux desktop at work. I use synergy to have an extended desktop, share the mouse and the keyboard across different computers over the network. It's really cool!</div>
<div>
<br /></div>
<div>
<b>Automated backups:</b></div>
<div>
<br /></div>
<div>
<a href="http://www.apple.com/macosx/what-is-macosx/time-machine.html">Time Machine</a> - I have an external hardisk which backs up pretty much everything automatically when I connect it to my macbook. Things in my desktop are backed up by a central procedure implemented in <a href="http://www.idiap.ch/">my research institute</a>. </div>
<div>
<br />
<b>Update:</b> I still use Time Machine in one computer, but now I rely more on cloud storage. I use Google Drive for some documents, PicasaWeb for pictures and use either Github or Bitbucket for source code or latex papers.<br />
<br /></div>
<div>
<b>Shell tools:</b></div>
<div>
<br /></div>
<div>
cat, head, tail, cut, tr, grep, sort, uniq.... sometimes sed and awk...</div>
<div>
I mostly use this to manipulate data files before feeding them to gnuplot and make some graphics.</div>
<div>
<br /></div>
<div>
<b>Document preparation system:</b></div>
<div>
<br /></div>
<div>
<a href="http://www.latex-project.org/">latex</a> - this is the standard in the scientific community and there are good reasons for that.</div>
<div>
<a href="http://www.bibtex.org/">bibtex</a> - to do proper citations to other people's articles or books.</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<b>Source code documentation:</b></div>
<div>
<br /></div>
<div>
<a href="http://www.doxygen.org/">doxygen</a> - I don't really develop libraries for other people to use, but generating documentation automatically from your source code can help you improve it. If you use doxygen with <a href="http://www.graphviz.org/">graphviz</a> you can for example see the class hierarchies and dependencies of your code.</div>
<div>
<br />
<br />
<br /></div>
<div>
<br /></div>
<div>
What tools do you use? Do you have any recommendations for me? I guess that the OS, editor and programming language are the most polemic! But, what about the others? Any ideas?</div>
Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com3tag:blogger.com,1999:blog-3637605010144131610.post-88691067050729082242009-11-01T06:07:00.000-08:002009-11-12T09:01:12.164-08:00Open PhD and Postdoc positions<div><br /></div><div>My <a href="http://www.idiap.ch/%7Efleuret/">supervisor</a> is leading a new European project called <a href="http://mash-project.eu/">MASH</a>, which stands for "Massive Sets of Heuristics". There are open positions here in Switzerland, as well as in France, Germany and Czech Republic.</div><div><br /></div><div>The goal is to solve complex vision and goal planning problems in a collaborative way. It will be tested in 3D video games and also in a real robotic arm. Collaborators will submit pieces of code (heuristics) that can help the machine solving the problem at hand. In the background, machine learning algorithms will be running to choose the best heuristics.</div><div><br /></div><div>If you are interested in: probabilities, applied statistics, information theory, signal processing, optimization, algorithms and C++ programming, you might consider applying! </div>Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com1tag:blogger.com,1999:blog-3637605010144131610.post-48181243016868770422009-10-14T00:24:00.000-07:002009-10-14T00:44:03.909-07:00Gmail Machine LearningI just quickly tried the new Gmail Labs feature "Got the wrong Bob"? and it actually works quite nicely! I put some email addresses of family members, followed by the address of an old professor, who has the same first name of one of my cousins, and... Gmail found it! :) It suggested right way to change to the correct person, based on context!<div>The other new feature, called "Don't forget Bob", is probably simpler, but quite useful as well. As I typed names of some close friends, I got more suggestions of friends I often email jointly with the previous ones.</div><div>I wonder if the models to run this feature are very complicated. Probably they are not. I guess one just has to estimate the probability of each email address in our contacts to appear in the "To:" field, given the addresses we have already typed. To estimate these, you just have to use a frequentist approach and count how many times this happened in the past. With this in hands, "Got the wrong Bob?" will notice unlikely email addresses and "Don't forget Bob" will suggest likely ones that are missing.</div><div><br /></div><br /><div>I think it's a really cool idea, in the same spirit of "Forgotten Attachement Detector". A bit of machine learning helping daily life! </div>Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com0tag:blogger.com,1999:blog-3637605010144131610.post-53659804519113004192009-10-05T12:03:00.000-07:002009-10-05T12:25:15.204-07:00Schools kill creativity<div><br /></div><div>My good friend Miguel called my attention to a TED talk that you might also find interesting:</div><div><br /></div><br /><object width="334" height="326"><param name="movie" value="http://video.ted.com/assets/player/swf/EmbedPlayer.swf"><param name="allowFullScreen" value="true"><param name="wmode" value="transparent"><param name="bgColor" value="#ffffff"> <param name="flashvars" value="vu=http://video.ted.com/talks/dynamic/SirKenRobinson_2006-medium.flv&su=http://images.ted.com/images/ted/tedindex/embed-posters/SirKenRobinson-2006.embed_thumbnail.jpg&vw=320&vh=240&ap=0&ti=66&introDuration=16500&adDuration=4000&postAdDuration=2000&adKeys=talk=ken_robinson_says_schools_kill_creativity;year=2006;theme=bold_predictions_stern_warnings;theme=master_storytellers;theme=top_10_tedtalks;theme=how_the_mind_works;theme=how_we_learn;theme=the_creative_spark;event=TED2006;&preAdTag=tconf.ted/embed;tile=1;sz=512x288;"><embed src="http://video.ted.com/assets/player/swf/EmbedPlayer.swf" pluginspace="http://www.macromedia.com/go/getflashplayer" type="application/x-shockwave-flash" wmode="transparent" bgcolor="#ffffff" width="334" height="326" allowfullscreen="true" flashvars="vu=http://video.ted.com/talks/dynamic/SirKenRobinson_2006-medium.flv&su=http://images.ted.com/images/ted/tedindex/embed-posters/SirKenRobinson-2006.embed_thumbnail.jpg&vw=320&vh=240&ap=0&ti=66&introDuration=16500&adDuration=4000&postAdDuration=2000&adKeys=talk=ken_robinson_says_schools_kill_creativity;year=2006;theme=bold_predictions_stern_warnings;theme=master_storytellers;theme=top_10_tedtalks;theme=how_the_mind_works;theme=how_we_learn;theme=the_creative_spark;event=TED2006;"></embed></object><br /><div><br /></div><div><div>Ken Robinson argues that "schools kill creativity", because kids are not given the chance to discover their interests and talents. Since very soon, students get a negative reward for making mistakes, which makes them too risk averse. He goes further, saying that the educational system is built to create university professors, leaving the majority of the students behing along the way. More space should be given to other forms of expressing intelligence, such as the arts or sports.</div><div><br /></div><div>I strongly recommend this video. Besides the interest of the subject, the presentation is actually quite funny, it somehow resembles a British-style stand-up comedy! </div><div><br /></div></div><div><br /></div>Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com0tag:blogger.com,1999:blog-3637605010144131610.post-22733081538985331702009-08-02T11:25:00.000-07:002009-08-02T11:49:18.701-07:00(My) ideal society<span style="font-weight: bold;">Each individual is respected as such and has the freedom and the means to pursue its own interests without having to harm the others.</span><br /><br /> Don't know how it looks like. It's a pretty simple (non-constructive) definition, however.<br />I'm sure mathematicians like it!<br /><br /> Read more at my webpage:<br /><a href="http://hpenedones.googlepages.com/thoughtsonlife">http://hpenedones.googlepages.com/thoughtsonlife<br /></a><br /> Note: This essay will be in beta version, longer than any Google product.Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com1tag:blogger.com,1999:blog-3637605010144131610.post-8818991737039899222009-07-22T05:56:00.000-07:002009-07-22T06:50:39.882-07:00Personal productivity, happiness and optimization algorithmsI spend lots of time wondering about the best ways to be both more productive and happy. Curiously, I'm coming to the conclusion that this is exactly what I should not do.<br /><br />Being productive, like being happy, requires living the present moment, not thinking about it.<br /><br />If you want to complete a task, the best strategy is just doing it! You might start by setting up a plan, a sequence of smaller actions that lead you to your goal, but once you have this, just do it. Spending too much energy re-planning and judging yourself along the way is just counter-productive.<br /><br />Curiously, this is not easy! Our brain seems to have some bad habits hard-wired. Want it or not, we start thinking about the past or making predictions about the future. Worse, we start multi-tasking (as you read this blog, you might also be listening to music, doing some work, or chatting with your friends in facebook)<br />Perhaps the only solution is to re-train our neuron connections. One way to do it would be meditating or repeatedly performing a task that requires one to be focused on the present. Feeling, not thinking. After enough <a href="http://www.amazon.com/Practicing-Mind-Bringing-Discipline-Focus/dp/0977657205">practicing</a>, the brain should start rewiring.<br /><br />I recently came across this famous Hemingway sentence:<br /><h1 style="margin: 0pt; font-size: 12px;"><br /></h1><h1 style="margin: 0pt; font-size: 12px;">“Happiness in intelligent people is the rarest thing I know.”</h1><br />Perhaps intelligent people have the tendency to plan too much? Planning involves predicting the reward associated with a set of possible actions and choosing the best ones. What if the reward function is not easily predictable? Perhaps the best optimization algorithm in this case is a <a href="http://en.wikipedia.org/wiki/Greedy_algorithm">greedy</a> one. Don't plan to be happy only next year or next month or even tomorrow. You are dealing with a real-time multi-agent system, you have only partial and noisy data about the world, the system is recursive, and finding the optimal reward is probably NP-hard-as-it-can-be!Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com7tag:blogger.com,1999:blog-3637605010144131610.post-7951181074336788662009-07-22T05:41:00.000-07:002009-07-22T05:52:01.895-07:00Increasing the scopeIn the past it happened that I didn't publish some potentially interesting thoughts in this blog, just because they didn't exactly fit the "about intelligence" topic.<br />I'm fed up of this self-imposed censorship. In the future the scope will be broader.Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com0tag:blogger.com,1999:blog-3637605010144131610.post-72749003273540966622009-05-06T13:44:00.000-07:002009-05-06T13:51:29.865-07:00Machine Learning to AIJohn Langford wrote a very interesting post on the failures of Artificial Intelligence research and why Machine Learning has been a safer bet. Read it <a href="http://hunch.net/?p=703">here</a>.Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com0tag:blogger.com,1999:blog-3637605010144131610.post-65102023585165328092009-04-01T03:07:00.000-07:002009-05-04T12:07:22.222-07:00Google CADIE vs Wolfram Alpha<a href="http://en.wikipedia.org/wiki/Google%27s_hoaxes">Google already has a tradition of April fool's jokes</a>: this year they are introducing an Artificial Intelligence brain!<br /><br />They describe the development process of their so called <a href="http://books.google.com/intl/en_us/landing/cadie/">CADIE : Cognitive Autoheuristic Distributed-Intelligence Entity</a> like this:<br /><br />"<span style="font-style: italic;">For several years now a small research group has been working on some challenging problems in the areas of neural networking, natural language and autonomous problem-solving. Last fall this group achieved a significant breakthrough: a powerful new technique for solving reinforcement learning problems, resulting in the first functional global-scale neuro-evolutionary learning cluster.</span>"<br /><br />Remember, this is an April fool's hoax. But now compare it with Wolfram's <a href="http://blog.wolfram.com/2009/03/05/wolframalpha-is-coming/">announcement</a> of the new Wolfram Alpha:<br /><br /><span style="font-style: italic;">"I wasn’t at all sure it was going to work. But I’m happy to say that with a mixture of many clever algorithms and heuristics, lots of linguistic discovery and linguistic curation, and what probably amount to some serious theoretical breakthroughs, we’re actually managing to make it work."</span><br /><br />I find them quite similar! ;)<br /><br />Now more seriously: I don't doubt Wolfram Alpha will have interesting features, but please don't try to sell it like the ultimate AI search engine. By the way, Daniel Tunkelang has a <a href="http://thenoisychannel.com/2009/03/31/wolfram-alpha-first-hand-impressions/">recent and well informed post on this topic</a>.<br /><br />Update: Indeed this <a href="http://www.youtube.com/watch?v=hYhLsQPHNas">sneak preview of Wolfram Alpha</a> shows some cool features! In the meanwhile <a href="http://www.youtube.com/watch?v=9Qt2n34VEr4">Google also gave some steps in the direction of better public data/statistics visualization</a>.Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com5tag:blogger.com,1999:blog-3637605010144131610.post-84883119417926883982009-03-28T12:12:00.000-07:002009-03-31T01:59:42.356-07:00Machine Learning artworkToday I tried out a great site to generate tag clouds, it is called <a href="http://www.wordle.net/">wordle.net</a>. I rendered some images just by copy-pasting the text from <a href="http://en.wikipedia.org/wiki/Machine_learning">wikipedia about machine learning</a>.<br /><br />The results were pretty cool and I guess one could print awesome t-shirts with them. What do you say?<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRi-ph9qui0QFSs2e7lDMU-JaTttTpjTmGeiAtbR49yOAAVYbJ4RftPoN6hPpQUfaFB3mSbua8eUtnK7Yvt2G3B_JHdPbihXt7UStQhN7PXP_xf-GBxJEmWByjaLi9nKRkBVoPpKfuHfA/s1600-h/Picture+26.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 320px; height: 165px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRi-ph9qui0QFSs2e7lDMU-JaTttTpjTmGeiAtbR49yOAAVYbJ4RftPoN6hPpQUfaFB3mSbua8eUtnK7Yvt2G3B_JHdPbihXt7UStQhN7PXP_xf-GBxJEmWByjaLi9nKRkBVoPpKfuHfA/s320/Picture+26.png" alt="" id="BLOGGER_PHOTO_ID_5318349337061816370" border="0" /></a><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjK55dtJ5O-KbI9m1icgyFw1oDOLQLVHdFhxsAt-EsX5Dehbg2NMJ1TeOM4xoL3Gmm2ijop451LG1KSsCJ6u7ZPuHRJnp9_4UPFpqujBk3N4VomoxsX_mKG_ODPIAoP45U1SpCFDD7sS9g/s1600-h/Picture+15.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 320px; height: 186px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjK55dtJ5O-KbI9m1icgyFw1oDOLQLVHdFhxsAt-EsX5Dehbg2NMJ1TeOM4xoL3Gmm2ijop451LG1KSsCJ6u7ZPuHRJnp9_4UPFpqujBk3N4VomoxsX_mKG_ODPIAoP45U1SpCFDD7sS9g/s320/Picture+15.png" alt="" id="BLOGGER_PHOTO_ID_5318321904798264466" border="0" /></a><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDwvV3RcgRTHClMvrmAnFRREb7r5znrBD_-WqSlt3MoginP4He7qLfprO6Q0fUrk3jj4eKCVj8RMV0ia046zy2LGq30jzQ-WWdVl79CMWOLZ8gIGcFjZix4eyCU7VueQZcyPYEP_Sdo4g/s1600-h/Picture+7.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 320px; height: 205px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDwvV3RcgRTHClMvrmAnFRREb7r5znrBD_-WqSlt3MoginP4He7qLfprO6Q0fUrk3jj4eKCVj8RMV0ia046zy2LGq30jzQ-WWdVl79CMWOLZ8gIGcFjZix4eyCU7VueQZcyPYEP_Sdo4g/s320/Picture+7.png" alt="" id="BLOGGER_PHOTO_ID_5318320997281405234" border="0" /></a><br /><br /><div style="text-align: center;">This one became officially my computer wallpaper:<br /></div><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgUPrwNw79zVJd-36b1jDFzozyJWFbJqAmVGWlxQ3jPGWnwn1TrbA2LfpafNdxKs7YKGGVvB2ITIhBn84Mgda3KxifvvMzjgT85XxzFDye8YbqGplvTwy2XhEYc6vnEd3LztWPboaqTaNU/s1600-h/Picture+16.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 320px; height: 153px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgUPrwNw79zVJd-36b1jDFzozyJWFbJqAmVGWlxQ3jPGWnwn1TrbA2LfpafNdxKs7YKGGVvB2ITIhBn84Mgda3KxifvvMzjgT85XxzFDye8YbqGplvTwy2XhEYc6vnEd3LztWPboaqTaNU/s320/Picture+16.png" alt="" id="BLOGGER_PHOTO_ID_5318321902198129346" border="0" /></a><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZFpuwWFOYXxmVgn-K-p51gVPuUNvsNqXrnmtvzO1VXNogHKyMsTIP0haV8VKv_GPwWroryW03Vlbyqj8VDJ-gtjYY6ta2q0YBKSCkf3IefvPtDClOApJs0kda1QJ7Oes0mCjqkpeMOEU/s1600-h/Picture+18.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 320px; height: 194px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZFpuwWFOYXxmVgn-K-p51gVPuUNvsNqXrnmtvzO1VXNogHKyMsTIP0haV8VKv_GPwWroryW03Vlbyqj8VDJ-gtjYY6ta2q0YBKSCkf3IefvPtDClOApJs0kda1QJ7Oes0mCjqkpeMOEU/s320/Picture+18.png" alt="" id="BLOGGER_PHOTO_ID_5318321456507960514" border="0" /></a><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivyr6OL7BkqEukajS0bA_5zo7u3gH1I30okRzWkGSlEnwMtlXF5iTan07_qYCOfs5mBt_a2uqe9Dki5WJ3eYSWhQD52F3vLv7St1viOmynfR6w2E1LBX7ntEachdXDbsmMu1YgL8OTC2U/s1600-h/Picture+17.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 320px; height: 205px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivyr6OL7BkqEukajS0bA_5zo7u3gH1I30okRzWkGSlEnwMtlXF5iTan07_qYCOfs5mBt_a2uqe9Dki5WJ3eYSWhQD52F3vLv7St1viOmynfR6w2E1LBX7ntEachdXDbsmMu1YgL8OTC2U/s320/Picture+17.png" alt="" id="BLOGGER_PHOTO_ID_5318321444796498834" border="0" /></a><br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrqDCvZY1Vbu-e3vns5iJ1d4GcDU0zsnhr1x53W7k4cvlSDJSzBfk7Dh_1KXU6vucVIOhOc87uHgWsZw1ZAAYxbctAZk1WJqNHVD3h1C4-cG5_QTLGv9YNik000PYFxzA5hAWylPLe1oI/s1600-h/Picture+12.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 320px; height: 193px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrqDCvZY1Vbu-e3vns5iJ1d4GcDU0zsnhr1x53W7k4cvlSDJSzBfk7Dh_1KXU6vucVIOhOc87uHgWsZw1ZAAYxbctAZk1WJqNHVD3h1C4-cG5_QTLGv9YNik000PYFxzA5hAWylPLe1oI/s320/Picture+12.png" alt="" id="BLOGGER_PHOTO_ID_5318321441757827410" border="0" /></a><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihntj7thnzOClRS8NS0o-scfjXF_rLUD-9iXaa3xfOmtVUmRVC2OIobSfBELY3k8jOVhqXdTSpZ5GO8-p3dlilIQzulJxNeTodeK7WenOFWHiM9ltG-h5gZBZdi5sjUv0xWKhA8ltP8PU/s1600-h/Picture+11.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 320px; height: 174px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihntj7thnzOClRS8NS0o-scfjXF_rLUD-9iXaa3xfOmtVUmRVC2OIobSfBELY3k8jOVhqXdTSpZ5GO8-p3dlilIQzulJxNeTodeK7WenOFWHiM9ltG-h5gZBZdi5sjUv0xWKhA8ltP8PU/s320/Picture+11.png" alt="" id="BLOGGER_PHOTO_ID_5318321437136453746" border="0" /></a><br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDDKm1erMkpCVFyfhZfe9rzH3UdpmHHOIDk_5F4E7zr9hlehZJiV-CB5nV3AYBLSnIgoNKGmU8Tg4UxOcvwYqSO1fwXPUWkxxmb8aywPCiU7_1OQ7Kf8-AlHxPy5YtFsMTTG9hBJkIG4s/s1600-h/Picture+8.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 320px; height: 211px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDDKm1erMkpCVFyfhZfe9rzH3UdpmHHOIDk_5F4E7zr9hlehZJiV-CB5nV3AYBLSnIgoNKGmU8Tg4UxOcvwYqSO1fwXPUWkxxmb8aywPCiU7_1OQ7Kf8-AlHxPy5YtFsMTTG9hBJkIG4s/s320/Picture+8.png" alt="" id="BLOGGER_PHOTO_ID_5318321004830546914" border="0" /></a><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgLyDtZwjWdnY3x_ozOd6uXmqbcnEfIyWvVlunm6ztKz1Cft6KcXyc_mIGT8Tb7xvXZW0gG9eVXnk7SSDPqFtRHQcPtEFgdRPHvDTxb9GCx-QWS-P4p5-5CrKJ6m38VU4E2nlMNSbNeBU8/s1600-h/Picture+6.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 320px; height: 211px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgLyDtZwjWdnY3x_ozOd6uXmqbcnEfIyWvVlunm6ztKz1Cft6KcXyc_mIGT8Tb7xvXZW0gG9eVXnk7SSDPqFtRHQcPtEFgdRPHvDTxb9GCx-QWS-P4p5-5CrKJ6m38VU4E2nlMNSbNeBU8/s320/Picture+6.png" alt="" id="BLOGGER_PHOTO_ID_5318321002008374466" border="0" /></a><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqsQnMx_ssL_kuPnNiiFs1WB1RHd8fX1MhDyc3OHzWGLA_w4MQbeYZ99pOCzLKYuNj3hxjERthH3z5R-UpWp7icL1fvJjsqAipQDt8_PDKTwniMXIx2Im7b6xjIxntVrh-ACenKudV9aI/s1600-h/Picture+9.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 320px; height: 174px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqsQnMx_ssL_kuPnNiiFs1WB1RHd8fX1MhDyc3OHzWGLA_w4MQbeYZ99pOCzLKYuNj3hxjERthH3z5R-UpWp7icL1fvJjsqAipQDt8_PDKTwniMXIx2Im7b6xjIxntVrh-ACenKudV9aI/s320/Picture+9.png" alt="" id="BLOGGER_PHOTO_ID_5318320997608968850" border="0" /></a><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDwvV3RcgRTHClMvrmAnFRREb7r5znrBD_-WqSlt3MoginP4He7qLfprO6Q0fUrk3jj4eKCVj8RMV0ia046zy2LGq30jzQ-WWdVl79CMWOLZ8gIGcFjZix4eyCU7VueQZcyPYEP_Sdo4g/s1600-h/Picture+7.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 320px; height: 205px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDwvV3RcgRTHClMvrmAnFRREb7r5znrBD_-WqSlt3MoginP4He7qLfprO6Q0fUrk3jj4eKCVj8RMV0ia046zy2LGq30jzQ-WWdVl79CMWOLZ8gIGcFjZix4eyCU7VueQZcyPYEP_Sdo4g/s320/Picture+7.png" alt="" id="BLOGGER_PHOTO_ID_5318320997281405234" border="0" /></a><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhCpqCfA6VzE-dSqFybonf4PnPFuHi0YA7AXU77xE7saG3oef5CqD0bxfO_t5NQ7AlZkk7p2eKYB_Nr9WoERK9PKet7tNU7U_7nkHUTn1aySylXs4N_ytthaJW60glr4bqEpw8ZaYdHDUs/s1600-h/Picture+13.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 320px; height: 233px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhCpqCfA6VzE-dSqFybonf4PnPFuHi0YA7AXU77xE7saG3oef5CqD0bxfO_t5NQ7AlZkk7p2eKYB_Nr9WoERK9PKet7tNU7U_7nkHUTn1aySylXs4N_ytthaJW60glr4bqEpw8ZaYdHDUs/s320/Picture+13.png" alt="" id="BLOGGER_PHOTO_ID_5318321444314164962" border="0" /></a><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgwDvX4XD1lWrs-eytT1OZN5hsSUs9nSgNLvdaYZXBIyV9a3QDAcj3_Om6HiYBsIJOR_FkDS-J854PnRhyphenhyphenB3y3VEJdY1m3LPKgRFioKLtMKHB9nxbdTgnKfiaab2m33dhG5E73xtVGwC04/s1600-h/Picture+22.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 320px; height: 196px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgwDvX4XD1lWrs-eytT1OZN5hsSUs9nSgNLvdaYZXBIyV9a3QDAcj3_Om6HiYBsIJOR_FkDS-J854PnRhyphenhyphenB3y3VEJdY1m3LPKgRFioKLtMKHB9nxbdTgnKfiaab2m33dhG5E73xtVGwC04/s320/Picture+22.png" alt="" id="BLOGGER_PHOTO_ID_5318349334415928306" border="0" /></a><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZLRo8Vo_XpDP1ZDsZBlx_rm37a-bZccpCcyel_Kq9yhMwFLecmNB_Qn8pta1f2zbO0omebxOesm1NbEkhyphenhyphenvInGybwzblL102RkDzXN9YBzCeA0pCC9AnvKVUgPcXFMn4OnVJg118PSK0/s1600-h/Picture+24.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 194px; height: 320px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZLRo8Vo_XpDP1ZDsZBlx_rm37a-bZccpCcyel_Kq9yhMwFLecmNB_Qn8pta1f2zbO0omebxOesm1NbEkhyphenhyphenvInGybwzblL102RkDzXN9YBzCeA0pCC9AnvKVUgPcXFMn4OnVJg118PSK0/s320/Picture+24.png" alt="" id="BLOGGER_PHOTO_ID_5318349331233394002" border="0" /></a><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihL5GhT9auflvR6afk3cn5NGo_Qv2qa65Agr_YxCTvoLSYEZldHDqkx2eeJVQCG8u9dx6wJ3nlI0sx2IVl1hgvM_FpbGWnHj8yY30yk6edaAgN0fDN5iImehmIUY3qmQfNOP222s65jIE/s1600-h/Picture+21.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 223px; height: 320px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihL5GhT9auflvR6afk3cn5NGo_Qv2qa65Agr_YxCTvoLSYEZldHDqkx2eeJVQCG8u9dx6wJ3nlI0sx2IVl1hgvM_FpbGWnHj8yY30yk6edaAgN0fDN5iImehmIUY3qmQfNOP222s65jIE/s320/Picture+21.png" alt="" id="BLOGGER_PHOTO_ID_5318349323029116274" border="0" /></a>Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com0tag:blogger.com,1999:blog-3637605010144131610.post-49272135937666222352009-03-18T04:13:00.000-07:002009-03-25T07:30:01.256-07:00ACM Paris Kanellakis Theory and Practice Award 2008The 2008 <a href="http://awards.acm.org/kanellakis/">ACM Paris Kanellakis Theory and Practice Award</a> was awarded to <a href="http://research.google.com/pubs/author121.html">Corinna Cortes</a> and <a href="http://www.clrc.rhul.ac.uk/people/vlad/">Vladimir Vapnik</a> "<span style="font-style: italic;">for the development of Support Vector Machines, a highly effective algorithm for classification and related machine learning problems</span>".<br /><br />It's not the first time this award is given to Machine Learning people. In 2004 it was awarded to <a href="http://www.blogger.com/www.cse.ucsd.edu/%7Eyfreund/">Yoav Freund</a> and <a href="http://www.cs.princeton.edu/%7Eschapire/">Robert Schapire</a> "<i>for the development of the theory and practice of boosting and its applications to machine learning."<br /><br /></i>I found a bit weird that they left <a href="http://www.eecs.berkeley.edu/%7Eboser/">Bernhard Boser</a> and <a href="http://www.clopinet.com/isabelle/">Isabelle Guyon</a> out of the prize, because they were Vapnik's co-authors in the 1992 paper "<a href="http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.21.3818">A training algorithm for optimal margin classifiers</a>", which I guess is <a href="http://www.clopinet.com/isabelle/Papers/index.html">considered</a> to be the first paper on Support Vector Machines...<br /><br />Anyway, congratulation to the winners. These are indeed elegant algorithms with sound theoretical foundations and numerous sucessful applications to vision, speech, natural language and robotics, to name just a few.<br /><i><br />---------------------------<br />Remarks:<br /><br /></i>Thanks to my cousin <a href="http://ruiaf.org/tag/computer-science/">Rui</a> for the link to this news.<br /><br />---------------------------<br />Related post:<br /><br /><a href="http://aboutintelligence.blogspot.com/2009/01/vapniks-picture-explained.html">Vapnik's picture explained</a>.Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com0tag:blogger.com,1999:blog-3637605010144131610.post-59024349243691832952009-02-06T07:48:00.000-08:002009-02-06T08:00:42.562-08:00Social features on this blogThe readers of this blog can now:<br /><br />1. Easily subscribe to the RSS feed with their reader of choice [left panel].<br />2. Decide to become a visible "follower" of this blog [left panel].<br />3. Rate each blog entry from 1 to 5 stars [end of each post].<br /><br />I would be particularly happy to see people rating the posts. It's less informative than writing comments but still it's very good feedback for me.<br /><br />Thanks!Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com0tag:blogger.com,1999:blog-3637605010144131610.post-54762643442513792952009-01-28T12:19:00.000-08:002009-02-01T09:25:19.128-08:00Vapnik's picture explained<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhNsDKM24TbhSTujsJBpQ5Mk4a0SN6q1Me0uC7LT4FYIvaXIc4L5kH4oaeKVN1xkJmR2LxKWaM8TaDta6UG_cxSpF63Fzk85rUH34nan_e6sfSpizIXrxlVCV353ZYdHWdzl0YQNzyIjds/s1600-h/vapnik.jpg"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 320px; height: 240px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhNsDKM24TbhSTujsJBpQ5Mk4a0SN6q1Me0uC7LT4FYIvaXIc4L5kH4oaeKVN1xkJmR2LxKWaM8TaDta6UG_cxSpF63Fzk85rUH34nan_e6sfSpizIXrxlVCV353ZYdHWdzl0YQNzyIjds/s320/vapnik.jpg" alt="" id="BLOGGER_PHOTO_ID_5297876863674530946" border="0" /></a><br /><br />This is an extremely geek picture! :) Let's try to explain it:<br /><br />First of all, as many of you know, the gentleman in the picture is <a href="http://www.ccls.columbia.edu/Vapnik-Bio.html">Prof. Vladimir Vapnik</a>. He is famous for his fundamental contributions to the field of Statistical Learning Theory, such as the Empirical Risk Minimization (ERM) principle, VC-dimension and Support Vector Machines.<br /><br />Then we notice the sentence in the board: it resembles the famous "<a href="http://en.wikipedia.org/wiki/All_your_base_are_belong_to_us">All your base are belong to us</a>"! This is a piece of geek culture that emerged after a "broken English" translation of a Japanese video game for Sega Mega Drive .<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1zC1k8Ig3Ml-peEqqIU_WKPyKCwYo4bfbnGG-7w7M5t_AaDqVXeAlUPn15rknDZPs3p3P-5uc4m4El-gNgNlJMXPWKwyifYNZQqQK_sNHEnlMWlasI8YVRCUCVQjgA_UXa_E31MhnVTY/s1600-h/Aybabtu.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 240px; height: 160px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1zC1k8Ig3Ml-peEqqIU_WKPyKCwYo4bfbnGG-7w7M5t_AaDqVXeAlUPn15rknDZPs3p3P-5uc4m4El-gNgNlJMXPWKwyifYNZQqQK_sNHEnlMWlasI8YVRCUCVQjgA_UXa_E31MhnVTY/s320/Aybabtu.png" alt="" id="BLOGGER_PHOTO_ID_5297873540595007026" border="0" /></a><br /><br />Wait, but they replaced the word "Base" by "Bayes"!?<br />Yes, that <a href="http://en.wikipedia.org/wiki/Thomas_Bayes">Bayes</a>, the British mathematician known for the <a href="http://en.wikipedia.org/wiki/Bayes%27_theorem">Bayes' theorem</a>.<br />Okay, seems fair enough, we are dealing with people from statistics...<br /><br />By the moment we think things can not get more geeky, we realize there is scary inequality written on the top of the white board:<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9A35AsgbtagiWfcACvF2Y6qGi1A88kB8jaaOzQBTHXEjMm1bMrgGbaeUNWsQYQVWF4YMQOlYxEwx0CfdU4msf12qXuJvq-Ezae11pRgO7zbgwEeVdDMvuhm-PxyUh1RBbtVXORDcVt5U/s1600-h/bound.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 400px; height: 51px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9A35AsgbtagiWfcACvF2Y6qGi1A88kB8jaaOzQBTHXEjMm1bMrgGbaeUNWsQYQVWF4YMQOlYxEwx0CfdU4msf12qXuJvq-Ezae11pRgO7zbgwEeVdDMvuhm-PxyUh1RBbtVXORDcVt5U/s400/bound.png" alt="" id="BLOGGER_PHOTO_ID_5297124426132640354" border="0" /></a>My goodness, what's this?! Okay, that's when things get really technical:<br />This is a probabilistic bound for the expected risk of a classifier under the ERM framework. In simple terms, it relates the classifier's expected test error with the training error on a dataset of size l and in which the cardinality of the set of loss functions is N.<br />If I'm not mistaken, the bound holds with probability (1 - eta) and applies only to loss functions bounded above by 1.<br /><br />Sweet! Now that we got the parts, what's the big message?<br /><br />Well, it's basically a statement about the superiority of Vapnik's learning theory over the Bayesian alternative. In a nutshell, the Bayesian perspective is that we start with some prior distribution over a set of hypothesis (our beliefs) and we update these according to the data that we see. We then look for an optimal decision rule based on the posterior distribution.<br />On the other hand, in Vapnik's framework there are no explicit priors neither we try to estimate the probability distribution of the data. This is motivated by the fact that density estimation is a <a href="http://en.wikipedia.org/wiki/Ill-posed">ill-posed</a> problem, and therefore we want to avoid this intermediate step. The goal is to directly minimize the probability of making bad decision in the future. If implemented through <a href="http://en.wikipedia.org/wiki/Support_vector_machine">Support Vector Machines</a>, this boils down to finding the decision boundary with maximal margin to separate the classes.<br /><br />And that's it, folks! I hope you had fun decoding this image! :)Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com2tag:blogger.com,1999:blog-3637605010144131610.post-80131918843786257962009-01-28T09:37:00.000-08:002009-01-28T10:05:26.880-08:00Computer Vision vs Computer GraphicsIf I had to explain what computer vision is all about, in just one snapshot, I would show you this:<br /><br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhNUz9RHEUFcyE791h96sqgew9KDg7kkoWv6vLgYH7H1QIf48Vdcgib0q7_WyUOXNub6gI4PY0lFlvIcPWTCXKNBWjmHmApVhQAjsyb8H60brLuylUjp9EmK6i9fyd5JfURdjgje-QUNvY/s1600-h/computer_vision_graphics.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 191px; height: 200px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhNUz9RHEUFcyE791h96sqgew9KDg7kkoWv6vLgYH7H1QIf48Vdcgib0q7_WyUOXNub6gI4PY0lFlvIcPWTCXKNBWjmHmApVhQAjsyb8H60brLuylUjp9EmK6i9fyd5JfURdjgje-QUNvY/s200/computer_vision_graphics.png" alt="" id="BLOGGER_PHOTO_ID_5296400168428800258" border="0" /></a><br /><br />Computer Graphics algorithms go from the parameter space to the image space (rendering), computer vision algorithms do the opposite (inverse-rendering). Because of this, computer vision is basically a (very hard) problem of statistical inference.<br />The common approach nowadays is to build a classifier for each kind of object and then search over (part of) the parameter space explicitly, normally by scanning the image for all possible locations and scales. The remaining challenge is still huge: how can a classifier learn and generalize, from a finite set of examples, what are the fundamental characteristics of an object (shape, color) and what is irrelevant (changes in illumination, rotations, translations, occlusions, etc.).<br />This is what is keeping us busy! ;)<br /><br />PS - Note that changes in illumination induce apparent changes in the color of the object and rotations induce apparent changes in shape!Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com0tag:blogger.com,1999:blog-3637605010144131610.post-36752541797015540742009-01-08T11:13:00.000-08:002009-01-08T11:38:45.460-08:00Stationary Features - Google Tech TalkFrançois Fleuret, my PhD advisor, recently gave a talk about object detection at Google (Zurich offices).<br />You can now see it online:<br /><br /><br /><object width="425" height="344"><param name="movie" value="http://www.youtube.com/v/-w72_VwSj6A&hl=en&fs=1"><param name="allowFullScreen" value="true"><param name="allowscriptaccess" value="always"><embed src="http://www.youtube.com/v/-w72_VwSj6A&hl=en&fs=1" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="425" height="344"></embed></object><br /><br />If you wonder where my research will try to extend the work done so far, just go to minute 45:30!Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com0tag:blogger.com,1999:blog-3637605010144131610.post-33222414588756672412008-09-24T05:05:00.000-07:002008-09-24T05:45:09.673-07:00Machine Learning Summer SchoolHeld in Ile de Ré (France), 1-15th September, this school counted with some famous names within the Machine Learning and Artificial Intelligence communities: <a href="www.cs.ualberta.ca/%7Esutton/">Rich Sutton</a> (co-author of the widely adopted book on Reinforcement Learning), <a href="www.clopinet.com/isabelle/">Isabelle Guyon</a> (co-author of the first paper on Support Vector Machines) and <a href="yann.lecun.com">Yann LeCun</a> (known for the convolutional neural network, energy based models and the <span helvetica="" style="font-size:100%;">DjVu </span><span helvetica="" style="font-size:100%;">image compression technique</span>).<br /><br />You can check the (almost) complete list of lecturers <a href="http://mlss08.futurs.inria.fr/confirmed-lecture-courses">here</a>. I found the course given by <span class="link-external"><a set="yes" linkindex="17" href="http://www.cs.uwaterloo.ca/%7Eshai/">Shai Ben-David</a></span>, on <i>the Theoretical Foundations of Clustering"</i> quite interesting and intriguing. Clustering seems to be *really* lacking solid theoretical support, which is surprising, given the importance of the problem. Some <a href="http://www.cs.cornell.edu/home/kleinber/nips15.pdf">atempts</a> are being done to axiomatize it, but there are a lot of <a href="http://www.cs.uwaterloo.ca/%7Eshai/LuxburgBendavid05.pdf">open questions</a>: what exactly is the class of clustering algorithms? how can you compare different clustering algorithms? why is a partition better than other?<br />Hope to see more developments in this area in the coming years.Hugo Penedoneshttp://www.blogger.com/profile/02746022526894210415noreply@blogger.com0