My Research

I am the co-head of the LIACS Imagery and Media Research cluster (20+ members), which is one of the four main research groups at the computer science department at Leiden University. In addition, as a member of the LIACS Scientific Council, I am responsible for steering the general direction of computer science research at Leiden University. My group's mission is to investigate new paradigms for the retrieval and understanding of multimedia information, and to advance the frontiers of social computing and social intelligence.

Research Summary

My research interests lie at the intersection between humans and computer science, that is, what are the next frontiers in both human-computer interaction and also social computing? Some examples: How can we search for multimedia information based on the pictorial contents of the media? How can we create an artificial imagination? What are the ramifications of social intelligence? By studying the computational mixture composed of humans, artificial intelligence, network modeling, probabilistic reasoning, neuroscience, and multi-modal analysis, we may be able to provide answers to the questions in our lifetimes.

Teaching/Courses

• Multimedia Information Retrieval, Wednesday, 13:45-15:30, Rm. 405, - Leiden Univ./Delft TU BioInformatics Program

• Multimedia Systems Wednesday, 13:45-15:30, - Voted by students as the most interesting course - Ranking
         - official student rating of 8.2 in January 2011 (one of the highest at LIACS)

• LIACS Research Seminar, Thursday, 16:00-17:00, Rm. 174

• Bachelor Class, Fridays, 11:15-12:30

• Computer Graphics, Tuesday, 13:45-15:30

• Editor-in-Chief of the International Journal of Multimedia Information Retrieval, Springer, London.

• Most cited multimedia research paper worldwide in the highest impact multimedia journal (from ISI Web of Science, Thomson-Reuters), 2009

• Published over a dozen books and 100 peer-reviewed scientific articles

• ACM Multimedia Steering Committee Member (1 of 6 members), 2011-2013

• General Co-Chair of the IEEE Pacific-Rim Conference on Multimedia, Shanghai, 2010

• Invited Speaker, ACM International Workshop on Connected Media, Florence, 2010

• Invited Speaker and Panelist, ACM Multimedia Information Retrieval Conference, Philadelphia, 2010

• Chair of the Steering Committee for ACM International Conference on Multimedia Retrieval, 2009-

• Keynote speaker for the 9th International Conference on Visual Information Systems - VISUAL 2007, 28-29 June 2007, Shanghai, China

• General Chair of the ACM International Conference on Multimedia Information Retrieval, Vancouver, Canada, October 30-31, 2008

• Chair of the Steering Committee for International Conference on Image and Video Retrieval - CIVR (jointly held by ACM, IEEE & IEE), 2007-
         - co-chair with John Eakins, 2002-2006

• Chair of the ACM SIGMM Multimedia Information Retrieval Steering Committee, 2003-

• Advisory Committee Member for TRECVID 2003-2006. TRECVID was founded as an independent evaluation in 2003.

• Co-Chair of the ACM Multimedia Technical Awards Committee, 2005

• Chair Professor at Tsinghua University - The Chinese MIT, Beijing, China (2003-)

• Organizing/Steering Committee Member for the Masters in Media Technology Program at Leiden University, 2001-2006

• Biography selected by the editors for Who's Who in the World, (Marquis 26th Edition)
         - The New York Times on Marquis Whos Who

• co-Head of the Imagery and Media Research Cluster - One of the 4 research themes in the Computer Science (CS) dept.
• Member of the Board of Examiners (ExamenCommissie) - oversees the quality of all CS dept. courses
• Member of the Curriculum Committee - guides the direction and content of the CS dept. courses
• Member of the Organizing Team for the LIACS Research Seminar
• co-Director of the LIACS Media Lab - advises and guides research and teaching in multimedia technology within the CS dept.
• Member of the LIACS Scientific Council - steering committee for all research in the CS dept.

• BRICKS Project (member of the project board)
• Cyttron (co-leader of the BioSearch Project)
• Advanced Information Processing in Bioinformatics, NBIC Biorange IV - VL-e (investigator)
• VIRSI Project (project leader)
• Philips Research Project on Bio-Medical Analysis (project leader)
• Philips Research Project on Texture Analysis (project leader)

• Editor-in-Chief of the International Journal of Multimedia Information Retrieval, Springer, London.

• Editorial Board member for the Journal of Multimedia, 2005-
• Editorial Board member for the Journal of Ambient Intelligence and Smart Environments , 2008-
• Guest Editor for the Journal of Computer Vision and Image Understanding special issue on Visual Concept Detection
• Guest Editor for the ACM Transactions on Multimedia Computing, Comm., and Apps Special Issue on Multimedia Information Retrieval
• Guest Editor for the ACM Multimedia Systems Journal Special Issue on Machine Learning Approaches in Multimedia Information Retrieval
• Guest Editor for the Image and Vision Computing Journal Special Issue on Human Computer Interaction
• Guest Editor for the ACM Multimedia Systems Journal Special Issue on Systems and Architectures of Multimedia Information Retrieval
• Guest Editor for the Journal of Computer Vision and Image Understanding special issue on Video Retrieval and Summarization
• Guest Editor for the Journal of Computer Vision and Image Understanding Special Issue on Similarity Matching in Computer Vision and Multimedia

• Book Editor, Principles of Visual Information Retrieval, Springer-Verlag, Heidelberg, 2001.
• Book Editor, Image and Video Retrieval (LNCS 2383), Springer-Verlag, Heidelberg, 2002.
• Book, Robust Computer Vision, Kluwer (ISBN: 1-4020-1293-4), London, April 2003.
• Book, Proc. 5th ACM SIGMM Int. Workshop on Multimedia Information Retrieval, ACM Press, New York, 2003.
• Book Editor, Image and Video Retrieval (LNCS 2728), Springer-Verlag, Heidelberg, 2003.
• Book Editor, Computer Vision in Human Computer Interaction (LNCS 3058), Springer-Verlag, Heidelberg, 2004.
• Book, Proc. 6th ACM SIGMM Int. Workshop on Multimedia Information Retrieval, ACM Press, New York, 2004.
• Book Editor, Computer Vision in Human Computer Interaction (LNCS 3766), Springer-Verlag, Heidelberg, 2005
• Book Editor, Image and Video Retrieval (LNCS 3568), Springer-Verlag, Heidelberg, 2005
• Book Editor, Computer Vision in Human Computer Interaction (LNCS 3979), Springer-Verlag, Heidelberg, 2006
• Book Editor, Human Computer Interaction (LNCS 4796), Springer-Verlag, Heidelberg, 2007
• Book Editor, Proc. 1st ACM International Conference on Multimedia Information Retrieval, ACM Press, New York, 2008
• Book Editor, Proc. of the 11th Pacific-Rim Conference on Multimedia Springer, London, 2010

Conference Organization

• General Co-Chair of the IEEE Pacific-Rim Conference on Multimedia, Shanghai, 2010
• General Chair of the ACM International Conference on Multimedia Information Retrieval - MIR2008, Vancouver, Canada, October 30-31, 2008, [Archive.org cache]
• Organizing Committee Member for the ACM Multimedia Conference, 2005, 2012, 2013
• General Co-Chair for the IEEE International Workshop on Human-Computer Interaction 2007 - HCI2007
• General Co-Chair for the IEEE International Workshop on Human-Computer Interaction 2005 - HCI2005 , Beijing, China, October 15-21, 2005.
• General Co-Chair for the ACM SIGMM International Workshop on Multimedia Information Retrieval - MIR2004 , New York, NY, USA, October 2004.
• General Co-Chair for the International Workshop on Human Computer Interaction - HCI2004 (in conjunction with ECCV'2004), Prague, Czech, May, 2004.
• General Co-Chair for the International Conference on Image and Video Retrieval - CIVR2003, Beckman Institute at University of Illinois at Urbana-Champaign, USA, July 24-25, 2003 (jointly held by IEEE & IEE).
• General Co-Chair for the ACM SIGMM International Workshop on Multimedia Information Retrieval - MIR2003, Berkeley, CA, USA, November 7, 2003. ACM Proceedings
• Program Chair for the International Conference on Image and Video Retrieval - CIVR2002, London, UK, July 18-19, 2002.
• Program Co-Chair for the International Conference on Image and Video Retrieval - CIVR2005, Singapore, 2005.
• Program Co-Chair for the International Workshop on Human Computer Interaction - HCI2006, Graz, Austria, 2006.

Scientific Conference Program Committees

• Program Committee Member for ACM-CIVR - International Conference on Image and Video Retrieval, 2002-2009
• Program Committee Member for ACM-CMM - ACM International Workshop on Connected Multimedia, 2010
• Program Committee Member for ACM-CVDB - ACM SIGMOD Workshop on Computer Vision Meets Databases, 2004
• Program Committee Member for ACM-HCM - ACM Workshop on Human Centered Multimedia, 2006-2007
• Program Committee Member for ACM-SIGIR - ACM SIGIR Conference, 2009-2010
• Program Committee Member for ACM-MDM - ACM KDD Workshop on Multimedia Data Mining, 2006
• Program Committee Member for ACM-MIRW (workshop) - ACM Workshop on Multimedia Information Retrieval, 2003-2007
• Program Committee Member for ACM-MIR (conference) - ACM International Conference on Multimedia Information Retrieval, 2008-2010
• Program Committee Member for ACM-MM - ACM Multimedia, 2003-2008, 2010, 2011
• Program Committee Member for ACM-MMM - International Multimedia Modeling Conference, 2008-2009, 2012
• Program Committee Member for DMS - International Conference on Distributed Multimedia Systems, 2005-2008
• Program Committee Member for FCST - International Conference on Frontiers of Computer Science and Technology, 2011
• Technical Committee Member for ICASSP - IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005
• Program Committee Member for ICETE - International Conference on E-business and Telecommunication Networks, 2005
• Program Committee Member for ICPR - International Conference on Pattern Recognition, 2004
• Program Committee Member for IEEE-CVPR - IEEE Conf. on Computer Vision and Pattern Recognition, 2006, 2008
• Program Committee Member for IEEE-EMMA - IEEE International Workshop on Managing Data for Emerging Multimedia Applications, 2005
• Program Committee Member for IEEE-HCI IEEE International Workshop on Human-Computer Interaction, 2005-2009
• Program Committee Member for IEEE-ICCV - IEEE International Conference on Computer Vision, 2007 (invited, review phase 1)
• Program Committee Member for IEEE-ICIP - IEEE International Conference on Image Processing, 2003-2006
• Program Committee Member for IEEE-ICMCS - IEEE International Conference on Multimedia Computing and Systems, 1999, 2009
• Program Committee Member for IEEE-ICME - IEEE Conf. on Multimedia and Expo, 1999-2008
• Program Committee Member for IEEE-MMSP - IEEE International Workshop on Multimedia Signal Processing, 2011
• Program Committee Member for IEEE-PCM - IEEE/ACM Pacific-Rim Conference on Multimedia, 2004-2007, 2010
• Program Committee Member for SPIE-Internet - SPIE Internet Multimedia Management Systems VI (IT108), October 2006
• Program Committee Member for SPIE-MCAMR - SPIE Multimedia Content Analysis, Management, and Retrieval, 2006
• Program Committee Member for SPIE-MCAAS - SPIE Multimedia Content Access: Algorithms and Systems IV, 2007-2010
• Program Committee Member for SPIE-SRMAM - SPIE Storage and Retrieval Methods and Applications for Multimedia, 2006
• Program Committee Member for TIDSE - International Conference on Technologies for Interactive Digital Storytelling and Entertainment, 2006
• Program Committee Member for VIP - Visual Information Processing, 2007
• Program Committee Member for VipIMAGE - First ECCOMAS Thematic Conference on Computational Vision and Medical Image Processing, 2005-2007
• Program Committee Member for VISAPP - International Conference on Computer Vision Theory and Applications, 2006-2008
• Program Committee Member for VISUAL - VISUAL - Conference on Visual Information Systems, 2003-2008

Representative Publications (over 100 peer-reviewed in ACM, IEEE, and LNCS)

Content-Based Multimedia Information Retrieval: State of the Art and Challenges, Citations: 500 in Google Scholar - Cache 2/2011

Michael S. Lew, Nicu Sebe, Chabane Djeraba, Ramesh Jain, ACM Transactions on Multimedia Computing, Communications, and Applications, vol. 2, issue 1, 2006, pp. 1-19.
-> New ideas and paradigms in multimedia information retrieval

Image Retrieval using Wavelet-based Salient Points, Citations: 100 in Google Scholar - Cache 1/2010

M.S. Lew, et al., Journal of Electronic Imaging, Vol. 10, No. 4, October, 2001, pp. 835-849.
-> Earlier version: "amongst the best submitted to the SPIE Conf. on Storage and Retrieval of Media Databases" - SPIE Program Committee

Principles of Visual Information Retrieval, Citations: 141 in Google Scholar - Cache 12/2009

Michael S. Lew, Springer-Verlag, London, ISBN 1-852333-381-2, January 2001.
-> Covers the state of the art in visual information retrieval. It has been used in more than 7 universities worldwide including Columbia University (USA), Tsinghua University (China), etc.

Next Generation Web Searches for Visual Content, Citations: 115 in Google Scholar - Cache 4/2010

Michael S. Lew, IEEE Computer, November, 2000, pp. 46-53.
-> Describes the ImageScape System which performs semantic multimedia searching on an index of over 25 million images covering the Web

Learning and Feature Selection in Stereo Matching, Citations: 100 in Google Scholar

Michael S. Lew, Thomas S. Huang, Kam W. Wong, IEEE Transactions on Pattern Analysis and Machine Intelligence on Learning in Computer Vision, September, 1994, pp. 869-881.
-> This paper was about learning which features would be best for matching. It also touches on a core interest of mine - to integrate methods from artificial intelligence into computer vision and image understanding.

The Distributed ASCI Supercomputer Project, Citations: 100 in Google Scholar - Cache

M. Lew, et al., Operating Systems Review, 2000,
-> Describes the Netherlands distributed supercomputer

Authentic Facial Expression Analysis, Citations: 100 in Google Scholar

M.S. Lew, et al., Image and Vision Computing, Vol. 25, No. 12, 2007, pp. 1856-1863.
-> Arguably the first evaluation on machine detection of authentic human emotions

Toward Improved Ranking Metrics, Citations: 100 in Google Scholar

M.S. Lew, et al., IEEE Transactions on Pattern Analysis and Machine Intelligence October, 2000, pp. 1132-1143.
-> Despite most papers assuming that noise is Gaussian, this paper shows that it is not!

The MIR FLICKR Retrieval Evaluation, Citations: 100 in Google Scholar

M.S. Lew, et al., ACM International Conference on Multimedia Information Retrieval (MIR'08), Vancouver, Canada, 2008.
-> A well annotated image retrieval test set and evaluation

Evaluation of salient point techniques, Citations: 50 in Google Scholar

M. Lew, et al., Image and Vision Computing, Vol. 21, No. 13-14, 2003, pp. 1087-1095.
-> A nice evaluation of salient point techniques

The State of the Art in Image and Video Retrieval, Citations: 60 in Google Scholar

M. Lew, et al., Proceedings of the International Conference on Image and Video Retrieval (CIVR), Urbana, IL, 2003.
-> A VIR overview

Information Theory and Face Detection, Citations: 51 in Google Scholar

Michael S. Lew, Nies Huijsmans, Proceedings of the International Conference on Pattern Recognition, Vienna, Austria, August 25-30, 1996, pp. 601-605.
-> This paper contains one of the most surprising results I've seen

Emotion Recognition Using a Cauchy Naive Bayes Classifier, Citations: 55 in Google Scholar

M.S. Lew, et al., Proceedings of the International Conference on Pattern Recognition, Quebec, Canada, 2002, pp. 17-20.
-> This paper is an early work on automatic emotion recognition

Publications in 2004

Older Publications

Recent Department Activities

   (1) 4 Semester length courses, 24 ECs (Jan - Dec 2010)
   (2) LIACS Media Lab for education/research (hardware/software facilities, knowledge)
       - Guidance, programming expertise and system administration for at least 5 courses & 30 students per year
   (3) LIACS/Mathematics/I-Group Information Screens & Website
   (4) LIACS Research Seminar (rebooted 2011)
   (5) Eureka Paper (at request of Educational Director)
   (6) Open Day Activities - Talks & Demonstrations
   (7) Science Based Business collaboration - video interviews
   (8) Exam Committee
   (9) Curriculum Committee
   (10) Public Relations Committee
       - Social Media Initiative (i.e. YouTube)
       - Google results initiative

Recent Graduate Students

• Susan Laraghy
• Zhenyang Li
• Ran Tao
• Simon Zaaijer
• Ard Oerlemans (graduated with Ph.D. on December 22nd, 2011)
• Bart Thomee (graduated with Ph.D. on November 3rd, 2010)

Computer Vision, Image, and Video Links

• Computer Vision Conferences
• Computer Vision Homepage

• Tom Huang's Homepage
• Ramesh Jain's Homepage

• 2009-2010 Fall - Masters
• 2009-2010 Spring - Masters
• 2008-2009 Fall - Masters
• 2008-2009 Spring - Masters
• 2007-2008 Fall - Masters
• 2007-2008 Spring - Masters
• 2006-2007 Fall - 3rd Years
• 2006-2007 Spring - 3rd Years
• 2006-2007 Fall - Masters
• 2006-2007 Spring - Masters
• 2005-2006 3rd Years
• 2005-2006 Masters

Other Activities: Student Advising and Research Grants (Click Here)

Contact information

Main Contact:

Telephone: 31-71-527-7034

Fax: 31-71-527-6985

Postal Address:

    Leiden Institute of Advanced Computer Science
    Leiden University
    Niels Bohrweg 1
    2333 CA Leiden
    The Netherlands

ImageScape: A multimedia AltaVista-like program which allows the user to search the world wide web for both images and text. Over 25 million images indexed

ImageSearch: A Java image query system that allows the user to create sketches and place icons which represent relevant objects such as faces, trees, sky, etc.

MIP Face Detection: An executable program for SGI workstations which automatically detects the locations of faces in complex backgrounds

MIP Face Detection

SD: An executable program for SGI workstations which reads MPEG-1 video files and automatically detects the locations of shot breaks.

Shot Break Detection

Google.com - A good WWW search engine for general information and guides

Altavista.com - A great search engine for specific documents

Download.com - Demo and Shareware Downloads

Old group repository

Private page

Dilbert

Perldoc.com - Perl Documentation

Multimedia Conferences 2011

ACM TOMCCAP MIR Survey (vol. 2, issue 1, pp. 1-19, 2006)