SlideShare a Scribd company logo
ediaEval 2012
Mul$media	
  Benchmark	
  Workshop	
  
   Sante	
  Croce	
  in	
  Fossabanda	
  
              Pisa,	
  Italy	
  
        4	
  October	
  2012	
  
Overview
•  What	
  is	
  (who	
  are)	
  MediaEval?	
  
•  Where	
  are	
  we	
  now	
  in	
  year	
  five?	
  
•  What	
  to	
  expect	
  today?	
  
The MediaEval Benchmark…
•  …evaluates	
  new	
  algorithms	
  for	
  mul$media	
  
   access	
  and	
  retrieval.	
  	
  
•  …emphasizes	
  the	
  "mul$"	
  in	
  mul$media:	
  
   speech,	
  audio,	
  visual	
  content,	
  tags,	
  users,	
  
   context.	
  	
  
•  …innovates	
  new	
  tasks	
  and	
  techniques	
  focusing	
  
   on	
  the	
  human	
  and	
  social	
  aspects	
  of	
  
   mul$media	
  content.	
  
MediaEval Tasks 2012
•    Placing	
  Task	
  (6)	
  
•    Social	
  event	
  detec$on	
  Task	
  (5)	
  
•    Affect	
  Task:	
  Violent	
  Scenes	
  Detec$on	
  Task	
  (8)	
  
•    Spoken	
  Web	
  Search	
  Task	
  (9)	
  
•    Visual	
  Privacy	
  Task	
  (3)	
  
•    Tagging	
  Task	
  (5)	
  



Number	
  of	
  teams	
  crossing	
  the	
  finish	
  line	
  
this	
  year	
  is	
  in	
  parentheses.	
  
Working Notes Papers
60	
  

50	
  

40	
  

30	
  

20	
                                           2011	
  



10	
  
                     2010	
  


  0	
  

                2010	
                    2011	
                    2012	
  
     Black:	
  Par$cipant	
  papers;	
  Grey:	
  Task	
  overview	
  papers	
  
MediaEval Organizers 2012	
  
Workshop Organizing Committee
•  Proceedings	
  Chairs:	
  Sebas$an	
  Schmiedeke	
  and	
  Pascal	
  
   Kelm,	
  Technische	
  Universität	
  Berlin,	
  Germany	
  	
  
•  Prac00oners	
  Chair:	
  Xavier	
  Anguera,	
  Telefonica,	
  Spain	
  	
  
•  Awards	
  Chair:	
  Mohammad	
  Soleymani,	
  Imperial	
  College	
  
   London,	
  UK	
  
•  Mul0media	
  Chair:	
  Florian	
  Metze,	
  CMU,	
  USA	
  (Supported	
  by	
  
   Babak	
  Loni,	
  TU	
  Dele)	
  	
  
•  Logis0cs	
  Chair:	
  Saskia	
  Peters,	
  TU	
  Dele,	
  Netherlands	
  
•  Local	
  Organiza0on	
  support:	
  Alessio	
  Bazzica,	
  Università	
  
   degli	
  Studi	
  di	
  Firenze)	
  
•  General	
  Chair	
  and	
  contact	
  person:	
  Martha	
  Larson	
  (TU	
  
   Dele,	
  Netherlands)	
  
MediaEval History	
  
•  MediaEval	
  was	
  founded	
  as	
  VideoCLEF	
  in	
  2008.	
  
•  Ran	
  in	
  2008	
  and	
  2009	
  at	
  the	
  Cross	
  Language	
  Evalua$on	
  
   Forum	
  (CLEF)	
  and	
  became	
  MediaEval	
  in	
  2010.	
  
•  Ran	
  in	
  2010	
  and	
  2011	
  as	
  an	
  independent	
  benchmark	
  
   under	
  the	
  sponsorship	
  of	
  the	
  PetaMedia	
  Network	
  of	
  
   Excellence.	
  
•  In	
  2012,	
  ran	
  as	
  a	
  completely	
  “boiom-­‐up	
  benchmark”	
  with	
  
   no	
  parent	
  project.	
  
Where are we now in year five?	
  
•  Wider	
  MedialEval	
  community	
  has	
  grown:	
  148	
  
   people	
  took	
  $me	
  to	
  fill	
  in	
  the	
  2012	
  survey.	
  
•  From,	
  2008-­‐1012	
  we	
  have	
  generated	
  100	
  
   papers	
  (VideoCLEF	
  +	
  MediaEval	
  Working	
  Notes	
  
   Proceedings	
  plus	
  other	
  workshop,	
  conference	
  
   and	
  journal	
  papers	
  that	
  use	
  MediaEval	
  data).	
  
•  We	
  now	
  have	
  a	
  new	
  “Brave	
  New	
  Task”	
  track	
  to	
  
   incubate	
  promising	
  tasks	
  and	
  allow	
  them	
  to	
  
   mature	
  to	
  full	
  tasks.	
  
MediaEval 2012 Opening
MediaEval 2012 Opening
Where are we now in year five?	
  
•  Wider	
  MedialEval	
  community	
  has	
  grown:	
  148	
  
   people	
  took	
  $me	
  to	
  fill	
  in	
  the	
  2012	
  survey.	
  
•  From,	
  2008-­‐1012	
  we	
  have	
  generated	
  100	
  
   papers	
  (VideoCLEF	
  +	
  MediaEval	
  Working	
  Notes	
  
   Proceedings	
  plus	
  other	
  workshop,	
  conference	
  
   and	
  journal	
  papers	
  that	
  use	
  MediaEval	
  data).	
  
•  We	
  now	
  have	
  a	
  new	
  “Brave	
  New	
  Task”	
  track	
  to	
  
   incubate	
  promising	
  tasks	
  and	
  allow	
  them	
  to	
  
   mature	
  to	
  full	
  tasks.	
  
Brave New Tasks
•  User	
  Account	
  Matching	
  (1)	
  
•  Search	
  and	
  Hyperlinking	
  (6)	
  
•  Musiclef:	
  Mul$modal	
  Music	
  Tagging	
  (1)	
  




Brave	
  New	
  Tasks	
  is	
  an	
  
“incuba$ng”	
  track	
  that	
  
serves	
  to	
  breed	
  new	
  tasks.	
  

                                                           Flickr	
  Mads	
  Boe	
  
Familiar from past years
•  Wireless	
  internet	
  not	
  reliable	
  
•  We	
  are	
  recording	
  audio	
  (Florian).	
  
•  Consent	
  forms	
  for	
  recorded	
  audio	
  (Babak):	
  
   Please	
  sign	
  and	
  return	
  your	
  form	
  
•  T-­‐Shirts:	
  Nearly	
  random	
  
•  The	
  restroom	
  is	
  across	
  the	
  courtyard.	
  
Familiar from last year
•  Short	
  (10	
  minute)	
  presenta$ons:	
  	
  
    –  Please	
  respect	
  the	
  bell.	
  
    –  Please	
  load	
  your	
  presenta$on	
  in	
  the	
  break	
  right	
  
       before	
  your	
  session.	
  
    –  The	
  Pillory:	
  What	
  happens	
  if	
  you	
  run	
  over	
  your	
  
       $me	
  slot	
  
•  The	
  caterer	
  expects	
  us	
  to	
  keep	
  on	
  schedule.	
  
MediaEval 2012 Opening
Multimedia@MediaEval 2012	
  
•  We	
  not	
  only	
  analyze	
  mul$media	
  we	
  create	
  it.	
  
•  Please	
  take	
  photos:	
  Babak	
  Loni	
  will	
  collect	
  
   them	
  on	
  the	
  final	
  day	
  and	
  upload	
  them	
  to	
  the	
  
   Mediaeval	
  Flickr	
  account.	
  
More multimedia 2012	
  
•  John	
  N.A.	
  Brown	
  is	
  our	
  filmmaker	
  this	
  year.	
  
•  To	
  join	
  us	
  he	
  takes	
  a	
  break	
  from	
  his	
  ‘other	
  life’	
  working	
  to	
  
   finishing	
  his	
  PhD	
  in	
  Erasmus	
  Mundus	
  Joint	
  Doctoral	
  
   Programme	
  in	
  Interac$ve	
  and	
  Cogni$ve	
  Environments.	
  
•  He	
  may	
  ask	
  you	
  to	
  say	
  a	
  few	
  
   words	
  about	
  MediaEval	
  in	
  front	
  
   of	
  the	
  camera.	
  
Keynote Speakers
•  Invited	
  Talk	
  by	
  Jana	
  Eggink,	
  BBC	
  Research	
  and	
  
   Development,	
  UK	
  
•  Invited	
  Talk	
  by	
  Nicola	
  Ferro,	
  University	
  of	
  Padova,	
  
   Italy	
  (PROMISE	
  Co-­‐ordinator)	
  
•  MediaEval	
  Code	
  of	
  Conduct,	
  Invited	
  Talk,	
  by	
  
   Adam	
  Rae,	
  Yahoo!	
  Research,	
  Spain	
  
•  The	
  Virtual	
  Kitchen	
  for	
  benchmarking:	
  Invited	
  
   Talk,	
  Florian	
  Metze,	
  CMU,	
  USA	
  
•  Special	
  appearance	
  by	
  Carol	
  Peters,	
  ISTI-­‐CNR,	
  
   Italy	
  (Co-­‐ordinator	
  emeritus	
  of	
  the	
  CLEF	
  Cross-­‐
   Language	
  EvaluaCon	
  Forum)	
  
Looking back; looking forward
•  Collec$ng	
  ideas	
  for	
  MediaEval	
  2013	
  	
  
   Please	
  fill	
  out	
  and	
  return	
  the	
  form	
  to	
  Gareth	
  
•  Pitch	
  ideas	
  for	
  new	
  tasks	
  for	
  next	
  year	
  at	
  
   Lightening	
  session	
  (3	
  min.)	
  
    –  Many	
  current	
  tasks	
  
    –  The	
  2012	
  BNTs	
  
    –  Bogdan,	
  Jana,	
  Alberto,	
  	
  
       Chritoph	
  	
  


                                              Flickr	
  Echoes89	
  	
  
Poster Session: Today 18:30
•  Crowdsourcing	
  Social	
  Fashion	
  Images:	
  a	
  Pilot	
  Study	
  
     –  Authors:	
  Maria	
  Menendez,	
  Babak	
  Loni,	
  Martha	
  Larson	
  et	
  al.	
  from	
  the	
  
        CUbRIK	
  project	
  
     –  Ins$tutes:	
  University	
  of	
  Trento,	
  TU	
  Dele,	
  Politecnico	
  die	
  Milano	
  
•  Enhanced	
  Seman0c	
  Similarity	
  using	
  Seman0c	
  Web	
  Technology	
  
     Authors:	
  Tom	
  De	
  Nies	
  et	
  al.	
  
     –  Affilia$ons:	
  Ghent	
  University	
  -­‐	
  IBBT	
  -­‐	
  Mul$media	
  Lab	
  and	
  KAIST	
  -­‐	
  IVY	
  Lab	
  
•  Axes:	
  Access	
  to	
  Audio-­‐Visual	
  Archives	
  
     –  Authors:	
  Shu	
  Chen,	
  Robin	
  Aly,	
  Gareth	
  Jones,	
  Roeland	
  Ordelman	
  
     –  Affilia$ons:	
  Dublin	
  City	
  University	
  and	
  University	
  of	
  TRent	
  
•  Distant-­‐speech	
  Interac0on	
  for	
  Robust	
  Home	
  Applica0ons	
  
     –  Authors:	
  Alberto	
  Abad	
  at	
  al.	
  from	
  the	
  Dirha	
  project	
  
     –  Affilia$ons:	
  INESC-­‐ID	
  	
  	
  
Group Photo: Before lunch Friday
Santa Croce in Fossabanda
Ciao!	
  




            Flickr	
  Marius	
  B	
  	
  

More Related Content

MediaEval 2012 Opening

  • 1. ediaEval 2012 Mul$media  Benchmark  Workshop   Sante  Croce  in  Fossabanda   Pisa,  Italy   4  October  2012  
  • 2. Overview •  What  is  (who  are)  MediaEval?   •  Where  are  we  now  in  year  five?   •  What  to  expect  today?  
  • 3. The MediaEval Benchmark… •  …evaluates  new  algorithms  for  mul$media   access  and  retrieval.     •  …emphasizes  the  "mul$"  in  mul$media:   speech,  audio,  visual  content,  tags,  users,   context.     •  …innovates  new  tasks  and  techniques  focusing   on  the  human  and  social  aspects  of   mul$media  content.  
  • 4. MediaEval Tasks 2012 •  Placing  Task  (6)   •  Social  event  detec$on  Task  (5)   •  Affect  Task:  Violent  Scenes  Detec$on  Task  (8)   •  Spoken  Web  Search  Task  (9)   •  Visual  Privacy  Task  (3)   •  Tagging  Task  (5)   Number  of  teams  crossing  the  finish  line   this  year  is  in  parentheses.  
  • 5. Working Notes Papers 60   50   40   30   20   2011   10   2010   0   2010   2011   2012   Black:  Par$cipant  papers;  Grey:  Task  overview  papers  
  • 7. Workshop Organizing Committee •  Proceedings  Chairs:  Sebas$an  Schmiedeke  and  Pascal   Kelm,  Technische  Universität  Berlin,  Germany     •  Prac00oners  Chair:  Xavier  Anguera,  Telefonica,  Spain     •  Awards  Chair:  Mohammad  Soleymani,  Imperial  College   London,  UK   •  Mul0media  Chair:  Florian  Metze,  CMU,  USA  (Supported  by   Babak  Loni,  TU  Dele)     •  Logis0cs  Chair:  Saskia  Peters,  TU  Dele,  Netherlands   •  Local  Organiza0on  support:  Alessio  Bazzica,  Università   degli  Studi  di  Firenze)   •  General  Chair  and  contact  person:  Martha  Larson  (TU   Dele,  Netherlands)  
  • 8. MediaEval History   •  MediaEval  was  founded  as  VideoCLEF  in  2008.   •  Ran  in  2008  and  2009  at  the  Cross  Language  Evalua$on   Forum  (CLEF)  and  became  MediaEval  in  2010.   •  Ran  in  2010  and  2011  as  an  independent  benchmark   under  the  sponsorship  of  the  PetaMedia  Network  of   Excellence.   •  In  2012,  ran  as  a  completely  “boiom-­‐up  benchmark”  with   no  parent  project.  
  • 9. Where are we now in year five?   •  Wider  MedialEval  community  has  grown:  148   people  took  $me  to  fill  in  the  2012  survey.   •  From,  2008-­‐1012  we  have  generated  100   papers  (VideoCLEF  +  MediaEval  Working  Notes   Proceedings  plus  other  workshop,  conference   and  journal  papers  that  use  MediaEval  data).   •  We  now  have  a  new  “Brave  New  Task”  track  to   incubate  promising  tasks  and  allow  them  to   mature  to  full  tasks.  
  • 12. Where are we now in year five?   •  Wider  MedialEval  community  has  grown:  148   people  took  $me  to  fill  in  the  2012  survey.   •  From,  2008-­‐1012  we  have  generated  100   papers  (VideoCLEF  +  MediaEval  Working  Notes   Proceedings  plus  other  workshop,  conference   and  journal  papers  that  use  MediaEval  data).   •  We  now  have  a  new  “Brave  New  Task”  track  to   incubate  promising  tasks  and  allow  them  to   mature  to  full  tasks.  
  • 13. Brave New Tasks •  User  Account  Matching  (1)   •  Search  and  Hyperlinking  (6)   •  Musiclef:  Mul$modal  Music  Tagging  (1)   Brave  New  Tasks  is  an   “incuba$ng”  track  that   serves  to  breed  new  tasks.   Flickr  Mads  Boe  
  • 14. Familiar from past years •  Wireless  internet  not  reliable   •  We  are  recording  audio  (Florian).   •  Consent  forms  for  recorded  audio  (Babak):   Please  sign  and  return  your  form   •  T-­‐Shirts:  Nearly  random   •  The  restroom  is  across  the  courtyard.  
  • 15. Familiar from last year •  Short  (10  minute)  presenta$ons:     –  Please  respect  the  bell.   –  Please  load  your  presenta$on  in  the  break  right   before  your  session.   –  The  Pillory:  What  happens  if  you  run  over  your   $me  slot   •  The  caterer  expects  us  to  keep  on  schedule.  
  • 17. Multimedia@MediaEval 2012   •  We  not  only  analyze  mul$media  we  create  it.   •  Please  take  photos:  Babak  Loni  will  collect   them  on  the  final  day  and  upload  them  to  the   Mediaeval  Flickr  account.  
  • 18. More multimedia 2012   •  John  N.A.  Brown  is  our  filmmaker  this  year.   •  To  join  us  he  takes  a  break  from  his  ‘other  life’  working  to   finishing  his  PhD  in  Erasmus  Mundus  Joint  Doctoral   Programme  in  Interac$ve  and  Cogni$ve  Environments.   •  He  may  ask  you  to  say  a  few   words  about  MediaEval  in  front   of  the  camera.  
  • 19. Keynote Speakers •  Invited  Talk  by  Jana  Eggink,  BBC  Research  and   Development,  UK   •  Invited  Talk  by  Nicola  Ferro,  University  of  Padova,   Italy  (PROMISE  Co-­‐ordinator)   •  MediaEval  Code  of  Conduct,  Invited  Talk,  by   Adam  Rae,  Yahoo!  Research,  Spain   •  The  Virtual  Kitchen  for  benchmarking:  Invited   Talk,  Florian  Metze,  CMU,  USA   •  Special  appearance  by  Carol  Peters,  ISTI-­‐CNR,   Italy  (Co-­‐ordinator  emeritus  of  the  CLEF  Cross-­‐ Language  EvaluaCon  Forum)  
  • 20. Looking back; looking forward •  Collec$ng  ideas  for  MediaEval  2013     Please  fill  out  and  return  the  form  to  Gareth   •  Pitch  ideas  for  new  tasks  for  next  year  at   Lightening  session  (3  min.)   –  Many  current  tasks   –  The  2012  BNTs   –  Bogdan,  Jana,  Alberto,     Chritoph     Flickr  Echoes89    
  • 21. Poster Session: Today 18:30 •  Crowdsourcing  Social  Fashion  Images:  a  Pilot  Study   –  Authors:  Maria  Menendez,  Babak  Loni,  Martha  Larson  et  al.  from  the   CUbRIK  project   –  Ins$tutes:  University  of  Trento,  TU  Dele,  Politecnico  die  Milano   •  Enhanced  Seman0c  Similarity  using  Seman0c  Web  Technology   Authors:  Tom  De  Nies  et  al.   –  Affilia$ons:  Ghent  University  -­‐  IBBT  -­‐  Mul$media  Lab  and  KAIST  -­‐  IVY  Lab   •  Axes:  Access  to  Audio-­‐Visual  Archives   –  Authors:  Shu  Chen,  Robin  Aly,  Gareth  Jones,  Roeland  Ordelman   –  Affilia$ons:  Dublin  City  University  and  University  of  TRent   •  Distant-­‐speech  Interac0on  for  Robust  Home  Applica0ons   –  Authors:  Alberto  Abad  at  al.  from  the  Dirha  project   –  Affilia$ons:  INESC-­‐ID      
  • 22. Group Photo: Before lunch Friday
  • 23. Santa Croce in Fossabanda Ciao!   Flickr  Marius  B