The document summarizes the MediaEval 2012 multimedia benchmark workshop. Some key points:
- MediaEval is in its 5th year of evaluating new algorithms for multimedia access and retrieval, emphasizing the "multi" aspects of multimedia.
- The 2012 workshop saw 6 tasks and over 30 participant teams.
- MediaEval has grown its community to over 140 participants and has generated over 100 academic papers since 2008.
- The workshop highlights included keynote speakers from BBC research and Yahoo research, as well as a filmmaker documenting the event.
2. Overview
• What
is
(who
are)
MediaEval?
• Where
are
we
now
in
year
five?
• What
to
expect
today?
3. The MediaEval Benchmark…
• …evaluates
new
algorithms
for
mul$media
access
and
retrieval.
• …emphasizes
the
"mul$"
in
mul$media:
speech,
audio,
visual
content,
tags,
users,
context.
• …innovates
new
tasks
and
techniques
focusing
on
the
human
and
social
aspects
of
mul$media
content.
4. MediaEval Tasks 2012
• Placing
Task
(6)
• Social
event
detec$on
Task
(5)
• Affect
Task:
Violent
Scenes
Detec$on
Task
(8)
• Spoken
Web
Search
Task
(9)
• Visual
Privacy
Task
(3)
• Tagging
Task
(5)
Number
of
teams
crossing
the
finish
line
this
year
is
in
parentheses.
7. Workshop Organizing Committee
• Proceedings
Chairs:
Sebas$an
Schmiedeke
and
Pascal
Kelm,
Technische
Universität
Berlin,
Germany
• Prac00oners
Chair:
Xavier
Anguera,
Telefonica,
Spain
• Awards
Chair:
Mohammad
Soleymani,
Imperial
College
London,
UK
• Mul0media
Chair:
Florian
Metze,
CMU,
USA
(Supported
by
Babak
Loni,
TU
Dele)
• Logis0cs
Chair:
Saskia
Peters,
TU
Dele,
Netherlands
• Local
Organiza0on
support:
Alessio
Bazzica,
Università
degli
Studi
di
Firenze)
• General
Chair
and
contact
person:
Martha
Larson
(TU
Dele,
Netherlands)
8. MediaEval History
• MediaEval
was
founded
as
VideoCLEF
in
2008.
• Ran
in
2008
and
2009
at
the
Cross
Language
Evalua$on
Forum
(CLEF)
and
became
MediaEval
in
2010.
• Ran
in
2010
and
2011
as
an
independent
benchmark
under
the
sponsorship
of
the
PetaMedia
Network
of
Excellence.
• In
2012,
ran
as
a
completely
“boiom-‐up
benchmark”
with
no
parent
project.
9. Where are we now in year five?
• Wider
MedialEval
community
has
grown:
148
people
took
$me
to
fill
in
the
2012
survey.
• From,
2008-‐1012
we
have
generated
100
papers
(VideoCLEF
+
MediaEval
Working
Notes
Proceedings
plus
other
workshop,
conference
and
journal
papers
that
use
MediaEval
data).
• We
now
have
a
new
“Brave
New
Task”
track
to
incubate
promising
tasks
and
allow
them
to
mature
to
full
tasks.
12. Where are we now in year five?
• Wider
MedialEval
community
has
grown:
148
people
took
$me
to
fill
in
the
2012
survey.
• From,
2008-‐1012
we
have
generated
100
papers
(VideoCLEF
+
MediaEval
Working
Notes
Proceedings
plus
other
workshop,
conference
and
journal
papers
that
use
MediaEval
data).
• We
now
have
a
new
“Brave
New
Task”
track
to
incubate
promising
tasks
and
allow
them
to
mature
to
full
tasks.
13. Brave New Tasks
• User
Account
Matching
(1)
• Search
and
Hyperlinking
(6)
• Musiclef:
Mul$modal
Music
Tagging
(1)
Brave
New
Tasks
is
an
“incuba$ng”
track
that
serves
to
breed
new
tasks.
Flickr
Mads
Boe
14. Familiar from past years
• Wireless
internet
not
reliable
• We
are
recording
audio
(Florian).
• Consent
forms
for
recorded
audio
(Babak):
Please
sign
and
return
your
form
• T-‐Shirts:
Nearly
random
• The
restroom
is
across
the
courtyard.
15. Familiar from last year
• Short
(10
minute)
presenta$ons:
– Please
respect
the
bell.
– Please
load
your
presenta$on
in
the
break
right
before
your
session.
– The
Pillory:
What
happens
if
you
run
over
your
$me
slot
• The
caterer
expects
us
to
keep
on
schedule.
17. Multimedia@MediaEval 2012
• We
not
only
analyze
mul$media
we
create
it.
• Please
take
photos:
Babak
Loni
will
collect
them
on
the
final
day
and
upload
them
to
the
Mediaeval
Flickr
account.
18. More multimedia 2012
• John
N.A.
Brown
is
our
filmmaker
this
year.
• To
join
us
he
takes
a
break
from
his
‘other
life’
working
to
finishing
his
PhD
in
Erasmus
Mundus
Joint
Doctoral
Programme
in
Interac$ve
and
Cogni$ve
Environments.
• He
may
ask
you
to
say
a
few
words
about
MediaEval
in
front
of
the
camera.
19. Keynote Speakers
• Invited
Talk
by
Jana
Eggink,
BBC
Research
and
Development,
UK
• Invited
Talk
by
Nicola
Ferro,
University
of
Padova,
Italy
(PROMISE
Co-‐ordinator)
• MediaEval
Code
of
Conduct,
Invited
Talk,
by
Adam
Rae,
Yahoo!
Research,
Spain
• The
Virtual
Kitchen
for
benchmarking:
Invited
Talk,
Florian
Metze,
CMU,
USA
• Special
appearance
by
Carol
Peters,
ISTI-‐CNR,
Italy
(Co-‐ordinator
emeritus
of
the
CLEF
Cross-‐
Language
EvaluaCon
Forum)
20. Looking back; looking forward
• Collec$ng
ideas
for
MediaEval
2013
Please
fill
out
and
return
the
form
to
Gareth
• Pitch
ideas
for
new
tasks
for
next
year
at
Lightening
session
(3
min.)
– Many
current
tasks
– The
2012
BNTs
– Bogdan,
Jana,
Alberto,
Chritoph
Flickr
Echoes89
21. Poster Session: Today 18:30
• Crowdsourcing
Social
Fashion
Images:
a
Pilot
Study
– Authors:
Maria
Menendez,
Babak
Loni,
Martha
Larson
et
al.
from
the
CUbRIK
project
– Ins$tutes:
University
of
Trento,
TU
Dele,
Politecnico
die
Milano
• Enhanced
Seman0c
Similarity
using
Seman0c
Web
Technology
Authors:
Tom
De
Nies
et
al.
– Affilia$ons:
Ghent
University
-‐
IBBT
-‐
Mul$media
Lab
and
KAIST
-‐
IVY
Lab
• Axes:
Access
to
Audio-‐Visual
Archives
– Authors:
Shu
Chen,
Robin
Aly,
Gareth
Jones,
Roeland
Ordelman
– Affilia$ons:
Dublin
City
University
and
University
of
TRent
• Distant-‐speech
Interac0on
for
Robust
Home
Applica0ons
– Authors:
Alberto
Abad
at
al.
from
the
Dirha
project
– Affilia$ons:
INESC-‐ID