Mulsemedia — multiple sensorial media — towards understanding user and quality of experience of future media applications
Niall Murray
Athlone Institute of Technology (AIT), Ireland
Abstract. Recently, there has been an emerging interest and research activity in in sensory technologies that stimulate all of the human senses vision, hearing, touch, smell, and taste. Critical to the success of such experiences is to understand how users perceive these ‘new’ enhanced multimedia experiences. Only by addressing this challenge systematically from technical and perceptual perspectives, can we meaningfully design multi-sensory multimedia experiences. In this talk, we provide an introduction to state of the art in Mulsemedia quality of experience (QoE) and user experience (Ux) research which considers different evaluation methodologies and approaches. The talk will also provide some discussion on the research carried by us through developing our own prototypes and executing user trials related to tactile, olfactory enhanced audio-visual experiences.
Biography. Dr. Niall Murray (www.niallmurray.info) is a Lecturer and researcher with the Faculty of Engineering and Informatics in the Athlone Institute of Technology (AIT), Ireland. He received his BE (Electronic and Computer Engineering) from National University of Ireland, Galway (2003), MEng (Computer and Communication Systems) from the University of Limerick (2004) and PhD from the Software Research Institute (SRI) in the Athlone IT in 2014. Since 2004, he has worked in R&D roles across a number of industries: Telecommunications, Finance, Health and Education. In 2014 He founded the Truly Immersive and Interactive Multimedia Experiences lab (tIIMEx). His research interests include Immersive Multimedia Communication, Multisensory Multimedia, Quality of Experience and Multimedia Synchronization. In this context, tIIMEx builds and evaluates from a user perceived quality perspective, end-to-end communication systems and novel immersive and interactive applications. He is a Funded Investigator with the Science Foundation Ireland (SFI) “Adapt Centre: the Global Centre of Excellence for Digital Content and Media Innovation” and a Funded Investigator with the SFI “Confirm Centre for Smart Manufacturing” where he applies his expertise in QoE, AR, VR and Mulsemedia across a range of application domains.
The art of saliency modeling for multimedia applications
Patrick Le Callet
Université de Nantes, France
Abstract. An extensive research work has been done in the last years to develop Visual Attention (VA) models for 2D, stereoscopic 3D images and videos or more recently for Virtual Reality and 360º. Reliable VA models are helpful in order to design efficient approaches for several applications, such as coding, streaming, foveated rendering, cinematography, movie editing, and Quality of Experience (QoE) evaluation. In this talk, I will review the current status on VA: advances and challenges from user study to modeling and benchmarking. A special focus will be dedicated to omnidirectional content. I will also illustrate how studying visual attention deployment of visual impaired people can help to improve VA computational modeling.
Biography. (IEEE fellow) received both an M.Sc. and a PhD degree in image processing from Ecole polytechnique de l’Université de Nantes. He was also a student at the Ecole Normale Superieure de Cachan where he sat the “Aggrégation” (credentialing exam) in electronics of the French National Education. He worked as an Assistant Professor from 1997 to 1999 and as a full time lecturer from 1999 to 2003 at the Department of Electrical Engineering of Technical Institute of the University of Nantes (IUT). Since 2003, he teaches at Ecole polytechnique de l’Université de Nantes (Engineering School) in the Electrical Engineering and the Computer Science departments where is now a Full Professor. He led for ten years (2006-16) the Image and Video Communication lab at CNRS IRCCyN and was one of the five members (2013-16) of the Steering Board of CNRS IRCCyN (250 researchers). Since January 2017, he is one of the seven members of the steering Board the CNRS LS2N lab (450 researchers), as representative of Polytech Nantes. He is also since 2015 the scientific director of the cluster “Ouest Industries Créatives”, a five year program gathering more than 10 institutions (including 3 universities). “Ouest Industries Créatives” aims to strengthen Research, Education & Innovation of the Region Pays de Loire in the field of Creative Industries. He is mostly engaged in research dealing with the application of human vision modeling in image and video processing. His current centers of interest are Quality of Experience assessment, Visual Attention modeling and applications, Perceptual Video Coding and Immersive Media Processing. He is co-author of more than 250 publications and communications and co-inventor of 16 international patents on these topics. He serves or has been served as associate editor or guest editor for several Journals such as IEEE TIP, IEEE STSP, IEEE TCSVT, SPRINGER EURASIP Journal on Image and Video Processing, and SPIE JEI. He is serving in IEEE IVMSP-TC (2015- to present) and IEEE MMSP-TC (2015-to present) and one the founding member of EURASIP SAT (Special Areas Team) on Image and Video Processing.
Mulsemedia in 360 VR
George Ghinea
Brunel University, London
Abstract. Previous research has shown that adding multisensory media—mulsemedia—to traditional audiovisual content has a positive effect on user Quality of Experience (QoE). However, the QoE impact of employing mulsemedia in 360 videos has remained unexplored. Accordingly, in this talk, Prof. Ghinea is going to present results of a recently concluded QoE study for watching 360 videos—with and without multisensory effects—in a full free-viewpoint Virtual Reality.
Biography. Dr. Gheorghita (George) Ghinea is a Professor in Mulsemedia Computing in the Department of Computer Science, at Brunel University. Dr. Ghinea’s research activities lie at the confluence of Computer Science, Media and Psychology. In particular, his work focuses on the area of perceptual multimedia quality and how one builds end-to-end communication systems incorporating user perceptual requirements. To this end, recognising the infotainment duality of multimedia, Dr. Ghinea proposed the Quality of Perception metric as a more complete characterisation of the human side of the multimedia perceptual experience. Dr. Ghinea has applied his expertise in areas such as eye-tracking,telemedicine, multi-modal interaction, and ubiquitous and mobile computing, leading a team of 8 researchers in these areas. He has over 300 publications in his research field and is the lead Brunel investigator of a H2020 project NEWTON (http://www.newtonproject.eu/) applying mulsemedia to STEM learning across Europe.
Hyperknowledge Graphs – Enabling Knowledge Graphs at scale with the Nested Context Model
Renato Cerqueira
Senior Research Manager, Natural Resources Solutions
IBM Research Brazil
Abstract. With the recent advances in Artificial Intelligence, the use of knowledge graphs is becoming pervasive. A new generation of AI-based applications is emerging, which relies on KGs to represent and manage knowledge. These KGs are consumed by advanced Machine Learning algorithms and reasoning engines. Despite of this increasing importance to the industry, there is a lack of methods and technologies to manage the lifecycle of KGs, especially to support reusability and maintainability. In this presentation we describe our work on the Hyperknowledge technology, which relies on the Nested Context Model to provide a flexible knowledge representation mechanism with high-level abstractions and expressiveness to enrich structure, enable reuse, and enhance interpretability.
Biography. Renato Cerqueira is Senior Research Manager and Senior Technical Staff Member at IBM Research Brazil, where he leads the Natural Resources Solutions group, aimed at investigating and developing new technologies to advance the ongoing digitalization of industries like Oil & Gas and Mining. With 25 years of experience with R&D projects in different industries, in the recent years Renato and his research group have been working together with different industry partners to create AI-based technologies for data and knowledge intensive processes, mainly in subsurface exploration and oil production. He is member of the IBM Industry Academy since 2017. Prior to join IBM Research in 2011, Renato was professor at the Department of Informatics at PUC-Rio, and research manager at Tecgraf/PUC-Rio, where he participated and led several R&D projects with partners from industry and academia. During 2001, he was Visiting Researcher at the University of Illinois at Urbana-Champaign, working with Prof. Roy H. Campbell on middleware technologies for Ubiquitous Computing. He has published more than 100 papers and advised several Ph.D. and M.Sc. students. He received his Ph.D. (2000) and M.Sc. (1996) degrees in Computer Science and his Computer Engineer (1993) degree from PUC-Rio.