Sergi Jordà /

Associate Professor

Music Technology Group
Dept. of Information and Communication Technologies
Universitat Pompeu Fabra, Barcelona

sergi.jorda@upf.edu


Overview


My scientific and professional career path rand its particular multidisciplinary approach can be better comprehended by understanding my long-term passion for scientific research, music and artistic creativity, and the desire to intertwine those elements in innovative ways.

I hold a Bachelor degree in Fundamental Physics (1986). I combined my undergraduate studies in the early 80s with music studies at the Conservatory and free jazz saxophone playing. After discovering computer programming during my third year of Physics at the university, I commited to dedicate myself to computer music improvisation (a completely novel area of practice and research in the mid 1980s in Spain). After graduation I thus worked as a software developer in the industry while self-studying computer music with an Amiga computer in the evenings. In this period I also started to collaborate with media artists and to perform live computer music with the "no-wave" and experimental jazz music group, Clónicos. Throughout the 1990s I conceived and developed award winning interactive installations, multimedia performances [7] and a collaborative music creation project over the Internet [5,6], in collaboration with internationally renowned Catalan artists such as Marcel·lí Antunez or La Fura dels Baus. I also continued performing and improvising live computer music, both alone and with the group FMOL Trio, as well as composing commisions for movies, media shows or specific musical ensembles.

I resumed my scientific and academic career in 1999, when the University Pompeu Fabra initiated the Computer Engineering Faculty (DTIC), together with a research group in computer music, the Music Technology Group (MTG), headed by Dr. Xavier Serra. This allowed me to start a researcher and teacher career, with the full support of the university. Hence, although my academic career officially started in 1999, my subsequent research could not be comprehended without the knowledge, vision and experience gained in the professional and artistic world throughout the 1990s.

In 2005 I was awarded with a PhD [8]. Within the MTG, I have coordinated two European projects and do currently lead a Music and Advanced Interaction team with 4 PhD students that focuses in Human Computer Interaction (HCI), with a special emphasis on real time music performance, tangible interaction and BCI interfaces. A first result of this convergence between computer music performance and HCI, was the project on tangible interfaces Reactable [9,10], which in 2007 gained international repercussion when Icelandic singer Björk incorporated a Reactable in her world tour Volta. With hundreds of articles in international magazines and general press, hundreds of lectures and presentations around the world, national and international presentations on television, the project has accomplished unparalleled mass popularity for a project of academic origin, having received many prestigious international awards. The technology transfer generated by the Reactable was also quite relevant (between 2006 and 2008, UPF sold 12 Reactable units to institutions on three continents). Within this growing trend, in March 2009, the creators team decided to create Reactable Systems, the third spin-off of UPF, with the participation of the university and Xavier Serra, head of the MTG research group. While two of the Reactable’s original researchers are working full-time at the company, I decided to stay within the academy, finally obtaining a tenure-track Professor position at UPF in 2011 and pursuing a longer-term research within the framework of the MTG.

After the Reactable project, I have become one of the world leading experts in Tangible Interaction (TI), with papers with 310+ citations [9], chairing twice the international conference on “Tangible Embedded and Embodied Interaction” (TEI) and becoming a member of TEI's steering committee. My team and I have also entered non-musical domains, investigating some of the special areas in which tangible interaction may be more profitable [11], creating tangible programming languages for children [3], studying the potential benefits of tabletops within special users such as autistic children [17], refining techniques for the tracking of objects in 3D space, exploring augmented communication and computer-supported collaborative work using tabletops and human physiology, studying implicit interaction and multimodal display of users' affective and cognitive states [14,15,16], developing open-source tabletop programming SDKs for helping others building their own tabletop applications [13], analyzing the potential for creativity in tabletop interaction [2] or creating frameworks and methods for the evaluation of complex interaction techniques such as the ones taking place on a collaborative tabletop [1,18,21]).

I am also especially interested in education, and I feel it is now time for me, for a new, ambitious and domain oriented direction, and at the same time for a fully multidisciplinary and holistic approach. I am thus starting to work on what, in my opinion, constitutes nowadays the most relevant challenge in Human Computer Interaction (HCI), the study of interaction design for children’s education. Young children, which are still mostly free of the burdens of 25 years on WIMP and GUI interaction, will be shaping our world very soon, and it is our big responsibility to provide them with a richer and more creative education - away from the burdens of the 19th century education we are still relying on - that will allow them to be freer, more conscious, open-minded, creative, and more prepared for the inescapable radical cultural, socio-economical and ecological shifts we are already starting to experiment.


The following references are not necessarily my most relevant publications. They have been selected in order to give an overview of my different lines of research, with a special focus on the more recent years and the newer work being carried together with the PhD candidates I'm currently supervising.

[1] Bosi, M. and Jordà S. (2012).Towards fast multi-point force and hit detection in tabletops using mechanically intercoupled Force Sensing Resistors. NIME'12.

[2] Catala, A., Jaen, J., van Dijk, B. and Jordà, S. (2012). Exploring Tabletops as an Effective Tool to Foster Creativity Traits. TEI 2012, Sixth international conference on tangible, embedded and embodied interaction. Kingston, Ontario, Canada.

[3] Gallardo, D., Julià, C.F. and Jordà, S. (2008). TurTan: A tangible programming language for creative exploration. Proceedings of the 3d IEEE international workshop on horizontal interactive human-computer systems “Tabletops & Interactive Surfaces” Amsterdam, Nederlands.

[4] Gallardo, D., Julià C. F. and Jordà S. (2013). Using MTCF for live prototyping on tablet and tangible tabletop devices. Seventh International Conference on Tangible and Embedded Interaction (TEI’13). Universitat Pompeu Fabra, Barcelona. Proceedings of the ACM Seventh International Conference on Tangible and Embedded Interaction (TEI), x-x.

[5] Jordà, S. (1999). "Faust Music On Line: (FMOL) An Approach To Real-Time Collective Composition On The Internet", Leonardo Music Journal 9:5-12.

[6] Jordà, S. (2002). "FMOL: Toward User-Friendly, Sophisticated New Musical Instruments". Computer Music Journal Number 26-3: 23-39.

[7] Jordà, S. (2002) Afasia: the Ultimate Homeric one-Man-Multimedia-Band, New Interfaces for Musical Expression Conference (Nime 2002). MediaLab, Dublin (Ireland). Proceedings of the New Interfaces for Musical Expression Conference (Nime 2002), ISBN1-87465365-8, 132-137.

[8] Jordà, S. (2005). Digital Lutherie: Crafting Musical Computers for New Musics’ Performance and Improvisation. Ph.D. thesis, Universitat Pompeu Fabra, Barcelona.

[9] Jordà, S., Geiger, G., Alonso, M. and Kaltenbrunner, M. (2007) Exploring the Synergy between Live Music Performance and Tabletop Tangible Interfaces. First International Conference on Tangible and Embedded Interaction (TEI’07). Louisiana State University, Baton Rouge, LA, USA. Proceedings of the ACM First International Conference on Tangible and Embedded Interaction (TEI), 139-146.

[10] Jordà, S. (2008). "On Stage: the Reactable and other Musical Tangibles go Real". International Journal of Arts and Technology, 1-3/4: 268-287.

[11] Julià, C.F. and Jordà, S. (2008). SongExplorer: a tabletop application for exploring large collections of songs. 10th International Society for Music Information Retrieval Conference ISMIR, 10th International Society for Music Information Retrieval Conference. Kobe, Japan. Proceedings of the 2008 IEEE international workshop on horizontal interactive human-computer systems.

[12] Julià, C. F., Jordà S. and Earnshaw N. (2013). GestureAgents: An Agent-Based Framework for Concurrent Multi-Task Multi-User Interaction. Seventh International Conference on Tangible and Embedded Interaction (TEI’13). Universitat Pompeu Fabra, Barcelona. Proceedings of the ACM Seventh International Conference on Tangible and Embedded Interaction (TEI), x-x.

[13] Julià, C.F., Gallardo, D. and Jordà, S. (2011). MTCF: A framework for designing and coding musical tabletop applications directly in Pure Data. 11th International Conference on New Interfaces for Musical Expression, NIME’11. Oslo, Norway. Proceedings of the 11th International Conference on New Interfaces for Musical Expression, NIME’11.

[14] Mealla, S., Valjamae, A., Bosi, M. and Jordà, S. (2011). Let Me Listen to Your Brain: Physiology-based Interaction in Collaborative Music Composition CHI 2011: 29th ACM Conference on Human Factors in Computing Systems. Vancouver, BC, Canada. Proceedings of the 2011 ACM SIGCHI 29th ACM Conference on Human Factors in Computing Systems, ACM 978-1-4503-0268-5/11/05.

[15] Mealla, S., Valjamae, A., Bosi, M. and Jordà, S. (2011). Listening to Your Brain: Implicit Interaction in Collaborative Music Performances. 11th International Conference on New Interfaces for Musical Expression, NIME’11. Oslo, Norway. Proceedings of the 11th International Conference on New Interfaces for Musical Expression, NIME’11.

[16] Mealla, S., Valjamae, A. and Jordà, S. (2011). Sonification of Brain and Body Signals in Collaborative Tasks Using a Tabletop Musical Interface. International Conference on Auditory Display (ICAD). Budapest, Hungary.

[17] Villafuerte, L., and Jordà, S. (2012). Acquisition of Social Abilities through music Tangible User Interface: children with Autism Spectrum condition and the Reactable. ACM SIGCHI, Seattle (USA).

[18] Xambó, A., Laney, R., Dobbyn, C. and Jordà, S. (2011). Collaborative music interaction on tabletops: An HCI approach. BCS HCI 2011 Workshop - When Words Fail: What can Music Interaction tell us about HCI? The 25th BCS Conference on Human-Computer-Interaction. Northumbria University, Newcastle Upon Tyne (UK). Proceedings of the 25th BCS Conference on Human-Computer-Interaction.

[19] Xambó, A., Laney R., Dobbyn C. and Jordà S. (2011). Multi-touch interaction principles for collaborative real-time music activities: towards a pattern language. International Computer Music Conference.

[20] Xambó, A., Laney R., Dobbyn C., & Jordà S. (2012).??Towards a taxonomy for video analysis on collaborative musical tabletops. BCS HCI 2012 Workshop on Video Analysis Techniques for HCI.

[21] Xambó, A., Laney, R., Dobbyn, C. and Jordà, S. (2013) Video analysis for evaluating music interaction: musical tabletops. Book chapter in “Music interaction”, Simon Holland, Katie Wilkie, Allan Seago and Paul Mulholland, editors. Springer Cultural Computing Series, Springer Verlag.


1/7

Next project: → Publications

Previous project: ← PhDs and Master theses supervised

Overview
Overview