BEGIN:VCALENDAR VERSION:2.0 PRODID:-//Talks.cam//talks.cam.ac.uk// X-WR-CALNAME:Talks.cam BEGIN:VEVENT SUMMARY:Forward and Inverse models in Human-Computer Interaction: Physical simulation and machine learning for inferring 3D finger pose - Professor Roderick Murray-Smith\, University of Glasgow DTSTART:20200206T110000Z DTEND:20200206T120000Z UID:TALK139369@talks.cam.ac.uk CONTACT:Prof Neil Lawrence DESCRIPTION:I will give a brief overview of the research activities in the Information\, Data & Analysis Section at the University of Glasgow\, touc hing on some of our recent applications of Machine Learning in optics\, nu clear physics & gravitational waves\, and our Closed-loop Data Science res earch project.\n\nI will then outline the role of computational methods in the design of human-computer interaction\, and more specifically the role of forward and inverse modelling approaches. Causal\, forward models tend to be easier to specify and simulate\, but the inverse problem (‘what w as the user’s intention’) is what typically needs to be solved in an H CI context. We illustrate the core issues in a case study\, where we quant ify the accuracy with which single finger 3D position (x\,y\,z) and pose ( pitch and yaw) can be inferred in real time on a mobile device using recen t developments in capacitive sensing technology which can sense the finger up to 5cm above a mobile phone screen. We use machine learning to develop data-driven models to infer position\, pose and sensor readings\, based o n three approaches to gathering training data: 1. data generated by robots \, 2. data from electrostatic simulators and 3. human-generated data. A fo rward model is trained on this data using a deep convolutional network. Th is emulation can accelerate the electrostatic simulation performance with a speedup factor of 2.4 million. We compare forward and inverse model appr oaches to inference of finger pose and fuse them with a probabilistic filt er. This combination of forward and inverse models improves performance ov er previous inverse-only approaches\, giving the most accurate reported re sults on inferring 3D position and pose with a mobile phone capacitive sen sor. I will then give an outlook on how we can improve on this with recent ly developed variational inference approaches\, and discuss the potential of these methods for moving towards a more constructive human model-based pipeline for design of human-computer systems.\n\nSome related publication s:\nInverse methods via ML & for HCI\nFrancesco Tonolini\, Jack Radford\, Alex Turpin\, Daniele Faccio\, Roderick Murray-Smith\, Variational Inferen ce for Computational Imaging Inverse Problems\, Jan 2020 https://arxiv.org /abs/1904.06264\n\nR. Murray-Smith\, Stratified\, computational interactio n via machine learning\, The Eighteenth Yale Workshop on Adaptive and Lear ning Systems\, Ed. K. Narendra\, Yale\, June 21-23rd\, 2017. http://www.dc s.gla.ac.uk/~rod/publications/Mur17.pdf\n\nJ. Müller\, A. Oulasvirta\, R. Murray-Smith\, Control Theoretic Models of Pointing\, ACM Transactions on Computer-Human Interaction\, Vol. 24\, No. 4\, August 2017. http://www.dc s.gla.ac.uk/~rod/publications/MueOulMur17.pdf\n\nMurray-Smith\, Control Th eory\, Dynamics and Continuous Interaction\, in Computational Interaction Design\, eds. A. Oulasvirta\, P. O. Kristensson\, A. Howes\, X. Bi\, Oxfor d University Press\, 2018. http://www.dcs.gla.ac.uk/~rod/publications/Mur1 8.pdf\n\nBoland\, D.\, R. McLachlan\, R. Murray-Smith. Engaging with mobil e music retrieval\, Proceedings of the 17th International Conference on Hu man-Computer Interaction with Mobile Devices and Services. ACM\, 2015. The "B&O Beomoment system":https://www.youtube.com/watch which resulted from this work.\n\nB. Vad\, Boland\, D.\, Williamson\, J.\, Murray-Smith\, R.\ , and Steffensen\, P. B.\, Design and evaluation of a probabilistic music projection interface\, In: 16th International Society for Music Informatio n Retrieval Conference\, Malaga\, Spain\, 26-30 Oct 2015. pdf\n\nF Tonolin i\, BS Jensen\, R Murray-Smith\, Variational Sparse Coding\, Uncertainty i n Artificial Intelligence\, 2019\n\nOptics & ML:\nP Caramazza\, O Moran\, R Murray-Smith\, D Faccio\, Transmission of natural scene images through a multimode fibre\, Nature communications 10 (1)\, 1-6\, 2019. \n\nO. Mora n\, P. Caramazza\, D. Faccio\, R. Murray-Smith\, Deep\, complex\, invertib le networks for inversion of transmission effects in multimode optical fib res\, Advances in Neural Information Processing Systems 31 (NeurIPS 2018)\ , Montreal. pdf\n\nN. Radwell\, S. D. Johnson\, M. P. Edgar\, C. F. Higham \, R. Murray-Smith\, and M. J. Padgett Deep learning optimized single-pixe l LiDAR\, Applied Physics Letters 115\, 231101 (2019). pdf\n\nC.F. Higham\ , R. Murray-Smith\, M.J. Padgett\, M.P. Edgar\, Deep learning for real-tim e single-pixel video\, Nature Scientific Reports\, 2018. pdf\n\nNuclear ph ysics & Gaussian processes:\nD. G. Ireland\, M. Döring\, D. I. Glazie r\, J. Haidenbauer\, M. Mai\, R. Murray-Smith\, and D. Rönchen\, Kaon Pho toproduction and the Λ Decay Parameter α−\, Phys. Rev. Lett. 123\, 182 301\, October 2019. pdf\n\nGravitational waves:\nH Gabbard\, C Messenger\, IS Heng\, F Tonolini\, R Murray-Smith\, Bayesian parameter estimation usi ng conditional variational autoencoders for gravitational-wave astronomy\, Sept 2019 https://arxiv.org/abs/1909.06296\n\nBIOGRAPHY\nRoderick Murray- Smith is a Professor of Computing Science at Glasgow University\, leading the Inference\, Dynamics and Interaction research group\, and heads the 6 0-strong Section on Information\, Data and Analysis\, which also includes the Information Retrieval\, Computer Vision & Autonomous systems and KDE B ig Data groups. He works in the overlap between machine learning\, interac tion design and control theory. In recent years his research has included multimodal sensor-based interaction with mobile devices\, mobile spatial i nteraction\, AR/VR\, Brain-Computer interaction and nonparametric machine learning. Prior to this he held positions at the Hamilton Institute\, NUIM \, Technical University of Denmark\, M.I.T. (Mike Jordan’s lab)\, and Da imler-Benz Research\, Berlin\, and is the Director of SICSA\, the Scottish Informatics and Computing Science Alliance (all academic CS departments i n Scotland). He works closely with the mobile phone industry\, having work ed together with Nokia\, Samsung\, FT/Orange\, Microsoft and Bang & Olufs en. He was a member of Nokia's Scientific Advisory Board and a member of t he Scientific Advisory Board for the Finnish Centre of Excellence in Compu tational Inference Research. He has co-authored three edited volumes\, 38 journal papers\, 21 book chapters\, and over 100 conference papers.\n\nhtt p://www.dcs.gla.ac.uk/~rod/\nhttp://www.dcs.gla.ac.uk/~rod/Publications.ht m\nhttps://twitter.com/IDAglasgow\nhttps://twitter.com/MurraySmithRod\n LOCATION:Computer Laboratory\, William Gates Building\, Room FW26 END:VEVENT END:VCALENDAR