The success of any Machine Learning strategy depends on the conceptual and algorithmic Representation that is selected for Encoding and Processing Information. Further, the chosen encoding/representation completely determines the questions that can be asked, analyses that can be performed, and the answers that can be obtained. Ultimately, the effectiveness and efficiency of any ML strategy depend on Information Theoretic choices related to what Information we chose to encode (and store), the form in which we choose to encode that Information, and the method by which that encoded Information is processed. My view is that by rooting the development of Machine Learning/Artificial Intelligence and Physics-Based Modeling in the fundamental perspectives and language of Information Theory, we can hope to achieve the most rapid progress in the Domain Sciences. While my thoughts may perhaps be speculative, I do not think I am alone in thinking this way, as evidenced by ML literature related to Information Bottleneck theory, and also to the fundamentals of Computational Science.
Hoshin Vijai Gupta is Regents Professor of Hydrology and Atmospheric Sciences at The University of Arizona. He received his BS in Civil Engineering from IIT Bombay, and MS and PhD degrees in Systems Engineering from Case Western Reserve University. His broad interest is in how “Learning” happens through the development and use of “Models”, and more specifically in how to combine Physics-Based Knowledge with Machine Learning (via Information Theory) for developing Earth & Environmental Systems Models that can progressively learn from interactions with the environment.
In 2017 and 2018, Hoshin was ranked in the top 1% on the Clarivate “Highly Cited Researchers List” for Environment/Ecology. He is a Fellow of the American Geophysical Union and the American Meteorological Society, recipient of AMS’s RE Horton Lecture Award (2017) and EGU’s Dalton Medal (2014), and has served as an Editor of Water Resources Research (2009-2013).
Hoshin teaches an introductory class on “The Bare Minimum” one needs to know about the physics-based approach to Environmental Systems modeling, and an advanced-elective class on “How We Learn from Data” that integrates relevant concepts from Statistics, Information-theory, Machine-Learning, Deep-Learning, and Physics-Based model development.