HAS Discussion Group on Info Theory and Machine Learning

Join us for our first weekly discussion group to explore the relevance of Info Theory and Machine Learning to our discipline and science in general.

We will meet in person and also try to accommodate those who wish to attend virtually via zoom.

Preparation for First Meeting

• Do some investigation of ChatGPT and see what you can find out about how it works
• Think up a list of topics you feel would be valuable to discuss
• (Update 4/3) A short and very high-level overview that could be relevant to our wednesday group discussions. 
  
  https://youtu.be/aQguO9IeQWE. In addition, a key ML technology that is relevant is the so-called "transformers" and "attention" mechanism. So, you may want to look into that as well. 

Agenda for April 5

• Brief discussion of format, content, and potential topics*
• As an initial topic to jump start the conversation:
  • What does ChatGPT actually do--how it works, strengths and weaknesses
  • How can ChatGPT be used to support HAS investigations

*Potential List of Topics for Future Discussion

• Fundamental Concepts in Machine Learning (Attention, Features, Directed Graphs, Representational Architectures, Convolution, Types of Memory, Auto-encoding, Regularization, Recognition versus Prediction, Training, Generalization …)
• Info Theory and Measurement Theory, Measurement as Encoding
• Representational Architectures / universal principles / composition / regularization
• What are the kinds of Info, and how are they represented/stored in data & models?
• What is “memory” as an encoding / source of “useful” information
• Pruning / lottery ticket hypothesis
• How can/do Theory and Data interact?
• How to ensure that ML is consistent with Physical understanding
• Please make suggestions!