Abstract

Recent years have seen an unprecedented level of technological uptake and engagement by the mainstream. From deepfakes for memes to recommendation systems for commerce, machine learning (ML) has become a regular fixture in society. This ongoing transition from purely academic confines to the general public is not smooth as the public does not have the extensive expertise in data science required to fully exploit the capabilities of ML. As automated machine learning (AutoML) systems continue to progress in both sophistication and performance, it becomes important to understand the 'how' and 'why' of human-computer interaction (HCI) within these frameworks. This is necessary for optimal system design and leveraging advanced data-processing capabilities to support decision-making involving humans. It is also key to identifying the opportunities and risks presented by ever-increasing levels of machine autonomy.
This tutorial provides an expansive perspective on what constitutes an automated/autonomous ML system and how humans interact with such systems. The authors also focus on the following questions: (i) What does HCI currently look like for state-of-the-art AutoML algorithms? (ii) Do the expectations of HCI within AutoML frameworks vary for different types of users and stakeholders? (iii) How can HCI be managed so that AutoML solutions acquire human trust and broad acceptance? (iv) As AutoML systems become more autonomous and capable of learning from complex open-ended environments, will the fundamental nature of HCI evolve? To consider these questions, the authors project existing literature in HCI into the space of AutoML and review topics such as user-interface design, human-bias mitigation, and trust in artificial intelligence (AI). Additionally, to rigorously gauge the future of HCI, they contemplate how AutoML may manifest in effectively open-ended environments. Ultimately, this tutorial serves to identify key research directions aimed at better facilitating the roles and modes of human interactions with both current and future AutoML systems.

Tutorial Outline

Goals: The main goal of this tutorial is to provide the audiences with an overview of how AutoML systems evolve to become steadily more autonomous over time, as well as the roles and modes of human interactions with such AutoML/AutonoML systems.

1. Introduction to AutoML, AutonoML, and related topics (35 mins)
   • Existing challenges of data science (5 mins)
   • ML workflow and the need for AutoML and Autonomous ML systems (10 mins)
   • Fundamental components of AutoML and AutonoML systems (10 mins)
   • AutoML Toolboxes (5 mins)
   • Questions (5 mins)
2. Interacting with AutoML Systems: Current Practices (45 mins)
   • Types of stakeholders (5 mins)
   • Roles and Modes within the machine learning workflow (10 mins)
   • The user interface: Many modalities (5 mins)
   • Improving the outcomes of interactions (10 mins)
   • The user interface: Key requirements (5 mins)
   • Questions (5 mins)
   • Break (5 mins)
3. Interacting with AutoML Systems: Constrained but Fully Automated (35 mins)
   • An introduction to AutonoML systems and their fundemental components (15 mins)
   • A solution for building constrained but fully automated systems: (5 mins)
   • Roles and Modes when human involvement is no longer required (8 mins)
   • What lies beyond the Constraints (2 mins)
   • Questions (5 mins)
4. Interacting with AutoML Systems: Open-ended Environments (35 mins)
   • The challenges of learning in an open world (5 mins)
   • One possible future: AutonoML and reasoning (10 mins)
   • Roles and Modes in relation to autonomous open-world systems (10 mins)
   • Questions (5 mins)
   • Break (5 mins)
5. Critical Discussion and Future Directions (30 mins)
   • Critical discussions (10 mins)
   • A case study of AutonoML in an end-to-end digital twin of bioprocesses for monoclonal antibody manufacturing (10 mins)
   • Potential research directions (5 mins)
   • Questions (5 mins)

Target Audience

This tutorial is aimed at Machine learning researchers and developers who have experience in building AutoML solutions and are interested in exploring the potential of AutoML in improving their system building process. This tutorial is also appropriate for computer science researchers and specific domain experts in understanding how they can interact with AutoML systems and leverage them for their works. Recommended prerequisites are:

• Machine Learning: Understanding of classical and deep machine learning models. Knowledge of AutoML system development will be a plus, but not required.
• Data Science: Understanding of basic concepts in Data Science and Artificial Intelligence. Basic experience with human-machine interactions will be a plus.

Materials

All of the slides and other accompanying materials are available on the GitHub repo.

The primary content of this tutorial is drawn from our recent comprehensive review papers and monographs as follows:

1. T. T. Khuat, D. J. Kedziora, and B. Gabrys, "The roles and modes of human interactions with automated machine learning systems: A critical review and perspectives," Foundations and Trends® in Human-Computer Interaction, vol. 17, no. 3-4, pp. 195-387, 2023.
2. D. J. Kedziora, K. Musial, and B. Gabrys, "Autonoml: Towards an integrated framework for autonomous machine learning," Foundations and Trends® in Machine Learning, vol. 17, no. 4, pp. 590-766, 2024.
3. A. Scriven, D. J. Kedziora, K. Musial, and B. Gabrys, "The technological emergence of automl: A survey of performant software and applications in the context of industry," Foundations and Trends® in Information Systems, vol. 7, no. 1-2, pp. 1-252, 2023.
4. X. Dong, D. J. Kedziora, K. Musial, and B. Gabrys, "Automated deep learning: Neural architecture search is not the end," Foundations and Trends® in Machine Learning, vol. 17, no. 5, pp. 767-920, 2024.
5. T. T. Khuat, R. Bassett, E. Otte, A. Grevis-James, and B. Gabrys, "Applications of machine learning in antibody discovery, process development, manufacturing and formulation: Current trends, challenges, and opportunities," Computers & Chemical Engineering, vol. 182, pp. 108585, 2024

Venue

The tutorial will be located in Room 03, Level 3, RMIT University City Campus Building 80, 445 Swanston Street, Melbourne, VIC 3000, Australia.

Speakers

Prof. Bogdan Gabrys

Professor, UTS

• UTS Profile
• Google Scholar

Bogdan Gabrys is a data scientist and strategist with deep expertise in Data Science, Complex Adaptive Systems, Computational Intelligence, Machine Learning and Predictive Analytics gained from a broad range of application areas and industries. He is currently a Professor of Data Science and a Co-Director of the Complex Adaptive Systems Laboratory at the University of Technology Sydney, Sydney, Australia. Over the last 30 years, he has been working at various universities and research and development departments of commercial institutions. His research activities have concentrated on the areas of data science, complex adaptive systems, computational intelligence, machine learning, predictive analytics, and their diverse applications. He has published over 220 research papers, chaired conferences, workshops, and special sessions, and been on program committees of a large number of international conferences with the data science, computational intelligence, machine learning, and data mining themes. He is frequently invited to give keynote and plenary talks at international conferences and lectures at internationally leading research centres and commercial research labs.

Dr. Thanh Tung Khuat

Research Fellow, UTS

• UTS Profile
• Google Scholar

Thanh Tung Khuat is now working as a Postdoctoral Research Fellow at the Complex Adaptive Systems Lab at the University of Technology Sydney (UTS), focusing on building modern explainable, transparent, and robust machine learning algorithms. He obtained a Ph.D. in Data Science in 2021 from UTS. He has previously been a Senior software engineer at the FPT Software Danang, Vietnam (Jan-2018 to Jun-2018), has been a project officer at the School of Computer Science and Engineering, Nanyang Technological University, Singapore (Aug-2016 to Mar-2017). Dr. Khuat is also an active reviewer for many prestigious journals and international conferences such as IEEE TNNLS, IEEE TIE, IEEE TSMC: Systems, Soft Computing, AI Review, ASE, IEEE Access, and ICONIP. He is currently a member of the IEEE, IEEE Computational Intelligence Society, and IEEE Computer Society. Khuat's research interests include machine learning, interpretable machine learning, fuzzy systems, knowledge discovery, evolutionary computation, intelligent optimization techniques, and applications in biopharmaceuticals, medicine, finance, and agriculture. He has authored and co-authored over 40 peer-reviewed publications in the areas of machine learning and computational intelligence.

Acknowledgment

This tutorial is partly supported under the Australian Research Council's Industrial Transformation Research Program (ITRP) funding scheme (project number IH210100051). However, the vast majority of the materials, including four monographs, had been generated and published prior to and independent from the Hub.

The ARC Digital Bioprocess Development Hub is a collaboration between The University of Melbourne, University of Technology Sydney, RMIT University, CSL Innovation Pty Ltd, Cytiva (Global Life Science Solutions Australia Pty Ltd) and Patheon Biologics Australia Pty Ltd.

For more information on the Centre, please visit the website at https://digitalbioprocesshub.org.au.