AI for Software Engineering and GUI Testing

Project Overview

Graphical user interfaces (GUIs) are the primary gateway through which users access software functionality and business logic. As software systems become increasingly complex across platforms, devices, runtime environments, and usage scenarios, conventional GUI testing techniques struggle to understand interface behavior and the business semantics behind it.

This project will investigate a knowledge-guided quality assurance framework for software GUIs. It will integrate multimodal information from software requirements, design documents, user manuals, GUI screenshots, source code, execution logs, and historical test reports. By combining large language models, multimodal learning, knowledge graphs, and software testing techniques, the project aims to improve the completeness, transferability, and defect-detection capability of GUI testing, with a strong emphasis on generalization across platforms and applications.

Research Directions

The doctoral researcher will work with the project lead on the following interconnected topics:

1. Extraction and organization of software knowledge for testing scenarios;
2. Cross-platform and cross-application GUI test understanding and generation;
3. Scenario-aware generation of GUI test oracles;
4. Scenario-specific applications of GUI testing.

Key Responsibilities

• Design and implement knowledge-guided GUI testing methods, algorithms, and prototype tools;
• Build and maintain multimodal software datasets and experimental infrastructure;
• Conduct reproducible empirical studies and systematically compare the proposed approaches with existing methods;
• Write and publish high-quality academic papers;
• Participate in international conferences and collaborative research activities;
• Open-source representative datasets, models, and research tools;
• Fulfil TUM's doctoral training and degree requirements.

About Us

The Chair of Software Engineering & AI is part of TUM's School of Computation, Information and Technology and is led by Professor Chunyang Chen. The group conducts interdisciplinary research at the intersection of software engineering, deep learning, and human-computer interaction. Its research spans AI-assisted software development, large language models, automated code and interface generation, GUI and functional testing, bug reproduction, software usability and accessibility, and the reliability, security, and privacy of AI-enabled software.

Professor Chen received his Ph.D. from Nanyang Technological University, previously worked at Monash University, and joined TUM in 2024. His research has received distinctions including the ACM SIGSOFT Early Career Researcher Award and the ACM SIGSOFT Distinguished Paper Award. The group maintains extensive collaborations with universities and industry worldwide, providing doctoral researchers with an environment that combines theoretical depth, engineering practice, and international exchange.

Dr. Shengcheng Yu, the principal investigator, is currently a postdoctoral researcher at TUM. He received his bachelor's and doctoral degrees from the Software Institute at Nanjing University in 2020 and 2024, respectively. His research interests include intelligent software testing, automated GUI testing, crowdsourced testing, GUI automation, human-agent collaboration, and LLM personification. He leads a DFG Individual Research Grant and an ACM SIGSOFT-funded Winter School project, among other initiatives. His honours include the Nanjing University Outstanding Ph.D. Dissertation Award, the Nanjing University–HPI Research Scholarship, and the Nanjing University President's Special Scholarship for Ph.D. Students. He serves on the program committees of major conferences in software engineering, human-computer interaction, and artificial intelligence, including ICSE, FSE, ASE, ISSTA, IJCAI, ICSME, MSR, and IUI, and reviews for journals including ACM TOSEM, IEEE TSE, IEEE TDSC, JSS, EMSE, IEEE TRel, and IEEE TIV. He has published more than ten full-length papers as first or corresponding author in leading software engineering journals and conferences, including CSUR, TOSEM, TSE, ICSE, and FSE. He has also developed a suite of tools for intelligent GUI testing and published a series of industry-track and tool papers; these technologies have seen initial adoption at companies including Infineon Technologies.

About the Technical University of Munich

The Technical University of Munich is a German University of Excellence and one of Europe's leading research universities. TUM is ranked 25th worldwide, first in Germany, and first in the European Union in the QS World University Rankings 2027. It is ranked 27th worldwide and likewise first among universities in Germany and the European Union in the Times Higher Education World University Rankings 2026.

TUM is committed to high-level research, education, and technological innovation across computer science, engineering, natural sciences, life sciences, medicine, management, and the social sciences. It fosters interdisciplinary collaboration and the transfer of research into practice. Its international community and close links with universities, research institutions, industry, and the high-technology start-up ecosystem provide an open, diverse, and highly international environment for early-career researchers.

Qualifications

Applicants should have:

• A master degree in software engineering, artificial intelligence, human-computer interaction or a related discipline, obtained before the position begins;
• A solid foundation in software engineering and artificial intelligence;
• Strong programming and system implementation skills;
• A strong interest in research and the ability to independently analyse problems and conduct experiments;
• Good English reading, writing, and academic communication skills;
• A rigorous, proactive approach and strong teamwork skills.

Experience in one or more of the following areas is desirable:

• Software testing, program analysis, GUI automation, or mobile and web application testing;
• Large language models or multimodal large language models;
• Knowledge graphs, ontology modelling, or graph neural networks;
• First-author or student-first-author publications at venues such as TSE, TOSEM, ICSE, FSE, ASE, ISSTA, TPAMI, IJCV, ICLR, CHI, or UIST; participation in research projects; or experience developing open-source software.

Applicants are not expected to have prior experience in every area listed above. We particularly value strong fundamentals, research potential, and the ability to learn new techniques.

Research Environment

The project will use TUM's computing and research infrastructure, including GPU servers, high-performance workstations, high-performance computing resources at the Leibniz Supercomputing Centre (LRZ), and a range of resources for large language model experiments. The doctoral researcher will have opportunities to participate in international collaborations, publish in leading conferences and journals in software engineering, human-computer interaction, and artificial intelligence, and attend relevant academic conferences.