- DataDrivenAEC
- Posts
- Automating BIM with AI Agents: An Interview with TUM Researchers Changyu Du and Zihan Deng
Automating BIM with AI Agents: An Interview with TUM Researchers Changyu Du and Zihan Deng
Week 31 – Aug 3, 2025
The world of AI is moving at lightning speed, and its impact on the AEC industry is a topic of intense discussion. While many applications focus on analyzing existing data, a new frontier is emerging: AI agents that can actively perform tasks within complex software.
I recently sat down with Changyu Du, a PhD student at the Technical University of Munich (TUM), and Zihan Deng, a Master's student soon to begin his PhD at TUM. Their latest research, presented at ICML 2025, explores using computer use agents for automated BIM authoring. We discussed the motivation behind their work, the current hurdles, whether AI will replace architects (spoiler: they don't think so), and what's next for this groundbreaking research.
The Elevator Pitch: Why Use AI Agents for BIM?
DataDrivenAEC: What was the initial motivation for this research? What's the elevator pitch?
Changyu: Our main goal was to explore applying computer use agents to BIM authoring software. We've seen them work impressively in gaming and desktop automation, but nobody had really applied them to our domain.
Zihan: As BIM users ourselves, we know how complex these programs can be—they come with thousands of pages of documentation. We wanted to find a more general, vision-based way to simplify the process. Instead of relying on APIs, like in previous Text-to-BIM work, we're exploring how an AI agent can use the graphical user interface (GUI) just like a human would. The ultimate vision is to make everyone a professional user of BIM software.
*Caption: An overview of the BIMGENT framework, illustrating the agent's workflow from design to execution. Source: BIMGENT Project Website
A Long Way to Go: Current Challenges and Low Success Rates
DataDrivenAEC: Your paper mentions that the results are still preliminary. How do you measure success, and how far are we from seeing this in practice?
Changyu: We're still in a very early phase of research. To be transparent, our framework achieved a 32% end-to-end success rate on our mini-building benchmark, which is quite low. The success rate for individual tasks, like creating a single building element, is even lower.
Beyond performance, there are significant hurdles like safety and privacy. You're essentially letting an agent take control of your computer and software, which is a risky proposition that not every designer would be comfortable with.
Zihan: Right now, our tests are limited to simple, one-floor buildings. We haven't yet tackled real-world complexities like building regulations or compliance checks. So, while the potential is huge, we have a long way to go before this is ready for a production environment.
The Chicken-and-Egg Problem: Models vs. Data
DataDrivenAEC: What's the bigger bottleneck right now: the AI models themselves or the lack of AEC-specific training data?
Changyu: It's really both. We tested off-the-shelf models from major AI labs, but they don't work well out of the box. They're designed for general web Browse or desktop tasks and need significant domain adaptation to understand the nuances of BIM.
Zihan: Data is the other massive challenge. Unlike booking a hotel, where a successful click is a clear signal, design lacks a "significant signal" of correctness. There's no simple ground truth to tell the model, "Yes, that was the right design choice." Is a building "correct"? Does it meet the design intent? This ambiguity makes it incredibly difficult to create the high-quality, labeled data needed for training a robust model. Our next big step is to figure out how to collect and label this kind of data effectively.
Advice for Firms: The Cost of Innovation
DataDrivenAEC: Given these challenges, what's your advice for AEC firms wanting to adopt AI? What's the first step?
Changyu: It's an interesting dynamic. We, as researchers, have the methodologies but lack massive, real-world datasets. Companies have troves of data hidden in their daily workflows. The key is for firms to start asking if AI can genuinely bring more revenue or reduce costs. If the answer is yes, then it's worth investing in data scientists who can mine that data for useful applications.
Zihan: There's a fundamental conflict: AI development is expensive—it requires money, GPUs, and talent. Conversely, AEC firms are primarily focused on reducing project costs. This is why the biggest AI advancements happen at large tech companies, not yet in our industry.
We believe the best path forward is collaboration between industry and academia. Firms provide the data and domain problems, and we provide the research expertise. It’s a win-win.
Will AI Replace Architects?
DataDrivenAEC: This is the question on everyone's mind. With models like GPT-4o generating impressive floor plans, do you see AI taking over the architect's job?
Zihan: I admit, when I saw the quality of floor plans GPT-4o could generate, it was a "wow" moment. It made me think about the future of design jobs. From that perspective, it seemed possible.
Changyu: I see it differently. I don't think we will replace architects. Our goal is to build systems that support architects in making better designs. Architectural design is a super-complex cognitive activity. A floor plan might look good to an engineer like me, but a senior architect would critique it from a dozen different angles we haven't even considered. We have to respect the deep complexity of the profession.
Zihan: Exactly. We need to treat AI as a tool. A tool helps us reduce time, not reduce people.
The Road Ahead: Next Steps for the Research
DataDrivenAEC: What's next for your project?
Zihan: I'll be starting my PhD in October to continue this work. Our roadmap is focused on three key areas:
Establish a Benchmark: First, we need to create a true benchmark for evaluating BIM authoring tasks. Without a standardized test, it's hard to measure progress.
Train Our Own Models: We plan to move beyond existing models and train our own, possibly using reinforcement learning, to achieve higher accuracy and speed.
Tackle Real-World Complexity: We want to integrate building regulations and complex design requirements into the framework. The goal is to move from modeling simple boxes to complex structures like hospitals or schools that are beautiful, functional, and compliant.
About the Researchers
Changyu Du is a PhD Student at the Technical University of Munich, funded by the Nemetschek Group. His research focuses on predictive modeling and generative AI for BIM. He has a background in Civil Engineering and previously worked at GOLDBECK and FARO Technologies.
Zihan Deng is completing his Master of Science in Information Technologies for the Built Environment at TUM and will begin his PhD this fall. His background is in Building Environment and Energy Application, and he has experience from internships at Tesla and Drees & Sommer.
You can learn more about their work here:
Here are the most relevant tech news highlights from the past week:
AEC Firms Invest in Technology for a Competitive Edge AEC firms are enhancing their data management, analytics, and visualization capabilities to boost efficiency and competitiveness.
Acelab is Hiring an AEC Technology Manager Acelab streamlines material selection and management for architects using AI-powered recommendations.
Meta Data Center to Use AI-Optimized Concrete Mix Meta's newest data center will utilize an AI-optimized concrete mix to enhance strength and efficiency.
Nemetschek Group as a SaaS Powerhouse in AEC Tech Nemetschek leverages SaaS and AI technologies to substantially grow its revenue in the AEC tech sector.
New AI Tool Aims to Modernize Permitting and Slash Timelines A new AI-driven tool is set to significantly expedite permitting processes in architecture, engineering, and construction.
Leveraging Human-Centered Design for Social Connection Among Older Adults Human-centered design (HCD) is used to innovate solutions that foster social connections for older adults in housing.
Fortifying Blueprints: Ransomware Defense Strategies for AEC As AEC projects incorporate IoT and AI, there's an increased focus on defending against ransomware through strategic cybersecurity.
PropTech Market Trends The PropTech market is set to grow significantly, driven by AI, machine learning, and blockchain innovations.
Visionstate Corp. Advances Technology and ESG Investments Visionstate enhances its technology and environmental, social, and governance (ESG) initiatives, aimed at better facility management.
Why AEC Leaders Should Treat AI as Imperative AI offers predictive analytics that can mitigate project risks; AEC leaders are encouraged to lead with AI advancements.
Shaping Ethical AI-Powered Digital Tools for Youth Human-centered design aids in creating ethical guidelines for AI-driven mental health tools in the youth sector.
Flexible Workplaces and Sensory-Conscious Design Facilities now focus on thoughtful, sensory-conscious designs that foster generational belonging without high costs.
Until next time,
Shicong Cao, DataDrivenAEC
Reply