Adaptive Human–Robot Collaborative Assembly: Integrating Planning, Behavior, and Gaze

Abstract

Human–robot collaborative assembly has high potential for manufacturing with high product variety, frequent product changes, and small production volumes. In such settings, assembly systems must be fexible. Rigid and highly specialized automation is often not suitable. Instead, assembly processes must be adapted quickly to new products, variants, and changing task conditions. Human–robot collaboration combines human fexibility and decision-making with robotic precision, repeatability, and physical support. However, this requires effcient planning, reliable interpretation of human behavior, and adaptive robot responses in dynamic environments. This thesis addresses these challenges and presents key approaches for adaptive human–robot collaborative assembly.

First, an Extract–Enrich–Assess–Plan–Review (E2APR) framework is introduced to automate the generation of assembly sequence plans from heterogeneous engineering data, including CAD models, technical drawings, and assembly instructions. The framework supports task allocation between humans and robots, expert-guided refnement, and the generation of multiple collaborative assembly strategies.

Second, this thesis presents an anomaly detection framework for collaborative assembly based on an LSTM autoencoder. Instead of explicitly classifying all possible worker actions, the system learns normal assembly behavior and detects deviations during execution. By combining reconstruction-error-based anomaly detection with object detection and the Assembly Sequence Plan, the framework distinguishes between valid alternative assembly paths and actual assembly errors.

Third, a gaze-based intention recognition approach is proposed to enable more proactive collaboration. Eye gaze is interpreted as a non-verbal signal of worker attention and intention and is categorized into fxation, scanning, and task-switching behaviors. Experimental results demonstrate promising classifcation performance across all three categories, indicating that gaze behavior can provide useful contextual information for anticipating human actions.

Finally, the thesis investigates adaptive robot path planning and communication in shared workspaces. Human arm movements are integrated as dynamic obstacles into the planning scene, and robot state changes are communicated through visual, auditory, and light-based modalities. A pilot user study indicates that communication does not negatively affect execution time, while light-based feedback reduces perceived frustration.

In total, this thesis contributes a coherent approach for linking planning, perception, and interaction in human-robot collaborative assembly. The presented methods provide a foundation for collaborative systems that are not only automatically planned, but also capable of interpreting human behavior and adapting robot actions in a transparent and human-centered manner.