- Conference Article
- 10.1109/automation66991.2025.11389982
Bridging Physical and Virtual Robotics: Real-Time Digital Twin with Vision-Based Identification
- Nov 28, 2025
- Suphasit Raoruja + 5 more +5
This paper presents a real-time digital twin framework for robotic manipulation that integrates ROS2 with Unity for synchronized execution and monitoring. The system combines a HIWIN RA610-1869 robotic arm, a CCV camera, and a YOLO-based perception pipeline to classify and manipulate ten classes of industrial components placed randomly in a workspace. Upon receiving a command, the robot identifies, localizes, and performs pick-and-place tasks, while Unity mirrors all motions in real time, enhancing transparency and operator validation. Experimental trials demonstrate high accuracy under controlled conditions and high reliability in manipulation. A comparative study of two YOLO models trained on different datasets further highlights the challenge of domain adaptation: the real-image model achieved stable validation (precision <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\approx 0.90$</tex>, recall <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\approx 0.92$</tex>, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{mAP50} \approx 0.90$</tex>), whereas the synthetic-only model exhibited strong in-domain results but <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\approx 90 \%$</tex> mislabeling in real-world scenes. These findings confirm system robustness while underscoring the need for domain adaptation in digital twinenabled robotics.
Read more