Cooperating Robots

Study of the kinematic and dynamic planning of robotic systems that collaborate with each other and can exchange tools and components without intervention. Robotic systems with dual or multiple arms (dual arm / multi-arm robots). These are systems with a high degree of freedom (10-15) that exhibit significant complexity in terms of kinematic and dynamic behavior.

Industrial applications with cooperating robots for the flexible assembly

REFERENCES/VIDEOS:

Dual Arm Robots

  • Use human-like dual arm robots in cooperation with people in the factory to enable flexible manufacturing.  
  • Research in motion planning, task planning, human-robot interaction, human safety, intuitive and interactive robot programming. 

REFERENCES/VIDEOS:

  • P.Tsarouchi, S. Makris, G. Michalos, M.Stefos, K. Fourtakas, K. Kaltsoukalas, D. Kontovrakis, G. Chryssolouris, “Robotized assembly process using Dual arm robot”, (CATS 2014), 13-14 November, Dresden, Germany, Volume 23, pp. 47-52 (2014)
  • N. Kousi, G. Michalos, S. Aivaliots, S. Makris,”An outlook on future assembly systems introducing robotic mobile dual arm workers”, (CMS2018), Procedia CIRP, Volume 72, pg.33-38, 16-18 May, Stockholm, Sweden (2018)
  • X-act Project Demo

 

High payload collaborative robot in Human-Robot Collaborative assemblies

  • Highly efficient during heavy payload collaborative applications. Tested in an assembly case study from the automotive industry for the manipulation of vehicle’s motor and gearbox parts (Up to 130 Kg). 
  • High-speed mode is dynamically managed, when collaboration mode is not required. 
  • Collision avoidance solution for both robot and tools.

Assembly of multiple products using low payload collaborative robots

  • Easy integration with other hardware components such as: 
    • Automated tool change solutions. 
    • Gripping solutions thanks to available Digital I/O signals of robot’s controller. 
    • Vision sensors. 
  • Definition of safety areas where robot speed is reduced. 
  • Easy programming of robot through AR. 

 

Mobile robots

Autonomous robotic vehicles used for transporting products/components and implementing production processes. These vehicles are mainly electric and equipped with a multitude of sensors for autonomous navigation and mapping of the environment (lidars, laser scanners, RGBd cameras, etc.). They can be equipped with robotic arms (mobile manipulators) or other systems for performing various tasks. 

REFERENCES/VIDEOS:

Intelligent mechatronics

Dexterous mechanisms and handling devices for components: the area includes design, detailed modeling, simulation (two-dimensional and three-dimensional representation), construction, and control of reconfigurable devices capable of grasping, manipulating, assembling, or processing complex components. The range of mechanisms varies both by design principle (type of actuators, degrees of freedom, type of interaction with the component) and by the type of product (continuous/discrete, flexible/compact, etc.). Design and synthesis of multi-degree-of-freedom mechanisms using electric, pneumatic, or hydraulic actuators and software development to optimize their control through digital controllers. 

Pneumatic gripper for in-hand manipulation and feeding of lightweight complex parts

 

REFERENCE: G. Michalos, K. Dimoulas, K. Mparis, P. Karagiannis, S. Makris,”A novel pneumatic gripper for in-hand manipulation and feeding of lightweight complex parts”, International Journal of Advanced Manufacturing Technology, Volume 97, Issue 9–12, pp 3735–3750, (2018)

Reconfigurable Gripper for Dexterous Manipulation in Flexible Assembly

REFERENCES: J. Spiliotopoulos, G. Michalos, S. Makris,”A Reconfigurable Gripper for Dexterous Manipulation in Flexible Assembly”, MDPI-Open Access Journal Inventions, Volume 3, Issue 1, (2018)

Robot gripper with high speed, in-hand object manipulation capabilities

Reference: Andronas, D., Xythalis, S., Karagiannis, P., Michalos, G., Makris, S., “Robot gripper with high speed, in-hand object manipulation capabilities”, (CATS 2020), Procedia CIRP, Volume 97, pg. 482-486 , 29 Sept – 1 Oct, Athens, Greece (2020)

 

Cognitive Mechatronic Devices for Reconfigurable Production of Complex Parts

Reference: Karagiannis, P., Michalos, G., Andronas, D., Matthaiakis, A.-S., Giannoulis, C., & Makris, S. (2021). Cognitive Mechatronic Devices for Reconfigurable Production of Complex Parts. Applied Sciences, 11(11), 5034 (link). 

 

Design, and implementation of flexible production stations

integrated stations used for the evaluation and verification of the correct operation of all mechanical systems and peripheral equipment required for assembly, processing, inspection, etc. Examples include systems developed for the automotive industry (dashboards, pumps, suspension systems), aerospace (manufacture and inspection of wings, engine blades), production of electrical goods and household appliances (refrigerators, washing machines, etc.), and consumer products (e.g., electric shavers), which have also been used as pilot applications for European research projects. 

REFERENCES/VIDEOS:

Modeling of fabrics and textile materials

Handling deformable materials, and especially fabrics, remains a significant challenge within the industrial landscape. Current methods, whether manual or automated, often fall short due to the complex behaviors of these materials. An advanced mass-spring model has been developed by LMS to simulate fabric behavior in real-time. The solution’s adaptability ensures precise representation of intricate shapes across various fabric and textile types. [1]

Design and optimization of digital twins and cyber-physical systems

In the world of cyber-physical systems and the industrial metaverse, it is crucial to accurately depict the real-world behavior of objects. Through the implementation of advanced optimization algorithms and artificial intelligence (AI), LMS has devised methodologies that seamlessly link the digital representation of an object to its tangible, real-world behavior. [3]

Design of mechatronics and multifunctional robot end-effectors for flexible materials

In today’s manufacturing landscape, automating tasks that demand precision and gentle handling—such as manipulating flexible materials—remains a significant challenge. LMS leads the way in crafting innovative multifunctional end-effectors to tackle industry challenges. Enhancing robot dexterity drives progress in efficiency, product quality, ergonomics, and worker well-being. [4]

Model-based manipulation of flexible materials

A crucial aspect of enhancing robots with cognitive abilities involves enabling them to interact with objects in the industrial environment and respond effectively. LMS leads the way in collaborative manipulation of deformable objects, achieving this through the development of model-based planning frameworks for robot agents, integrating advanced algorithms and strategies. [2]

Modelling and assembly of cable structures

Manipulating non-rigid components, especially cable structures like cable harnesses, poses several challenges in today’s manufacturing environment. To address these challenges, LMS develops sophisticated models using intricate mathematical equations to make accurate predictions about the deformation and twisting behavior of cables. Additionally, advanced simulation strategies are employed to enhance the capabilities of robotic manipulation in this context [6-8]

REFERENCES:

  1. Model-Based Robot Control for Human-Robot Flexible Material Co-Manipulation
  2. On deformable object handling: Model-based motion planning for human-robot co-manipulation
  3. Cyber-physical systems in non-rigid assemblies: A methodology for the calibration of deformable object reconstruction models
  4. On deformable object handling: multi-tool end-effector for robotized manipulation and layup of fabrics and composites
  5. On modelling and handling of flexible materials: A review on Digital Twins and planning systems
  6. Twist modelling for assembly of wiring harnesses (2021)
    • model-based prediction of shape
    • modelling wiring harness mechanical behavior
    • The latest of 1-3 basically describing the same problem
  7. Simulating robotic manipulation of cabling and interaction with surroundings (2018)
  8. Prediction of cabling shape during robotic manipulation (2015)

High payload collaborative applications – conditional space sharing

  • Cognition:
    • Dynamic rescheduling of tasks and assignment to pool of resources and operators
  • Perception:
    • Depth camera for operator gesture recognition
  • Safety:
    • Speed & Separation monitoring
    • Overhead camera (SafetyEye) supervising different areas
    • Emergency buttons
  • Interaction:
    • Gesture recognition to denote start/end of human task
    • Physical buttons
    • Force/torque sensor for manual guidance

Reference: G. Michalos, S. Makris, J. Spiliotopoulos, I. Misios, P. Tsarouchi, G. Chryssolouris, “ROBO-PARTNER: Seamless Human-Robot Cooperation for Intelligent, Flexible and Safe Operations in the Assembly Factories of the Future”, (CATS 2014), 13-14 November, Dresden, Germany, Volume 23, pp. 71-76 (2014)

High payload collaborative applications – full space and part sharing

  • Cognition:
    • Production monitoring and real-time rescheduling and assignment of tasks to robot/operator – reduction of physical interference
  • Perception:
    • Operator position detection
    • Prediction of human movement and adaptation of robot trajectory
    • Assembly validation – detection of assembly errors and provision of visual clues
  • Safety:
    • Elimination of physical fences – space and part sharing
    • Dynamic adjustment of robot speed – switch between industrial and collaborative speed
    • Power & Force Limiting – sensing of collisions with the robot body and with the manipulated part
    • Emergency buttons
    • ISO15066 Certified installation
  • Interaction:
    • Voice commands to denote start/end of human task
    • F/T based manual guidance – F/T mounted on the robot flange
    • Augmented reality provision of instructions and information using AR glasses
  • Industrial installation and validation:
    • Demonstrated reduction of ~18% in the cycle time
    • Improvement of ergonomic score of the whole procedure improved,
    • Reduction of maximum weight handled by the operators down to 2kg compared to up to 42kg (handed by two operators)
    • Potential employment of operators with special restrictions.

References: Solar thermal collectors production using high payload collaborative robots

Low payload collaborative applications – full space and part sharing

  • Cognition:
    • Production monitoring and real-time re-scheduling / assignment of tasks to ro-bot/operator – reduction of physical interference
  • Perception:
    • Operator position detection
    • Prediction of human movement and adaptation of robot trajectory
    • Assembly validation – detection of assembly errors and provision of visual clues
  • Safety:
    • Elimination of physical fences – space and part sharing
    • Dynamic adjustment of robot speed – switch between industrial and collaborative speed
    • Power & Force Limiting
    • Emergency buttons
    • ISO15066 Certified installation
  • Interaction:
    • Voice commands to de-note start/end of human task
    • Motor current deviation for manual guidance
    • Smartwatch application
    • Augmented reality provision of instructions / in-formation using 2D projectors
  • Industrial installation and validation:
    • Demonstrated reduction of ~40% in the cycle time
    • Reduction of assembly errors
    • Reduction of time to introduce a new product variant in the assembly station by ~40%
    • ROI of less than 5 years

Reference: Industrial modules assembly using low payload collaborative robot

Collision detection for collaborative assembly operations on high-payload robots

  • collision detection on high-payload manipulators using a F/T sensor on the robot flange as well as the robot motor currents.
  • Detection of collisions with the parts manipulated by the robot
  • Redundant sensors, data flow and processing
  • Enabling non-conditional space sharing
  • Certified by ISO15066

References:  K. Katsampiris-Salgado, K. Haninger, C. Gkrizis, N. Dimitropoulos, J. Krüger, G. Michalos, S. Makris, “Collision detection for collaborative assembly operations on high-payload robots”, Robotics and Computer-Integrated Manufacturing, Volume 87, (2023)

AR-based operator support

Augmented Reality (AR) tools have been implemented in aid of operators being in a hybrid, human and robot collaborative industrial environment. The system aims to provide production and process-related information and enhance the operators’ immersion in the safety mechanisms, dictated by the collaborative workspace. The developed system has been integrated with a service-based station controller, which is responsible for orchestrating the flow of information to the operator, according to the task execution status. The tool has been applied to a case study from the automotive sector, resulting in an enhanced operator’s integration with the assembly process.

References:

  • Michalos, G., Karagiannis, P., Makris, S., Tokçalar, Ö., Chryssolouris, G., 2015, Augmented reality (AR) applications for supporting human-robot interactive cooperation, 48th CIRP Conference on Manufacturing Systems, Naples Italy, (2015) PROCEDIA.
  • Makris, S., Karagiannis, P., Koukas, S., Matthaiakis, A.-S., 2016. Augmented reality system for operator support in human–robot collaborative assembly. CIRP Annals 65, 61–64. https://doi.org/10.1016/j.cirp.2016.04.038
  • Michalos, G., Kousi, N., Karagiannis, P., Gkournelos, C., Dimoulas, K., Koukas, S., Mparis, K., Papavasileiou, A., Makris, S., 2018. Seamless human robot collaborative assembly – An automotive case study. Mechatronics 55, 194–211. https://doi.org/10.1016/j.mechatronics.2018.08.006
  • Papanastasiou, S., Kousi, N., Karagiannis, P., Gkournelos, C., Papavasileiou, A., Dimoulas, K., Baris, K., Koukas, S., Michalos, G., Makris, S., 2019. Towards seamless human robot collaboration: integrating multimodal interaction. Int J Adv Manuf Technol 105, 3881–3897. https://doi.org/10.1007/s00170-019-03790-3
  • Karagiannis, P., Togias, T., Michalos, G., Makris, S., 2021. Operators Training Using Simulation And VR Technology. Procedia CIRP 96, 290–294. https://doi.org/10.1016/j.procir.2021.01.089
  • Indirect Human-Robot interaction using AR and smart devices

Augmented Reality based framework assisting operators in human-robot collaborative assembly operations

The efficient interaction and collaboration of human and robot resources in such production lines enables the combination of both types of resources for the balanced distribution of the manufacturing tasks.  LMS develops Augmented Reality applications for operators’ support including different features such as:

  • Assembly guidance provision through textual information or holograms
  • Hologram-based robot controlling
  • Interaction with gripping tools
  • Security breach notifications
  • Quality inspection results visualization
  • Resilience of the production system in case of unexpected events
  • Safety awareness

References:

Aivaliotis, S., Papavasileiou, A., Konstantinou , C., Anastasiou, T., Gkournelos, C., Koukas, S., Makris, S., An interactive Augmented Reality based framework assisting operators in human-robot collaborative assembly operations, 17th CIRP Conference on Intelligent Computation in Manufacturing Engineering

Use of wearable devices in manufacturing

Wearable devices support operators in shared industrial workplaces where humans and robots coexist. The tool was developed as a software application for wearable devices, such as smartwatches, and provides functionalities for direct interaction with the robot. Interfaces to audio commands, manual guidance applications and AR visualization systems have been implemented. LMS focuses on the development of the smartwatch application and its integration with the network of resources under the ROS framework. The tool has been tested on two pilot cases, one in the automotive and one in the white goods production sectors. The results indicate that the approach can significantly enhance the operators’ integration in the hybrid assembly system.

References:

  • Gkournelos, C., Karagiannis, P., Kousi, N., Michalos, G., Koukas, S., Makris, S., 2018. Application of Wearable Devices for Supporting Operators in Human-Robot Cooperative Assembly Tasks. Procedia CIRP 76, 177–182. https://doi.org/10.1016/j.procir.2018.01.019
  • Michalos, G., Kousi, N., Karagiannis, P., Gkournelos, C., Dimoulas, K., Koukas, S., Mparis, K., Papavasileiou, A., Makris, S., 2018. Seamless human robot collaborative assembly – An automotive case study. Mechatronics 55, 194–211. https://doi.org/10.1016/j.mechatronics.2018.08.006
  • Papanastasiou, S., Kousi, N., Karagiannis, P., Gkournelos, C., Papavasileiou, A., Dimoulas, K., Baris, K., Koukas, S., Michalos, G., Makris, S., 2019. Towards seamless human robot collaboration: integrating multimodal interaction. Int J Adv Manuf Technol 105, 3881–3897. https://doi.org/10.1007/s00170-019-03790-3

Generative AI and Human-Robot Interaction

  • Development of AI algorithms for real-time interpretation and adaptation to human actions.
  • Enhancement of collaborative efficiency between humans and robots in manufacturing.
  • Usage of Large Language Models (LLMs) for enabling natural Human Robot Collaboration

https://www.youtube.com/watch?v=WyHZkatFOfA&t=175s

Sensing, Vision-Based Robot Guidance

  • Advanced vision-based robot guidance using deep learning for superior accuracy.
  • Application in quality inspection and complex manufacturing environments.

Reference on safety:

  • Karagiannis, P., Kousi, N., Michalos, G., Dimoulas, K., Mparis, K., Dimosthenopoulos, D., Tokçalar, Ö., Guasch, T., Gerio, G.P., Makris, S., 2022. Adaptive speed and separation monitoring based on switching of safety zones for effective human robot collaboration. Robotics and Computer-Integrated Manufacturing 77, 102361. https://doi.org/10.1016/j.rcim.2022.102361
  • https://www.youtube.com/watch?v=G7i0I56jzRM

AI-based spatial object detection for automotive parts assembly

LMS develops detection algorithms to achieve highly accurate detection of parts. A custom solution for complex parts’ detection based on holes is presented below. A high payload robot is able to detect the central axle hole of a vehicle motor part in order to later approach it with a gearbox and align the gearbox’s axle with the motor.

References:

Karaklas, N., Papavasileiou, A., Makris, S., Empowering Precision-Guided Automotive Assembly Operations: A Flexible Robot Vision Framework, 10th CIRP Conference on Assembly Technology and Systems.

Motion Planning

Task Planning

  • Innovative AI-driven methodologies for optimal human-robot task distribution.
  • Focused on orchestrating complex manufacturing processes with precision and adaptability.

Artificial Intelligence (AI) based module for task planning process in HRC assembly processes

The efficient allocation of tasks to human and robotic resources in Human-Robot Collaborative production lines is very critical in terms of cycle time and resource idle time reduction but also strenuous tasks’ execution by robotic resources to increase human ergonomy. LMS develops search based decision-making solutions for tasks allocation in HRC workcells. These solutions are based on Artificial Intelligence algorithm based on custom heuristic functions for the quick generation of the alternative task plans. Task plans’ evaluation is a multi-criteria decision making process based on user defined criteria and a simulation engine for task plans’ simulated execution and validation. The used is able to interact with the task planner module through a custom Graphical User Interface (GUI) developed by LMS based on the requirements from the corresponding use case.

References:

Papavasileiou, A., Aivaliotis, S., Glykos, C., Koukas, S., Makris, S., An AI-based decision-making framework with task planning and dynamic reconfiguration capabilities, industrial session in European Robotics Forum (ERF) 2024

https://www.youtube.com/watch?v=neD449clmLY

Selected references:

Human Motion & Intention Prediction

  • Robot teleoperation through human posture detection techniques
  1. Detection of human posture using various vision sensors.  
  2. AI based algorithms for human posture classification. 
  3. Easy to-be trained classification algorithm able to support several human postures. 
  4. Integration with different robotic manipulator through Robotic Operating System (ROS) interfaces. 

Human-centric Ergonomics Evaluation

Digital Human Modelling & Simulation

Power and Force Limiting on industrial robots

A method for limiting the forces applied by an industrial robotic manipulator by detecting the collision with its surroundings without the use of external sensors has been developed by LMS. Using time-invariant dynamic models and supervised feedforward input-delay neural networks on signal processing, the required current signals for a certain arm trajectory are predicted. The predicted nominal current signals are compared with the actual current signals of the motors that are continuously measured by the robot controller. The discussed models can be extended for application in the implementation of human robot collaborative applications.

References:

  • P. Aivaliotis, S. Aivaliotis, C. Gkournelos, K. Kokkalis, G. Michalos, S. Makris,”Power and force limiting on industrial robots for human-robot collaboration”, Robotics and Computer-Integrated Manufacturing, Volume 59, pg. 346-360, (2019)
  • P.Aivaliotis, E. Papalitsa, G. Michalos, S. Makris, “Identification of dynamic robot’s parameters using physics-based simulation models for improving accuracy”,  (CIRPe 2020), Procedia CIRP, Volume 96, pg. 254-259 , 14-16 Oct, Leuven, Belgium (2020)
  • Z. Arkouli, P. Aivaliotis, S. Makris, “Towards accurate robot modelling of flexible robotic manipulators”, (CATS 2020), Procedia CIRP, Volume 97, pg. 497-501 , 28 Sept – 1 Oct, Athens, Greece (2020)
  • S. Makris, P. Aivaliotis, “Framework for accurate simulation and model-based control of hybrid manufacturing processes”, (CATS 2020), Procedia CIRP, Volume 97, pg. 470-475 , 28 Sept – 1 Oct, Athens, Greece (2020)
  • K. Kokkalis, G. Michalos, P. Aivaliotis, S. Makris, “An approach for implementing power and force limiting in sensorless industrial robots”, (CATS 2018), Procedia CIRP Volume 76, pg. 138-143, 10-12 May, Tianjin, China,(2018)

Predictive Maintenance in Manufacturing

Remaining Useful Life (RUL) of machinery equipment by utilizing physics-based simulation models and Digital Twin concept, in order to enable predictive maintenance for manufacturing resources using Prognostics and health management (PHM) techniques is area of LMS expertise. The resources and the properties of them are first modelled in digital environment able to simulate the real machine’s behaviour. Data are gathered by machines’ controllers and external sensors to be used for the Digital Twin enabling via synchronous tuning of the digital models and their simulation.

References:

  • Aivaliotis, Z. Arkouli, K. Georgoulias, S.Makris, “Degradation curves integration in physics-based models: Towards the predictive maintenance of industrial robots”, Robotics and Computer-Integrated Manufacturing, Volume 71, 102177, (2021)
  • Aivaliotis, K. Georgoulias, G. Chryssolouris, “The use of Digital Twin for predictive maintenance in manufacturing”, International Journal of Computer Integrated Manufacturing, Volume 32, Issue 11, pg. 1067-1080, (2019)
  • Aivaliotis, Z. Arkouli, D. Kaliakatsos-Georgopoulos, S. Makris, “Prediction assessment methodology for maintenance applications in manufacturing”, 54th CIRP Conference on Manufacturing Systems (CIRP CMS 2021), Procedia CIRP, Volume 104, pg. 1494-1499, 22–24 September (2021)
  • Aivaliotis, K. Georgoulias, K. Alexopoulos,”Using digital twin for maintenance applications in manufacturing: State of the Art and Gap analysis”, International Conference on Engineering Technology and Innovation (ICE IEEE), Available Online, 17-19 June, Sophia Antipolis, France (2019)
  • Aivaliotis, K. Georgoulias, Z. Arkouli, S. Makris, “Methodology for enabling Digital Twin using advanced physics-based modelling in predictive maintenance”, 52nd CIRP CMS Conference, (CIRP CMS 2019), Procedia CIRP,Volume 81, pp.417-422, 12-14 June, Ljubljana, Slovenia (2019)
  • Aivaliotis, K. Georgoulias, G. Chryssolouris, “A RUL calculation approach based on physical-based simulation models for predictive maintenance”, International Conference on Engineering Technology and Innovation (ICE IEEE), 27-29 June, Madeira Island, Portugal,(2017)