(General context) Today, using robots has grown dramatically in various applications; such as manufacturing, logistics, medicine, home, education, defense, factory, etc. Robot arms are one of the important types of robots that are used in factories to assemble or produce goods. Furthermore, the robots don’t have any brain so they need to be controlled. So “robot controlling” is a challenging problem among researchers and engineers. In the past, most of the robots were controlled, manually. But today, automated methods are more common.
Robot arms are one of the more useful robots in production lines where they are used to do some works. It is because the benefits are plentiful: higher quality, faster production, less waste, and greater safety (removing employees from hazardous working conditions, handling heavy parts).
These kinds of robots are used mostly in automotive and aerospace, especially for the handling and assembly of large parts such as Drilling and Fastening, Inspection, Welding, Painting and Sealing, Collaborative Assembly, etc. Thus it is important for an automatic robot arm to detect objects and reach them as well. In some cases, usually the target has a random position, thus it will be a challenging problem to reach to a random target.
(Problem) Automatically random target reaching by robot arms not only is a challenging problem but also is a popular topic among researchers. Actually, there are several solutions to solve this challenging problem. Recently, the researcher uses reinforcement learning algorithm to solve it. According to the methods, they apply a reinforcement learning algorithm to robot arms in order to learn how the end effector can reach to the random target. Q-learning (QL), deep Q network (DQN), Deep Deterministic Policy Gradient (DDPG), etc. are some of the algorithms that have been used by researchers recently.
(Main Idea) In this project, we are going to use a vision-based investigation to find a random target and also, we will use the SARSA algorithm as a reinforcement learning algorithm to reach the target.
(Method) Now, here, we apply a strategy to reach the goal (position of random target). At first we should find the position of target by image processing of the picture that come from top view camera. Next, we will divide the picture to some cells (Discretization). Then the position of the target will be used as a goal. Actually, we considered the robot arm, random target, position of the end effector and Up-Right-Down-Left moving (in each cell) as an agent, state and actions respectively. As mentioned, SARSA reinforcement learning is considered to learn the robot how the end effector can reach to the random target. This method is one of the on-policy TD control methods. In this method, the first step is to learn an action-value function rather than state-value function. We must estimate for the current behavior policy 𝜋 and for all states “s” and actions “a”. Also, we consider transitions form state-action pair to state-action pair, and learn the value of state-action pairs. The update will be done after every transition from a nonterminal state “”. If is terminal, then is defined as zero. This rule uses every element of the quintuple of events .
(Result) We can automatically assemble goods in products line, using a robot arm that controls by a vision-based SARSA reinforcement algorithm. Also, we can modify and then operate this approach with other robots such as autonomous vehicles and any other robot that need to detect and reach to a random target.
(Contribution) According to as mentioned above, this project mainly uses in product lines of factories to help workers, thus we can contribute with some companies such as POSCO company, HUNDA, KIA, IKC (Iranian automaker), SIPA (Iranian automaker) and any other companies that use robot arm in their product lines.
Ali's Researches 