TY - JOUR
T1 - A novel hybrid path planning method based on q-learning and neural network for robot arm
AU - Abdi, Ali
AU - Adhikari, Dibash
AU - Park, Ju Hong
N1 - Funding Information:
This paper presents a new hybrid approach to path planning based on Q-learning a neural network for robot arms. To this end, in order to prepare the required information and a neural network for robot arms. To this end, in order to prepare the required infor-for the hybrid path planning method, the location of the start point, the obstacle, and the mation for the hybrid path planning method, the location of the start point, the obstacle, target were obtained via image processing using the KNN algorithm. Then, in the first and the target were obtained via image processing using the KNN algorithm. Then, in the component of hybrid path planning, inspired by the windy grid world problem, the Q- first component of hybrid path planning, inspired by the windy grid world problem, the learning algorithm was used to find a sequence of actions such as up, down, left, and right Q-learning algorithm was used to find a sequence of actions such as up, down, left, and to reach the target cell and avoid collision with the obstacle in a gridded 2D workspace. right to reach the target cell and avoid collision with the obstacle in a gridded 2D work-Using simple action-state spaces led us to model the agent-environment interaction only in space. Using simple action-state spaces led us to model the agent-environment interaction programming software instead of linking it with simulator software, which significantly only in programming software instead of linking it with simulator software, which signif-simplified the path planning problem. Then, we use a trained neural network to convert the icantly simplified the path planning problem. Then, we use a trained neural network to convert the found actions to the corresponding joints angles, which provides desired movement from a cell to its adjacent cells. Since the weights were trained already before path planning, using them during the angle-finding step is not time-consuming at all. We test this hybrid method both using a simulator and a real robot arm. The results show that this hybrid approach significantly improves the slowness and the complexity of the path ApluatnhnoirnCgo pnrtroibbluetmio.n s: A.A. surveyed the backgrounds of this research, raised the idea of hybrid path planning, designed the neural network, performed the simulations, and wrote the manuscript. ADu.Ath. porreCpoanretdribthuetieoxnpse:rAim.Aen. staulrsveetyuepda tnhde ibmacpklegmroeunntedds tohfethmisetrheosedaorcnht,hreairseeadl-wthoerlidderaoboof th. yJ.bHri.Pd. psuapthe rpvliasnendinangd, dseuspigpnoerdte tdheth niseustruadl ny.etAwlloarku,thpoerrsfohramveedrethaed sainmdualagtrieoends,t aontdh ewpruobtel itshhee dmvaenrussiocrnipotf. Dth.eAm. parneupsacrreidpt t.he experimental setup and implemented the method on the real-world robot. J.H.P. supervised and supported this study. All authors have read and agreed to the published version of Funding: This research was supported by the MSIT (Ministry of Science and ICT), Korea, under the manuscript. the ICT Creative Consilience program (IITP-2020-2011-1-00783) supervised by the IITP (Institute for FInufnodrminagt:i oTnh&is rceosmeamrcuhn wicaatsi osunps pToecrhtendo bloygtyhPe lManSnITin (gM&inEisvtarlyuoafti Socni)e.nce and ICT), Korea, under the ICT Creative Consilience program (IITP-2020-2011-1-00783) supervised by the IITP (Institute for Information & communications Technology Planning & Evaluation).
Funding Information:
Funding: This research was supported by the MSIT (Ministry of Science and ICT), Korea, under the ICT Creative Consilience program (IITP-2020-2011-1-00783) supervised by the IITP (Institute for Information & communications Technology Planning & Evaluation).
Publisher Copyright:
© 2021 by the authors.
PY - 2021/8/1
Y1 - 2021/8/1
N2 - Path planning for robot arms to reach a target and avoid obstacles has had a crucial role in manufacturing automation. Although many path planning algorithms, including RRT, APF, PRM, and RL-based, have been presented, they have many problems: a time-consuming process, high computational costs, slowness, non-optimal paths, irregular paths, failure to find a path, and complexity. Scholars have tried to address some of these issues. However, those methods still suffer from slowness and complexity. In order to address these two limitations, this paper presents a new hybrid path planning method that contains two separate parts: action-finding (active approach) and angle-finding (passive approach). In the active phase, the Q-learning algorithm is used to find a sequence of simple actions, including up, down, left, and right, to reach the target cell in a gridded workspace. In the passive phase, the joints angles of the robot arm, with respect to the found actions, are obtained by the trained neural network. The simulation and test results show that this hybrid approach significantly improves the slowness and complexity due to using the simplified agent-environment interaction in the active phase and simple computing the joints angles in the passive phase.
AB - Path planning for robot arms to reach a target and avoid obstacles has had a crucial role in manufacturing automation. Although many path planning algorithms, including RRT, APF, PRM, and RL-based, have been presented, they have many problems: a time-consuming process, high computational costs, slowness, non-optimal paths, irregular paths, failure to find a path, and complexity. Scholars have tried to address some of these issues. However, those methods still suffer from slowness and complexity. In order to address these two limitations, this paper presents a new hybrid path planning method that contains two separate parts: action-finding (active approach) and angle-finding (passive approach). In the active phase, the Q-learning algorithm is used to find a sequence of simple actions, including up, down, left, and right, to reach the target cell in a gridded workspace. In the passive phase, the joints angles of the robot arm, with respect to the found actions, are obtained by the trained neural network. The simulation and test results show that this hybrid approach significantly improves the slowness and complexity due to using the simplified agent-environment interaction in the active phase and simple computing the joints angles in the passive phase.
KW - Hybrid method
KW - Neural network
KW - Obstacle avoidance
KW - Path planning
KW - Q-learning
KW - Robot arm
KW - Target reaching
UR - http://www.scopus.com/inward/record.url?scp=85111693516&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85111693516&partnerID=8YFLogxK
U2 - 10.3390/app11156770
DO - 10.3390/app11156770
M3 - Article
AN - SCOPUS:85111693516
SN - 2076-3417
VL - 11
JO - Applied Sciences (Switzerland)
JF - Applied Sciences (Switzerland)
IS - 15
M1 - 6770
ER -