Design and Control of a Highly Redundant Rigid-Flexible Coupling Robot to Assist the COVID-19 Oropharyngeal-Swab Sampling
Yingbai Hu, Jian Li, Yongquan Chen, Qiwen Wang, Chuliang Chi, Heng Zhang, Qing Gao, Yuanmin Lan, Zheng Li, Zonggao Mu, Zhenglong Sun, Alois Knoll
11 Design and Control of a Highly RedundantRigid–Flexible Coupling Robot to Assist theCOVID-19 Oropharyngeal-Swab Sampling
Yingbai Hu , ‡ , Jian Li , ‡ , Yongquan Chen , ∗ , Qiwen Wang , , Chuliang Chi , , Heng Zhang , , Qing Gao , ,Yuanmin Lan , , Zheng Li , , Zonggao Mu , , Zhenglong Sun , , Alois Knoll Abstract —The outbreak of novel coronavirus pneumonia(COVID-19) has caused mortality and morbidity worldwide.Oropharyngeal-swab (OP-swab) sampling is widely used for thediagnosis of COVID-19 in the world. To avoid the clinical stafffrom being affected by the virus, we developed a 9-degree-of-freedom (DOF) rigid–flexible coupling (RFC) robot to assist theCOVID-19 OP-swab sampling. This robot is composed of a visualsystem, UR5 robot arm, micro-pneumatic actuator and force-sensing system. The robot is expected to reduce risk and free upthe clinical staff from the long-term repetitive sampling work.Compared with a rigid sampling robot, the developed force-sensing RFC robot can facilitate OP-swab sampling procedures ina safer and softer way. In addition, a varying-parameter zeroingneural network-based optimization method is also proposed formotion planning of the 9-DOF redundant manipulator. Thedeveloped robot system is validated by OP-swab sampling onboth oral cavity phantoms and volunteers.
Index Terms —Medical Robots and Systems, Task and MotionPlanning, Redundant Robots, Deep Learning for Visual Percep-tion
I. I
NTRODUCTION T HE outbreak of novel coronavirus pneumonia (COVID-19) is affecting the entire world, which has caused alarge number of deaths with an increase in the spread ofCOVID-19. To control the spread of COVID-19 at the earlystage, oropharyngeal-swab (OP-swab) sampling is commonlyadopted with respect to sample collection and specimensources for diagnosis [1]. However, protecting the safetyof medical staff during the sampling process raises a newchallenge because of susceptibility to infection from person toperson through respiratory droplets and contact transmissionin an unprotected environment [2]. A variety of related studieshave reported that respiratory droplets, feces and urine are the * This work was supported by The Chinese University of Hong Kong, Shen-zhen, and the Shenzhen Institute of Artificial Intelligence and Robotics for So-ciety. (Corresponding author: Yongquan Chen, Email: [email protected]) ‡ These authors contributed equally to this work. Robotics and Intelligent Manufacturing & School of Science and Engi-neering, The Chinese University of Hong Kong, Shenzhen, 518172, China. Shenzhen Institute of Artificial Intelligence and Robotics for Society,518129, China. Chair of Robotics, Artificial Intelligence and Real-time Systems, Tech-nische Universit¨at M¨unchen, M¨unchen, 85748, Germany. Department of surgery, and Chow Yuk Ho Technology Centre forInnovative Medicine, The Chinese University of Hong Kong, Hong Kong. School of Mechanical Engineering, Shandong University of Technology,Zibo, 255000, China. Longgang District People’s Hospital of Shenzhen, 518172, China.
Fig. 1: 9-degree-of-freedom (DOF) redundant rigid–flexiblecoupling robotroutes of transmission [3]. To address these issues in OP-swabsampling, robotics could play a key role in disease prevention.Considering the high risk of infection of COVID-19 andnegative tests of nasopharyngeal swabs (NP-swab) causedby irregular sampling, it is necessary to design an OP-swabsampling robot to assist healthcare staff through remote access.Robot-assisted OP-swab sampling is a promising techniquebecause it relieves the burden from medical staff, is convenientfor large-scale deployment, is cost-effective, and offers sam-pling standardization. As reported in [4], an OP-swab robotcould speed up the sampling process especially because thereis a lack of qualified healthcare workers. In [5] and [6], asemi-automatic OP-swab robot (compared to the traditionalhuman method) was developed with a teleoperation schemethat achieved good performance in clinical practice with a95% success rate. The OP-swab robot system mainly includesa snake-like manipulator, two haptic devices(Omega.3 andOmega.6) for teleoperation, an endoscope for visualization(assistance operating in the oral cavity) and a force-sensingsystem for safety protection. All OP-swab sampling processeswere controlled by experts’ teleoperation, where one hapticdevice (Omega.3) control the tongue depressor and anotherhaptic device (Omega.6) operate the manipulator. Wang et al. a r X i v : . [ c s . R O ] F e b Fig. 2: Control framework of COVID-19 OP-swab sampling with a redundant rigid–flexible coupling robot[7] have designed a low-cost robot for assistance in samplingof NP-swab; a swab gripper is attached to the rotation link withits extruded active 2-degree-of-freedom (DOF) end-effector foractuating the swab and a generic 6-DOF passive arm for globalpositioning. In [8], the team from the University of SouthernDenmark and the Lifeline Robotics company developed thefirst automatic swab robot for COVID-19 sampling. In [8], therobot is integrated with a UR5 manipulator, a visual systemand dexterous rotatable rigid connectors assembled with OP-swab which could perform the sampling task quickly using anautomated scheme.Of note, medical security remains vital to OP-swab sam-pling because human throat is fragile and easily injured.Nonetheless, although many desirable results were obtained,the abovementioned end-effectors were designed on the basisof a rigid body structure, which may cause physical injuriesof the oral cavity during improper operation or other medicalmalpractices. On the basis of our previous work [9], a novelmicro-pneumatic actuator (MPA) for throat swab sampling isdeveloped to achieve flexible collection and integrate a forcesensor, which provides safe, stable, and reliable samplingexperience, as shown in Fig. 1. Furthermore, force-sensingactuation offers compliance, which is helpful against shocks,particularly during interaction with oral cavity. Compared withthe rigid body-based sampling robots [5], [7] and [8], thedeveloped MPA is made of soft material, which is safer andlighter. Moreover, MPA is smaller and has a 7.5 mm cross-sectional diameter, which is convenient for working in humanoral cavity.To avoid contact with the OP-swab specimen and collisionwith the manipulator, it is essential to consider the constraintsof the oral cavity space during motion planning. Inspired byour previous work [10] on remote center of motion (RCM)techniques in minimally invasive surgical robots, the secondto the last link of the OP-swab sampling robot is constrainedwith the oral cavity center (OCC) constraint, which guaranteesthe absence of collision between the inserted end-effector withthe oral cavity. Although mechanical implementation is usuallysafer for an open minimally invasive surgery, programmableRCM is cost effective and convenient to be implemented[11]. The traditional Cartesian adaptive control in [11] was applied for surgical control with the RCM constraint butwithout considering the physical limits of the manipulator,such as the joint angle and velocity. For practical applications,we propose an optimization strategy for the multi-constraintproblem, where the joint angle and velocity, and the OCCconstraint-based kinematic model are derived as a quadraticprogramming problem. In this study, a comparison betweenthe varying-parameter zeroing neural network (VP-ZNN) andthe traditional gradient descent method [12] is proposed for theoptimization problem that has achieved superior performance,which can converge at the finite time owing to the optimalsolution changing over time.To test the developed OP-swab sampling robot, which isshown in Fig. 1, the experiments are simultaneously conductedin both the human oral cavity phantom and volunteers, regard-ing the sampling time, operation complexity for medical staff,safety and effectiveness. The framework of COVID-19 OP-swab sampling using the RFC robot is shown in Fig. 2. Thecontributions of this study can be summarized as follows.1) A rigid–flexible coupling robot is designed to assistCOVID-19 OP-swab sampling, where the MPA for OP-swab sampling is developed to achieve flexible samplingand thereby provides compliant, safe, stable, and reliablesampling.2) An improved motion planning method is proposed forthe 9-DOF highly redundant robot-assisted OP-swabsampling system, which guarantees efficiency and ac-curacy, where the multi-constraint kinematics model isderived as an optimization problem; then, the VP-ZNNis applied to solve the optimization problem.3) Substantive practical experiment testing of the developedOP-swab sampling robot system is demonstrated in thehuman oral cavity phantom and volunteers.II. D
ESIGN OF THE R IGID –F LEXIBLE C OUPLING R OBOT
A. Redundant Rigid–Flexible Coupling Robot
The rigid–flexible coupling robot for OP-swab sam-pling consists of a UR5 manipulator and a self-developedrigid–flexible coupling manipulator. The overall structure ofthe 9-DOF redundant rigid–flexible coupling robot is shown inFig. 1. The self-developed manipulator has 3 DOFs, including a linear motor (prismatic joint), a servo– α motor (revolutejoint), and an MPA that can change the offset distance of throatswab from the center.Fig. 3: (a) Cross section of the micro-pneumatic actua-tor(MPA). (b) bending process of MPA. and (c) 2D modelingof MPA. B. Design, Modeling, and Fabrication of MPA
The design of MPA refers to the wrinkle shape of theelastomer robot [13]. Figure 3 shows that MPA consists of n effective air chambers. The sectional view of MPA is shownin Fig. 3(a), which contains a chamber part and a cover part.These two parts are bonded together to form a single-DOFMPA. b is the wall thickness of the air chamber; c is thedistance between the air chambers, and l f is the total lengthof MPA. When MPA is inflated, the air chambers will expandand repel each other, which causes MPA to bend. The bendingprocess of MPA is shown in Fig. 3(b). (a) (b) Fig. 4: Experimental results of MPA. (a) Deformation of MPAwith a 50-shore hardness under different air pressures. (b)Relationship between bending angle γ and air pressure Assuming that all air chambers have the same bending underthe same pressure, it is easy to obtain the geometric relation: θ = β/n = 2 γ/n (1)Owing to the small value of θ after bending, the chord lengthis approximately equal to the arc length at the non-tensilelayer. The distance between the center line and the non-tensilelayer is m , and the arc length c (cid:48) is also approximately equalto the chord length; thus, c (cid:48) is c (cid:48) = c + 2 m sin( θ/ (2)The overall motion model is shown in Fig. 3 (c), where thechord length d is d = c (cid:48) sin( nθ/ θ/ (3)The forward kinematics of MPA is modeled as x = d sin (cid:0) n θ (cid:1) + h sin ( β ) y = d cos (cid:0) n θ (cid:1) + h cos ( β ) (4)According to the abovementioned information, we designMPA with 12 air chambers. However, the number of effectiveair chambers, i.e., the number of effective joints is n = 11 because only half of the head chamber and tail chamber willbend. The wall thickness b of each chamber is 1.5 mm , thewidth of each air chamber c is 4.5 mm and the total length l f is 80 mm.The chamber part and the cover part of MPA are madeby 3D printing with a 50-shore hardness. The air pressureis controlled by the proportional valve ITV1050-312L. Then,a deformation experiment of MPA was conducted, and theexperimental results are shown in Fig. 4(a), where the gravityis directly perpendicularly to the plane of the paper. Therelationship between the bending angle γ and air pressure isexpressed in Fig. 4(b). It is observed that the bending angle γ is approximately proportional to the air pressure. After datafitting, the linear equation for the angle γ ( ◦ ) and pressure P (kPa) is written as γ = 0 . P − . (5)The correlation coefficient R is 0.99990 and the standarddeviation S is 0.6464, which confirms that γ and P are linearlydependent. By combining equations (1) and (5), we obtain θ = 0 . P − . (6)Finally, the end-point position of MPA can be calculated bycombining equations (2), (3), (4), and (6). C. Force Sensing
To detect the contact force F during OP-swab sampling,a strain gauge sensor (BF350-3AA23T0) attached to MPA isadopted. It is necessary to explore the mapping relationshipbetween the force and voltage for force measurement calibra-tion. Actually, the current output voltage U now is related to twofactors, i.e., voltage ∆ U generated by the force, and voltage U F =0 caused by the air pressure without a load. Therefore,we have ∆ U = U now − U F =0 . After 20 group experiments, Fig. 5: Relationship between F and ∆ U the relationship between no-load voltage and air pressure isobtained by linear fitting: U F =0 = − . P + 2770 . (mV). Force calibration was performed using the electronicbalance (CX-668). The relationship between ∆ U and F isapproximately linear; thus, we know that F = 0 . U + 0 . (7)The details are shown in Fig. 5. The correlation coefficient R is 0.99461 and the standard deviation S is 2.65733.III. O PTIMIZATION C ONTROL FOR THE
EDUNDANT M ANIPULATOR
A. Manipulator Kinematics Model with OCC Constraints
The coordinate system of the 9-DOF redundant RFC ma-nipulator is shown in Fig. 6. The forward kinematics model[14], [15] of the 9-DOF RFC robot can be defined as follows: J ˙ q = ˙ l d (8)where J ∈ R m × n denotes the Jacobian matrix. The joint angleand velocity constraints are defined as follows: q − i ≤ q i ≤ q + i (9) ˙ q − i ≤ ˙ q i ≤ ˙ q + i (10)where q − i and q + i present the lower and upper bounds of q i , respectively; and ˙ q − i and ˙ q + i denote the lower and upperbounds of the joint velocity ˙ q , respectively.Actually, the constraints in (9) can be converted as σ (cid:0) q − i − q i (cid:1) ≤ ˙ q i ≤ σ (cid:0) q + i − q i (cid:1) (11)where σ is the positive constant. Therefore, according to(10) and (11), the joint angle and velocity constraints can berewritten as a new constraint in the velocity level: ρ − i ≤ ˙ q i ≤ ρ + i (12) ρ − i = max (cid:8) ˙ q − i , σ (cid:0) q − i − q i (cid:1)(cid:9) ρ + i = min (cid:8) ˙ q + i , σ (cid:0) q + i − q i (cid:1)(cid:9) Because the RFC robot is a 9-DOF highly redundant ma-nipulator, there are infinite solutions of ˙ q in (8) by the inversekinematics. However, the convergence rate, accuracy, and Fig. 6: Coordinate system of the 9-DOF redundant RFCmanipulator.computational complexity of the pseudoinverse-type solutionin inverse kinematics cannot satisfy the requirements. Conse-quently, we need to identify an optimization solution undermultiple constraints; thus, the inverse kinematics problem canbe expressed as a new optimization problem. The first-priorityoptimization problem associated with OP-swab sampling isexpressed as follows:min
12 ˙ q T W ˙ q (13)s.t. J ( q ) ˙ q = ˙ l d (14) ρ − ≤ ˙ q ≤ ρ + (15)where ˙ l d represents the reference velocity associated with OP-swab sampling tasks. The weight matrix M is set as theidentity matrix.Considering the OCC constraint, the link- L n − of the RFCmanipulator needs to pass through the OCC, and L n performsthe sampling tasks, which is different from the traditionalRCM constraint in the last link. The geometric relationshipis shown in Fig. 7. For the n -DOF OP-swab sampling robot,the forward kinematics mapping function of Cartesian position l n − ∈ R m and l n − ∈ R m can be defined as follows: l n − = f n − ( q ) l n − = f n − ( q ) (16)Fig. 7: Constraint with the oral cavity center.Unlike common RCM constraints, l occ should be always onthe straight line straight line between l n − and l n − (secondto the last link), where l n − is the end position of the link L n − and l n − is the end position of the link L n − . Duringthe actual OP-swab sampling, we want to keep the error of theOCC constraint E occ as small as possible. Lines 1 and 2 areconstructed as follows: −−−−−−→ l n − r n − = l n − − l n − , −−−−−→ l n − l occ = l occ − l n − , respectively. Utilizing the relationship between E occ and the vector projection, E occ can be further written asfollows: E occ = −−−−−→ l n − l occ × −−−−−→ l n − l n − L (17)where L = (cid:107) l n − − l n − (cid:107) is the length of the second to thelast link. The derivative of E occ in (17) with respect to timeis reformulated as follows: J occ ˙ q = ˙ E occ (18)where J occ ∈ R m × n denotes the Jacobian matrix of the OCCconstraint. Regarding the OCC constraint task, we want tokeep the OCC error E occ at the minimum value.In this section, it is necessary to find a feasible solution formultiple constraints and guarantee that the tracking error andOCC error always remain in a small range. For COVID-19sampling tasks, we aim to reformulate OCC (18) and jointphysical limits (15) in an optimization scheme and design amethod to solve the optimization problem. Consequently, bysimultaneously taking OCC, end-effector task, and joint phys-ical constraints into account, the new multi-task optimizationproblem can be formulated asmin
12 ˙ q T W ˙ q (19)s.t. J ˙ q = v d J occ ˙ q = v occ ρ − ≤ ˙ q ≤ ρ + where v d = ˙ l nd and v occ = 0 . Without the explicit expressionof l n and e occ , the actual trajectory will drift and the positionerror cannot converge to zero from a random initial position.To overcome this issue in (19), a feedback item associatedwith position signals is integrated into v d and v occ : v d = − k ( f n ( q ) − l nd ) + ˙ l nd (20) v occ = − k ( l occ ) (21)Of note, we should manage the priority strategy for themultiple-task optimization problem by different weights.Therefore, the objective function is defined as F ( ˙ q ) = ε q T ˙ q + ε (cid:107) J ˙ q − v d (cid:107) + ε (cid:107) J occ ˙ q − v occ (cid:107) (22)The optimization problem in (22) can be rewritten asfollows: min F ( ˙ q ) (23)s.t. J ˙ q = v d J occ ˙ q = v occ ρ − ≤ ˙ q ≤ ρ + where ε > , ε > and ε > are the constants employedto prioritize different tasks. B. Neural Network Design
In this section, we describe the design of the VP-ZNN,which is used to solve the optimization problem in (22). First,the multiple-task optimization problem in (22) is convertedinto an equivalent problem; thus, the VP-ZNN is employed tosolve it.The Lagrange function of (22) constraints is formulated asfollows: L ( ˙ q, ξ , ξ ) = ε (cid:107) J ˙ q − v d (cid:107) + ε (cid:107) J occ ˙ q − v occ (cid:107) ε q T ˙ q + ξ T ( v d − J ˙ q ) + ξ T ( v occ − J occ ˙ q ) (24)where ξ ∈ R m and ξ ∈ R m ; ∇L = (cid:104) ∂ L ∂ ˙ q , ∂ L ∂ξ , ∂ L ∂ξ (cid:105) T indicates the gradient of (24). As the KKT condition definedin [16], if ∇L is continuous, the optimization solution satisfiesthe following condition: ∇L = 0 (25)The state decision variable l ( t ) = [ ˙ q, ξ , ξ ] T ∈ R n +2 m . Theproblem in (25) is equivalent to the following: B ( t ) l ( t ) = P ( t ) (26)where B ∈ R ( n +2 m ) × ( n +2 m ) and P ( t ) ∈ R ( n +2 m ) . Thebounds of state variable ˙ q b are expressed as ˙ q b = ρ + , ˙ q > ρ + ρ, ρ − ≤ ˙ q ≤ ρ + ρ − , ˙ q < ρ − Therefore, the bound of state variable l ( t ) are defined as l b ( t ) = (cid:2) ˙ q b ξ b ξ b (cid:3) , ξ b , ξ b ∈ R The error model of the novel VP-ZNN is expressed as e ( t ) = B ( t ) l ( t ) − P ( t ) (27)To ensure the model error convergence to zero, we define thefollowing formulation, de ( t ) dt = − µ exp ( t ) Ψ ( e ( t )) (28) Ψ ( e ( t )) = (1 + exp( − δ )) (1 − exp( − δe i ( t )))(1 − exp( − δ )) (1 + exp( − δe i ( t ))) where µ > is the constant that can adjust the convergencerate; Ψ ( e i ( t )) denotes the activation function, and ξ ≥ ,which makes ≤ | e i ( t ) | ≤
1. Clearly, the error in (28)converges to zero with exponential convergence.The function in (28) is expanded as B ( t ) ˙ l ( t ) = − ˙ B ( t ) l ( t ) + ˙ P ( t ) − µ exp ( t ) Ψ (cid:0) l ( t ) − l b ( t ) + B ( t ) l ( t ) − P ( t ) (cid:1) (29)We further modify the VP-ZNN in (29) as ˙ l ( t ) = ( I − B ( t )) ˙ l ( t ) − ˙ B ( t ) l ( t ) + ˙ P ( t ) − µ exp ( t ) Ψ (cid:0) l ( t ) − l b ( t ) + B ( t ) l ( t ) − P ( t ) (cid:1) (30)For comparison, the traditional gradient descent-based re-current neural network is denoted as ˙ l ( t ) = µ ( − l ( t ) + P Ω [ l ( t ) − ( B ( t ) l ( t ) − P ( t ))]) (31) Fig. 8: Demonstration with an oral cavity phantom. In ourexperiments, the 1:1 human oral cavity is tested.IV. E
XPERIMENTS
In this section, we present the tests with an oral cavityphantom and volunteers using the OP-swab robot system,approval from the Institutional Review Board of The Chi-nese University of Hong Kong, Shenzhen was obtained(IRBnumber CUHKSZ-D-20210002). Then, the advantages anddisadvantages are summarized and discussed. The parametersof the VP-ZNN are set as: ε = 0 . , ε = 10 , ε = 10 , µ = 0 . , k = 10 , and k = 10 . A. Demonstration with Phantom Experiments
For safety reasons, first, we conduct the experiments with anoral cavity phantom, which will help to confirm the safety ofthe OP-swab robot system. The oral cavity phantom has a 1:1size of human oral cavity, which enables natural simulationof workflow across OP-swab sampling and allows us to workextremely close to conditions in practice. The procedures ofexperiments are as follows: • The phantom oral cavity is detected and segmented withthe RealSense Camera, which is configured on the RFCmanipulator. Because the sampling areas are the left andright tonsils and the palate area, we recognize and locatethe target position by Mask R-CNN. • The desired sampling trajectories are obtained by onlinemotion planning with an oral cavity constraint, in whichwe locate the target position in the first step and thengenerate the trajectories. The desired trajectories area piecewise straight line from the palate area to theleft tonsils to the right tonsils in the Cartesian space,where the relationship between each axis and time is theminimum jerk curve. • The robot is driven to perform the sampling tasks. Inthis phase, the desired joint trajectories are obtainedfrom optimization control methods. Moreover, multipleprotection mechanisms with force sensing are activatedduring robot operation.The experimental scenarios of the oral cavity phantom areshown in Fig. 8. We collect the dataset for the oral cavityphantom which is trained by Mask R-CNN [17]. To obtainthe category and location of the oral sampling areas, an oralvisual detector is trained and obtained. First, an oral cavity image dataset is created by collecting from RealSense D435i.Second, the Mask R-CNN with a Inception v4 module as itsbackbone is trained on the oral cavity image dataset and anoral cavity detector is obtained. Finally, oral cavity coordinatesare mapped to depth images to obtain the depth values of theoral cavity. We achieved a 95% recognition success rate onthe model, and the sampling time is less than 20 s.
B. Demonstration with Volunteers Experiments
On the basis of many trials of the OP-swab RFC robotsystem with the oral cavity phantom, the robustness and safetywere significantly improved. Then, we conducted a numberof volunteer experiments and achieved milestone significancefor the future clinical application of OP-swab sampling. Theprocedures of experiments are the same as those described inSec. IV-A. The dataset of the human oral cavity is collectedby 29 volunteers and trained using Mask R-CNN.The experimental scenarios of different subjects are shownin Fig. 9. The desired trajectories are the same as in Sec. IV-A.We design the piecewise straight-line trajectories from thepalate to the right tonsils to the left tonsils in the Cartesianspace, and the relationship between each axis and time is theminimum jerk curve. Figure 10 shows the tracking resultsassociated with one of the trials of planning trajectories.The motion trajectories in the Cartesian space are shownin Fig. 10(a). Figures 10(b) and 10(c) show that the jointtrajectories located in the physical limits and joint velocity aresmooth within the velocity limits, respectively. Figure 10(d)shows the tracking error in the Cartesian space, where allerrors rapidly converge to a small value (0.02 mm). In addition,we added a comparison experiment shown in Fig. 10(d),where the proposed method has a faster convergence rate andsmaller errors than [10]. The average recognition rate, controlprecision, sampling time, and sampling contact force arerecorded and shown in Table. I. The comparative experimentswere conducted to examine the effectiveness of robotic OP-swab sampling. The swab quality was verified according tothe threshold cycle (Ct) value of the selected reference gene(RNase P) by the RT-PCR test [18]. Figure 11 shows the RT-PCR test results compared with those for the manual scheme.OP-swab with Ct values of ≤ and > are considered asqualified and unqualified samples, respectively, and the detailsare shown in Table II. The test results show that the samplesare qualified.TABLE I: Average sampling parameters. Metric Types Phantom VolunteersRecognition rate 95% 93%Control precision ≤ ≤ ± ± ≈
150 mN ≈
150 mN
TABLE II: Ct distribution.
Ct (24–27) (27–30) (30–33) (33–37) >
37 Qualified rateManual 11 16 3 0 0 100%Robotic 5 21 3 0 1 96.67%
Fig. 9: Sampling process from vision detection to sampling tasks. In our experiments, many volunteers were tested, and therecognition rate, control precision, sampling time, and sampling contact force were recorded. (a) Motion trajectories. (b) Joint trajectories.(c) Joint angle velocity. (d) Tracking errors.
Fig. 10: Tracking results: the manipulator track of the desired trajectories (sampling tasks) in the Cartesian space. (a) The 3Dtrajectories of the manipulator; (b) the joint angle trajectories; (c) the joint velocity trajectories (located in bounds); and (d)comparison experiments with [10].
Fig. 11: RT-PCR test results.V. C
ONCLUSION AND F UTURE W ORK
In this study, we designed a 9-DOF redundant RFC robotto assist the COVID-19 OP-swab sampling. Moreover, weformulate the sampling tasks, physical limits, and OCC con-straints as a novel optimization problem. Then, a VP-ZNN isproposed to solve the multi-constraint optimization problemonline. The experimental results of the oral cavity phantomand volunteer experiments demonstrate the effectiveness ofthe designed robot system and proposed control methods. Theaverage sampling time on phantoms and volunteers are 18 sand 20 s respectively. To maintain the sterility of the arm andeffector between patients, the MPA is designed as a disposabledevice with a quick connector that is easily to be replaced,where the disposable protective film is attached to the MPA.In the future, we will focus on improving the robustness ofthe system and move forward to clinical testing.A
CKNOWLEDGMENT
The authors would like to thank the Prof. Xi Zhu, Mr.Yi Liang, Ms. Kerui Yi and Mr. Xiaochuan Lin for theirsuggestions and engineering contributions to this work. Thiswork was supported by the Shenzhen Fundamental Researchgrant (JCYJ20180508162406177, JCYJ20190806142818365)and the National Natural Science Foundation of China(U1613216, 62006204, 61903100) from The Chinese Univer-sity of Hong Kong, Shenzhen. This work was also partiallysupported by the Shenzhen Institute of Artificial Intelligenceand Robotics for Society.R
EFERENCES[1] X. Wang, L. Tan, X. Wang, W. Liu, Y. Lu, L. Cheng, and Z. Sun,“Comparison of nasopharyngeal and oropharyngeal swabs for sars-cov-2 detection in 353 patients received tests with both specimenssimultaneously,”
International Journal of Infectious Diseases , 2020.[2] B. Xu, M. U. Kraemer, B. Gutierrez, S. Mekaru, K. Sewalk, A. Loskill,L. Wang, E. Cohn, S. Hill, A. Zarebski et al. , “Open access epidemiolog-ical data from the covid-19 outbreak,”
The Lancet Infectious Diseases ,vol. 20, no. 5, p. 534, 2020.[3] J. Hindson, “Covid-19: faecal–oral transmission?”
Nature Reviews Gas-troenterology & Hepatology , vol. 17, no. 5, pp. 259–259, 2020. [4] G.-Z. Yang, B. J. Nelson, R. R. Murphy, H. Choset, H. Christensen,S. H. Collins, P. Dario, K. Goldberg, K. Ikuta, N. Jacobstein et al. ,“Combating covid-19—the role of robotics in managing public healthand infectious diseases,” 2020.[5] S.-Q. Li, W.-L. Guo, H. Liu, T. Wang, Y.-Y. Zhou, T. Yu, C.-Y. Wang,Y.-M. Yang, N.-S. Zhong, N.-F. Zhang et al. , “Clinical application of in-telligent oropharyngeal-swab robot: Implication for covid-19 pandemic,”
European Respiratory Journal , 2020.[6] H. Liu, “Application of artificial intelligence robots in respiratorydiseases,” 2000. [Online]. Available: https://wx.vzan.com/live/tvchat-76585257?ver=637228232672628287 arXivpreprint arXiv:2005.12679
Engineering , vol. 1,no. 1, pp. 073–078, 2015.[10] H. Su, Y. Hu, H. R. Karimi, A. Knoll, G. Ferrigno, and E. De Momi,“Improved recurrent neural network-based manipulator control withremote center of motion constraints: Experimental results,”
NeuralNetworks , vol. 131, pp. 291–299, 2020.[11] H. Su, C. Yang, G. Ferrigno, and E. De Momi, “Improved human–robot collaborative control of redundant robot for teleoperated minimallyinvasive surgery,”
IEEE Robotics and Automation Letters , vol. 4, no. 2,pp. 1447–1453, 2019.[12] S. S. Mirrazavi Salehian, N. Figueroa, and A. Billard, “A unified frame-work for coordinated multi-arm motion planning,”
The InternationalJournal of Robotics Research , vol. 37, no. 10, pp. 1205–1232, 2018.[13] Z. Wang, Y. Torigoe, and S. Hirai, “A prestressed soft gripper: design,modeling, fabrication, and tests for food handling,”
IEEE Robotics andAutomation Letters , vol. 2, no. 4, pp. 1909–1916, 2017.[14] Z. Li, B. Huang, A. Ajoudani, C. Yang, C.-Y. Su, and A. Bicchi,“Asymmetric bimanual control of dual-arm exoskeletons for human-cooperative manipulations,”
IEEE Transactions on Robotics , vol. 34,no. 1, pp. 264–271, 2017.[15] Z. Li, J. Li, S. Zhao, Y. Yuan, Y. Kang, and C. P. Chen, “Adaptiveneural control of a kinematically redundant exoskeleton robot usingbrain–machine interfaces,”
IEEE transactions on neural networks andlearning systems , vol. 30, no. 12, pp. 3558–3571, 2018.[16] F. Facchinei, C. Kanzow, and S. Sagratella, “Solving quasi-variationalinequalities via their kkt conditions,”
Mathematical Programming , vol.144, no. 1-2, pp. 369–412, 2014.[17] K. He, G. Gkioxari, P. Doll´ar, and R. Girshick, “Mask r-cnn,” in
Proceedings of the IEEE international conference on computer vision ,2017, pp. 2961–2969.[18] A. Radoni´c, S. Thulke, I. M. Mackay, O. Landt, W. Siegert, andA. Nitsche, “Guideline to reference gene selection for quantitative real-time pcr,”