Optimal crater landmark selection based on optical navigation performance factors for planetary landing

Yi XIU, Shengying ZHU,*, Rui XU, Maoeng LI

a School of Aerospace Engineering, Beijing Institute of Technology, Beijing 100081, China

b Key Laboratory of Autonomous Navigation and Control for Deep Space Exploration, Ministry of Industry and Information Technology, Beijing 100081, China

c Key Laboratory of Dynamics and Control of Flight Vehicle, Ministry of Education, Beijing 100081, China

d Science and Technology on Space Intelligent Control Laboratory, Beijing Institute of Control Engineering, Beijing 100091, China

KEYWORDS Crater-based optical navigation;Crater landmark selection;Crater matching;Measurement uncertainty;Planetary landing

Abstract Planetary craters are natural navigation landmarks that widely exist and are easily observed.Optical navigation based on crater landmarks has become an important autonomous navigation method for planetary landing.Due to the increase in observed crater landmarks and the limitation of onboard computation, the selection of good crater landmarks has gradually become a research hotspot in the field of landmark-based optical navigation.This paper designs a fast crater landmark selection method, which not only considers the configuration observability of crater subsets but also focuses on the influence on navigation performance arising from the measurement uncertainty and the matching confidence of craters, which is different from other landmark selection methods.The factor of measurement uncertainty, which is anisotropic, correlated and nonidentically distributed, is quantified and integrated into selection based on crater pairing detection and localization error evaluation.In addition, the concept of the crater matching confidence factor is introduced,which reflects the possibility of 2D projection measurements corresponding to 3D positions.Combined with the configuration observability factor, the crater landmark selection indicator is formed.Finally,the effectiveness of the proposed method is verified by Monte Carlo simulations.

1.Introduction

Planetary landing is an important way to understand the solar system and the universe and is also a very challenging space activity.1,2However, due to the remoteness of the planets and the complexity of the deep space dynamic environment such as the gravitational fields,the real-time characteristic and safety of traditional ground-based methods cannot be guaranteed.3,4In addition,the traditional navigation scheme based on IMU cannot meet the requirements of precision landing in the future due to the problem of cumulated errors.For surface terrains such as craters, the absolute position with respect to the target body-fixed coordinate system can be obtained in advance,which is helpful to determine the spacecraft position.Craters on the surfaces of interplanetary bodies are numerous and widely distributed with high visibility and distinguishability.5Optical navigation by crater landmarks provides strong autonomy and excellent navigation performance and has become the main navigation method in planetary exploration missions.6,7

For the optical navigation of planetary landing,the general process includes optical imaging of the overall outline or the surface of the target planet through the optical sensor, extraction of optical features by image processing algorithms, and state estimation of the spacecraft combined with other navigation information.The observed optical features mainly include the centroid of the target planet, the outline of the planet’s edge,and the feature points on the planetary surface.The feature point information usually needs to be obtained through pre-observation,and its position in the target body-fixed coordinate system needs to be determined.During the landing phase, the target planet cannot be fully observed in the field of view,and only a certain part of the planet or even the planetary surface is full of images.At this time, reference objects such as the edge of part of the planet body, craters and rocks on the surface are suitable as observation information for optical navigation.

In recent years, as far as landmark-based navigation missions are concerned,Jet Propulsion Laboratory(JPL)put forward the thought of position and attitude estimation of a spacecraft based on surface topographic features.During the process of Mars landing,NASA’s Spirit and Opportunity successfully matched landmarks in two adjacent images and estimated the horizontal velocity of the lander relative to the Mars surface.This was the first successful application of optical navigation in a planetary soft-landing mission in deep space.8,9In NASA’s another Mars-bound mission,the Mars 2020 mission,the terrain absolute navigation is designed by matching landmarks extracted from onboard images against landmarks selected from a prior map.10The Hayabusa from JAXA released artificial navigation beacons on the surface of Itokawa during the landing phase,and obtained the optical information of the target landing area through the optical navigation camera.Then, the image and distance information are fused to recursively implement the state of the lander to achieve soft landing.11A similar navigation scheme with 5 beacons was also applied to the Hayabusa2.12During the 2 h impact of the Deep Impact impactor on the Temple 1 comet, the target comet was also imaged by the optical navigation camera,and the onboard computer processed and calculated the attitude information.By calculating the relative position in real time, the impactor performed three autonomous orbital maneuvers during the impact phase, and finally realized the first human comet impact.13The OSIRIS-REx builds a three-dimensional map of Bennu using camera imaging and laser altimeter scanning.Combined with multi-spectrometer,the spacecraft can autonomously navigate based on the terrain features of the asteroid surface.14

In the optical navigation of planetary landings,the terrainlandmark-based autonomous optical navigation schemes have gradually become a research hotspot.Scholars divide such schemes into Terrain-Relative Relative Navigation (TRRN)and Terrain-Relative Absolute Navigation (TRAN).TRRN determines the relative motion of the spacecraft based on feature tracking between image sequences taken in real time during orbiting and/or landing phases.TRAN consists in detecting features in the camera image and in mapping them into an on-board database indexed by an absolute bodyfixed coordinate system.Then, the link between the 2D information of the terrain landmarks and the 3D position stored in the database map is established, so as to determine the absolute pose of the spacecraft with respect to the target bodyfixed coordinate system.15This paper is based on the TRAN scheme.

For optical navigation researches based on terrain landmarks, Rowell et al.performed position and attitude estimation of orbiting asteroid spacecraft based on optical recognition of known surface landmarks.16Cheng and Ansar proposed a classic crater-based method for absolute navigation, craters in the landing area are detected during the orbiting phase, and a database is established.Then, during the descent and landing phase, craters in landing images are matched with the craters in database, and then the position and attitude of the spacecraft are determined.This algorithm has been applied in exploration missions to Mars and the moon.17Cui et al.developed an innovative visual navigation scheme using crater edge curves during the descent and landing phases.In their study, the edge curves of the craters tracked from two sequential images were utilized to determine the relative attitude and position of the lander through a normalized method.18Delaune et al.proposed a vision-aided inertial navigation system for pinpoint planetary landing.The system uses measurements from a novel image-to-map matcher to update the state of an extended Kalman filter through a tight data fusion scheme propagated with inertial data.19Maass et al.presented an adaptive crater navigation scheme with three different crater matching methods for absolute terrain navigation.They pointed out that even under perfect imaging conditions,there would be small differences in the localization and ellipse parameters between detected craters and the projection of the same craters from a database on the image plane.The existence of perturbation exacerbates this deviation, so the matching thresholds are assigned according to the magnitude of the disturbance, and different matching methods are developed to adapt to different imaging conditions.20

Although it is preferred to adopt more landmarks for higher-accuracy pose estimation, the usage of abundant landmarks (for example, Rosetta defined up to 1157 landmarks through observations during the exploration for Comet 67P proximity21) brings the expense of intensive computation considering the limited onboard computer storage and capacity.With regard to the selection of navigation landmarks,the position configuration, gray-level information and overlap situation are studied.The pose estimation based on the pixel information of observed landmark localization is a key technology in the field of landmark navigation and directly affects the calculation efficiency and autonomous navigation capability of spacecraft.22–28Tian and Yu proposed a three-crater selection mechanism based on a one-step greedy look-ahead strategy by maximizing mutual Fisher information gain to reduce navigation estimation uncertainty.22Burschka et al.studied a navigation landmark selection method of mobile robots by using the condition number of the image Jacobian matrix and analyzed the influence of the navigation landmark configuration on the observability and controllability.23Mousavi and Motee proposed a feature selection method based on a random sampling algorithm,24which can effectively shorten the selection time compared to the greedy algorithm.Based on a first-order approximation of the error propagation from image measurements to the pose parameters,Lerner et al.used the trace of the covariance matrix of the image measurements to evaluate the pose estimation uncertainty generated by the combination of navigation landmarks.25An optimal navigation landmark selection method based on the evaluation of the observability of landmarks by a Line-of-Sight (LoS) angle observation matrix was proposed, and the influence of landmark distribution on navigation accuracy was analysed.26Similarly, the landmark selection method proposed by Hu et al.also takes the observability of the navigation system into consideration.27Xu et al.presented a prior-to-flight landmark selection algorithm based on linear covariance.28This algorithm selects the landmark subset that minimizes the positional uncertainty of the lander into the landmark database, which actually also optimizes the landmarks’localization configuration for the best observability of the navigation system.

In feature selection for image matching, in order to solve the tracking problem of affine transformed images, Shi and Tomasi pointed out that the corner point with the largest eigenvalue in the image should be selected, which can significantly improve the tracking quality and reduce the computational cost.29Anandan and Irani demonstrated that the image points have the characteristics of uncertainty in observation errors.The observation errors have uncertainties in scalar and in the two directions of the image, which can be transformed into a covariance-weighted data space by Singular Value Decomposition (SVD).30

From the research discussed above,current landmark selection methods mainly focus on the configuration observability of landmark localization,and other factors affecting pose estimation are ill-considered.For this issue, the key contribution of this paper is improving the landmark selection method.Considering that the measurement error uncertainty of crater localization is anisotropic, correlated and nonidentically distributed,a way to evaluate the localization uncertainty of crater measurement is presented.At the same time, the matching confidence of each crater is also taken into account,thus forming a rapid landmark selection method of planetary craters by creatively integrating the measurement uncertainty,configuration observability and matching confidence.

The structure of the remainder of this paper is as follows:Section 2 gives the measurement model and navigation scheme.Section 3 introduces the crater pairing detection method and the localization error evaluation method.Section 4 illustrates the concepts of measurement error uncertainty,configuration observability and matching confidence and constructs an evaluation indicator of the three factors, thereby forming a fast selection method.Section 5 presents simulations and analysis, and Section 6 draws some conclusions.

2.Measurement model and navigation scheme

Autonomous optical navigation can reduce the operation complexity,enhance the autonomous survivability and expand the space application potential of the lander.31Therefore,autonomous optical navigation has played an increasingly important role in completing landing missions.Landmark-based autonomous optical navigation exhibits good orbit determination performance.This section presents a landmark-based measurement model and navigation scheme.

2.1.Measurement model

Optical navigation cameras carried by the lander can perform oriented shooting to a planetary surface to obtain a navigation landmark image.Information such as pixels and lines of landmarks can be extracted from the image,and then the landmark oriented direction is obtained with respect to the lander navigation camera coordinate system.The camera imaging model used in this paper is a perspective projection model.A schematic diagram of the camera observation model is shown in Fig.1.

In Fig.1, the following coordinate systems are introduced for better understanding: Op-uv is the image plane coordinate system Σi, and B-XbYbZbrepresents the lander body-fixed coordinate system Σb.The camera coordinate system C-XCYCZCis donated as ΣC, and O-XfZfZfis the target body-fixed coordinate system Σf.

Fig.1 Sketch map of navigation measurement model.

Assuming that the camera coordinate system ΣCcoincides with the lander body-fixed coordinate system Σb, the landmark’s 2D localization in the image plane can be expressed as

2.2.Crater-based autonomous navigation scheme

In general, a landmark-based optical autonomous navigation scheme includes the following steps: The spacecraft preimages the target planetary body through the optical navigation camera in the initial orbiting stage and completes the observation of craters on the planetary surface with the support of the ground station.As a result, a high-precision crater database map is established for the target planetary surface.On this basis, craters are detected in new navigation images of subsequent landing missions and matched with the database.Then,the matched craters provide support for the absolute position and attitude estimation of the lander with respect to the target body-fixed coordinate system Σf.

However, there are constraints in the onboard storage and computation capabilities of the lander.Among the many craters observed in images,selecting craters with good observability, reliable matching confidence and low measurement uncertainty as navigation landmarks not only saves calculation resources compared with navigation by all detected craters but also improves navigation accuracy compared with selection considering only configuration observability.

To solve the above problems, this paper designs an autonomous navigation scheme based on crater landmarks, as shown in Fig.2.Initially, a new crater image is captured by an optical navigation camera equipped on the lander.With the help of image processing, the crater pairing detection method is used to achieve accurate crater detection, and the uncertainty of the crater localization error can be obtained according to the 2D observation information of the crater image.Then, the detected crater landmarks are matched with the stored crater map to calculate the matching confidence and assign the corresponding 3D position.Subsequently, considering the crater localization error uncertainty, the distribution configuration of crater localization and crater matching confidence, the optimal navigation landmarks can be selected by minimizing the landmark selection indicator.Finally, the absolute position and attitude of the lander are estimated based on the 2D image measurements and the matched 3D positions of selected craters.For this navigation scheme, optimal crater selection is the focus of this paper.

3.Crater localization and error evaluation

The establishment of the optimal landmark selection indicator is closely related to the localization coordinates of the crater center, and the final pose estimation result is also inseparable from the accurate 2D crater localization.Therefore, this section first gives a method for pairing detecting craters on planetary surfaces without prior sunlight information.The approach is realized by constructing a pairing indicator of illuminated and shadow regions and transforming the pairing problem into minimizing the pairing indicator.On this basis,the detected outer edge of the crater is used to perform ellipse fitting and center localization.In addition,by constructing the localization error matrix,a universal localization error evaluation method is derived to provide support for optimal crater landmark selection and pose estimation.

Fig.2 Crater-based autonomous navigation scheme.

3.1.Crater localization based on pairing detection

If the direction of sunlight is not perpendicular to the planetary surface, the crater imaging characteristics under sunlight are shown in Fig.3(a).Because most impacted craters are bowl-shaped,5,22the crater lip blocks the light, and pseudoregions occur, which need to be removed in image processing.Due to the different angles of incident light, the ratio of the shadow and illuminated regions changes, but the combination is always approximately an ellipse after projection; this shape coincides with the projected crater in an image.A diagram of the projected crater’s illuminated and shadow regions in an image in the direction of sunlight is shown in Fig.3(b).

The illuminated and shadow regions of craters in a navigation image can be easily obtained by image segmentation methods.32,33For the schematic diagram shown in Fig.4(a),based on 2D Otsu algorithm segmentation, the global image is preliminarily segmented into shadow and illuminated region images,which have the largest amount of information in their respective images.Then, shadow and illuminated regions are separated from the background images by the 2D maximum entropy threshold segmentation algorithm.34The crater binary images of the shadow and illuminated regions extracted by image segmentation are shown in Fig.4(b) and Fig.4(c),respectively.The numbers of detected shadow regions and illuminated regions are recorded as nSand nI, respectively.

Craters need to be detected by pairing the illuminated and shadow regions that belong to the same crater.Under a group of parallel lights, the direction from the shadow region to the illuminated region belonging to the same crater should be consistent with the direction of sunlight.Therefore, the directions of all correctly paired shadow and illuminated regions should be roughly identical.In addition,the distance between the shadow and illuminated regions belonging to the same crater is assumed to be the shortest in this method.Based on the shadow and illuminated regions obtained from image segmentation, a search window is designed with Rpas the search radius and the shadow region center CSpas the circle center.Potential illuminated regions are searched for in this search window to reduce the computational cost of global pairing in the whole image.The region area and region center can be calculated as follows:

Suppose that the center of the p-th(p=1,2,...,nS)shadow region is CSp=[uSp,vSp]and that the center of the q-th(q=1,2,...,nI)illuminated region is CIq=[uIq,vIq].For each region,the area Apixelof a shadow or illuminated region is the total number of pixels contained in that region.Then, the coordinates of the area center Ccenter= [ucenter, vcenter] are shown as

Considering that all vectors of correctly paired shadow and illuminated regions should satisfy constraints that all directions coincide and that the distance summation is the minimum, the pairing indicator Jpis constructed as shown in Eq.(7).That is,the problem of pairing the crater shadow and illuminated regions is transformed into minimizing the pairing indicator Jpas follows:

Fig.3 Crater illuminated and shadow regions and their projections under sunlight.

Fig.4 Pairing schematic diagram of illuminated and shadow regions.

where d is the set of distances between shadow and illuminated regions within all search windows; K is used to measure the degree of consistency in directions from shadow regions to illuminated regions;||?||is the Euclidean norm;var(?)denotes variance; α is the weighting factor; Apis the area of the p-th shadow region; W is the decision matrix, where the elements in W are wpq? {0,1 } and wpq= 1 and wpq= 0 mean pairing and not pairing the p-th shadow region with the q-th illuminated region, respectively; rank(?) is the rank of the matrix.Constraint Eqs.(13)–(16) ensure that l paired shadow regions and illuminated regions have a one-to-one correspondence to avoid repeated paired regions.

Let L be the pairing combination solution of the above problem, and KLbe the set of direction vectors corresponding to L.Then, the direction of sunlight can be solved by KLas

The above method can be used to process Fig.4(a).The pairing result is shown in Fig.4(d), where the solved direction of sunlight is marked in green in the top left corner of the figure.

Using the searched optimal solution L,for the j-th crater of the l detected craters,a least-square ellipse fitting is performed on the outer edge of the combined region of the shadow and illuminated regions.35The center of the fitting ellipse can be regarded as the localization of the detected crater center by approximating that the crater edges on the interplanetary body surface are circles.36,37

3.2.Crater localization error

After craters are detected, the localization of crater centers is of great interest.The problem of crater center localization can be approximated as ellipse fitting and ellipse center determination.However, for 2D observations of crater landmarks from navigation cameras, crater localization inevitably exhibits observation errors, which come from imaging sensors,environmental conditions, target planetary bodies’selfmovement and so on.38These factors cause the localization error of a crater center to be anisotropic, correlated and nonidentically distributed.29,30The localization error is usually ignored and regarded as independent and identically distributed in most crater-based navigation methods.In this way,the accuracy of position and attitude estimation decreases due to the propagation of the error uncertainty.

Considering that the crater localization error is one of the factors that affect the navigation performance, a method for evaluating the localization error based on the measurement information of observed craters is proposed.39

In the image coordinate system Σi, we suppose that the quadratic polynomial of the fitting ellipse for each crater is

Then, the center coordinates (x0j, y0j) of the fitting ellipse can be solved by a least-square algorithm as

where Bj,Cj,Dj,Ej,and Fjare the coefficients of the j-th crater fitting ellipse.

Since the crater center is determined by the coefficients of the fitting ellipse, which are solved according to crater edge points xij, the localization error of the crater center depends on the characteristics of the edge points.Then, nj equations like Eq.(18) can be written.The residual of the elliptic equation is Vj= [v1j, v2j,...,vnj]T, and a matrix is used to represent Eq.(18) with the residual as

4.Optimal crater landmark selection

With the improvement of the accuracy of optical sensors and image detection, a large number of crater landmarks may be detected in the field of view.These crater landmarks are input into the pose estimation algorithm,which certainly brings benefits to improving navigation accuracy but also leads to a sharp increase in onboard computer calculations.Starting from three factors affecting navigation performance, which are the measurement uncertainty, the configuration observability and matching confidence of craters, an optimal navigation landmark selection method is proposed to optimize navigation information by discarding redundant craters.Tabu search is adopted to achieve fast selection of navigation landmarks.

4.1.Measurement uncertainty factor

As discussed in Section 3,the lander images the planetary surface during landing.Due to the influence of various factors,observation errors occur in the localization of crater centers in the landing image.The localization error is an important factor that affects the lander’s navigation performance.Therefore, the optimal crater landmark selection should take the impact of crater measurement uncertainty on pose estimation into consideration.Based on the localization error matrix,craters with good observation quality should be selected, and those with large measurement error uncertainty should be excluded to provide crater landmarks with high localization accuracy for the position and attitude estimation algorithm.

Based on the localization error matrix R given in Eq.(25)of a single crater,if l craters are detected in the landing image,the measurement uncertainty factor R of all detected craters is given as follows:

4.2.Configuration observability factor

The observability of the navigation system is one of the key performance drivers associated with navigation capability,which reveals the ability to determine the state of the system through observational information.Aiming to solve the problems that the localization configuration of crater landmarks affects navigation performance, scholars have proposed selection methods based on the optimal navigation landmark configuration.The use of global landmarks can be decreased by selecting optimally distributed landmarks based on pixel positions, thereby reducing the calculation complexity of navigation and saving on-board computing sources.22–28

Since the configuration of crater landmarks impacts the observability and performance of the navigation system, it is necessary to actively optimize the geometric configuration relation of selected craters.Note that the observation h is a variable related to the position r = [X, Y, Z]Tand attitude σ=[φ,θ,ψ]Tof the lander,and Eq.(2)is linearized by a Jacobian matrix that contains all of hj’s partial derivatives.Then,the observation matrix of the j-th crater landmark is shown as

4.3.Matching confidence factor

Crater matching connects crater detection and pose estimation of the lander.Falsely detected crater landmarks in the landing image can also be filtered and excluded by matching.Incorrect crater matching results will input abnormal crater location information to the pose estimation algorithm, which will lead to poor estimation results.Since matching outliers are much more likely to affect the outputs, the matching results, as one of the important factors, need to be quantified and taken into account during crater landmark selection.This section uses the matching distance to describe the matching confidence of detected craters between the stored crater database map and the landing image.

Many image matching methods have been developed to date, such as SIFT41and SURF.42Through these methods,the homography matrix between two images with overlapping areas can be easily obtained to determine the correspondences.Let the homography transformation matrix be T.The matched crater centers in the map (IA) and the landing image (IB) have the following relationship:

It can be seen from Eq.(30)that under ideal conditions,the coordinates of an image point in one image can be transformed into another image by T and should coincide with the coordinates of the corresponding matching image point.However,in practical applications, due to the existence of errors, there are often differences between the transformed coordinates and the original coordinates.This difference can be regarded as the matching score to a certain extent.Here,the matching distance is used to describe that difference,as shown in Fig.5.The larger the matching distance of the correspondences, the greater the possibility of the correspondences being mismatched.

As shown in Fig.5,in IA,the original coordinates of crater centers are denoted as CA, and the transformed crater center coordinates from IBare CB’:

where i’is the matched crater number of the j-th crater of IBin IA.

This section provides a quantitative description of the matching confidence of the detected crater in the landing image, which can avoid the influence of falsely detected landmarks on pose estimation.The smaller the matching distance of the crater correspondences is, the higher the matching confidence of that crater correspondences is, and the greater the possibility that the corresponding craters are correctly matched is.Therefore,the matching confidence can be considered one of the crater selection factors.

Fig.5 Schematic diagram of crater matching confidence.

4.4.Fast selection method for optimal crater landmarks

The first-order estimates Θ of the position and attitude covariance matrix can be derived by Ref.25

where R is the measurement error uncertainty matrix, which reflects the localization errors in a 2D image.If R is simplified as isotropic, independently and identically distributed errors,which means that R is a diagonal matrix with constant values along the diagonal, Θ = σ2(HTH)–1can be regarded as the observability considering only the configuration distribution(H-only).Then, Θ is a measure of the configuration observability of the navigation system and only changes with the selected crater location configuration.

Considering the influence of the measurement uncertainty factor R, the configuration observability factor H and the matching confidence factor M on the performance of the lander’s position and attitude estimation, the pose covariance matrix Θ can be rewritten as

Then, the evaluation indicator Jscan be constructed based on Θ, that is, the quality of the crater landmark combination can be measured by the pose covariance matrix Θ.The smaller the eigenvalues of Θ are,the smaller the state deviation vectors corresponding to the feature vector or the errors corresponding to a linear combination of the state deviation vectors are,and the higher the accuracy of position and attitude estimation is.Since the trace of Θ is equal to the summation of its eigenvalues, the trace of Θ is used as the evaluation indicator to select crater landmarks, thus optimizing the lander’s navigation performance.

The selection criteria are shown as follows: when l crater landmarks are detected and the desired number of crater landmarks is N,select N crater landmarks from l detected craters to minimize the evaluation indicator Js.The model of the optimal landmark selection problem is shown as

where tr[?]represents the trace of the matrix, wjis the decision variable, and Λ is a decision matrix that is used to randomly select N craters from l craters.The solution that satisfies Eqs.(37)–(39) represents the selected optimal navigation landmarks.

If an exhaustive method is used to solve the above optimization problem, the computational time will increase explosively with an increase in the detected number of crater landmarks l or the desired number of crater landmarks N.Considering the influence of l and N on the search efficiency,a Tabu search optimization algorithm is used to search for the set of optimal crater combinations that can minimize Jsto improve the search speed of the optimal crater landmarks.

Tabu search is an extension of a local neighborhood search algorithm that can avoid falling into local optima.43As shown in Fig.6,first,l detected craters are numbered from 1 to l,and then N craters are randomly assigned a status of 1,which indicates that the landmark is selected.Then,the status of the rest of the craters is set to 0.The above assignment is the initial solution of the crater landmark combination,and the Tabu list is set to blank.At the beginning of an iteration, two crater states are randomly swapped to generate a specified number of new neighborhood solutions.In this algorithm, the two swapped craters’states are forced to be one selected (status 1)and the other unselected (status 0)to generate more diverse solutions and speed up the search process.Subsequently, all evaluation indicator Jsvalues in this set of solutions are calculated with the information of R, H, and M, and the solution with the smallest Jsis selected as a candidate solution.Then,follow the general process of the Tabu algorithm to judge,iterate and update the best-so-far solution, and finally output the set of craters that can minimize Js.The termination condition adopted in this paper is the maximum number of iterations.

5.Simulation analysis

To verify the effectiveness of the optimal crater landmark selection method based on the optical navigation performance factors described above, this section constructs a simulation scene based on crater images of real planet surfaces.Then,crater pairing detection was performed on a transformed image.Subsequently, the method mentioned in Section 4 is used to select the optimal landmarks among all detected craters.Finally,Monte Carlo simulations are used to compare the navigation performance differences among the optimal landmarks considering the measurement uncertainty, the configuration observability and the matching confidence (MHR) and the landmarks considering only the configuration observability(H-only).

5.1.Simulation scene

A navigation image is used to simulate a planetary scene, as shown in Fig.7(a),which is transformed from a real planetary surface image of Mercury,44as shown in Fig.7(b).The simulation scene includes the lander, the navigation camera, and planetary craters, as shown in Fig.7(c).

In the simulation scene, the positions of crater landmarks on the planetary surface are customized based on the reference image information of Fig.7(b), so the 3D coordinates of craters with respect to the target body-fixed coordinate system Σfare known.When a new crater image (see Fig.7(a)) is captured by the navigation camera,crater detection,crater matching and crater selection are performed to Fig.7(a), and the camera pose is estimated.The simulation parameters used in the pairing detection, the optimal landmark selection and the pose estimation with respect to Σfare shown in Table 1.

Fig.6 Flowchart of fast selection method for optimal crater landmarks.

Fig.7 Schematic diagram of simulation scene.

Table 1 Simulation conditions.

5.2.Pairing detection results

To verify the crater pairing method,Fig.7(a)is used for detection.The processing results are shown in Fig.8, where Fig.8(a) and Fig.8(b) show the shadow and illuminated regions,respectively.The blue points are the centers of the respective regions.

From the pairing processing results,the problem of pairing the correct shadow and illuminated regions of the crater landmarks is solved by searching for the combination solution of the shadow and illuminated regions with the smallest pairing indicator Jp.As shown in Fig.8(c),the algorithm in this paper can effectively pair shadow or illuminated regions, and additionally, the average direction (marked in green in Fig.8(c))from the centers of the shadow regions to the centers of the illuminated regions is basically consistent with the direction of sunlight.The validity of the shadow-illumination pairing algorithm presented in Section 3 is verified.The final pairing results are shown in Table 2.

As shown in Fig.8(d),the Canny detector is used to extract the outer edge of the combined region of the correctly paired shadow and illuminated regions.The extracted outer edges are shown by yellow points, which represent the contour points of craters.The red ellipses are fitted from the crater outer edge points,and the green dots indicate the center localization of craters.

5.3.Selection results

5.3.1.Measurement uncertainty

Fig.8 Crater pairing detection results for landing image.

Table 2 Pairing results of shadow and illuminated regions.

Fig.9 Error ellipses of center localization of detected craters.

Based on the above detected craters, the localization error matrix of each crater center can be calculated by the method in Section 3.2.The localization error matrix R derived in Eq.(25) of the crater center determines the error ellipse, which is an uncertainty elliptical region located at (x0, y0).The error ellipse parameters can be obtained by applying the error ellipse calculation formulas given in Ref.39.By plotting error ellipses on the detection result, the uncertainty direction and uncertainty size of the fitting ellipse centers are intuitively shown in Fig.9.

It can be seen from Fig.9 that the error ellipse depends on the edge distribution of the detected crater under the same image noise.Due to the difference in observation information of different craters,the error ellipses of large craters are usually small.As a result, large craters should be given priority in the selection process.

5.3.2.Matching confidence

After craters in the landing image are detected, it is necessary to match them with craters in the stored crater map to obtain the location information.Here,the reference image(see Fig.7(b))is used as that map for simulation, and SIFT41is adopted for image matching.The crater matching results of 49 detected craters are shown in Fig.10(a),where all craters detected in the landing image are matched with correct crater landmarks in the reference image.The matching confidence M can be calculated through the method described in Section 4.3,and the difference in crater matching confidence is intuitively shown in Fig.10(b).

From Fig.10, although detected craters in the landing image are correctly matched, the matching confidence of each crater varies.The crater matching confidence determines the correspondence of the 2D image information and the 3D position information.Mismatches will lead to a very poor pose estimation result.Therefore,the matching confidence is a nonignorable factor when selecting craters.

5.3.3.Selected crater landmarks

The pose estimation method used here is EPnP,which requires at least 4 image points to solve the camera pose.45If the landing mission needs to select 4,5,6,and 7 crater landmarks from the 49 detected craters in the navigation image for subsequent position and attitude estimation, the selection is compared between an evaluation indicator of H-only25and MHR, as proposed in Section 4.4.The simulation comparison results are shown in Fig.11.

The selection results show that to make the position and attitude estimation accuracy as high as possible, the selected craters are basically distributed away from the center of the image.This is because if the selected craters come together,their LoS will form a very narrow bundle, which will in turn lead to a very inaccurate pose estimation compared to a combination of craters far from each other.In addition, the selection method proposed in this paper takes into account the observation quality,the matching confidence of each crater individual and the overall localization distribution.The MHR method excludes craters with large fitting errors or large matching distances in the H-only method.This approach increases the localization accuracy and the matching confidence of each landmark at the expense of sacrificing a partial effect of the overall configuration, thereby improving the accuracy of subsequent position and attitude estimation.

Fig.10 Crater matching results between reference image and landing image.

Fig.11 Crater landmark selection results of H-only and MHR.

Table 3 Comparison of running time for optimal landmark selection.

In addition,to verify the speed of the algorithm proposed in this paper, the running time for the optimal crater landmarks selected by the fast selection method and the exhaustive method is calculated.The CPU of the computer used in this simulation is an AMD Ryzen 9 5900HS, the main frequency is 3.30 GHz, and the simulation software is MATLAB 2019b.The results are shown in Table 3.

As shown in Table 3, for the selection from 49 crater landmarks, as the number of craters to be selected increases, the crater combinations become increasingly complicated.The exhaustive method needs to calculate the evaluation index Jsfor all kinds of combinations.As a result, the running time of the exhaustive method greatly increases.By virtue of the advantage of avoiding repeated searches, the running time of Tabu search is almost unchanged and is stable between 0.18 s and 0.20 s.In addition,the Tabu search algorithm guarantees the calculation speed,and the selected results are always consistent with those of the exhaustive method,thereby verifying the speed and effectiveness of the method proposed in Section 4.4.

5.4.Position and attitude estimation results

Assuming that the 3D positions of the 49 detected crater landmarks captured by the navigation camera are known, the corresponding 2D image plane coordinates can be obtained by the crater detection and localization method.The value of the image observation noise is 1, and the EPnP method is used to solve the position and attitude of the lander.45

Fig.12 Position and attitude error results with 4 craters from a 1000 run Monte Carlo simulation.

Fig.13 Position and attitude error results with 5 craters from a 1000 run Monte Carlo simulation.

Supposing that the number of required crater landmarks is 4, 5, 6, and 7, 1000 run Monte Carlo simulations are performed for H-only selection and MHR selection.The simulation results are shown in Figs.12–15.The two statistical characteristics of the mean values (Mean) and Standard Deviations (STD) of the errors are used to evaluate the accuracy and stability of the navigation performance by the two different crater selection indicators.The estimation results are shown in Tables 4–7.

From the above simulation results,although the number of selected landmarks changed,the STDs of position and attitude estimation errors by MHR were always better than those of the H-only method.This is because the localization error uncertainty of the crater landmarks selected by MHR is small,while the landmarks selected by the H-only method have large fitting and localization errors, which lead to unstable crater landmark coordinates in the image plane during pose estimation.

It should be noted that since all craters are correctly matched with craters in the reference image, the impact of the matching confidence cannot be reflected in this pose estimation simulation.If there is a mismatch and the H-only method selects the mismatched landmark, the pose estimation by these crater landmarks will give poor results.In addition,in a longitudinal comparison,as the number of crater landmarks increases, the observation information used for position and attitude estimation gradually increases.Regardless of which method is used, the overall estimation results improve, and the gap between the two methods gradually narrows.

Fig.14 Position and attitude error results with 6 craters from a 1000 run Monte Carlo simulation.

Fig.15 Position and attitude error results with 7 craters from a 1000 run Monte Carlo simulation.

Table 4 Mean and standard deviation values of position and attitude errors with 4 craters.

Table 5 Mean and standard deviation values of position and attitude errors with 5 craters.

Table 6 Mean and standard deviation values of position and attitude errors with 6 craters.

Table 7 Mean and standard deviation values of position and attitude errors with 7 craters.

The simulation results presented in Section 5 indicate that the optimal crater landmark selection based on optical navigation performance factors for planetary landing proposed in this paper is effective.This method can pair shadow and illuminated regions without information on the direction of sunlight to achieve autonomous crater detection.Additionally,in view of the problem that there are many crater landmarks observed in an image, considering the measurement error uncertainty of crater localization, the craters’configuration observability and the crater matching confidence, the optimal crater landmarks can be accurately selected.The simulation results of position and attitude estimation verify the accuracy and stability.

6.Conclusions

For the problem of an excessive number of craters in the field of view, this paper presents a novel crater landmark selection method for optical navigation in planetary landing.This method can rapidly select the optimal crater landmarks by minimizing an evaluation indicator consisting of craters’measurement uncertainty factor,configuration observability factor and matching confidence factor.Importantly, to handle the anisotropic, correlated and nonidentically distributed errors in images,a crater pairing detection method is given for crater localization, and an error evaluation method is presented to describe the measurement uncertainty.The selection results verified that the landmark selection process fully considers the three factors mentioned above and retains a crater subset with small localization errors,scattered distribution,and small matching distance as the optimal craters.The navigation performance based on these craters is better than that of the crater landmarks selected only by the observability configuration.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

This work was supported by the National Key Research and Development Program of China (No.2019YFA0706500), the National Natural Science Foundation of China (No.61873302,61973032,U20B2055 and U2037602),the Basic Scientific Research Program of China (No.JCKY2018602B002),and the Space Debris Program of China (No.KJSP2020020302).