T 1990/16 (Mobile robots and 3D maps/SAMSUNG) of 29.7.2021

Summary of Facts and Submissions

I. This case concerns the appeal of the patent proprietor (henceforth, "appellant") against the decision of the opposition division revoking the European patent on the ground of lack of inventive step (Articles 100(a), 52(1) and 56 EPC).

II. The following prior-art documents are relevant to the board's decision:

E1: M. Kurisu et al.: "Development of a Laser Range

Finder for 3D Map-Building in Rubble - The 2nd

Report: Development of the 2nd Prototype",

Proceedings of the IEEE International Conference

on Mechatronics & Automation, Niagara falls,

Canada, July 2005, pp. 1842-1847.

E15: R. Siegwart et al.: "Introduction to Autonomous

Mobile Robots", MIT press, 2004, pp. 48-51,

90-117, 154-165 and 250-256.

III. The appellant requests that the decision under appeal be set aside and that the opposition be rejected, i.e. that the patent be maintained as granted (main request).

Alternatively, the appellant requests that the patent be maintained in amended form on the basis of one of auxiliary requests 1 to 5, all as filed with the statement of grounds of appeal. Auxiliary requests 1 to 4 were submitted during opposition proceedings and were refused by the opposition division on the ground of lack of inventive step. Auxiliary request 5 was filed for the first time during the appeal proceedings.

IV. The respondent requests that the appeal be dismissed.

V. Oral proceedings were held on 29 July 2021 only in the presence of the appellant. The respondent had announced in advance that they would not be represented at those oral proceedings.

At the end of the oral proceedings, the chair announced the board's decision.

VI. Claim 1 of the main request reads as follows:

"A three-dimensional map-generating apparatus, comprising:

an odometer (110) detecting a pose of a mobile robot;

a distance-measuring sensor (120), including a light source module (122) disposed on an upper surface of the mobile robot to emit light upward relative to a movement direction of the mobile robot and a camera module (124), disposed on the upper surface of the mobile robot, to capture an image formed by the light as reflected from an obstacle, measuring a distance to the obstacle for the mobile robot using the captured image; and

a map-generating unit (130) generating a

three-dimensional map using the distance measured by the distance-measuring sensor while changing the pose of the mobile robot."

VII. Claim 1 respectively of auxiliary requests 1 to 4 differs from claim 1 as granted in that the following wording is added to the end of the claim:

Claim 1 of auxiliary request 1:

", wherein the map-generating unit accumulates three-dimensional data obtained by reflecting the pose of the mobile robot to measured

two-dimensional distance data while changing the pose of the mobile robot, thereby generating the three-dimensional map".

Claim 1 of auxiliary request 2:

"; and a plane-extracting unit (210) extracting a plane from the generated three-dimensional map".

Claim 1 of auxiliary request 3:

"; a plane-extracting unit (210) extracting a plane from the generated three-dimensional map; and

a feature-map-generating unit (220) extracting feature points from the extracted plane to generate a feature map".

Claim 1 of auxiliary request 4:

"; a plane-extracting unit (210) extracting a plane from the generated three-dimensional map;

a feature-map-generating unit (220) extracting feature points from the extracted plane to generate a feature map; and

a position-detecting unit (230) comparing the feature points extracted from the map of the mobile robot at a current position with previously stored feature points on the feature map to detect the position of the mobile robot".

VIII. Claim 1 of auxiliary request 5 reads as follows (with amendments with respect to claim 1 as granted underlined by the board):

"A three-dimensional map-generating apparatus, comprising:

an odometer (110) detecting a pose of a mobile robot, wherein the pose of the mobile robot includes position and direction angle of the mobile robot;

a distance-measuring sensor (120), including

a light source module (122) disposed on an upper surface of the mobile robot to emit light upward relative to a movement direction of the mobile robot, wherein the light source module is a line laser, and

a camera module (124), disposed on the upper surface of the mobile robot, to capture an image formed by the light as reflected from an obstacle,

measuring a distance to the obstacle for the mobile robot using the captured image; and

a map-generating unit (130) generating a

three-dimensional map using the distance measured by the distance-measuring sensor while changing the direction angle at a fixed position of the mobile robot."

Reasons for the Decision

1. Technical context and interpretation of terms

1.1 The opposed patent concerns the field of robotics, and in particular a three-dimensional (3D) map-generating apparatus for inclusion in a mobile robot. The basic principle is that light is emitted upward relative to a movement direction of the robot, and an image formed by light reflected from an obstacle (e.g. a wall or ceiling of a building) is captured by a camera disposed on the upper surface. In this way, the distance to the obstacle can be determined. By repeatedly changing the "pose" of the robot (see below), a 3D map can be created.

1.2 The 3D map-generating apparatus as claimed comprises an "odometer [for] detecting a pose of a mobile unit". The patent defines the term "pose" to mean "the position [i.e. location] and direction angle [i.e. orientation] of the mobile robot" (cf. paragraph [0027]), and this interpretation is adopted by the board. An "odometer" is considered to be any means for estimating the pose by means of motion sensors.

2. Main request - claim 1 - novelty and inventive step (Articles 100(a) and 52(1) EPC)

2.1 Claim 1 of the main request (i.e. as granted) includes the following limiting features (feature labelling as used by the opposition division in the impugned decision):

A) A 3D map-generating apparatus, comprising:

B) an odometer detecting a pose of a mobile robot;

C) a distance-measuring sensor, including

D) a light source module disposed on an upper surface

of the mobile robot to emit light upward relative to

a movement direction of the mobile robot;

E) a camera module, disposed on the upper surface of

the mobile robot, to capture an image formed by the

light as reflected from an obstacle;

C') measuring a distance to the obstacle for the mobile

robot using the captured image;

F) a map-generating unit generating a 3D map using the

distance measured by the distance-measuring sensor

while changing the pose of the mobile robot.

2.2 The closest prior-art document is considered to be E1, which describes methods for building a 3D map for a robot moving through a rubble environment. E1 discloses first and second prototypes of a laser range finder mounted on a robot. Both prototypes include a "laser beam module" in which a ring laser beam is generated by using a conical mirror, so that the laser beam is, inter alia, radiated in an upward direction (cf. Figs. 1 and 3). A CCD camera with a hyperbolic mirror captures the reflected image of the ring laser on the interior surfaces of the rubble environment. As the mobile robot moves inside the rubble environment, a 3D map is generated.

2.3 The respondent argued that E1 disclosed all the features of claim 1, so that its subject-matter lacked novelty (Article 54 EPC). In contrast, the appellant argued that E1 did not disclose features B, D and E.

2.4 Re feature B:

2.4.1 In one experimental embodiment of E1 (cf. page 1846, section IV.B), the position (i.e. location) of the robot is determined based on velocity, and the robot is driven straight forwards (cf. E1, page 1846, right-hand column, lines 2-12). However, it is not stated that the velocity is measured by a sensor (as stated, an odometer requires a motion sensor). Furthermore, there is no disclosure of a sensor for determining the direction-angle aspect of the "pose". The board therefore concludes that feature B is not disclosed in E1.

2.4.2 The respondent counter-argued that E1 disclosed that the robot determines its position using a so-called

SLAM-algorithm ("Simultaneous Localisation and Map Building"), and that it was common general knowledge that an odometer was required for the SLAM scheme. Thus, E1 implicitly disclosed an odometer.

2.4.3 This argument is however not convincing. E1 merely states that "we are investigating a method to estimate the position of robot in the SLAM framework" (cf. page 1846, right-hand column, lines 16-18). Since there is no enabling disclosure of a robot implementing a SLAM algorithm in E1, the inclusion of this feature is a matter to be considered in connection with inventive step rather than novelty (see point 2.7 ff. below). Furthermore, the appellant disputed that it was implicit that an odometer must be used when implementing the SLAM scheme, this being merely an option.

2.5 Re features D and E:

2.5.1 In E1, the range finder is mounted on the robot, i.e. on an "upper surface" thereof (cf. Fig. 19). Hence both the "light source module" and the "camera module" can be regarded as being mounted on the upper surface of the robot. Furthermore, the light beam is emitted upwards (via the conical mirror, cf. Fig. 3) to illuminate an upper surface, e.g. a ceiling, and the image reflected therefrom is captured by a CCD camera (via the hyperbolic mirror).

2.5.2 The appellant argued that the conical and hyperbolic mirrors were respectively external to the light source and camera modules. The presence of conical and hyperbolic mirrors in the modules is however not excluded by the wording of claim 1, since a "module" is a term to be interpreted broadly.

2.6 Therefore, in agreement with the opposition division, the only distinguishing feature between the

subject-matter of claim 1 and the disclosure of E1 is an "odometer" which detects a "pose", i.e. both a position (location) and a direction angle (orientation) of the robot.

2.7 The objective technical problem starting out from E1 can be considered as "how to improve 3D map building in a rubble environment by enabling the robot to determine its position using a SLAM algorithm".

2.8 The setting of the objective problem itself does not contribute to inventive step because E1 already states that the use of the SLAM algorithm for position determination is under investigation.

2.9 In order to solve this problem, the skilled person in the field of mobile robots would consult document E15, which comprises excerpts taken from a text book published in 2004 entitled "Introduction to Autonomous Mobile Robots". This book can be considered as representing common general knowledge at the patent's priority date (i.e. 19 November 2007). One application of robots considered in E15 is within buildings, since these have certain recognisable features, such as lines, corners and planes (cf. pages 161 to 163, section 4.3.1.4, especially page 163, lines 1-5). The skilled person would consider that E15 is relevant for a robot operating in a rubble environment within the meaning of E1, because E1 is related to a project for earthquake disaster mitigation in urban areas, and hence the rubble may obviously be that resulting from collapsed or partially collapsed buildings. In such an environment, line, corner and plane features (e.g. walls, ceilings and doorways) may still be at least partially recognisable.

E15 further discloses that typical sensors used in mobile robotics applications consist of, inter alia, "Wheel/motor sensors (wheel/motor speed and position)" and "Heading sensors (orientation of the robot in relation to a fixed reference frame)" (cf. page 91, Table 4.1). Implicitly, the use of such sensors in this context concerns odometry. In accordance with E15, a general schematic for "concurrent localisation and map building" (cf. page 251, Fig. 5.38), i.e. SLAM, shows inter alia a block entitled "Prediction of Measurement and Position (odometry)".

2.10 The skilled person wishing to perform SLAM would therefore be taught to make use of odometry in order to help estimate the robot's position in a global reference frame. Types of odometer that the skilled person would obviously consider, based on E15 (cf. point 2.9 above), include sensors reflecting the robot location (wheel sensors) and orientation (heading sensors), noting that a knowledge of the robot's orientation is still needed even if the robot intends to move in a straight line in order that it does not deviate from the line. By incorporating such sensors, the skilled person would arrive in an obvious manner at the subject-matter of present claim 1.

2.11 The appellant counter-argued that the use of odometry was not obvious in the rubble environment of E1 in view of the passage on page 1846, right-hand column, lines 14-18, which reads:

"... In the practical use, the range finder is mounted on the mobile robot whose position cannot be measured directly when the robot is inside rubble. We are investigating a method to estimate the position of robot in the SLAM framework ...".

In this respect, the appellant submitted that the indication that the position "cannot be measured directly" taught away from using odometry. Furthermore, the skilled person would not use odometry in view of the bumpy surface of rubble, since this would give rise to large errors.

2.12 The board finds these arguments unconvincing. Firstly, this passage of E1 does not at all teach away from using odometry, since, in general, an odometer uses sensors which do not directly measure a position, but indirectly. Secondly, a SLAM algorithm is stochastic (cf. E15, page 250, section 5.8.1), i.e. is based on a probability estimate rather than requiring accurate data. Such a stochastic process is thus eminently suitable for a "bumpy surface".

2.13 Consequently, the subject-matter of claim 1 does not involve an inventive step (Articles 52(1) and 56 EPC).

3. Auxiliary request 1 - claim 1 - inventive step

3.1 Claim 1 of auxiliary request 1 comprises the additional limiting feature that "the map-generating unit accumulates three-dimensional data obtained by reflecting the pose of the mobile robot to measured two-dimensional distance data while changing the pose of the mobile robot, thereby generating the

three-dimensional map".

3.2 This feature is disclosed in document E1, since the robot is moved forwards in order to obtain the 3D map data (cf. E1, page 1846, right-hand column, lines 1-4, noting that "changing the pose" embraces moving the robot only forwards with the same orientation (cf. paragraph [0037] and Fig. 7 of the patent). That notwithstanding, changes in orientation are suggested by the robot path taken in the embodiment shown in Fig. 14 of E1.

3.3 The appellant argued that, in the embodiment of Fig. 14 of E1, "no mobile robot is involved", because the robot was moved manually. However, that is self-evidently only because "the position estimation system was not implemented yet" (cf. page 1846, section IV.A). When equipped with a position estimation system as taught by E15, it would be obvious that a mobile robot may follow the same weaving path as shown in Fig. 14.

3.4 The subject-matter of claim 1 of auxiliary request 1 therefore does not involve an inventive step (Articles 52(1) and 56 EPC).

4. Auxiliary request 2 - claim 1 - inventive step

4.1 With respect to claim 1 as granted, claim 1 of auxiliary request 2 comprises the additional limiting feature of "a plane-extracting unit [for] extracting a plane from the generated three-dimensional map".

4.2 E15 discloses the feature of extracting typical "plane" building features including walls and ceilings from 3D data (cf. E15, pages 161 to 163, section 4.3.1.4; in particular page 163, lines 1-5). Even in a rubble environment, parts of walls, ceilings and floors may still be intact. Consequently, this feature does not contribute to inventive step.

4.3 The appellant argued that E15 taught that features such as lines and planes were extracted from the pixel data delivered by the CCD cameras, but were not extracted from a 3D map. Furthermore, in accordance with E15, the map was a 2D map (cf. page 254, Fig. 5.40).

4.4 This argument is however not convincing. In accordance with Fig. 5.38 of E15, sensor data (e.g. features derived from pixel data) are matched against predicted features extracted from the map ("predicted feature observations"). Figure 5.38 is moreover a general schematic which does not specify whether the map is two or three-dimensional. However, self-evidently, the skilled person starting out from E1 and wishing to solve the objective technical problem on the basis of E15 would be compelled to implement a 3D map, since the problem itself stipulates that there be a 3D map (cf. point 2.7 above).

4.5 The subject-matter of claim 1 of auxiliary request 2 therefore does not involve an inventive step either (Articles 52(1) and 56 EPC).

5. Auxiliary request 3 - claim 1 - inventive step

5.1 With respect to claim 1 of auxiliary request 2, claim 1 of auxiliary request 3 includes the additional limiting feature of "a feature-map-generating unit extracting feature points from the extracted plane to generate a feature map".

5.2 The general schematic for SLAM depicted in E15, page 251, Figure 5.38, shows a block for inter alia extracting feature points from the odometry, sensor and map data ("Map Building and Maintenance"). It is moreover obvious that a detected plane may have features, e.g. a doorway. Consequently, this feature also does not contribute to inventive step.

5.3 The subject-matter of claim 1 of auxiliary request 3 therefore does not involve an inventive step either (Articles 52(1) and 56 EPC).

6. Auxiliary request 4 - claim 1 - inventive step

6.1 With respect to claim 1 of auxiliary request 3, claim 1 of auxiliary request 4 includes the additional limiting feature of "a position-detecting unit comparing the feature points extracted from the map of the mobile robot at a current position with previously stored feature points on the feature map to detect the position of the mobile robot".

6.2 This additional feature is however disclosed as part of the SLAM schematic of E15, Figure 5.38 (cf. the "Matching" processing loop in the bottom left-hand quarter of the figure), as was already pointed out by the opposition division (cf. appealed decision, Reasons, 12.1, starting from "The additional feature I ..."). This feature therefore also does not contribute to inventive step.

6.3 Consequently, the subject-matter of claim 1 of auxiliary request 4 does not involve an inventive step (Articles 52(1) and 56 EPC) either.

7. Auxiliary request 5 - admittance (Article 12(4) RPBA 2007)

7.1 This auxiliary request has been filed for the first time in the appeal proceedings. Its admittance is therefore at the discretion of the board (Article 12(4) RPBA 2007).

7.2 Noting that the request was not convergent with the higher-ranking requests (cf. e.g. T 1903/13, Reasons 3.3.4 ff.), comprised a feature ("... while changing the direction angle at a fixed position of the mobile robot") which had never been previously claimed and thus entailed a "fresh case" (cf. e.g. T 1741/12, Reasons 5), and that the statement of grounds of appeal provided no substantiation as to why the amendments overcame the grounds for the decision (cf. T 217/10, Reasons 5; T 1732/10, Reasons 1.5), the board informed the appellant in their preliminary opinion that they saw no reason to admit that claim request.

7.3 The appellant counter-argued that the request was "not late-filed" because it was filed with the statement of grounds of appeal. Furthermore, the request should be considered because it was an appropriate response to the objections of the opposition division.

7.4 Neither of these reasons is however convincing. Requests filed with the statement of grounds of appeal may be disregarded in accordance with Article 12(4) RPBA 2007, as explained above. Further, if the request were indeed an appropriate response to the objections of the opposition division, it would have been incumbent on the appellant in the statement of grounds of appeal to explain how the objections (especially the objection of lack of inventive step) had been overcome rather than to leave that task to the board. It is noted that even after the board's preliminary opinion, no substantiation with respect to auxiliary request 5 has been submitted.

7.5 For the above reasons, the board did not admit auxiliary request 5 into the proceedings.

8. As there is no allowable claim request, it follows that the appeal must be dismissed.

Order

For these reasons it is decided that:

The appeal is dismissed.