T 0300/22 (Augmenting a simulation model/STEINBEIS INTERAGIERENDE SYSTEME) of 26.4.2024

Summary of Facts and Submissions

I. The appeal is against the decision of the examining division issued on 16 July 2021 to refuse European patent application no. 17 202 216.2.

II. The examining division refused the application - after a sole communication pursuant to Article 94(3) EPC (here­inafter: "the communication of the examining division") - on the basis that claim 1 of the sole request then on file did not meet the requirements of Article 123(2) EPC and claims 1 to 4 and 9 did not meet the requirements of Article 84 EPC. The decision cites documents D1 to D3 that were referred to in the European Search Opinion but they were not relied upon in the reasoning of the examining division.

III. With the statement of grounds of appeal, the appellant requested that the decision of the examining division be set aside and that a patent be granted on the basis of a new set of claims 1-7 filed with the statement.

In support of its position on Article 84 EPC, the appellant submitted the following documents with the statement of grounds of appeal:

NPL1: Centre for Connected and Autonomous Vehicles,

"BSI Connected and Automated vehicles -

Vocabulary", BSI Flex 1890 v3.0:2020-10, 2020,

NPL2: M. Hekmatnejad et al., "Encoding and

monitoring responsibility sensitive safety

rules for automated vehicles in signal

temporal logic", Proceedings of MEMOCODE'19,

9-11 October 2019, La Jolla, CA, USA,

NPL3: Mitre, Systems Engineering Guide, 2014.

IV. In a communication sent with the summons to oral proceedings, the board presented its preliminary opinion on the appeal. The board also introduced the following documents, Article 114(1) EPC:

D4: W. Wachenfeld et al., "Virtual assessment of

automation in field operation - A new runtime

validation method", 10. Workshop

Fahrerassistenzsysteme, 2015, XP093130401,

D5: U. Lages et al., "Automatic scenario generation

based on laser scanner reference data and

ad­vanced offline processing", 2013 IEEE

Intelli­gent Vehicles Symposium (IV), 23-26 June

2013, Gold Coast, Australia, pages 153-155,

XP032501963.

D4 is the document cited in paragraph [0005] of the present application; D5 is cited on page 2 of D4 as reference [6].

V. With a letter dated 30 March 2024, the appellant filed new sets of claims for auxiliary requests 1 to 3, reques­ted a full or partial reimbursement of the appeal fee in view of the infringement of its right to be heard, and submitted a further document in support of its position on Article 84 EPC:

D6: R. Jiang et al., "Ego-vehicle corridors for

vision-based driver assistance", IWCIA 2009,

LNCS 5852, pages 238 and 239.

VI. At the oral proceedings, the appellant requested that the decision under appeal be set aside and that a patent be granted on the basis of claims 1 to 7 as filed with the statement of grounds of appeal (main request), or on the basis of the claims of auxiliary requests 1 to 3 filed with letter dated 30 March 2024. The appellant also requested remittal of the case to the examining division and reimbursement of the appeal fee due to an alleged substantial procedural violation. At the end of the oral proceedings, the chairman announced the decision of the board.

VII. Claim 1 of the main request reads as follows:

"Computer-implemented method of augmenting a simulation model of a physical environment of an ego-vehicle,

characterized in

establishing a deficiency of the simulation model, the deficiency pertaining to a certain setting within the physical environment,

issuing instructions for capturing a scan of the setting,

receiving the scan, and

remedying the deficiency by complementing the simulation model with the scan,

wherein

the setting is defined in terms of a certain location, terrain, weather conditions, and physical object present within the setting and

the instructions call for certain sensor hardware to be employed in the capturing and certain boundary conditions to be met by the scan, such as relative height, speed, angle, or distance between the hardware and the object."

VIII. Claim 1 of auxiliary request 1 differs from claim 1 of the main request in that it includes the following additional features:

"wherein

the deficiency is established by a client and communicated to a server, by issuing the instructions, preferably through a Web portal, to a community of participants, the server mounts a field campaign among said community to capture the scan, and

upon following the instructions, at least one of the participants uploads the scan to the server".

IX. Claim 1 of auxiliary request 2 differs from claim 1 of auxiliary request 1 in that it includes the following additional features:

"wherein

the hardware comprises at least one primary sensor, such as a radar, lidar, optical, stereo-optical, or ultrasonic sensing system, and auxiliaries, such as a camera, GPS, distance, distance to ground, orientation, temperature, or humidity sensor, preferably integrated in a mobile device such as a smartphone, tablet PC, or optical head-mounted display".

X. Claim 1 of auxiliary request 3 differs from claim 1 of auxiliary request 2 in that it includes the following additional features:

"wherein,

when mounting the campaign, based on availability of prior scans, the server optionally substitutes the setting for a similar setting relevant to the pertinent deficiency".

Reasons for the Decision

The application

1. The application relates generally to the virtual testing of algorithms for an "ego-vehicle" (this term is further addressed below), e.g. advanced driver assistance system (ADAS) algorithms (paragraphs [0003], [0005], [0007]).

2. The virtual testing of such algorithms requires a simulation of the physical environment of the ego-vehicle in various "settings" (paragraph [0007]), where a setting may be defined inter alia in terms of location, terrain, weather conditions and/or a physical object present within the setting (paragraph [0011]).

3. The simulation of the physical environment in a particular setting is based on scans of corresponding real-world situations recorded by real vehicles.

4. A user of such virtual testing tools (e.g. an original equipment manufacturer) may have the wish to perform testing based on a particular setting of the physical environment (e.g. a particular driving route by rain) that is not yet covered by the simulation model or not at the desired model quality (paragraph [0010]). This is referred to in the application as a "deficiency of the simulation model" (paragraph [0011]).

5. The application proposes to provide a web-based portal via which users may communicate such a "deficiency". The portal is also accessed by a community of "people from anywhere in the world who share an interest in seeing or capturing real objects and phenomena from the angle of a machine" (paragraph [0013]).

Via this portal, users may place an "order" (including "instructions") that "pos[es] a challenge to be met among the community by, for example, capturing a scan of the respective object at a certain location with a specific scanning technology" (paragraph [0015]).

Whoever uploads the required scan to the portal may be rewarded after it has been verified that the scan meets the user requirements (paragraph [0015], [0021]).

6. The proposed approach rests on the idea of "decoupling the tasks of simulation or test and scanning of the setting both technically and economically", so that "real settings" may be scanned without expert knowledge in an economically feasible manner (paragraph [0009]).

Main request - Admittance (Article 12(4) RPBA)

7. Claims 1-7 filed with the statement of grounds of appeal differ substantially from the claims on which the contested decision was based.

A number of amendments based on the description, which had been made to claim 1 in reply to the communication of the examining division and had been objected to under Article 123(2) and 84 EPC in the decision, have been taken back in the present claims (e.g. "vehicle" changed back to "ego-vehicle", as in original claim 1), with claim 1 now corresponding to original claims 3.

8. The Board had discretion to admit such an amended set of claims filed for the first time with the statement of appeal under Article 12(4) RPBA.

9. The board notes that the contested decision was taken after a single communication pursuant to Article 94(3) EPC and that the appellant's right to be heard, Article 113(1) EPC, has not been respected at least in regard of the objection under Article 123(2) EPC raised in that decision (point 2).

9.1 In its communication, the examining division maintained all objections under Article 84 EPC that had been raised in the European Search Opinion ("ESOP", see point 1 of the decision), and thus in particular the objection that the term "ego-vehicle" used in original claim 1 was not clear and was therefore "interpreted in its broadest sense as any vehicle" (ESOP, points 2.1.1 and 3.1).

9.2 In reply to that communication, the appellant amended inter alia "ego-vehicle" to "vehicle".

9.3 The objection in the decision - taken directly after that reply - that this amendment infringes the requirements of Article 123(2) EPC because "the description explicitly differentiates between 'ego-vehicles', 'real vehicles', and 'self-driving vehicles', but specifically states that it presents a computer-implemented method of augmenting a simulation model of a physical environment of an 'ego-vehicle' only" (decision, point 2) could not have been expected given the interpretation suggested in the examining division's previous communication, and had not been communicated beforehand to the appellant. The appellant was thus effectively taken by surprise by this objection raised for the first time in the decision, which infringes its right to be heard, Article 113(1) EPC, and constitutes a procedural violation.

10. In view of these circumstances, and because the new set of claims was prima facie compliant with Article 123(2) EPC (see point 12 below), the board exercised its discretion under Article 12(4) EPC to admit this set of claims.

11. At the same time, the board did not consider it appro­priate to remit the case to the first instance without further examination (Article 111(1) EPC and Article 11 RPBA), as the violation of the appellant's right to be heard under Article 123(2) EPC was not causal for the appeal, earlier objections under Article 84 EPC having been maintained, too. Moreover, the validity of the objections in the decision could be readily reviewed on the basis of the new set of claims. The appellant did not object to the board proceeding in that way.

Main request - Article 123(2) EPC

12. The objection under Article 123(2) EPC to the use of the term "vehicle" instead of "ego-vehicle" in claim 1 (decision, point 2) has become moot by the reinsertion of that term in the claim. Claim 1 is a direct combination of original claims 1-3. Claims 2-7 are respectively based on original claims 4, 10, 12-15. Hence, claims 1-7 are compliant with the requirements of Article 123(2) EPC.

Main request - Article 84 EPC

13. Ego-vehicle

13.1 Claim 1 is directed to a "computer-implemented method of augmenting a simulation model of a physical environ­ment of an ego-vehicle", as was original claim 1.

13.2 During the examination proceedings, the examining division argued that the use of the term "ego-vehicle" rendered original claim 1 not clear, Article 84 EPC (see the communication, point 1, referring to the ESOP, points 2.1.1 and 3.1).

13.3 The appellant argued that the term "ego-vehicle" is well understood, in the context of connected and automated vehicles, to mean "subject connected and/or automated vehicle [...], the behaviour of which is of primary interest in testing, trialling or operational scenarios" (as evidenced by the definition given in NPL1, page 4), that is "the vehicle which is under consideration or evaluation" (as evidenced by NPL2, page 2, footnote 1). The term refers to "that vehicle whose environment is being modelled (i.e. the main actor)" (grounds of appeal, page 1). The appellant also pointed to D6, section 1, as showing that the term "ego-vehicle" meant "our car" in the context of improving simulation models for driver assistance systems (letter of 30 March 2024, page 2).

13.4 The board notes that NPL1 and NPL2 were published only after the filing date of the present application. Still, the board tends to agree with the appellant that already before that date the term "ego-vehicle" was commonly used; more specifically, in the context of the testing connected and/or automated vehicles, which is shared by the present application and NPL1, this term seems to have been used as defined in NPL1. D6, pub­lished before the filing date, shows that the term was also used in a similar sense in the context of impro­ving simulation models for driver assistance systems.

Claim 1 however specifies neither testing a vehicle, nor an algorithm for controlling a vehicle, nor im­pro­ving a simulation model for a driver assistance system. The board thus has doubts as to whether the term "ego-vehicle" without this context is clear. However, in view of the inventive step objection addressed below, this issue may be left open.

14. Results to be achieved

14.1 The examining division objected in the decision (points 3 and 4) that the then pending claim 1 "attempt[ed] to define the subject-matter in terms of a result to be achieved", which was only allowable under the conditions elaborated in the Guidelines for Examination in the EPO ("EPO Guidelines", presumably March 2021 edition) F-IV, 4.10. The examining division pointed out two specific "results to be achieved" in claim 1:

(i) "establishing [...] a deficiency of the

simulation model", and

(ii) "remedying the deficiency".

As it was "unclear which technical features [were] necessary for achieving [these] result[s]", claim 1 was considered not to meet the requirements of Article 84 EPC.

Similar objections were raised against the following "results to be achieved" recited in dependent claims:

(iii) establishing a deficiency by a client;

(iv) verifying the deficiency by re-simulating the

setting in an equally defective reference model;

(v) verifying and validating the scan.

14.2 The appellant argued in respect of (i) and (ii) that, "in line with the requirements of T 68/85, suitable technical implementations of both method steps vary considerably in a manner difficult to define whilst still providing the desired effect - namely, that the deficiency is established and remedied", so that "the subject-matter at hand cannot otherwise be defined more precisely without unduly restricting the scope of claim 1". Furthermore, these steps were per se common in the field of systems engineering, as evidenced by NPL3. Similar considerations applied to (iii), which was included in claim 2. The other contested features had been deleted in the new set of claims (grounds of appeal, page 2).

14.3 The board notes that the objections to features (iv) and (v) are indeed moot due to the deletion of the claims including these features.

14.4 The board considers that the examining division's objections to claims 1 and 2 in view of features (i) and (ii) and feature (iii), respectively, are not justified.

First, these features do not specify the technical problem underlying the invention, which is the situation addressed in EPO Guidelines F-IV, 4.10 on "results to be achieved". For this reason alone, reliance on this passage of the Guidelines does not appear appropriate. Furthermore, claims 1 and 2 do not require these steps to be carried out automatically: they could as well be performed by a human at a computer. At least in this case, the skilled person would know, from common general knowledge, many ways how each of (i), (ii) and (iii) could be realised.

These features are broad - which is to be taken into account in the assessment of novelty and inventive step - but not unclear or not supported in the sense of Article 84 EPC.

Main request - Inventive step

15. The appellant noted in the statement of grounds of appeal (pages 2 and 3) that while the contested decision did not address inventive step, objections under Articles 54 and 56 EPC had been raised in the European search opinion (ESOP) and the subsequent examination by the examining division. The appellant then argued why the examining division's interpretation of D1 was erroneous.

In the ESOP it had been considered that some claim features related to a method of doing business and were thus non-technical (ESOP, points 3.2 and 3.3) and objections of lack of novelty and inventive step based on D1, D2 and D3 had been raised against all original claims (points 4 and 5). These objections were maintained in the communication of the examining division (point 3).

In the preliminary opinion of the board, it was noted that the board tended to agree with the appellant's arguments regarding the disclosure of D1 but considered that, in view of the breadth of claim 1, a more fundamental objection of inventive step presented itself, starting from the prior art acknowledged in the application. Reference was made to D4 and D5.

16. D4 is the prior art document cited in paragraph [0005] and further discussed in paragraph [0007] of the present description. D4 discloses on page 2, second complete paragraph, that virtual scenarios for testing ADAS software can be generated by recording real world driving, citing D5 in this respect (publication [6] in D4). D4 states that D5 does not address "how to generate the necessary amount of relevant scenarios" (D4, page 2, second complete paragraph).

17. D5 explains that certain sensor technology mounted on a car may be used as Reference Sensor System (RSS) for automatically generating reference scenarios from laser scan data, The automatically constructed virtual scenarios may then be loaded and used in commercial simulators, such as the PreScan simulation tool for the development and validation of ADAS and active safety systems (D5, abstract, section II, first paragraph, sections III and IV). A scenario involving a traffic sign and an oncoming vehicle is shown as an example virtual scenario that could be generated using RSS and loaded into PreScan (section II, last paragraph, section III; figures 2 and 3).

D5 thus discloses that it was technically possible, at the filing date, to augment a given simulation model for advanced driver assistance systems by a specific virtual scenario generated from laser scan data of a test drive.

18. The board considers that, starting from the disclosure of D5, it would have been obvious to a developer of a simulation software, whenever a particular scenario is found to be missing in a commercial simulation software - be it by the developer or a user of the software, to see to it that a driver is found and asked to record scan data corresponding to the missing scenario by driving a car with an appropriate RSS system, to upload the scan data and to convert it into a virtual scenario that would then be added to the commercial simulation software. Any communication between the involved persons, including the communication of a specification of the desired scenario to the driver, would typically take place electronically.

19. This would amount, in the terms of claim 1, to a "computer-implemented method of augmenting a simulation model of physical environment of an ego-vehicle" (the commercial simulation software to be augmented by the new virtual scenario), comprising "establishing a deficiency of the simulation model, the deficiency pertaining to a certain setting within the physical environment" (identifying a missing scenario in the commercial software, by a developer or a user, where a scenario corresponds to a "setting"), "issuing instructions for capturing a scan of the setting" (commu­nication of a specification of the desired scenario to a driver), "receiving the scan" (upload of the recorded scan data by the driver), and "remedying the deficiency by complementing the simulation model with the scan" (addition of the virtual scenario generated from the scan data to the commercial simulation software).

20. It would be furthermore obvious that the instruction to the driver includes a specification of the desired scenario ("setting" in the terms of claim 1) and of the technology to be used for recording the scan data (e.g. so that a virtual scenario can be generated from it).

The claim feature that "the setting is defined in terms of a certain locations, terrain, weather conditions, and physical object present within the setting" merely defines obvious conditions that would, according to circumstances, be used to specify a desired scenario for testing an ADAS software (a scenario may for instance have to involve rain or a particular traffic sign, hence specify a "physical object" to be present).

The claim feature that "the instructions call for certain sensor hardware to be employed in the capturing and certain boundary conditions to be met by the scan, such as relative height, speed, angle, or distance between the hardware and the object" merely defines obvious ways of specifying means for recording a scenario for testing ADAS software, e.g. using the sensor technology indicated in D5 could be required.

The board considers furthermore that no technical effect is achieved in the context of claim 1 by the particular conditions included in the instruction. First, because they only characterise an instruction to a person (driver), i.e. a presentation of information within the meaning of Article 52(2)(d) EPC. It is also not verified whether the driver complies with the instructions. Secondly, following G 1/19 (see, in particular, reasons 110, 111 and 120), neither simulating a physical environment of an ego-vehicle nor a simulation model for that purpose is technical, and so augmenting a simulation model is also not in itself a technical effect. Features which do not contribute to the technical character of the claimed subject-matter cannot support the presence of an inventive step ("COMVIK approach", see T 641/00, headnote I).

21. The appellant argued that the common approach to generate relevant scenarios was to collect scan data from test drives over a huge number of kilometers, as described in D4, e.g. in section 2, second paragraph, or section 3.4, third sentence. In contrast thereto, the invention allowed the addition of a specific virtual scenario to the simulation model. The technical problem was to overcome the limitations of the prior art mentioned in paragraph [0007] of the description. Furthermore, neither D4 nor D5 discloses the specific "AND"-combinations of conditions regarding scenario ("setting") and hardware in the instruction.

The board is not convinced by these arguments.

First, collecting scan data over a huge number of kilo­meters and generating virtual scenarios therefrom, as described e.g. in D4, aims at covering as many relevant scenarios as possible, including scenarios that neither a developer nor a user would think of but which may still be very relevant because they arise in reality. The claimed method does not achieve the same as that approach, as which scenarios will be covered is entire­ly dependent on the mental decision-making of the per­son carrying out the step of "establishing a defi­cien­cy in the simulation model". It is thus comple­men­tary to that approach, rather than an alternative to it.

Secondly, paragraph [0007] of the description appears to assume that in the prior art a simulation software is limited to the virtual scenarios that have been included in it when the software was released. However, D5 shows that the simulation model of a commercial simulator software may be augmented by later-generated virtual scenarios. The invention does not require adaptation of the simulation software and/or the scan recording technology. The "decoupling" of "the tasks of simulation or test and scanning of the setting", mentioned in paragraph [0009], is already realised in the technology described in D5 and is thus not an contribution of the invention over the prior art.

Thirdly, the argument regarding the "AND"-combination has been addressed in the preceding section (see in particular the last paragraph).

21.1 The board considers thus that claim 1 does not involve an inventive step, Article 52(1) and 56 EPC.

Auxiliary requests 1 to 3 - Admittance (Article 13(2) RPBA)

22. The set of claims for the auxiliary requests 1 to 3 were filed in reply to the board's preliminary opinion, as a reaction to the objection of inventive step that was raised therein. As this objection was based on prior art introduced by the board, the board exercised its discretion under Article 13(2) RPBA to admit auxiliary requests 1 to 3.

Auxiliary request 1 - Inventive step

23. Claim 1 of auxiliary request 1 adds that "the deficiency is established by a client and communicated to a server, by issuing the instructions, preferably through a Web portal, to a community of participants, the server mounts a field campaign among said community to capture the scan, and upon following the instructions, at least one of the participants uploads the scan to the server".

24. The board considers that these features define a particular way of organising the search for a driver (essentially a crowd-sourcing approach), which is a business method within the meaning of Article 52(2)(c) EPC, and an obvious computer-implementation of this non-technical method (using conventional web-based portal to enable an interaction between a user and a community of potential drivers).

Put differently: that a third-party (the one running the web portal) facilitates the interaction of a user of a simulation software, looking for a particular virtual scenario to be added, with a community of potential drivers, for finding a driver who will generate scan data for the desired scenario, is a business idea, hence a non-technical aspect of the claim. An inventive step could only be based on the computer-implementation of this idea, which in the present case (provision of a web portal) is however obvious given the business idea. That the business idea may be considered to be given to the skilled person with the task of finding a suitable computer-implemen­tation is part of the "COMVIK-approach" to the assessment of inventions involving technical and non-technical features (T 641/00, headnote II).

25. Hence, claim 1 of auxiliary request 1 does not involve an inventive step, Article 52(1) and 56 EPC.

Auxiliary request 2 - Inventive step

26. Claim 1 of auxiliary request 2 adds that "the hardware comprises at least one primary sensor, such as a radar, lidar, optical, stereo-optical, or ultrasonic sensing system, and auxiliaries, such as a camera, GPS, distance, distance to ground, orientation, temperature, or humidity sensor, preferably integrated in a mobile device such as a smartphone, tablet PC, or optical head-mounted display".

27. The board considers this feature to merely further specifying the instruction to a driver. The considera­tions of point 20 above apply as well to this feature.

28. Hence, claim 1 of auxiliary request 2 does not involve an inventive step, Article 52(1) and 56 EPC.

Auxiliary request 3 - Inventive step

29. Claim 1 of auxiliary request 3 adds that "when mounting the campaign, based on availability of prior scans, the server optionally substitutes the setting for a similar setting relevant to the pertinent deficiency".

30. The board considers that claim 1 encompass with this feature the possibility that it is a human administra­tor at the server who may identify the possibility and perform the substitution. The board considers that - within the context of the aforementioned business idea - it would have been an obvious (and non-technical) measure to have a human administrator check whether a new scan is actually needed - based on available scans - for the desired virtual scenario, before a driver is sought to record a new scan.

31. The appellant argued that the claim wording required "the server", not a human being, to perform this step. The appellant pointed to paragraph [0014], in which it is said that "the portal checks whether similar scans are already available or whether one scan can be plausibly verified by a second scan".

The board notes first that its reasoning does not assume claim 1 to require a human administrator to perform this step: it is merely considered that claim 1 encompasses such a realisation, and that if that realisation is not inventive, so is the subject-matter of claim 1. As to this interpretation of "the server" in claim 1, the board notes that the claim does not require the server to perform the step automatically: it could be under human control.

As regards the use of the term "the portal" in the description, the board notes that in paragraph [0012] it is said that "from a business perspective, the stakeholder submitting the deficiency would typically be considered the primary client of the portal". The term "portal" is obviously used here not to refer to the server computer as a machine but rather to the party running it. The description also does not describe how the identification of a suitable prior scan could be carried out automatically. Hence, the narrow interpretation suggested by the appellant is also not supported by the description.

32. Hence, claim 1 of auxiliary request 3 does not involve an inventive step, Article 52(1) and 56 EPC.

Reimbursement of the appeal fee

33. As regards the appellant's request for a full or partial reimbursement of the appeal fee in view of an alleged substantial procedural violation, the board notes that Rule 103(1)(a) EPC does not provide for a partial reimbursement and requires the appeal to be allowable for a (full) reimbursement of the appeal fee. The appeal being not allowable, the request for reimbursement of the appeal fee is refused.

Order

For these reasons it is decided that:

The appeal is dismissed.