T 0534/21 (Searching and language translation/VÄÄNÄNEN) of 1.2.2023

Summary of Facts and Submissions

I. The appellant (applicant) appealed against the decision of the examining division refusing European patent application No. 10742816.1, which was published as international application WO 2011/020742.

II. The contested decision cited, inter alia, the following documents:

D1: |US 2008/0319962 A1, 25 December 2008;|

D3: |US 2005/0149498 A1, 7 July 2005; |

D4: |US 2009/0208116 A1, 20 August 2009; |

D6: |US 2008/0226119 A1, 18 September 2008;|

D8: |US 2006/0227992, 12 October 2006; |

D9: |US 6 560 529 B1, 6 May 2003; |

D10:|US 6 266 442 B1, 24 July 2001. |

The examining division decided on a main request and on third, fourth, fifth, sixth and seventh auxiliary requests. First and second auxiliary requests were not on file.

The main request and the fourth, fifth and sixth auxiliary requests did not meet the requirements of Article 84 EPC.

The third auxiliary request did not meet the requirements of Article 123(2) EPC.

The subject-matter of claim 1 of the main request and of the fifth and sixth auxiliary requests lacked inventive step over document D4.

The decision also contained statements to the effect that the third auxiliary request did not meet the requirements of Article 56 EPC in view of document D1, that the fourth auxiliary request did not meet the requirements of Article 56 EPC in view of document D3, and that the fifth and sixth auxiliary requests did not meet the requirements of Article 56 EPC in view of the common general knowledge. However, these statements appear to have considered only the features added in these requests.

The decision further contained statements to the effect that the seventh auxiliary request did not meet the requirements of Article 84 EPC, was not admitted into the proceedings under Rule 137(3) EPC, and did not meet the requirements of Article 56 EPC in view of document D1.

Documents D8, D9 and D10 were referred to as giving examples of certain features.

III. With his statement of grounds of appeal, the appellant filed a revised main request as his highest-ranking request and declared that "[a]ll other pending requests move down one step".

IV. In a communication accompanying the summons to oral proceedings, the board indicated that it considered the appellant's request to be that the decision under appeal be set aside and that a patent be granted on the basis of the claims of the revised main request or, in the alternative, of one of the main request and third to seventh auxiliary requests considered in the decision under appeal. It expressed the preliminary opinion that none of the appellant's requests complied with Articles 84 and 123(2) EPC and that the third to seventh auxiliary requests should not be admitted into the appeal proceedings under Article 12(5) RPBA 2020.

V. With a letter dated 25 January 2023, the appellant submitted the following document:

D14:|R. Belew: "Finding Out About - A Cognitive Perspective on Search Engine Technology and the WWW", Cambridge University Press, 2008, pp. 5-9.|

He requested "appointment of a substantively competent Board, at a level of the [person skilled in the art]" and postponement of the oral proceedings. He mentioned that he did not recall withdrawing requests and that thus "Main, 1-8 Auxiliary Requests" were on file.

VI. Oral proceedings were held as scheduled on 1 February 2023. At the beginning of the oral proceedings, the appellant's requests for a change of composition of the board and postponement of the oral proceedings were discussed and rejected. Subsequently, the appellant confirmed that his substantive requests were as indicated in the board's communication. At the end of the oral proceedings, the chairman announced the board's decision.

VII. The appellant's final requests were that the decision under appeal be set aside and that a patent be granted on the basis of the claims of the revised main request filed with the statement of grounds of appeal or, in the alternative, on the basis of one of the main request and third to seventh auxiliary requests considered in the decision under appeal.

VIII. Claim 1 of the revised main request reads as follows:

"A system comprising a search engine stored in a memory medium of at least one server computer (210, 240) and a mobile computer (210), wherein

-a mixed search input query, comprising at least one text term and at least one image, is extracted from visual photographic data arranged to be provided by a camera of the mobile computer (210), is received by said search engine,

-at least one document is configured to be searched using the said mixed search input query comprising the at least one image and at least one text term with at least one server computer (210, 240) by said search engine,

-most relevant search results obtained with the said mixed search input query are configured to be retrieved by said search engine, said photographic visual data include signs which are read, and said search results being configured to be contextually ranked."

IX. Claim 1 of the main request differs from claim 1 of the revised main request in that instead of the last bullet point it has the following text:

"-the search input query terms are divided to different computers (210),

-most relevant search results obtained with the said mixed search input query are configured to be retrieved from the network to the mobile computer (210) by said search engine, said photographic visual data may include signs which are read, and said search results being configured to be contextually ranked based on data open in an application on the mobile computer (210) of the user."

X. Claim 1 of the third auxiliary request reads as follows:

"A system comprising a search engine stored in a memory medium of at least one server computer (210, 240) and a mobile computer (210), wherein

-a mixed search input query, comprising at least one text term and at least one image, is extracted from visual photographic video data arranged to be provided by a camera of the mobile computer (210), is received by said search engine,

-at least one query term is synonym expanded[,]

-at least one document is configured to be searched using the said synonym expanded mixed search input query comprising the at least one image and at least one text term with at least one server computer (210, 240) by said search engine,

-the search input query terms are divided to different computers (210) by the search engine that are executed by different computers (210),

-most relevant search results obtained with the said mixed search input query are configured to be retrieved from computer network to the mobile computer (210) by said search engine, said photographic video data include signs which are read, and said synonym expanded search results being configured to be contextually ranked based on data open in an application on the mobile computer (210) of the user and contextual terms that are functional or conditional, and that context terms have ontologies."

XI. Claim 1 of the fourth auxiliary request reads as follows:

"A system comprising a search engine stored in a memory medium of at least one server computer (210, 240) and a mobile computer (210), wherein

-a mixed search input query, comprising at least one text term and at least one image, is extracted from visual photographic data arranged to be provided by a camera of the mobile computer (210), is received by said search engine,

-at least one query term is synonym expanded[,]

-at least one document is configured to be searched from the using the said synonym expanded mixed search input query comprising the at least one image and at least one text term with at least one server computer (210, 240) by said search engine,

-the search input query terms are divided to different computers (210),

-most relevant search results obtained with the said mixed search input query are configured to be retrieved from the network to the mobile computer (210) by said search engine, said photographic visual data may include signs which are read, and said synonym expanded search results being configured to be contextually ranked based on data open in an application on the mobile computer (210) of the user and contextual terms that are functional or conditional, and that context terms have ontologies, and

-at least one context ontology is dynamically derived and uses relative weights."

XII. Claim 1 of the fifth auxiliary request differs from claim 1 of the fourth auxiliary request in that the following text has been added at the end of the claim:

", and

-at least one context ontology weight is user assigned."

XIII. Claim 1 of the sixth auxiliary request differs from claim 1 of the fifth auxiliary request in that the word "traffic" has been inserted before "signs which are read" and in that "out loud" has been inserted after "signs which are read".

XIV. Claim 1 of the seventh auxiliary request reads as follows:

"A system comprising a search engine stored in a memory medium of at least one server computer (210, 240) and a mobile computer (210), wherein

-a mixed search input query, comprising at least one text term and at least one image, is extracted from visual photographic data arranged to be provided by a camera of the mobile computer (210), is received by said search engine,

-at least one query term is synonym expanded[,]

-at least one document is configured to be searched using the said synonym expanded mixed search input query comprising the at least one image and at least one text term with at least one server computer (210, 240) by said search engine,

- the mixed search input query is configured to be divided into different search queries by the Internet search engine that are executed by different computers,

-most relevant search results obtained with the said mixed search input query are configured to be retrieved from computer network to the mobile computer (210) by said search engine, said photographic visual data include signs which are read, and said synonym expanded search results being configured to be contextually ranked based on data open in an application on the mobile computer (210) of the user and contextual terms that are functional or conditional, and that context terms have ontologies, and

-at least one context ontology is dynamically derived and uses relative weights, and

-at least one context ontology weight is user assigned."

XV. The appellant's arguments, where relevant to this decision, are discussed in detail below.

Reasons for the Decision

1. Request for a change of the board's composition

1.1 The appellant requested the replacement of all board members because it was apparent from the clarity and added-matter objections raised in the board's communication that its members lacked basic knowledge both of the technical field of the application and of the English language.

1.2 The appellant's allegations, even if assumed to be correct, cannot lead to the conclusion that the board's composition has to be changed. In the present case, the legal basis that the board considers to be potentially applicable for the appellant's request for members of the board to be replaced is Article 24(3) EPC. According to this provision, members of a board of appeal may be objected to by any party for one of the reasons mentioned in Article 24(1) EPC or if suspected of partiality. But the appellant made no reference to Article 24 EPC and argued neither that there was a suspicion of partiality nor that any of the reasons mentioned in Article 24(1) EPC applied.

1.3 Even if the appellant's request for a change of composition had to be regarded as an objection under Article 24(3) EPC, it would have to be dismissed as inadmissible by the board in its original composition for being clearly unsubstantiated (see G 1/21 of 28 May 2021, Reasons 10 and 12). Indeed, without further explanations, which the appellant did not provide, it cannot be seen how a lack of basic knowledge of either the technical field of the application or the English language on the part of a member of the board could give rise to a suspicion of partiality (see T 355/13, Reasons 2.4.3).

1.4 For these reasons, the board rejects the appellant's request for a change of the board's composition.

2. Request for postponement of the oral proceedings

2.1 Since the appellant's request for a change of the board's composition is rejected, it cannot support the appellant's additional request for postponement of the oral proceedings.

2.2 At the oral proceedings, the appellant argued that the board needed more time to study the application and the technical field and that it should therefore not yet take a negative decision on clarity or added subject-matter. The oral proceedings should therefore be postponed.

2.3 Since the board is satisfied that it is in a position to decide on the allowability of the appeal, it rejects the appellant's request for postponement of the oral proceedings.

3. The application relates to data searching and language translation.

Revised main request

4. Admission into the appeal proceedings - Article 12(4) RPBA 2020

Since the amendments made in the revised main request overcome clarity objections which were raised for the first time during the oral proceedings before the examining division, the board has no objection to its admission under Article 12(4) RPBA 2020.

5. Added subject-matter - Article 123(2) EPC

5.1 Claim 1 of the revised main request is directed to a system comprising a search engine stored in a memory medium of at least one server computer and a mobile computer.

5.2 The first bullet point of the claim reads "a mixed search input query, comprising at least one text term and at least one image, is extracted from visual photographic data arranged to be provided by a camera of the mobile computer, is received by said search engine".

Hence a "mixed search input query" which includes at least one text term and at least one image is extracted from "visual photographic data" provided by a camera of the mobile computer.

In addition, the mixed search input query is "received by said search engine", i.e. by the search engine at the server computer.

5.3 The originally-filed claims merely disclose receiving a search query comprising at least one search term. According to the appellant's letter of 8 September 2016 filed with the examining division, the amended feature now forming the first bullet point of claim 1 is based on page 9 of the description, which describes phase 100 of Figure 1. In the appeal proceedings, the appellant did not rely on any other passage of the application as filed.

5.4 The passage on page 9 describing phase 100 of Figure 1 ("input search query") explains that a search query can be input by typing text, by speech, and "with an image when preferably pattern recognition is conducted by the computer to extract the query terms". It gives the following examples of how a user can input the same query:

- the user types "how to ship a box";

- the user says "how to ship a box", and speech recognition recognises the words and uses them as search terms;

- the user takes an image of a ship with a box on board, and pattern recognition software recognises the ship and the box and extracts these as search terms;

- the user takes an image of a sign reading "how to ship a box", and the computer uses pattern or character recognition to identify the characters and thus the words in the sign.

It then explains that these input methods can be used in a mix, for example the user writes "how to" and takes an image of the box on the ship, and the written words "how to" and the pattern-recognised words "ship" and "box" are combined to form the search query.

The passage adds that "[l]ikewise speech and text and image and speech, or in fact all three forms of input, can be used in a mix in accordance with the invention".

Finally, it explains that the terms are arranged in the order in which they arrive or are recognised, or can be ordered by the user, or that the computer can suggest entire search queries to the user based on the recognised search terms.

Hence the description of phase 100 in the application as filed leaves no doubt that the search query, which is input via text, speech, image or a combination of the three, is constructed from a number of words extracted from the text, speech and image input.

5.5 The passage on page 9 does not disclose the claim feature "a mixed search input query, comprising at least one text term and at least one image, is extracted from visual photographic data". If "visual photographic data" obtained from a camera are input, then words, not an image, are extracted from them, and those words are used as search terms.

5.5.1 In this respect, the appellant argued that the passage on page 9 disclosed that an image was taken with a camera, which meant that an image was extracted from the user's surroundings. The passage also disclosed that text was extracted from an image.

However, the claim requires both "at least one text term" and "at least one image" to be extracted from the same "visual photographic data" provided by a camera. This is not what the passage on page 9 discloses.

5.5.2 The appellant also referred to the sentence on page 9 stating that "[l]ikewise speech and text and image and speech, or in fact all three forms of input, can be used in a mix in accordance with the invention". He had selected the subset image+text, which did not extend beyond what was disclosed.

However, the subset image+text corresponds to the example of a user typing the text "how to" and taking an image of a ship with a box on board, from which pattern recognition software then extracts the words "ship" and "box". The sentence referred to by the appellant therefore does not disclose that both a text term and an image are extracted from "visual photographic data" and together make up a "mixed search query".

5.6 Nor does the passage on page 9 disclose the claim feature specifying that "a mixed search input query, comprising at least one text term and at least one image ... is received by said search engine".

5.6.1 As explained above, the passage on page 9 discloses that the search query constructed in phase 100 at the "computer" consists of text terms only and is therefore not a "mixed search query". The "computer" refers to the computer 210 in Figure 2, corresponding to the "mobile computer" of the claim. The search engine at the server 240, corresponding to the "server computer" of the claim, therefore receives a search query consisting of words, not a combination of a text term and an image.

5.6.2 The appellant argued that the board's interpretation that all inputs were text was in contradiction to the statement on page 9 that the input could be a mix of all three forms of input.

However, this misrepresents the board's interpretation of the passage on page 9 as set out in point 5.4 above.

5.6.3 The appellant also argued that mixed-type search queries went through all the phases 100 to 140 of Figure 1 and therefore of necessity ended up at the server 240 in phase 120 or later.

The board agrees with the appellant that the search engine at the server 240 is not involved in phase 100. However, since the query constructed in phase 100 consists of a number of words, no mixed-type search query arrives at the server 240.

5.7 The appellant took the view that the board did not apply the correct standard for the skilled person to whom the application was addressed. At the oral proceedings, the appellant submitted that if the application were given to ten persons having the expertise of the skilled person, more than five of them would build a system implementing search queries that included more than only text.

The board observes that "more than five out of ten" confirms that the standard by which compliance with Article 123(2) EPC is to be judged, namely that of direct and unambiguous disclosure, is not met. The appellant's argument therefore needs no further discussion.

5.8 In sum, the revised main request does not comply with Article 123(2) EPC.

Main request and third to seventh auxiliary requests

6. Admission into the appeal proceedings - Article 12(5) and (6) RPBA 2020

6.1 Under Article 12(5) RPBA 2020, the board has the discretion not to admit any part of a submission by a party, including requests on which the decision under appeal was based, which does not meet the requirements in Article 12(3) RPBA 2020 (see the explanatory remarks in Supplementary publication 2, OJ EPO 2020).

6.2 In its communication, the board observed that the statement of grounds of appeal appeared not to have addressed the objections raised against the third to seventh auxiliary requests in points 25, 28(ii) to (v), 33, 36 and 39 of the reasons for the contested decision, contrary to the requirement of Article 12(3) RPBA 2020 that the statement of grounds of appeal is to contain the appellant's complete case.

6.3 However, in the present case it is procedurally more efficient to deal with the merits of the third to seventh auxiliary requests than to investigate whether the statement of grounds of appeal complies with Article 12(3) RPBA 2020 in respect of these requests.

6.4 Likewise, the board sees no need to investigate under Article 12(6) RPBA 2020 the status of the seventh auxiliary request, which in the contested decision was both "not admitted" under Rule 137(3) EPC and found not to meet the requirements of Article 56 EPC (see point II. above).

6.5 The board therefore considers the third to seventh auxiliary requests, as well as the main request, to be in the appeal proceedings.

7. Added subject-matter - Article 123(2) EPC

7.1 The introductory clause and the first bullet point of claim 1 of the main request and of the fourth to seventh auxiliary requests are identical to the introductory clause and the first bullet point of claim 1 of the revised main request. These claims therefore include the same combination of features objected to in point 5. above under Article 123(2) EPC.

7.2 Claim 1 of the third auxiliary request differs only in that "visual photographic data" has been replaced with "visual photographic video data". This difference has no bearing on the reasoning set out in point 5. above.

7.3 At the oral proceedings, the appellant conceded that the board's objection under Article 123(2) EPC raised against the revised main request also applied to the main request and the third to seventh auxiliary requests.

7.4 Hence the main request and the third to seventh auxiliary requests do not comply with Article 123(2) EPC.

8. Since none of the requests on file is allowable, the appeal is to be dismissed.

Order

For these reasons it is decided that:

The appeal is dismissed.