T 1357/08 (Image browser/THOMSON LICENSING) of 1.8.2014

Summary of Facts and Submissions

I. The applicant (appellant) appealed against the decision of the Examining Division to refuse European patent application no. 03730116.5.

II. In the decision under appeal, the Examining Division found that the amendments to claim 1 filed with letter dated 18 January 2008 introduced subject-matter which extended beyond the content of the application as filed (Article 123(2) EPC). In particular, the Examining Division considered that there was no basis in the application for "a means to extract dominant semantic information of said semantic information sub-space defining [...]".

Furthermore, under the heading "Obiter dictum" the Examining Division raised objections under Articles 83, 84 and 56 EPC.

III. With the statement of grounds of appeal, the appellant filed a set of claims 1 to 8 (main request).

IV. In a communication dated 2 January 2014 accompanying the summons to oral proceedings, the Board noted that the wording of some features of claim 1 did not correspond to any of the passages of the description and expressed, inter alia, the preliminary opinion that the claimed subject-matter could not be clearly and unambiguously derived from the application as filed (Article 123(2) EPC).

V. In reply to the Board's communication, the appellant filed, with letter dated 18 July 2014, a new set of claims 1 to 8 as "Auxiliary request".

VI. On 1 August 2014 oral proceedings were held before the Board. During these proceedings, the appellant withdrew the main request and maintained the "Auxiliary request" as sole request.

VII. The appellant requested that the decision under appeal be set aside and that a patent be granted on the basis of the sole request entitled "Auxiliary request" and filed with the letter dated 18 July 2014.

VIII. Claim 1 according to the appellant's request reads as follows:

"Device for creating semantic browsing options comprising

- a means to select an initial set of images from an input database comprising a plurality of images, each image being associated with a semantic information out of a semantic information space,

- a means to determine a semantic information sub-space represented by the semantic information associated with said initial set of images,

characterized in that it comprises

- a means to select a reduced semantic information sub-space by

- making a cluster analysis of the words contained in the semantic information sub-space, by calculation of histograms made on single words or word combinations contained in said semantic information sub-space,

- extracting dominant words or dominant words [sic] combination from said histograms,

- defining the dimensions of the reduced semantic information sub-space as the extracted words and/or word combinations,

said means being further configured to associate with said reduced semantic information sub-space, images of said database upon a distance measure between said images associated semantic information and said reduced sub-space dimensions,

- a means to create image buttons for the images of the database associated with the semantic information of said reduced sub-space,

- a means to display said image buttons,

- a means to display iteratively, upon a selection of an image button,

- a new set of images, these images being the images of the database of which associated semantic information is strong in the same dimension as that one associated with the selected image button,

- image buttons representing the semantic information associated with the reduced semantic information sub-space generated from said new set of images."

Claims 2 to 6 are dependent on claim 1. Independent claim 7 is directed to a method which comprises steps corresponding to the features of claim 1. Claim 8 relates to a computer program product comprising program instructions for executing the steps according to claim 7 when loaded on a computer.

IX. In support of the clarity of the subject-matter of claim 1 the appellant essentially argued that the skilled person would know how to interpret the features of claim 1 and, on the basis of general knowledge common in the field of image browsers, would immediately understand how they should be implemented in the context of the invention. Hence, claim 1 fulfilled the requirements of Article 84 EPC.

Reasons for the Decision

1. The appeal is admissible.

2. The present application relates to a method and a device for creating semantic browsing options which are "represented and displayed by an automatic selection of images that are each representative for a certain semantic class" (application as published, page 2, lines 26 to 27).

In particular, "browsing options are proposed to the user in a visual, implicit manner represented by images such that the user can follow the options without explicit selecting of a semantic class, a category or a theme" (ibid. page 2, lines 29 to 31).

"By clicking on images, the user can be exposed to a visual excitation instead of a reasonable class name. Furthermore, subjective associations that may have nothing in common with the semantic class can stimulate the user to click on an image" (ibid. page 4, lines 23 to 26).

2.1 In other words, the gist of the present invention consists essentially in classifying and indexing images of a database by means of "semantic information" taken from a "semantic information space", and in proposing browsing options in the form of images ("image buttons") representative of certain "dimensions" of a "reduced semantic information sub-space".

Key aspects of the invention relate to the extraction of "dominant semantic dimensions" from an initial set of images of an image database, to the display of browsing options in the form of "image buttons" representative of the dominant semantic information associated with the images of the initial set and to the selection of a further set of images from the database in response to the user's selection of an "image button".

3. The appellant's sole request was submitted with letter dated 18 July 2014, that is more than two weeks after the time limit (one month before the date of the oral proceedings) for filing new requests indicated in the summons to oral proceedings. Hence, it is to be considered as late-filed.

3.1 In the exercise of its discretion under Article 13(1) RPBA, the Board has decided to admit the appellant's late request into the appeal proceedings as it appears to be a justifiable reaction to objections raised in the Board's summons.

4. Claim 1 according to the appellant's request is directed to a device comprising the following features (itemized for the sake of convenience):

(a) a means to select an initial set of images from an input database comprising a plurality of images, each image being associated with a semantic information out of a semantic information space,

(b) a means to determine a semantic information sub-space represented by the semantic information associated with said initial set of images,

(c) a means to select a reduced semantic information sub-space by

(i) making a cluster analysis of the words contained in the semantic information sub-space, by calculation of histograms made on single words or word combinations contained in said semantic information sub-space,

(ii) extracting dominant words or dominant words [sic] combination from said histograms,

(iii) defining the dimensions of the reduced semantic information sub-space as the extracted words and/or word combinations

(d) said means being further configured to associate with said reduced semantic information sub-space images of said database upon a distance measure between said images associated semantic information and said reduced sub-space dimensions,

(e) a means to create image buttons for the images of the database associated with the semantic information of said reduced sub-space,

(f) a means to display said image buttons,

(g) a means to display iteratively, upon a selection of an image button,

(i) a new set of images, these images being the images of the database of which associated semantic information is strong in the same dimension as that one associated with the selected image button,

(ii) image buttons representing the semantic information associated with the reduced semantic information sub-space generated from said new set of images.

5. According to the description (ibid. page 5, lines 16 to 17), the terms "semantic information" and "semantic information space" associated with images of a database are defined by "entities chosen among labels of semantic classes, words and word combinations". Keywords, such as "indoor", "outdoor", "people", "mountain" or "city" are given as examples of "semantic class labels" (ibid. page 8, lines 9 to 10).

Thus, "single words" and "word combinations" constitute the semantic information relevant to the present invention and, in particular, the dimensions of the "semantic information space" associated with the image database.

5.1 This interpretation of "semantic information" and "semantic information space" is consistent with feature (b) and features (c)(i), (ii) and (iii) of claim 1 which specify that "a semantic information sub-space" is represented by the semantic information associated with an initial set of images, and that "a reduced information sub-space" is selected on the basis of a cluster analysis of the words contained in the semantic information sub-space, by extracting dominant words or word combinations and by using them to define the dimensions of the reduced semantic information sub-space.

6. Feature (d) relates to the selection of images from the database to be associated with the reduced semantic information sub-space, and therefore deals with one of the key aspects of the present invention.

According to this feature, the "association" of images with the "reduced semantic information sub-space" is performed "upon a distance measure between said images associated semantic information and said reduced sub-space dimensions" (underlining added).

6.1 As pointed out above [see point 5.1], "words" and/or "word combinations" represent the dimensions of the semantic information space and the semantic information associated with each image. Thus, according to feature (d), images are selected on the basis of a "distance measure" between words or word combinations. As this does not appear to imply a clearly defined parameter, it is important to investigate how feature (d) is supposed to be implemented in the context of the invention.

6.2 The present application refers to a module 4 (cf. Figure 1) "of creation of semantic links" which is connected with a module 3 (responsible for the creation of the initial set of images) and selects the reduced semantic information sub-space. Module 4 creates "semantic links" to images of the database "of which associated semantic information is strong in the same dimension as that one of said reduced sub-space" (ibid. page 10, lines 3 to 5, underlining added). In particular the description specifies that the "images of the database, which correspond best to the reduced sub-space, are extracted from the database. The correspondence can be formulated by a distance measure and can be implemented in various manners" (ibid. page 10, lines 6 to 9, underlining added).

Furthermore, according to the preferred embodiment, "the correspondence is made by selecting the images of the database, of which the associated semantic information is strong in all dimensions of said reduced sub-space. If the number of images to be displayed is too high, a random selection or a ranking can be made. The distance measure may contain the notation of precision using at least one parameter that defines the relevance of the selected images with respect to the search keywords" (ibid. page 10, lines 11 - 17, underlining added).

6.3 Hence, the criteria for selecting an image from the database which can be derived from the application may be summed up as follows:

- the semantic information associated with a selected image should be "strong" in the same dimension as the one of the reduced semantic information sub-space;

- the images to be extracted from the database "correspond best" to the reduced semantic information sub-space;

- the correspondence can be formulated by "a distance measure" and can be implemented in "various manners";

- the correspondence is made by selecting the images whose associated semantic information is "strong in all dimensions" of said reduced semantic information sub-space;

- the distance measure may contain "the notation of precision" using at least one parameter that defines the relevance of the selected images with respect to search keywords.

6.4 The description does not give any definition of "distance measure". For semantic information and dimensions of a semantic information space expressed in terms of words or word combinations, a distance measure could simply indicate whether one or more dimensions (words) of the semantic information sub-space are associated with a certain image.

However, "a distance measure" based simply on the number of keywords an image has in common with a (reduced) semantic information sub-space would give no information as to the "relevance" of an image with respect to a keyword and, in particular, could not contain the "notation of precision" referred to in the description.

6.5 Furthermore, claim 1 relies on the concept of "strength" of semantic information in a certain dimension of a (reduced) semantic information sub-space to specify how images of the database are to be linked to the semantic information associated with a certain image button, and thus what result is achieved by the selection of a "browsing option" (see features (g)(i) and (ii)).

6.6 "Strong" is a relative term which implies that a certain image characteristic or feature may be present in various degrees. A keyword, however, is essentially a "label" that identifies an image as belonging to a certain group of images or as representing a certain theme. Apart from the trivial implication that an image associated with a certain keyword can be qualified as being "strong" in the dimension of the reduced sub-space identified by such keyword, the description does not clarify how and by what means the "strength" of an image with respect to a "semantic dimension" (i.e. "keyword") should be assessed (cf. feature (g)(i) of claim 1).

7. The appellant has essentially argued that a person skilled in the art would have known at the priority data of the present application how to implement the claimed device and in particular features (d) and (g)(i) of claim 1.

7.1 In the appellant's view, the skilled person would have understood that known techniques of image analysis and image segmentation would have made it possible to associate certain pixel areas with certain words or word combinations of the semantic information space. For instance, it was common knowledge that the image area occupied by a certain subject (i.e. mountains) could be used as a measure of the relevance of the image to the corresponding keyword ("mountain").

7.2 Furthermore, according to the appellant, the skilled person could also have envisaged an implementation of the claimed device whereby the user, when setting up an image database, would visually evaluate the relevance of an image with respect to certain keywords and quantify it by means of numerical parameters.

8. Although the appellant's examples may be regarded as plausible implementations of some essential features of the invention, there is no hint in the application that would direct the skilled person to implementing the claimed device as suggested by the appellant.

On the contrary, the application appears to discourage implementations of the invention that would involve any image processing. As specified on page 11, lines 27 to 31, in "existing relevance feedback systems, image search is formulated by visual similarity using measures of color, texture image layout or shape. In this invention, the search is based on textural metadata as for instance the words "people" or "mountain". The use of textural metadata can reduce drastically the computational cost of search formulation and retrieval".

8.1 Furthermore, all the passages of the application providing examples of "semantic information" and of the selection of images appear to indicate that the image content is only expressed in terms of keywords and that the selection of the images to be associated with a reduced semantic information sub-space is only implemented in terms of keywords (see application as published, page 12, lines 1 to 15).

8.2 In other words, the Board does not contest the appellant's argument that a skilled person, relying on general knowledge common in the field of image browsing at the priority data of the application, might have arrived at a viable implementation of the features of the claimed device. However, as pointed out above, the implementations suggested by the appellant find no support in the application as filed and even appear to be incompatible with the aim of the invention to provide an image browser directed to the inexperienced user and which can drastically reduce computational costs (cf. ibid. page 1, lines 16 to 17 and 29 to 30, and page 11, lines 27 to 31).

On the other hand, implementations of the invention relying only on keywords for the selection of images to be associated with a certain semantic information sub-space or with the image buttons, which would in principle be supported by the description, fail to shed any light on the meaning to be attributed to features (d) and (g) (i) of claim 1 in the context of the claimed device.

8.3 Finally, the Board notes that the present application comprises only a schematic description of possible embodiments of the invention, but gives no details as to how some essential features of the invention could be implemented. As acknowledged by the appellant, this excludes the possibility of clarifying claim 1 without violating Article 123(2) EPC.

8.4 In summary, the Board comes to the conclusion that claim 1 of the appellant's request does not satisfy the requirements of Article 84 EPC since some of its features are neither clear nor supported by the description.

9. As the appellant's sole request is not allowable, the appeal has to be dismissed.

Order

For these reasons it is decided that:

The appeal is dismissed.