T 0156/09 (Image content search III / SHARP K.K.) of 1.7.2010

Summary of Facts and Submissions

I. European patent application number 01 974 806.0 (publication number EP 1 331 576) concerns a search information transmitting apparatus transmitting or receiving search information for searching for moving image contents.

II. In the course of the examination, the examining division cited, among others, the following prior art documents:

D4: Girardot M. et al: "Efficient representation and streaming of XML content over the Internet medium" IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, vol. 1, 30 July 2000, pages 67-70;

D7: Grosso P. et al: "XML Fragment Interchange" W3C Working Draft, 30 June 1999, pp. 1 to 17 see at <http://www.w3.org/1999/06/WD-xml-fragment-19990630.html>.

The examining division refused the application in oral proceedings held on 17 July 2008. The decision, posted on 20 August 2008, was based on lack of clarity and, for all requests, added subject matter in claim 1. As an "obiter dictum" and "preliminary opinion" the examining division added the further objection that the claims even if amended on the basis of the first embodiment described at pp. 8 to 10 would not meet the requirement of inventive step because there was no technical feature which might distinguish the invention from the closest prior art, document D4.

III. The appellant lodged an appeal against the decision on 22 October 2008, paying the appeal fee on the same day. In a letter dated and filed on 22 December 2008 the appellant submitted amended sets of claims and a statement setting out the grounds of appeal.

IV. In a communication pursuant to Rule 100(2) EPC the Board raised various objections, among others, of lack of novelty and inventive step.

V. On 3 December 2009 and on 6 May 2010 by fax dated 30 April 2010, the appellant filed amended claims as main request and first to third auxiliary requests as well as observations in reply to the communication. According to the appellant the changes to the first auxiliary request over the main request, although minor, allowed for easier reading of the claim. The second auxiliary request was similar to the then existing requests. These requests did not change the "inventive theme". The third request was specifically directed to the second embodiment shown in figure 5 ff. of the application. This second embodiment was based on the first embodiment, but required the presence of first and second information analysing units, first and second filtering units and so on. The working of the two embodiments were similar.

VI. In oral proceedings that took place jointly with cases T 580/07 and T 581/07 on 30 June 2010 and 1 July 2010, the appellant submitted amended claims in accordance with new main and second auxiliary requests. The Board after examination of their admissibility decided not to admit these requests to the proceedings. The matter then still pending before the Board was discussed with the appellant.

VII. According to the appellant's requests as confirmed at the closure of the oral proceedings, the decision under appeal should be set aside and a patent be granted on the basis of claims 1 to 3 filed with letter dated 3 December 2009 (main request) or in the alternative on the basis of one of the set of claims filed with letter dated 30 April 2010 (auxiliary requests 1 to 3).

Claim 1 of the main request reads as follows:

"1. A search information transmitting apparatus transmitting or receiving search information (15) represented by a tree structure for searching for information of moving image contents, said tree structure comprising a root, a plurality of nodes and a plurality of leaves, comprising:

search information analyzing means (101) operable receive as an input and

analyzing said search information and output information of a plurality of search information elements (d) constituting said search information;

extraction processing means (102) for receiving as inputs said information of plurality of search information elements output from said search information analyzing means and extract, from said input plurality of search information elements, two or more said search information elements corresponding to a part of said search information, based an a prescribed condition; and

search information regenerating means (103) for generating a partial tree (151) of said search information represented by said tree structure, and comprising the root and said nodes and leaves corresponding to said two or more search information elements extracted by said extraction processing means."

Claim 1 of the first auxiliary request reads as follows:

"1. A search information transmitting apparatus transmitting or receiving search information (15) represented by a tree structure for searching for information of moving image contents, said tree structure comprising a root, a plurality of nodes and a plurality of leaves, comprising:

search information analyzing means (101) operable to receive and analyze said search information and output information of a plurality of search information elements (d) constituting said search information;

extraction processing means (102) for receiving said information of said plurality of search information elements output from said search information analyzing means and extract, from said plurality of search information elements, two or more said search information elements corresponding to a part of said search information, based an a prescribed condition; and

search information regenerating means (103) for generating a partial tree (151) of said search information represented by said tree structure, wherein said partial tree comprises the root of said tree structure and said nodes and leaves corresponding to said two or more search information elements extracted by said extraction processing means."

Claim 1 of the second auxiliary request reads as follows:

"1. A search information transmitting apparatus operable to transmit or receive search information (15) for searching moving image contents, comprising:

search information analyzing means (101) for receiving and analyzing said search information (15) of said moving image contents represented by a tree structure, and outputting information of a plurality of search information elements (d) constituting said search information, wherein said tree structure includes an original tree having a root, a plurality of nodes and a plurality of leaves;

extraction processing means (102) for receiving said information of said plurality of search information elements output from said search information analyzing means, and extracting part of said plurality of search information elements based on a prescribed condition; and

search information reorganizing means for forming partial search information represented by a partial tree, wherein the partial tree consists of:

the root of the original tree, at least one of the plurality of leaves of the tree structure of said search information (15),

and at least one of the plurality of nodes connecting the root of the original tree and said at least one of the plurality of leaves, based on said part of said plurality of search information elements extracted by said extraction processing means (102)."

Claim 1 of the third auxiliary request reads as follows:

"1. A search information transmitting apparatus operable to transmit or receive a plurality of pieces of search information represented by a tree structure for searching moving image contents, the tree structure comprising:

a root which corresponds to the entire moving image content, a plurality of nodes and a plurality of leaves, the search information transmitting apparatus comprising:

a search information analyzing unit operable to analyze the plurality of pieces of search information, and output a plurality of search information elements constituting the plurality of pieces of search information;

a condition setting unit operable to receive the plurality of search information elements, and set an extracting condition;

an extraction processing unit operable to receive the output from the search information analyzing unit, and extract from the plurality of search information elements part of the search information elements corresponding to the plurality of pieces of search information based on the extracting condition;

a search information regenerating unit operable to generate a partial tree, including the root, based on the search information elements corresponding to each of the extracted plurality of pieces of search information."

VIII. According to the observations and submissions made by the appellant, the search information consisted of a deep tree structure that represented the moving image contents in a hierarchical arrangement of nodes and leaves. The leaves at each level of the hierarchy represented semantic information elements such as the title of the entire moving image contents, the titles of individual scenes etc, or signal information elements such as the colour frequency and the motion intensity. The tree structure was analysed and a part of the search information elements was extracted, based on a prescribed condition as for example the type of elements, to generate a partial tree including the root of the tree structure and the nodes connecting the root to the leaves representing the part of the information elements extracted. Since the partial tree included only information reduced to the needs of the user, the search information could be efficiently transmitted, stored and utilised, saving bandwidth and resources at the transmitting as well as at the receiving side of the system.

The present invention was clearly distinguished from the Millau streaming system disclosed in document D4. The example of a structured multimedia document in section 3.2 showed a single segment related to the film "Gone with the Wind", having a shallow structure and only a single type of information element, namely the title of the segment. There was no clear disclosure of the structure of the multimedia document and no indication that the example lines represented more than a single segment. The example displayed actually was the segment produced as search result in searching for the title "Dinner scene".

The segment was essentially different from the search information elements extracted by the extraction processing means and the partial tree generated by the search information regenerating means according to the present invention; the examples of the search information elements extracted and the partial tree shown in figures 3 and 4, respectively, of the application clearly illustrated the differences.

Document D4 was also silent about any kind of prescribed extracting condition. The user merely issued a query to filter the multimedia stream, whereas the present invention generated a reduced search information tree before the search was conducted, thereby increasing the efficiency and speed of transmitting the search information.

Document D4 did not anticipate any of the essential features of the invention and did thus not support any of the objections concerning novelty or inventive step.

Reasons for the Decision

1. The appeal is admissible. The appeal, however, cannot be allowed since on the basis of the requests pending the application does not pass scrutiny under the patentability requirements.

2. The new main and second auxiliary requests submitted for the first time in the oral proceedings (see VI. above) have been filed late. The decision on admitting such late filed requests requires the balance between the merits of the case and the need for procedural economy. Considering that the amendments requested have no clear relevance for the questions in issue and moreover prima facie might introduce new subject matter into the application, the Board decides not to admit the amended claims to the proceedings.

3. The requests submitted in due time (see VII above) are not allowable for lack of patentability of the claimed invention. In the light of prior art document D4, the main request, the first auxiliary request and the second auxiliary request do not meet the requirement of novelty, Article 54(1) and (2) EPC 1973, whereas the third auxiliary request does not meet the requirement of inventive step, Article 56 EPC 1973.

Regarding the first group of requests, it is sufficient to consider only one of these requests in detail since the claims differ only slightly in wording but not in substance (see also V above). Claim 1 of the second auxiliary request is somewhat more precise than claim 1 of the other requests and is thus taken as basis for the following considerations. The conclusions, however, hold also for the preceding requests.

4. Second auxiliary request

4.1 It is common ground that document D4 is the closest prior art and a suitable starting point for assessing novelty and inventive step. The subject matter of claim 1 of the second auxiliary request is fully anticipated by this piece of prior art.

4.2 Document D4 discloses a so-called Millau streaming system based on a client-server architecture for efficient encoding and streaming of structured XML documents with text or multimedia data (see e.g. the Abstract). As described in section 3.2 The Millau browser at page 68 ff. of document D4 the system comprises sender and receiver and provides tools for browsing multimedia documents containing multimedia data like video or audio. The Millau system allows to exchange and stream such multimedia documents over the Internet and to search for relevant information in the document. Such a system meets the definition of a search information transmitting apparatus operable to transmit or receive search information for searching moving image contents, the generic object of claim 1.

4.3 For browsing large multimedia documents and searching for relevant information, document D4 proposes as solution to add an XML structure to the multimedia data, creating a "structured multimedia document" that can be streamed and browsed using tools of the Millau system (ibid). An "example" of a structured multimedia document for the movie "Gone with the Wind" and a segment titled "Dinner scene" is given at p. 68, right-hand column, lines 29-41. The XML document has a tree structure comprising a root (MOVIE), a plurality of nodes (SEGMENT ID = "SEGMENT #1", SHOT ID = "SHOT #1"), and a plurality of leaves (end nodes TITLE, KEY_FRAME, AUDIO and VIDEO). The exemplary XML document is thus search information in the format defined by the second paragraph of claim 1.

4.4 The structured multimedia document to which the "example" refers can be "streamed and browsed" (D4, p. 68, right-hand column, line 29 ff.). Therefore, the appellant's argument that document D4 did not disclose a full tree structure but merely displayed the search result for the word "dinner", and thus only a single segment, cannot be accepted by the Board. The tree structure of the multimedia document is immediately clear from the purpose and context of the "example" described in section 3.2 of document D4.

4.5 The remark at p. 69, left-hand column, line 25 ff. that "[t]he receiver could possibly find another `diner [sic] scene´ in the structure" implies that the structured multimedia document of the "example" includes more than a single segment #1. It would indeed be unreasonable to structure a movie like "Gone with the Wind" by using a single segment and a single shot. The skilled reader would immediately understand from the document that the "example" did not reproduce the complete XML structure of the multimedia document but a typical component of this structure, the complete structure comprising the root MOVIE, a sequence of child nodes of the type SEGMENT, and for each segment an end node or leaf of the type TITLE and a sequence of nodes of the type SHOT connected to end nodes or leaves of the types KEY_FRAME, AUDIO, and VIDEO. The full XML structure meets the definition of search information in terms of the second paragraph of claim 1.

4.6 As disclosed in document D4 (see e.g. sections 2.2 and 3.1 together with page 69, line 10 ff.) the sender of the Millau streaming system comprises a Millau parser that analyses the input stream received, for example for special global tokens, and outputs a plurality of XML elements transmitted on a separate structure stream. Operating on a structured multimedia document as illustrated by the "example" (see above), the Millau parser processes search information of moving image contents and outputs the XML elements, the search information elements constituting the search information; the system thus meets the definition of the search information analysing means of the second paragraph of claim 1.

4.7 As disclosed in the Abstract and section 2.3, for example, the sender in the Millau system can break down and reorder an XML document into "fragments". A fragment in the context of the XML standard is "a general term to refer to part of an XML document, plus possibly some extra information, that may be useful to use and interchange in the absence of the rest of the XML document" (see document D7, section 3 at page 3). In the Millau system the fragments consist of parts of the original XML document, individual nodes or subtrees, which may be transmitted in some order predecided between the client and the server. The structure of these partial trees or subtrees and the order of transmission are predetermined according to the user's preferences or the browser's capabilities so as to allow the system to transmit the most important information first and to delay or discard the less important information.

4.8 The fragmentation of an XML document requires an information extraction and regenerating process, namely selecting and extracting the XML components representing the most important information from the original XML document and reassembling the components extracted into the pre-specified partial trees or subtrees.

4.9 For the "example" (ibid.), the partial tree resulting from such an extraction and reassembly process is illustrated by the "first level structure stream" shown at p. 68, right-hand column, line 44 to p. 69, left-hand column, line 3. This partial tree consists of the root MOVIE, i.e. the root of the original tree, the leaf TITLE, i.e. one of the leaves of the original tree structure, and a node SEGMENT connecting the root MOVIE and the leaf TITLE. It is clear from the purpose and context of the example that the full structure of the "first level structure stream" comprises a sequence of segments (for example segment #1, segment #2, etc).

4.10 Moreover, the fragmentation and the reordering of the multimedia components are based on selections made by the user (see for example document D4, the Abstract or at p. 68, right-hand column, line 27 ff.). It can thus be said that the extraction process is based on a "prescribed condition".

4.11 Therefore, the Millau streaming system also meets the definition of the extraction processing means and search information reorganizing means in the third and fourth paragraphs of claim 1.

4.12 It follows that all features of claim 1 are anticipated by the Millau streaming system of document D4 disclosing a Millau browser for browsing and searching the structured multimedia document illustrated by the "example" at p. 68 f.

5. Therefore, the second auxiliary request, and accordingly the main request and first auxiliary request as well, do not meet the requirement of novelty as set out in Article 52(1) EPC and Article 54(1) and (2) EPC 1973.

6. Third auxiliary request

6.1 The third auxiliary request does not meet the requirement of inventive step. Claim 1 of this request differs from claim 1 of the preceding requests essentially in the features that the search information transmitting apparatus is operable to transmit or receive "a plurality of pieces of search information" and that the apparatus comprises "a condition setting unit operable to receive the plurality of search information elements, and set an extracting condition". According to the appellant (see V. above) these features limited the claimed invention to the second embodiment described in relation to figure 5 ff. of the application. The workings of the two embodiments were similar.

6.2 Claim 1 does not clearly define the specific features of the second embodiment, i.e. the parallel arrangement of two analysing, processing, and reorganising units of the first embodiment and a condition setting unit which "collates the result of analysis by first search information analyzing unit 501 with the result of analysis by second search information analyzing unit 504, extracts any common search information element, and applies the same to first filtering processing unit 502 and second filtering processing unit 505" (see the published application, paragraph 0041). If the appellant's interpretation of the third auxiliary request is nevertheless accepted it is still not clear what technical result is achieved by setting a search information element common to two different moving image contents as the extracting condition. Such an unclear feature cannot form a valid basis for inventive step. Therefore, the subject matter of claim 1 is considered not to involve an inventive step.

6.3 It could be added that a technical problem which might be in the background of such a double arrangement is speeding up the search process. Parallel processing, however, is a notorious solution for this purpose. It is in particular obvious in the light of the Millau streaming system of document D4 which allows for fragmentation and reordering of multimedia documents, which points to parallel processing of such large documents.

7. In summary, the main request and the first and second auxiliary requests do not meet the requirement of novelty ((Article 52(1) EPC and Article 54(1) and (2) EPC 1973), whereas the third auxiliary request does not meet the requirement of inventive step (Article 52(1) EPC and Article 56 EPC 1973).

ORDER

For these reasons it is decided that:

The appeal is dismissed.