T 0581/07 (Image content search I / SHARP K.K.) of 1.7.2010

Summary of Facts and Submissions

I. European patent application number 01 976 746.6 (publication number EP 1331 577) concerns a search information managing apparatus for searching for moving image contents.

II. In the course of the examination, the examining division cited, among others, the following prior art documents:

D1: Girardot M. et al: "Efficient representation and streaming of XML content over the Internet medium" IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, vol. 1, 30 July 2000, pages 67-70;

D2: Grosso P. et al: "XML Fragment Interchange" W3C Working Draft, 30 June 1999, pp. 1 to 17 see at <http://www.w3.org/1999/06/WD-xml-fragment-19990630.html>.

The examining division refused the application in oral proceedings held on 26 October 2006. The decision, posted on 17 November 2006, was based for all requests on the objections of lack of clarity, added subject matter and lack of inventive step in the light of document D1 as closest prior art.

III. The appellant lodged an appeal against the decision on 29 December 2006, paying the appeal fee on the same day. In a letter dated and filed on 22 March 2007 the appellant submitted amended sets of claims and a statement setting out the grounds of appeal.

IV. In a communication pursuant to Rule 100(2) EPC the Board gave a negative opinion in particular on novelty and inventive step.

V. On 17 December 2009, the appellant filed an amended main request and amended first and second auxiliary requests as well as observations in reply to the communication. According to the appellant the amendments were all clarifying in nature and served to better demonstrate the differences between the invention and the disclosure of document D1.

VI. In the oral proceedings that took place jointly with cases T 580/07 and T 156/09 on 30 June 2010 and 1 July 2010, the appellant submitted amended claims as new main and second auxiliary requests.

VII. According to the appellant's requests as confirmed at the closure of the oral proceedings, the decision under appeal should be set aside and a patent be granted on the basis of claim 1 according to the main request filed at the oral proceedings or claims 1 to 9 according to the first auxiliary request filed with letter dated 17 December 2009 or claim 1 according to the second auxiliary request filed at the oral proceedings.

VIII. Claim 1 of the first auxiliary request reads as follows:

"1. A search information managing apparatus (1) for managing search information (15), the search information comprising a tree structure corresponding to moving image contents, the apparatus comprising:

separating means (101) for separating <1> and extracting a plurality of search information elements from said search information, wherein each of the search information elements is given by a leaf (RF) of the tree structure, the plurality of search information elements given by the leaves are elements of the same type included in said search information, and nodes directly connected with the leaves are different in the tree structure;

unit of search information forming means (102, 103, 104) operable to form a unit of search information by arranging the plurality of search information elements of the same type extracted by the separating means, separate from search information elements of any other type, in accordance with a prescribed format; and

control means (105) operable to apply the unit of such information to a transmitting processor, a storing processor, or a processing portion for executing a search <2>."

Insertions <1> and <2> are added to indicate passages where the requests differ. The passages inserted are as follows:

In the main request and the second auxiliary request:

1 <the search information into structural search information representing the structure of the moving image contents, semantic search information representing characteristics in semantic context of the moving image contents, and signal search information representing physical characteristics of signals of the moving image contents>.

In the second auxiliary request:

2 <, wherein said prescribed format represents an arrangement of the plurality of search information elements extracted by the separating means, in accordance with an order of appearance representing how the leaves corresponding to the search information elements precede/succeed in the tree structure>.

IX. According to the observations and submissions made by the appellant, the present invention proposed for the first time the novel and inventive concept to handle together, as unit search information, search information elements of the same type from different parts of a tree structure regardless of the time structure of the corresponding moving image contents. By managing search information in this manner, the need to go through and analyse all types of search information elements present was avoided, the computational complexity reduced, and the efficiency of search improved.

The conventional manner of arranging search information was by dividing the search information in accordance with the time structure of the corresponding moving image contents and collecting together the elements succeeding a single parent node. In document D1 there was no clear teaching regarding the structure of multimedia documents and in particular no indication that the example of a structured multimedia document in section 3.2 represented more than a single segment and a single shot. The segment #1 and the shot #1 shown as example were not search information elements corresponding to moving image contents as proposed by the present invention but a piece of data produced as result of a search for the title "Dinner scene".

There was no indication of extracting a plurality of search information elements of the same type from remote positions in the tree structure of the original search information and arranging the elements extracted to form a unit of search information in a prescribed format, which could be stored or used for executing a search. Furthermore, there was no indication to separate the search information into the three types of structural, semantic, and signal search information, nor to arrange the search information elements extracted in accordance with the structure of the original tree structure. All these features of the claimed invention provided a novel and inventive contribution over the prior art.

Reasons for the Decision

1. The appeal is admissible. The appeal, however, cannot be allowed since on the basis of the requests presented the application does not pass scrutiny under the patentability requirements.

2. It is common ground that document D1 is the closest prior art and a suitable starting point for assessing novelty and inventive step. In the light of this piece of prior art, the first auxiliary request does not meet the requirement of novelty, Article 54(1) and (2) EPC 1973, whereas the main request and the second auxiliary request do not meet the requirement of inventive step, Article 56 EPC 1973.

3. First auxiliary request

3.1 Document D1 discloses a so-called Millau streaming system based on a client-server architecture for efficient encoding and streaming of structured XML documents with text or multimedia data (see e.g. the Abstract). As described in section 3.2 The Millau browser at page 68 ff. of document D1 the system comprises sender and receiver and provides tools for browsing and searching for relevant information in a large document containing multimedia data like video or audio. An XML structure is added to the multimedia data, creating a "structured multimedia document" that can be streamed and browsed using tools of the Millau system (ibid).

3.2 An "example" of a structured multimedia document of a segment titled "Dinner scene" for the movie "Gone with the Wind" is given at page 68, right-hand column, lines 29-41. The document in XML format has a tree structure comprising a root (MOVIE), a plurality of nodes (SEGMENT ID = "SEGMENT #1", SHOT ID = "SHOT #1"), and a plurality of leaves (end nodes TITLE, KEY_FRAME, AUDIO and VIDEO). Such a system meets the definition of a search information managing apparatus for managing search information, the search information comprising a tree structure corresponding to moving image contents, the generic object of claim 1 of all requests.

3.3 The structured multimedia document to which the "example" refers can be "streamed and browsed" (D1, page 68, right-hand column, line 29 ff.). Therefore, the appellant's argument that document D1 did not disclose a full tree structure but merely displayed the search result for the word "dinner", and thus only a single segment, cannot be accepted by the Board. The tree structure of the multimedia document is immediately clear from the purpose and context of the "example" described in section 3.2 of document D1.

3.4 The remark at page 69, left-hand column, line 25 ff. that "[t]he receiver could possibly find another `diner [sic] scene´ in the structure" implies that the structured multimedia document of the "example" includes more than a single segment #1. It would indeed be unreasonable to structure a movie like "Gone with the Wind" by using a single segment and a single shot. The skilled reader would immediately understand from the document that the "example" does not reproduce the complete XML structure of the multimedia document but a typical component of this structure, the complete structure comprising the root MOVIE, a sequence of child nodes of the type SEGMENT, and for each segment a leaf (i.e. an end node) of the type TITLE and a sequence of nodes of the type SHOT connected to leaves of the types KEY_FRAME, AUDIO, and VIDEO. The full XML structure meets the definition of search information in terms of the second paragraph of claim 1.

3.5 As disclosed in document D1 (see e.g. sections 2.2 and 3.1 together with page 69, left-hand column, line 10 ff.) the sender of the Millau streaming system comprises a Millau parser that analyses the input stream received, for example for special global tokens, and outputs a plurality of XML elements transmitted on a separate structure stream. Operating on a structured multimedia document as illustrated by the "example" (see above), the Millau parser processes search information of moving image contents and outputs the XML elements, the search information elements constituting the search information.

3.6 As disclosed in the Abstract and section 2.3, for example, the sender of the Millau system can break down and reorder an XML document into "fragments". A fragment in the context of the XML standard is "a general term to refer to part of an XML document, plus possibly some extra information, that may be useful to use and interchange in the absence of the rest of the XML document" (see document D2, section 3 at page 3). In the Millau system the fragments consist of parts of the original XML document, individual nodes or subtrees, which may be "transmitted in some order predecided between the client and the server". The structure of these partial trees or subtrees and the order of transmission are predetermined according to the user's preferences or the browser's capabilities so as to allow the system to transmit the most important information first and to delay or discard the less important information.

3.7 The fragmentation of an XML document requires an information separating process, namely selecting and extracting the XML components representing the most important information from the original XML document, and the reassembling of the components extracted into a unit of search information, forming the pre-specified partial trees or subtrees referred to in section 2.3 of document D1.

3.8 For the "example" (ibid.), the partial tree resulting from such a separating process is illustrated by the "first level structure stream" shown at page 68, right-hand column, line 44 to page 69, left-hand column, line 3. This partial tree consists of the root MOVIE, i.e. the root of the original tree, the leaf TITLE, i.e. one of the leaves of the original tree structure, and a node SEGMENT connecting the root MOVIE and the leaf TITLE. It is clear from the purpose and context of the example that the full structure of the "first level structure stream" comprises a sequence of segments (for example Segment #1, Segment #2, etc). The leaves of this structure, the TITLE elements, are elements of the same type. The nodes directly connected with these leaves, i.e. the SEGMENT elements Segment #1, Segment #2, etc, are different in the tree structure.

3.9 Therefore, the Millau browser and streaming system configured to stream and browse the "example" of a structured multimedia document in section 3.2 meet the definitions of the separating means in the second paragraph and the unit of search information forming means in the third paragraph of claim 1.

3.10 Finally, the receiver executes a search ("will look for the word diner ...") and keeps the fragments stored in memory so that the prior art system also meets the definition of control means in the last paragraph of claim 1.

3.11 For these reasons, the subject matter of claim 1 of the first auxiliary request lacks novelty (Article 52(1) EPC and Article 54(1) and (2) EPC 1973).

4. Main request

4.1 The main request does not meet the requirement of inventive step. Claim 1 of this request differs from claim 1 of the first auxiliary request in the additional feature that the search information is separated into structural search information, semantic search information, and signal search information representing physical characteristics of signals of the moving image contents. Indeed, document D1 does not describe physical characteristics of the signals of the moving image contents, whereas in D1 the segment ID and the TITLE elements are structural search information and semantic search information, respectively, in terms of claim 1 of the main request.

4.2 However, the wish to be able to search for physical characteristics of signals is of a non-technical character. Once this aim has been stated its technical implementation, for example by introducing corresponding information elements into the XML structure of the multimedia document and processing such elements in the same way as the structural and semantic information elements, is straightforward and does not involve an inventive step.

5. Second auxiliary request

5.1 The second auxiliary request does not meet the requirement of inventive step either. Claim 1 of this request differs from claim 1 of the main request only by the feature that the arrangement of search information elements is in accordance with the order in which the search information elements precede/succeed in the tree structure.

5.2 Since in document D1 the information elements are arranged in a "first level structure stream" the arrangement is automatically in accordance with the appearance of the elements in the original tree structure. This additional feature is hence anticipated by document D1 and can thus not establish patentability of the invention.

6. In summary, the main request and the second auxiliary request do not meet the requirement of inventive step (Article 52(1) EPC and Article 56 EPC 1973).

ORDER

For these reasons it is decided that:

The appeal is dismissed.