T 0580/07 (Image content search II / SHARP K.K.) of 1.7.2010

Summary of Facts and Submissions

I. European patent application number 01 976 747.4 (publication number EP 1 335 302) concerns a search information managing apparatus for searching for moving image contents.

II. In the course of the examination, the examining division cited, among others, the following prior art documents:

D1: Girardot M. et al: "Efficient representation and streaming of XML content over the Internet medium" IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, vol. 1, 30 July 2000, pages 67-70;

D2: Grosso P. et al: "XML Fragment Interchange" W3C Working Draft, 30 June 1999, pp. 1 to 17 see at <http://www.w3.org/1999/06/WD-xml-fragment-19990630.html>.

The examining division refused the application in oral proceedings held on 27 October 2006. The decision, posted on 17 November 2006, was based for all requests on the objections lack of clarity, added subject matter and lack of inventive step in the light of document D1 as closest prior art.

III. The appellant lodged an appeal against the decision on 29 December 2006, paying the appeal fee on the same day. In a letter dated and filed on 22 March 2007 the appellant submitted amended sets of claims and a statement setting out the grounds of appeal.

IV. In a communication pursuant to Rule 100(2) EPC the Board gave a negative opinion in particular on novelty and inventive step.

V. On 17 December 2009, the appellant filed an amended main request and amended first and second auxiliary requests as well as observations in reply to the communication. According to the appellant the amendments were all clarifying in nature and served to better demonstrate the differences between the invention and the disclosure of document D1. Further amendments were filed by the letter dated 26 April 2010.

VI. In the oral proceedings that took place jointly with cases T 581/07 and T 156/09 on 30 June 2010 and 1 July 2010, the appellant submitted amended claims according to new main and first auxiliary requests. The Board after examination of their admissibility decided not to admit the requests to the proceedings.

VII. According to the appellant's requests as confirmed at the closure of the oral proceedings, the decision under appeal should be set aside and a patent be granted on the basis of the claims according to the main or second auxiliary requests filed with letter dated 17 December 2009 or the first or third auxiliary requests filed with letter dated 26 April 2010.

Claim 1 of the main request reads as follows:

"1. A search information managing apparatus (1) managing search information (15), having a tree structure corresponding to moving image contents, comprising:

separating means (101) for separating and drawing out a plurality of search information elements from the search information, wherein each of the search information elements is given by a leaf (RF) of the tree structure, the <1> search information elements given by the leaves are elements of the same type included in said search information, and nodes directly connected with the leaves are different in the tree structure;

structural formation extracting means (102) for extracting, from said search information, structural information (50) corresponding to said plurality of search information elements drawn out by said separating means <2>; and

unit of search information forming means (103, 104) being operable to form a unit of search information by arranging the plurality of search information elements of the same type drawn out by said separating means and the corresponding structural information extracted by said structural information extracting means, separate from search information elements of any other type and their corresponding structural information, in accordance with a prescribed format."

Insertions <1> and <2> are added to indicate passages where the requests differ. The passages inserted are as follows:

In all auxiliary requests:

1 <plurality of>.

In the first and third auxiliary requests only:

2 <, wherein the structural information includes information representing a position before separation, of the search information elements drawn out by said separating means, in said tree structure of said search information>.

VIII. According to the observations and submissions made by the appellant, the present invention proposed for the first time the novel and inventive concept to handle together, as unit search information, search information elements of the same type from different parts of a tree structure regardless of the time structure of the corresponding moving image contents. By managing search information in this manner, the need to go through and analyse all types of search information elements present was avoided, the computational complexity reduced, and the efficiency of search improved.

Additionally, by including structural information and in particular information representing a position before separation of the search information elements in the units of search information, the search information as well as the search results may be recovered in the original tree structure.

The conventional manner of arranging search information was by dividing the search information in accordance with the time structure of the corresponding moving image contents and collecting together the elements succeeding a single parent node. In document D1, there was no clear teaching regarding the structure of multimedia documents and in particular no indication that the example of a structured multimedia document in section 3.2 represented more than a single segment and a single shot. The segment #1 and the shot #1 shown as example were not search information elements corresponding to moving image contents as proposed by the present invention but a piece of data produced as result of a search for the title "Dinner scene".

There was no indication of extracting a plurality of search information elements of the same type from remote positions in the tree structure of the original search information and arranging the elements extracted to form a unit of search information in a prescribed format, which could be stored or used for executing a search. Furthermore, there was no indication to rearrange the search information elements extracted in accordance with the structure of the original tree structure. All these features of the claimed invention provided a novel and inventive contribution over the prior art.

Reasons for the Decision

1. The appeal is admissible. The appeal, however, cannot be allowed since on the basis of the requests pending before the Board the application does not pass scrutiny under the patentability requirement of novelty.

2. The new main and first auxiliary requests submitted for the first time in the oral proceedings (see VI above) have been filed late. The decision on admitting such late filed requests requires the balance between the merits of the case and the need for procedural economy. Considering that the amendments requested have no clear relevance for the questions in issue and moreover prima facie might introduce new subject matter into the application, the Board decides not to admit the amended claims to the proceedings.

3. Document D1 discloses a so-called Millau streaming system based on a client-server architecture for efficient encoding and streaming of structured XML documents with text or multimedia data (see e.g. the Abstract). As described in section 3.2 The Millau browser at page 68 ff. of document D1 the system comprises sender and receiver and provides tools for browsing and searching for relevant information in a large document containing multimedia data like video or audio. An XML structure is added to the multimedia data, creating a "structured multimedia document" that can be streamed and browsed using tools of the Millau system (ibid).

4. An "example" of a structured multimedia document of a segment titled "Dinner scene" for the movie "Gone with the Wind" is given at page 68, right-hand column, lines 29-41. The document in XML format has a tree structure comprising a root (MOVIE), a plurality of nodes (SEGMENT ID = "SEGMENT #1", SHOT ID = "SHOT #1"), and a plurality of leaves (end nodes TITLE, KEY_FRAME, AUDIO and VIDEO). Such a system meets the definition of a search information managing apparatus managing search information having a tree structure corresponding to moving image contents, the generic object of claim 1 of all requests.

5. The structured multimedia document to which the "example" refers can be "streamed and browsed" (D1, page 68, right-hand column, line 29 ff.). Therefore, the appellant's argument that document D1 did not disclose a full tree structure but merely displayed the search result for the word "dinner", and thus only a single segment, cannot be accepted by the Board. The tree structure of the multimedia document is immediately clear from the purpose and context of the "example" described in section 3.2 of document D1.

6. The remark at page 69, left-hand column, line 25 ff. that "[t]he receiver could possibly find another `diner [sic] scene´ in the structure" implies that the structured multimedia document of the "example" includes more than a single segment #1. It would indeed be unreasonable to structure a movie like "Gone with the Wind" by using a single segment and a single shot. The skilled reader would immediately understand from the document that the "example" does not reproduce the complete XML structure of the multimedia document but a typical component of this structure, the complete structure comprising the root MOVIE, a sequence of child nodes of the type SEGMENT, and for each segment an end node (leaf) of the type TITLE and a sequence of nodes of the type SHOT connected to end nodes or leaves of the types KEY_FRAME, AUDIO, and VIDEO.

7. As disclosed in document D1 (see e.g. sections 2.2 and 3.1 together with page 69, left-hand column, line 10 ff.) the sender of the Millau streaming system comprises a Millau parser that analyses the input stream received, for example for special global tokens, and outputs a plurality of XML elements transmitted on a separate structure stream. Operating on a structured multimedia document as illustrated by the "example" (see above), the Millau parser processes search information of moving image contents and outputs the XML elements, the search information elements constituting the search information.

8. As disclosed in the Abstract and section 2.3, for example, the sender of the Millau system can break down and reorder an XML document into "fragments". A fragment in the context of the XML standard is "a general term to refer to part of an XML document, plus possibly some extra information, that may be useful to use and interchange in the absence of the rest of the XML document" (see document D2, section 3 at page 3). In the Millau system the fragments consist of parts of the original XML document, individual nodes or subtrees, which may be transmitted in some order predecided between the client and the server. The structure of these partial trees or subtrees and the order of transmission are predetermined according to the user's preferences or the browser's capabilities so as to allow the system to transmit the most important information first and to delay or discard the less important information.

9. The fragmentation of an XML document requires an information separating process, namely selecting and extracting the XML components representing the most important information from the original XML document, and the reassembling of the components extracted into a unit of search information, forming the pre-specified partial trees or subtrees referred to in section 2.3 of document D1.

10. For the "example" (ibid.), the partial tree resulting from such a separating process is illustrated by the "first level structure stream" shown at page 68, right-hand column, line 44 to page 69, left-hand column, line 3. This partial tree consists of the root MOVIE, i.e. the root of the original tree, the leaf TITLE, i.e. one of the leaves of the original tree structure, and a node SEGMENT connecting the root MOVIE and the leaf TITLE. It is clear from the purpose and context of the example that the full structure of the "first level structure stream" comprises a sequence of segments (for example Segment #1, Segment #2, etc). The leaves of this structure, the TITLE elements, are elements of the same type. The nodes directly connected with these leaves, i.e. the SEGMENT elements Segment #1, Segment #2, etc, are different in the tree structure. The first level structure stream hence comprises a plurality of search information elements, viz. the TITLE elements, separated and extracted by the Millau streaming system from the search information, viz. the complete XML structure of the "example" of the structured multimedia document.

11. By separating and extracting the TITLE elements, the system also separates and extracts structural information corresponding to the TITLE elements, namely the segment IDs, as well as implicitly the structural information defining the structure of the first level structure stream that corresponds to the moving image contents. Thus the structural information includes information representing the position of the TITLE elements in the original tree structure of the complete XML document before separation.

12. Therefore, the Millau browser and streaming system configured to stream and browse the "example" of a structured multimedia document in section 3.2 meets the definitions of the separating means and structural information extracting means in the second and third paragraphs and the unit of search information forming means in the fourth paragraph of claim 1 of all requests.

13. It follows that none of the requests meets the requirement of novelty (Article 52(1) EPC and Article 54(1) and (2) EPC 1973).

ORDER

For these reasons it is decided that:

The appeal is dismissed.