T 1393/12 (Analysis of audio, music or video data/QUEEN MARY AND WESTFIELD … of 16.10.2017

Summary of Facts and Submissions

I. The appeal lies from the decision of the Examining Division to refuse European patent application No. 06744249.1, which was filed as international application PCT/GB2006/002225 and published as WO 2006/134388, by a "decision according to the state of the file", using EPO Form 2061, referring to the communication dated 24 November 2011.

In that communication the Examining Division cited the following prior-art documents:

D1: Baumann, S. et al.: "Using Natural Language Input and Audio Analysis for a Human-Oriented MIR System", Second International Conference on WEB Delivering of Music (WEDELMUSIC'02), pages 74 to 81, Piscataway, NJ, USA, IEEE, 9 December 2002;

D2: WO 01/27793 A2 of 19 April 2001.

The Examining Division was of the opinion that the subject-matter of claim 1 of the sole request then on file was not inventive over the disclosure of document D1 in combination with either the common general knowledge of the skilled person or the disclosure of document D2, and that none of the claims recited inventive subject-matter.

II. In the statement of grounds of appeal, the appellant requested that the decision be set aside and that a patent be granted on the basis of the pending sole request considered in the appealed decision. With the notice of appeal the appellant requested reimbursement of the appeal fee.

III. In a communication accompanying a summons to oral proceedings, the Board questioned whether the feature "the processing unit including a maths processing unit and a logic processing unit" of claim 1 was clear, and was of the preliminary opinion that some claims lacked clarity and that the subject-matter of claim 1 did not involve an inventive step. The Board briefly assessed most of the features of the other claims and concluded that their subject-matter did not seem to be inventive either.

IV. With a letter of reply the appellant informed the Board that it would not make written submissions nor attend the oral proceedings.

V. Oral proceedings were held on 16 October 2017 in the absence of the appellant. At the end of the oral proceedings, the chairman pronounced the Board's decision.

VI. The appellant's final requests were that

- the contested decision be set aside and that a patent be granted on the basis of claims 1 to 35 of the request on which the appealed decision was based, which was filed with letter of 13 May 2010;

- the appeal fee be reimbursed.

VII. Claim 1 reads as follows:

"A method of analysing audio, music or video data, comprising the steps of:

(i) storing audio, music or video data in a database;

(ii) analysing the data in a processing unit to automatically generate meta-data, the processing unit including a maths processing unit and a logic processing unit;

(iii) the processing unit analysing the meta-data to generate meta-data that is semantically labelled in conformance with an ontology;

and in which the processing unit infers knowledge from any of the data, the meta-data and the semantically labelled meta-data, by applying logic."

VIII. The appellant's arguments relevant to this decision are discussed in detail below.

Reasons for the Decision

1. The appeal complies with the provisions referred to in Rule 101 EPC and is therefore admissible.

2. Invention

2.1 The invention concerns a knowledge-generation or information-management system designed for audio, music and video applications (page 10, lines 1 to 7 of the international publication), which supports complex reasoning (page 4, line 26 to page 5, line 13, page 16, lines 26 to 32).

2.2 The audio, music or video data is stored in a database or in an external source such as the Internet. A processing unit analyses the data and automatically generates metadata in conformance with an ontology to infer knowledge from the data and metadata (page 10, lines 9 to 13, page 11, lines 1 to 9).

All entities in a processing unit, which is referred to as a "knowledge machine" in the description, can be described by descriptors (e.g. a class of metadata) conforming to an ontology. According to the description, the entities include computations, the results of computations and inputs to those computations. All entities in the knowledge machine can be reasoned over, so that complex queries involving logical inference as well as mathematics, can be resolved (page 13, lines 27 to 33).

Lines 1 to 6 of page 14 read as follows:

"The ontology can be a collection of terms specific to the creation, production, recording, editing, delivery, consumption, processing of audio, video or music data and which provide semantic labels for the audio, music or video data and the meta-data. The ontology can include an ontology of one or more of the following: music, time, events, signals, computation, any other ontology available on the internet or the Semantic Web."

2.3 The knowledge machine according to the description brings together logic programming, semantic reasoning, mathematical processing, a (relational) database, and an ontology (page 17, last paragraph, Figure 4, pages 37 and 38) and is implemented using different technologies, as described on pages 37 and 38. The description explains, for example, that: "the ontology is coded in the description logic language OWL-DL"; "[t]he different components of the system, on the Semantic Web side, are integrated using Jena, an open source library for Semantic Web applications"; relational data models are stored in an RDBMS accessed via SQL managed by Jena; knowledge machines are based on SWI-Prolog; "the computational facet of the system" is built around a Prolog/Matlab interface; Matlab is used as an external engine to evaluate Prolog terms (page 37, lines 6 to 18).

3. Interpretation of claim 1

The terms "maths processing unit" and "logic processing unit" used in claim 1 are not explicitly defined in the application as originally filed. In the decision under appeal, the Examining Division interpreted the two terms as hardware processors for the specific purposes of processing mathematical and logic operations, respectively.

In the grounds of appeal, the appellant disagreed with that interpretation and argued that the "logic processing unit" had specific technical characteristics relating to the processing of formal systems of logic, and was capable of making sophisticated logical inferences by operating on symbolic data.

The Board has doubts that those two features have to be understood as maintained by the appellant on the basis of the claims alone. For the benefit of the appellant, the Board nevertheless opts to interpret those terms in the light of the description.

According to the application on page 17, lines 16 to 20, in the system there is "a tight binding and integration of a logic processing engine (such as Prolog) with a mathematical engine (such as Matlab, or compiled C++ code, or interpreted Java code)". Taking into account that passage of the application and the definitions provided by the appellant, in the following the Board interprets the terms "maths processing unit" and "logic processing unit" as referring, respectively, to program modules for numerical processing (e.g. using MatLab or C++) and logic processing (e.g. based on Prolog). Logic processing is interpreted as being based on logic predicates as the basic representational tool, and on information represented as propositions or as inference rules (page 21, lines 20 to 23).

4. Inventive step - claim 1

4.1 Document D1 discloses a music information retrieval (MIR) system which processes natural-language queries to large-scale music collections. It is based on meta-tag construction, content-based classification of audio, and music ontologies to represent musical knowledge (abstract, page 74, section 2).

4.2 With the grounds of appeal the appellant argued that document D1 was not even part of the state of the art because it was not an enabling disclosure. It was not clear whether the system was actually capable of answering the example queries it disclosed, as could be seen from the sentence "The presented work is still on its way" in Section 4 "Conclusion".

The Board agrees that document D1 describes an experimental system and that some features are disclosed as being planned for future implementation. However, the present application describes the invention at a similar level of abstraction to that of document D1. In particular, the present application does not describe in detail how the technical aspects of the invention are implemented, in most cases referring only in general terms to known technologies used in the implementation (see point 2.3 above). It mentions some components of the knowledge machine without however explaining in detail their function and how they interact. The appendices relate essentially to non-technical aspects, namely the music production ontology and the business model. Especially with regard to the processing of queries, the present application provides almost no technical details. The same applies to claim 1, which defines the invention in very broad terms.

Furthermore, the question of whether a disclosed system or method has been implemented is not directly determining to the questions of whether the disclosure is enabling or relevant for the assessment of inventive step. A disclosure, even if deficient, is enabling if the skilled person can understand its technical contents on the basis of her common general knowledge (see also T 230/01 of 26 April 2005, reasons 5.2) and thus can carry out the invention. In the Board's opinion, the skilled person was able to envisage ways to implement the queries disclosed in document D1 using known technologies. The Board cannot identify any deficiency in the disclosure of document D1 that might bring the reader to conclude that the disclosed system was incapable of answering the example queries.

For those reasons, the Board does not find the arguments of the appellant persuasive and is of the opinion that document D1 is an adequate starting point for the inventive-step assessment.

4.3 Document D1 discloses that the MIR system accesses the music data from an underlying database such as a CD collection and gathers information such as artist, title or volume from another database (section 2.2). The automatic audio analysis recognises "properties such as loud/quiet, fast/slow, etc as well as more sophisticated features for the determination of similarity" (page 76, section 2.3.2). Information about the audio data, such as artist, title, the sophisticated features, or the semantically labelled properties loud/quiet and fast/slow constitute metadata. The system of document D1 therefore stores audio and music data in a database (see also page 75, first four lines) and analyses the data to automatically generate metadata, including metadata that is semantically labelled. Furthermore, document D1 discloses that ontologies are used to express terms, relations and rules, e.g. acid-jazz is-a jazz, organ is-a keyboard, track part-of compilation, and member part-of band (page 75, section 2.1). It is clear that these concepts and relations are related to metadata, e.g. member, track, band, or compilation. That metadata is hence semantically labelled in accordance with an ontology.

4.4 Document D1 discloses inferring knowledge from the data, the metadata and the semantically labelled data (see section 3.1.4 "Questions", page 75, left column, section 2.3.2 "Automatic audio analysis"). Document D1 therefore also discloses a processing unit which analyses data and infers knowledge.

In the Board's opinion, any computer program processes numerical data and thus includes maths processing units. Furthermore, the specific types of analysis disclosed in document D1, for example the fingerprint compilation and audio-features extraction described in section 2.3.2, requires numerical processing. Consequently, the processing unit of document D1 also includes a maths processing unit.

Even though document D1 discloses ontology features which are typically used in logic processing and programming, such as concepts and relations expressed by the ontology, it does not explicitly disclose an implementation using a logic programming language or a deductive database to process such data. Interpreting the feature "logic processing unit" as explained above, the Board therefore recognises this feature as a distinguishing feature.

4.5 In the grounds of appeal the appellant questioned that document D1 inferred knowledge from all three types of data, i.e. data, metadata and semantically labelled metadata. The Board finds however that it is implicit from document D1 that all three types of data can be used in a query, and that knowledge is thus inferred from all three.

With regard to the example query cited by the Examining Division regarding which CDs Madonna had made between 1985 and 1995, the appellant stated that it did not require the use of the metadata, such as "loud/quiet", "fast/slow". The Board is however of the opinion that the artist and the years constitute metadata. A query to retrieve slow pop music from Madonna, which is based on query predicates disclosed in document D1 and which the skilled reader thus understands to be foreseen for the system of document D1, infers knowledge from all three types of data (slow is semantically labelled metadata automatically retrieved from the data itself, pop music is semantically labelled metadata in conformance with an ontology, and Madonna is metadata giving information about the artist). The example query "Something snappier from 2002!" given on page 80, left column, involves inferring knowledge from at least two types of data (2002 is metadata and snappier is semantic metadata which is either stored directly as semantically labelled metadata or inferred from other metadata).

4.6 Taking into account the interpretation of logic processing mentioned above, the subject-matter of claim 1 is considered to differ from the method of document D1 in that

(a) the semantically labelled metadata is generated from metadata;

(b) the processing unit includes a logic processing unit and applies logic when inferring knowledge.

4.7 In the Board's opinion, feature (a) directly reflects non-technical considerations regarding which knowledge can be obtained from an analysis of the metadata and whether it is useful for the specific non-technical field of an application. The Board notes that technically speaking metadata is also data and the question of whether generating semantically labelled metadata from metadata is useful depends solely on non-technical aspects such as the content of the data and of the metadata or the type of information the user wants to obtain. Feature (a) is hence not inventive.

Distinguishing features (b) merely result in semantically different data being returned by queries, or in semantically different types of queries being supported by the system. Since the claim covers all, including non-technical, types of data, knowledge and queries with regard to audio, music and video, those semantically different data and types of queries correspond to non-technical aims. Distinguishing features (b) therefore solve the problem of modifying the method of document D1 to be able to support semantically different data and queries.

It is standard practice to use logic processing in the area of information retrieval, e.g. in knowledge bases or deductive databases, to reason about facts and rules and deduce new facts. It would therefore be obvious for the skilled person trying to support different semantics and queries in the system of document D1 to modify the processing unit of document D1 to also include a logic processing unit and perform some steps of the method by applying logic.

4.8 Consequently, the subject-matter of claim 1 is not inventive (Article 56 EPC).

5. Concluding remarks

5.1 Since the sole request is not allowable, the appeal is to be dismissed.

5.2 As the Board explained in its preliminary opinion, the appellant did not reason its request for reimbursement of the appeal fee and the Board could not identify any ground justifying such a reimbursement. The appeal fee can in any case not be reimbursed because the appeal is not allowable (Rule 103(1)(a) EPC). The request for reimbursement of the appeal fee is therefore to be refused.

Order

For these reasons it is decided that:

1. The appeal is dismissed.

2. The request for reimbursement of the appeal fee is refused.