T 2022/15 () of 20.8.2020

Summary of Facts and Submissions

I. The appeal is against the decision to refuse European patent application No. 06 798 249.6, published as international application WO 2007/034918 A1.

II. The documents cited in the decision under appeal comprised the following document:

D1:|Thomas Wiegand et al., "Overview of the H.264/AVC Video Coding Standard", IEEE Transactions on Circuits and Systems for Video Technology, volume 13, number 7, July 2003, XP011221093|

III. The decision was based on the grounds that the subject-matter of claim 1 of the five requests then on file lacked inventive step (Article 56 EPC) in view of the disclosure of D1 combined with common general knowledge and that claim 1 of the first auxiliary request contained subject-matter which extended beyond the content of the application as filed (Article 123(2) EPC).

IV. The applicant ("appellant") filed notice of appeal. With the statement of grounds of appeal, it requested that the decision under appeal be set aside and that a patent be granted on the basis of the claims of the main request forming the basis for the decision under appeal or, in the alternative, on the basis of the claims of one of the first to fourth auxiliary requests filed with the statement of grounds of appeal, in this order of preference. The first and second auxiliary requests filed with the statement of grounds of appeal corresponded to the first and second auxiliary requests forming the basis for the decision. The appellant indicated a basis in the application as filed for the claimed subject-matter and provided arguments as to why the subject-matter of the claims involved an inventive step.

V. The board issued a summons to oral proceedings. In a communication under Article 15(1) RPBA 2020 (Rules of Procedure of the Boards of Appeal in the version of 2020, OJ EPO 2019, A63), annexed to the summons, the board introduced the following document ex officio into the appeal proceedings on the basis of Article 114(1) EPC 1973:

D4:|International standard ISO/IEC, "Information technology - Coding of audio-visual objects - Part 2: Visual", reference number ISO/IEC 14496-2:2001(E), second edition 1 December 2001, XP055664889|

The board gave its preliminary opinion that

(a) the subject-matter of claim 1 of all requests lacked inventive step (Article 56 EPC 1973) in view of the disclosure of document D4 combined with common general knowledge and

(b) claim 1 of the first auxiliary request lacked clarity (Article 84 EPC 1973) and contained subject-matter which extended beyond the content of the application as filed (Article 123(2) EPC).

VI. By letter dated 20 July 2020, the appellant commented on the preliminary opinion expressed in the board's communication and indicated that the first auxiliary request would not be prosecuted further and that the second, third and fourth auxiliary requests filed with the statement of grounds of appeal became the first, second and third auxiliary requests, respectively.

VII. On 20 August 2020 the oral proceedings before the board were held via videoconference, as agreed with the appellant.

The appellant's final requests were that the decision under appeal be set aside and that a European patent be granted on the basis of the claims of the main request on which the decision under appeal was based or, in the alternative, on the basis of the claims of one of the second to fourth auxiliary requests filed with the statement of grounds of appeal, in this order of preference and renumbered as first to third auxiliary requests.

At the end of the oral proceedings, the chairman announced the board's decision.

VIII. Claim 1 of the main request reads as follows:

"A moving image coding apparatus which divides a moving image into macroblocks and codes each of the macroblocks, said apparatus comprising:

a macroblock size determination means (1) for determining a size of each of the macroblocks on a sequence-by-sequence basis, wherein the sequence is a set of a plurality of continuous frames;

a macroblock dividing means (2) for dividing an inputted image into macroblocks each having the size determined by said macroblock size determination means (1);

a macroblock coding means (3) for determining a coding mode for each of said macroblocks, and for coding pixel values in each of said macroblocks in the determined coding mode; and

a macroblock size information multiplexing means (4) for multiplexing the macroblock size information determined by the macroblock size determination means (1) into a bit stream.".

IX. Claim 1 of the first auxiliary request reads as follows (features added to claim 1 of the main request are underlined):

"A moving image coding apparatus which divides a moving image into macroblocks and codes each of the macroblocks, said apparatus comprising:

a macroblock size determination means (1) for determining a size of each of the macroblocks on a sequence-by-sequence basis in units of each sequence, wherein the sequence is a set of a plurality of continuous frames;

a macroblock dividing means (2) for dividing an inputted image into macroblocks each having the size determined by said macroblock size determination means (1);

a macroblock coding means (3) for determining a coding mode for each of said macroblocks, and for coding pixel values in each of said macroblocks in the determined coding mode; and

a macroblock size information multiplexing means (4) for multiplexing the macroblock size information determined by the macroblock size determination means (1) into a bit stream in units of each sequence.".

X. Claim 1 of the second auxiliary request reads as follows (features added to claim 1 of the main request are underlined; deleted features are indicated by strikethrough):

"A moving image coding apparatus which divides a moving image into macroblocks and codes each of the macroblocks, the macroblocks being spatially divided from a slice, the moving image including the slice, the [deleted: said] apparatus comprising:

a macroblock size determination means (1) for determining a size of each of the macroblocks on a sequence-by-sequence basis, wherein the sequence is a set of a plurality of continuous frames and each of the continuous frames is temporally divided from the moving image;

a macroblock dividing means (2) for dividing [deleted: an inputted] the moving image into the macroblocks each having the size determined by the [deleted: said] macroblock size determination means (1);

a macroblock coding means (3) for determining a coding mode for each of the [deleted: said] macroblocks, and for coding pixel values in each of the [deleted: said] macroblocks in the determined coding mode; and

a macroblock size information multiplexing means (4) for multiplexing the macroblock size information determined by the macroblock size determination means (1) into a bit stream.".

XI. Claim 1 of the third auxiliary request reads as follows (features added to claim 1 of the second auxiliary request are underlined):

"A moving image coding apparatus which divides a moving image into macroblocks and codes each of the macroblocks, the macroblocks being spatially divided from a slice, the moving image including the slice, the apparatus comprising:

a macroblock size determination means (1) for determining a size of each of the macroblocks on a sequence-by-sequence basis in units of each sequence, wherein the sequence is a set of a plurality of continuous frames and each of the continuous frames is temporally divided from the moving image;

a macroblock dividing means (2) for dividing the moving image into the macroblocks each having the size determined by the macroblock size determination means (1);

a macroblock coding means (3) for determining a coding mode for each of the macroblocks, and for coding pixel values in each of the macroblocks in the determined coding mode; and

a macroblock size information multiplexing means (4) for multiplexing the macroblock size information determined by the macroblock size determination means (1) into a bit stream in units of each sequence.".

XII. The arguments submitted by the appellant, as far as relevant to the present decision, may be summarised as follows.

The objective technical problem formulated in section 4 of the board's communication ("how to reduce the number of overhead bits") was incorrect. The person skilled in the art would not have tried to solve this problem because, by reducing the number of overhead bits, some flexibility in the rate control would be lost. Instead, the objective technical problem should be formulated as how to improve coding efficiency. Indeed, for most realistic videos and for the same visual quality of the video reconstructed by the decoder, signalling the size of the macroblocks on a sequence-by-sequence basis, as specified by claim 1 of all requests, led to a shorter bit stream than signalling it for each Video Object Plane, as disclosed in document D4.

Assuming, arguendo, that the objective technical problem was formulated as how to reduce the number of overhead bits, the person skilled in the art starting from document D4 and faced with this problem would not have signalled the element "vop_reduced_resolution" on a sequence-by-sequence basis. Indeed, the person skilled in the art could have chosen almost any syntax element to be signalled less frequently or even not at all. There was no motivation in document D4 to specifically change the definition of the Video Object Plane, let alone to choose the element "vop_reduced_resolution" to be signalled less frequently.

Even if the person skilled in the art had come to the idea of signalling the syntax element "vop_reduced_resolution" less frequently, there would have been no motivation in document D4 to signal it in the Video Object Layer or the Group of Video Object Plane. Moreover, the person skilled in the art would not have signalled this syntax element in a higher layer because this would have resulted in a loss of rate-control flexibility. In any case, the Video Object Layer and the Group of Video Object Plane did not correspond to a plurality of continuous frames but rather to a plurality of continuous Video Object Planes.

Reasons for the Decision

1. The appeal is admissible.

2. The invention

2.1 In conventional video coding methods, the macroblock size is fixed and cannot be changed according to the content of the incoming video. In order to improve "coding efficiency", the patent application proposes relaxing this constraint (description as filed, paragraphs [0007] and [0008]).

2.2 More specifically, the patent application proposes determining and signalling the size of a macroblock for each of a plurality of coding units into which the video is divided. The coding unit may be a slice of macroblocks, a frame of the video, or a sequence of continuous frames (paragraphs [0036] and [0037]).

2.3 Claim 1 of all the requests forming the basis for this decision relates to the alternative in which a macroblock size is determined (and signalled) on a sequence-by-sequence basis.

3. Main request - Inventive step, Article 56 EPC 1973

3.1 According to Article 56 EPC 1973, an invention is to be considered as involving an inventive step if, having regard to the state of the art, it is not obvious to the person skilled in the art.

3.2 It is undisputed that document D4 may be considered as the closest state of the art in the context of the established "problem-solution approach" for the assessment of whether an invention involves an inventive step (Case Law of the Boards of Appeal of the European Patent Office, 9th edition 2019 ("Case Law"), I.D.2) (see the appellant's letter dated 20 July 2020, paragraph bridging pages 1 and 2).

3.3 Document D4 discloses a moving image coding apparatus (see paragraph 3.60 on page 5) which divides a moving image into macroblocks (see paragraph 3.107 on page 7) and codes each of the macroblocks (see paragraph 6.2.6 on pages 53 to 56).

Paragraph 3.107 on page 7 discloses that the size of a macroblock is determined depending on whether it is part of a "Reduced Resolution VOP" or not. Hence, the apparatus disclosed in document D4 comprises macroblock size determination means for determining a size of each of the macroblocks and macroblock dividing means for dividing an inputted image into macroblocks each having the size determined by said macroblock size determination means.

The apparatus disclosed in document D4 also comprises macroblock coding means for determining a coding mode for each of said macroblocks and for coding pixel values in each of said macroblocks in the determined coding mode. As shown in Tables B-6 and B-7 on page 338, the syntax element "mcbpc" mentioned on page 131 signals the macroblock pattern, in particular its "mb_type". It can further be derived from Table B-1 on page 336 that "mb_type" represents a "coding mode". Moreover, as disclosed in paragraph 6.2.7 on pages 58 and 59, the coding of a block depends on the value of the syntax element "mb_type".

The apparatus disclosed in document D4 further comprises a macroblock size information multiplexing means for multiplexing the macroblock size information determined by the macroblock size determination means into a bit stream. Indeed, according to page 41, a syntax element "vop_reduced_resolution" is signalled if some conditions are met.

3.4 The subject-matter of claim 1 of the main request differs from the disclosure of document D4 in that the size of each of the macroblocks is determined on a sequence-by-sequence basis, not for each Video Object Plane.

3.5 This difference has the technical effect that fewer values of the syntax element "vop_reduced_resolution" need to be signalled, hence the number of overhead bits is reduced.

3.5.1 At the oral proceedings before the board, the appellant submitted that, for most of the realistic videos, this difference had the effect that a video of a given visual quality was represented by a smaller amount of bits, i.e. it had the effect that the "coding efficiency" was improved.

3.5.2 The board is not convinced that this effect is achieved over the whole scope of the claim. Claim 1 does not specify how each macroblock is coded. It is thus impossible to assess the size of the final output bit stream. In the method disclosed in document D4, the macroblocks are coded in a predictive manner (see Table B-1 on page 336, "inter", "intra"). Even assuming, arguendo, that the apparatus defined in claim 1 would code the macroblocks in such a manner, there is no guarantee that it would represent most of the "realistic" videos with a smaller number of bits than the apparatus disclosed in document D4 for a given target visual quality. A macroblock size which is determined for a whole sequence of frames is usually less adapted to the image content than a macroblock size determined for a Video Object Plane. If the macroblock size is less adapted to the content in the apparatus defined in claim 1, the prediction errors that would be obtained would normally be larger - thus resulting in longer codes - than the prediction errors obtained by the apparatus disclosed in document D4. Overall, the apparatus defined in claim 1 would only achieve a better coding efficiency if the reduction of overhead bits obtained by signalling the macroblock size at the sequence level (instead of signalling it at Video Object Plane level) were to compensate for the decrease in prediction efficiency. However, there is no feature in claim 1 that controls the decrease in prediction efficiency and, therefore, no guarantee that it will be compensated by the reduction in the number of overhead bits. In particular, the content of the frames included in a sequence of frames is not limited by the wording of claim 1. The image content in an arbitrary sequence of frames in a "realistic" video may change significantly; some frames may be composed of smooth textures and little motion while others may be composed of complex textures and complex motion. It is, therefore, impossible to assess the increase in prediction error caused by determining the macroblock size only on a sequence-by-sequence basis. Accordingly, it cannot be concluded that this increase is compensated by the reduction in the number of overhead bits for most of the "realistic" videos. The board thus comes to the conclusion that the effect of increased coding efficiency alleged by the appellant cannot be considered as a credible technical effect for the assessment of inventive step.

3.6 Therefore, the objective technical problem starting from document D4 is formulated as how to reduce the number of overhead bits.

3.7 At the oral proceedings before the board, the appellant argued that the person skilled in the art would not have tried to solve this problem because reducing the number of overhead bits had the clear disadvantage of reducing the flexibility of the rate controller.

However, the objective technical problem must be formulated on the basis of the technical effect achieved by the claimed subject-matter. Since the appellant could not convince the board that an effect other than a reduction of overhead bits was achieved by the distinguishing features of the claim, the board maintained its problem as originally formulated.

3.8 The different levels at which a parameter may be adjusted represent obvious trade-offs between the number of overhead bits and the adaptivity of the coding apparatus to the image content. In particular, signalling the syntax element "vop_reduced_resolution" in the Video Object Layer (D4, section 6.2.3) or the Group of Video Object Plane (D4, section 6.2.4) instead of signalling it in the Video Object Plane (D4, section 6.2.5) has the obvious advantage of saving overhead bits and the obvious disadvantage of decreasing the adaptivity of the coding apparatus to the image content. The person skilled in the art of video coding would choose the level at which this syntax element is signalled depending on whether a finer adaptation to the image content or a reduction of overhead bits is desired, i.e. without involving any inventive activity. Moreover, as recalled in paragraph [0003], lines 27 and 28, of the description of the application as filed, and as affirmed by the appellant at the oral proceedings before the board, a Video Object Plane may correspond to a frame. In that obvious scenario, a Video Object Layer and a Group of Video Object Plane represent a plurality of continuous frames, i.e. a "sequence" within the meaning of the present application. Therefore, by choosing the obvious alternative of signalling the syntax element "vop_reduced_resolution" in the Video Object Layer (D4, section 6.2.3) or the Group of Video Object Plane, the person skilled in the art would have signalled it "on a sequence-by-sequence basis" within the meaning of the present application.

3.9 In its letter dated 20 July 2020, the appellant submitted the following arguments:

(a) The person skilled in the art starting from document D4 and faced with the problem of reducing the number of overhead bits could have chosen almost any syntax element to be signalled less frequently or even not at all. There was no motivation in D4 to specifically change the definition of the Video Object Plane, let alone to signal the element "vop_reduced_resolution" less frequently (page 2, third and fourth full paragraphs).

(b) Even if the person skilled in the art had come to the idea of signalling the syntax element "vop_reduced_resolution" less frequently, there would have been no motivation in document D4 to signal it in the Video Object Layer or the Group of Video Object Plane (paragraph bridging pages 2 and 3).

(c) The person skilled in the art would not have signalled the syntax element "vop_reduced_resolution" in a higher layer because this would have resulted in a loss of rate-control flexibility (page 3, second full paragraph).

(d) The Video Object Layer and the Group of Video Object Plane did not correspond to a plurality of continuous frames but rather to a plurality of continuous Video Object Planes.

3.10 The board does not find these arguments persuasive for the following reasons, respectively:

(a) A plurality of coding parameters are disclosed in document D4, including the flag "vop_reduced_resolution". Each of these coding parameters is signalled at a certain level of the coding structure. For example, the flag "vop_reduced_resolution" is signalled at the level of a Video Object Plane. It is obvious that signalling any one of these coding parameters at a finer level enables a finer adaptation to the image content at the price of additional overhead bits, and that signalling any one of them at a coarser level enables a reduction in the number of overhead bits at the expense of a coarser adaptation to the image content. Hence, the selection of one particular coding parameter, for example "vop_reduced_resolution", to arrive at a different compromise between adaptivity to image content and number of overhead bits merely corresponds to the selection of one obvious alternative from several.

(b) As indicated under point 3.8 above, the different levels at which a parameter may be adjusted represent obvious trade-offs between the adaptivity of the coding apparatus to the image content and the number of overhead bits. Therefore, the selection of a particular level (for example, the Video Object Layer or the Group of Video Object Planes) also corresponds to the selection of one obvious alternative from others to adjust this trade-off.

(c) In circumstances in which the reduction in the number of overhead bits was more important than the adaptivity, then, as explained above, the skilled person would have wanted to reduce the number of overhead bits at the expense of the adaptivity and, accordingly, of the flexibility of the coding apparatus.

(d) The description of the application as filed itself indicates that a Video Object Plane corresponds to a frame when the object is of rectangular shape (paragraph [0003], lines 27 and 28). As already mentioned above, in that obvious scenario, a Video Object Layer and a Group of Video Object Plane represent a plurality of continuous frames, i.e. a "sequence" within the meaning of the present application.

3.11 For the above reasons, the board arrives at the conclusion that the subject-matter of claim 1 of the main request does not involve an inventive step (Article 56 EPC 1973) in view of the disclosure of document D4 combined with common general knowledge.

4. First auxiliary request - Inventive step, Article 56 EPC 1973

4.1 Claim 1 of the first auxiliary request differs from claim 1 of the main request in that it further specifies that the macroblock size is determined "in units of each sequence" and multiplexed into a bit stream "in units of each sequence".

4.2 The fact that the macroblock size is determined and signalled in units of each sequence was already considered as implicit in claim 1 of the main request and was already taken into account in the reasoning laid out under point 3 above. Thus, this reasoning also applies to claim 1 of the first auxiliary request.

4.3 Neither in its letter dated 20 July 2020 nor at the oral proceedings did the appellant provide any arguments in addition to those put forward with respect to the main request.

4.4 For the above reasons, the board arrives at the conclusion that the subject-matter of claim 1 of the first auxiliary request also lacks inventive step (Article 56 EPC 1973) in view of the disclosure of document D4 combined with common general knowledge.

5. Second and third auxiliary requests - Inventive step, Article 56 EPC 1973

5.1 Claim 1 of the second and third auxiliary requests differs from claim 1 of the main request and second auxiliary request, respectively, in that it further specifies that the continuous frames forming the sequence are obtained by temporally dividing the moving image, that a slice is included in a moving image and that a macroblock is obtained by spatially dividing a slice.

5.2 The fact that the continuous frames are temporally divided from the moving image (i.e. from the video) was already considered an implicit feature of claim 1 of the main and first auxiliary requests in the above assessment of inventive step. Hence, this feature cannot contribute to an inventive step.

5.3 At the oral proceedings, the appellant argued that document D4 only anticipated the use of a slice layer to achieve a tighter rate control (paragraph L.1.3 on page 479). The person skilled in the art confronted with the problem of reducing the number of overhead bits would not have sought a tighter rate control and, thus, would not have used an additional slice layer on top of the macroblock layer and the Video Object Plane.

5.4 The board does not find this argument persuasive. Slices form part of the common general knowledge of the person skilled in the art. It is well known that slices are spatially distinct regions of a frame that are encoded separately from one another and that, as a result, errors or missing data from one slice cannot propagate to any other slice within the frame. Thus, although slices may be used as a rate-control tool, their main function is to provide for error resilience. The person skilled in the art would have used a slice layer on top of the macroblock layer to improve error resilience, irrespective of whether a finer or a coarser rate control is achieved thereby.

5.5 For the above reasons, the board considers that the subject-matter of claim 1 of the second and third auxiliary requests also lacks inventive step (Article 56 EPC 1973) in view of the disclosure of document D4 combined with common general knowledge.

6. Since none of the appellant's requests is allowable, the appeal is to be dismissed.

Order

For these reasons it is decided that:

The appeal is dismissed.