T 0709/16 () of 21.10.2020

Summary of Facts and Submissions

I. The appeal is against the decision dated 9 September 2015 to refuse European patent application No. 04 712 388.0, the second appeal against a decision to refuse this application. In the first appeal, the board decided to remit the case to the department of first instance for further prosecution (T 2137/09).

II. The documents cited in the decision under appeal comprised the following:

D5: D6: D7:|WIEGAND, Thomas, "Text of Final Committee Draft of Joint Video Specification (ITU-T REC. H.264 | ISO/IEC 14496-10 AVC)", ISO/IEC JTC1/SC29/WG11 MPEG02/N4920, July 2002, XP001100641 US 5 021 891 US 5 818 532|

III. The decision under appeal was based on the ground that the subject-matter of claim 1 of the single request then on file lacked inventive step in view of the disclosures of documents D5 and D6 and the common general knowledge of a person skilled in the art as "recalled e.g. in document D7 (column 15, line 55 - column 16, line 33)" (see points 1.5.1 and 1.5.2 of the impugned decision).

IV. The applicant (appellant) filed notice of appeal. In the statement of grounds of appeal, it requested that the decision under appeal be set aside and that a patent be granted on the basis of the claims of the request forming the basis of the decision under appeal (main request) or, in the alternative, that the decision under appeal be set aside and that the case be remitted to the examining division for further examination (first auxiliary request) or, in the alternative, that oral proceedings be held before the board (second auxiliary request). The appellant further provided arguments as to why the subject-matter of claim 1 involved an inventive step.

V. On 9 June 2020, the board issued a summons to oral proceedings. In a communication under Article 15(1) of the Rules of Procedure of the Boards of Appeal in the version of 2020 (RPBA 2020, OJ EPO 2019, A63), annexed to the summons, the board gave its preliminary opinion that the subject-matter of all claims of the main request lacked inventive step (Article 56 EPC 1973) in view of the disclosure of document D5 and the common general knowledge of the person skilled in the art, as recalled in document D7, column 15, line 55 to column 16, line 33.

VI. By letter dated 21 September 2020, the appellant replaced all claims previously filed with the claims of a new main request and an auxiliary request I. The appellant provided a basis for the amendments in the application as filed, as well as arguments as to why the subject-matter of the claims involved an inventive step.

VII. On 21 October 2020, oral proceedings were held before the board.

The appellant's final requests were that the decision under appeal be set aside and a European patent be granted on the basis of the claims of the main request, or, in the alternative, of auxiliary request I, both filed by letter dated 21 September 2020. As auxiliary request II, the appellant requested that the decision under appeal be set aside and the case be remitted to the examining division for continued examination.

At the end of the oral proceedings, the chairwoman announced the board's decision.

VIII. Claim 1 of the main request reads as follows:

"A picture coding method for coding picture data on a block-by-block basis, comprising:

selecting one of a first size and a second size as an orthogonal transformation size of a block on which orthogonal transformation should be performed;

orthogonal transforming a block having the selected orthogonal transformation size, and obtaining resulting transformation coefficients;

quantizing the transformation coefficients; and

variable length coding the quantized transformation coefficients, and outputting a resulting coded stream,

characterized in that, when inter picture coding is performed on each block in a macroblock, on which orthogonal transformation should be performed,

said selecting step includes (1) selecting the first size which is smaller than the second size as the orthogonal transformation size, in the case where a motion compensation block size is smaller than the second size, and (2) selecting the second size as the orthogonal transformation size in the case where a motion compensation block size is equal to or greater than the second size,

wherein,

a size of the macroblock is a size of 16x16 pixels, the first size is a size of 4x4 pixels, the second size is a size of 8x8 pixels, and the motion compensation block size is one of a size of 16x16 pixels, a size of 16x8 pixels, a size of 8x16 pixels, a size of 8x8 pixels, a size of 8x4 pixels, a size of 4x8 pixels, and a size of 4x4 pixels, and

said selecting step selects the second size of 8x8 pixels as the orthogonal transformation size in the case where the motion compensation block size is equal to or greater than a size of 8x8 pixels, and

said selecting step selects the first size of 4x4 pixels as the orthogonal transformation size in the case where the motion compensation block size is one of a size of 8x4 pixels, a size of 4x8 pixels, and a size of 4x4 pixels,

further characterized in that the quantizing the transformation coefficients is performed by use of a weighting matrix corresponding to the selected orthogonal transformation size, wherein said weighting matrix corresponding to the selected orthogonal transformation size is inserted in the coded stream."

IX. Claim 1 of auxiliary request I reads as follows (amendments with respect to claim 1 of the main request are underlined):

"A picture coding method for coding picture data on a block-by-block basis, comprising:

selecting one of a first size and a second size as an orthogonal transformation size of a block on which orthogonal transformation should be performed;

orthogonal transforming a block having the selected orthogonal transformation size, and obtaining resulting transformation coefficients;

quantizing the transformation coefficients; and

variable length coding the quantized transformation coefficients, and outputting a resulting coded stream,

characterized in that, when inter picture coding is performed on each block in a macroblock, on which orthogonal transformation should be performed,

said selecting step includes (1), for each block in the macroblock, on which orthogonal transformation should be performed, selecting the first size which is smaller than the second size as the orthogonal transformation size, in the case where a motion compensation block size is smaller than the second size, and (2) selecting the second size as the orthogonal transformation size in the case where a motion compensation block size is equal to or greater than the second size,

wherein,

a size of the macroblock is a size of 16x16 pixels, the first size is a size of 4x4 pixels, the second size is a size of 8x8 pixels, and the motion compensation block size is one of a size of 16x16 pixels, a size of 16x8 pixels, a size of 8x16 pixels, a size of 8x8 pixels, a size of 8x4 pixels, a size of 4x8 pixels, and a size of 4x4 pixels, and

said selecting step selects the second size of 8x8 pixels as the orthogonal transformation size in the case where the motion compensation block size is equal to or greater than a size of 8x8 pixels, and

said selecting step selects the first size of 4x4 pixels as the orthogonal transformation size in the case where the motion compensation block size is one of a size of 8x4 pixels, a size of 4x8 pixels, and a size of 4x4 pixels,

further characterized in that the quantizing the transformation coefficients is performed by use of a weighting matrix corresponding to the selected orthogonal transformation size, wherein said weighting matrix corresponding to the selected orthogonal transformation size is inserted in the coded stream."

X. The arguments submitted by the appellant with respect to the allowability of the main request, as far as relevant to the present decision, may be summarised as follows:

(a) On proper interpretation in the context of the application, lines 15 to 30 of claim 1 of the main request meant that a selection between a first and a second orthogonal transformation size was performed independently for the various motion compensation blocks within a macroblock, which could lead to different orthogonal transformation sizes being selected for different motion compensation blocks within a macroblock (appellant's letter dated 21 September 2020, page 4, third paragraph). The background of the invention was the ITU-T standard H.264/MPEG-4 AVC (page 3, lines 5 to 8 of the application as filed). According to this standard, each 8x8 motion compensation block within a macroblock can be independently partitioned into motion compensation blocks of size 8x4, 4x8 or 4x4 pixels. This was how the person skilled in the art would have understood the passage on page 15, lines 15 to 19 of the application as filed. Moreover, the passages on page 9, lines 8 to 18 and lines 26 to 29 of the application as filed indicated that a block size on which orthogonal transformation should be performed might not be fixed within a macroblock. Thus these passages meant that a macroblock might be partitioned into motion compensation blocks having different sizes. Consequently, the passage on page 15, lines 7 to 9 could only mean that a selection between a first and a second orthogonal transformation size was performed independently for the various motion compensation blocks within a macroblock, which might lead to different orthogonal transformation sizes being selected for different motion compensation blocks within a macroblock.

(b) Document D5 did not disclose the following features of claim 1 (statement of grounds of appeal, section 1, and appellant's letter dated 21 September 2020, page 9, point 2.2., second sentence):

(i) The size of 4x4 pixels is selected as an orthogonal transformation size in the case where the motion compensation block size is 8x4 or 4x8 ("Feature (a)").

(ii) The weighting matrix corresponding to the selected orthogonal transformation size is inserted in the coded stream ("Feature (b)").

(iii) A motion compensation block cannot have a larger size than 8x8 pixels ("Feature (c)").

(c) Feature (a) achieved a simplification of the coding apparatus because less switching between orthogonal transformation sizes was needed when there were motion compensation blocks of different sizes within the macroblock (appellant's letter dated 21 September 2020, page 6, lines 13 to 20). Feature (b) increased coding flexibility and improved coding efficiency (statement of grounds of appeal, page 4, point 2.2.). Feature (a) and Feature (b) had a synergistic effect because the improvement of coding efficiency achieved by Feature (b) compensated for the deterioration of coding efficiency caused by Feature (a).

(d) A person skilled in the art faced with the problem of simplifying the coding apparatus of document D5 would not have thought of reducing the number of possible orthogonal transformation sizes. Assuming arguendo that they would have considered simplifying the orthogonal transformation, they would simply have deleted the "adaptive block size transforms" tool described in document D5, section 12 and would have kept only one orthogonal transformation size - the 4x4 pixel size - to achieve the highest level of simplification. There was no incentive in document D5 to keep the 8x8 pixel size as well. Previous MPEG-1, MPEG-2 and MPEG-4 video coding standards taught using an orthogonal transform size not larger than the smallest motion compensation block size.

(e) Feature (b) should not be assessed separately from Feature (a) because these features, when considered together, solved the common objective technical problem of a simplification in coding and decoding while maintaining good coding efficiency (statement of grounds of appeal, point 2.3).

XI. The arguments submitted by the appellant with respect to the admissibility of auxiliary request I may be summarised as follows:

The amendment was a reaction to the comment made by the board in point 3.3.2 of its communication under Article 15(1) RPBA 2020. It was meant to clarify that a selection between a first and a second orthogonal transformation size was not made for only a single motion compensation block (appellant's letter dated 21 September 2020, paragraph bridging pages 11 and 12).

The appellant cited the passage from page 15, line 10 to page 16, line 9 as basis for the amendment.

Reasons for the Decision

1. The appeal is admissible.

2. The invention

2.1 The application relates to the compression and coding of videos. Conventionally, a signal is compressed by eliminating redundancies and by reducing the set of output values via a process called quantisation.

2.2 There are different ways of eliminating redundancies in a video:

One possibility is to predict the content of a block of pixels using neighbouring pixel values in the same image ("intra" prediction) or using pixel values of a corresponding block in a reference image ("inter" prediction, a.k.a. "motion compensation"). The prediction is then subtracted from the block to generate a prediction residual. According to MPEG standards, a picture to be compressed is divided into macroblocks of size 16x16. Each macroblock can be further divided into blocks on which intra or inter prediction is performed.

An additional possibility for eliminating redundancies is to transform the prediction residual into a frequency space where the energy of the signal is shared among a smaller number of coefficients. This transform may be an orthogonal transformation.

2.3 Conventionally, coefficients obtained after orthogonal transformation are quantised. For this, it is known to use a weighting matrix that changes the quantisation steps on a per-frequency component basis (application as filed, page 19, lines 4 to 9).

2.4 The background art mentioned in the application is an existing picture coding apparatus and an existing picture decoding apparatus (page 5, line 26 to page 7, line 32) in which the size of the orthogonal transformation is fixed (page 8, lines 10 to 20).

2.5 To accommodate different content, it is desirable to use an adaptive size (page 8, lines 20 to 29).

2.6 The claims of the main request and of auxiliary request I relate to a picture coding method and a picture decoding method in which the size of the orthogonal transformation is set to (i) 8x8 pixels in a motion compensation block of size 16x16, 16x8, 8x16 or 8x8 pixels, and (ii) 4x4 pixels in a motion compensation block of size 8x4, 4x8 or 4x4 pixels. Transform coefficients are quantised using a weighting matrix corresponding to the selected orthogonal transformation size. The weighting matrix is inserted in the coded stream.

3. Main request, interpretation of claim 1

3.1 The board finds that the interpretation adopted by the appellant (see point X.(a) above) contradicts the passages on page 14, lines 5 to 7 and page 16, lines 10 to 12 of the application as filed. Indeed, these passages teach that the finest level in the coding structure at which an orthogonal transformation size is selected is the macroblock level.

3.2 The board also finds that the passages cited by the appellant do not disclose that 8x8 motion compensation blocks within a macroblock are independently partitioned into motion compensation blocks of size 8x4, 4x8 or 4x4 pixels. In particular, they do not indicate that the partitioning of a macroblock complies with the standard H.264/MPEG-4 AVC.

3.3 The passage from page 15, line 7 to page 16, line 9, when read in combination with the passage on page 14, lines 5 to 7, can also be understood as meaning that all motion compensation blocks within a macroblock have the same size and that this size is used to determine the orthogonal transformation size on a per-macroblock basis.

3.4 The passage on page 9, lines 26 to 29 relates to an embodiment in which an additional size information is transmitted within the coded stream. This embodiment is to be distinguished from the selection of an orthogonal transformation size based on the motion compensation block size.

3.5 In view of the above, the board holds that, on proper interpretation within the context of the application, claim 1 of the main request does not necessarily imply that the selection between a first and a second orthogonal transformation size is performed independently for the various motion compensation blocks within a macroblock. It also covers the situation where all motion compensation blocks within a macroblock have the same size and only one selection of an orthogonal transformation size is made for the macroblock.

4. Main request, inventive step (Article 56 EPC 1973)

4.1 According to Article 56 EPC 1973, an invention is to be considered as involving an inventive step if, having regard to the state of the art, it is not obvious to a person skilled in the art.

4.2 The following assessment of whether the invention claimed in claim 1 of the main request involves an inventive step follows the established "problem and solution approach" (Case Law of the Boards of Appeal of the European Patent Office ("Case Law"), 9th edition 2019, I.D.2).

4.3 It is common ground that document D5 may be considered the closest prior art in the context of this approach (see point 1.1 of the impugned decision and point 2.1 of the statement of grounds of appeal).

4.4 It is also common ground that document D5 discloses a picture coding method in which the size of the orthogonal transformation is set to (i) 8x8 pixels in a motion compensation block of size 16x16, 16x8, 8x16 or 8x8 pixels, (ii) 8x4, 4x8 or 4x4 pixels in a motion compensation block of size 8x4, 4x8 or 4x4 pixels respectively (page 11, section 6.4, page 34, lines 1 to 3; page 128, section 12.1; and page 135, section 12.3.1.3, equations (12-1) and (12-2)). Different weighting matrices are used for quantisation of coefficients obtained with different orthogonal transformation sizes. Equation (8-41) on page 74 specifies the weighting matrix used in quantising a 4x4 transform block (see equation (8-48) on page 75), whereas equations (12-4) and (12-6) on pages 144 and 145 specify the weighting matrices used in quantising 8x8, 8x4 and 4x8 transform blocks.

4.5 It is also common ground that document D5 does not disclose Feature (a) or Feature (b).

4.6 However, the board does not agree that Feature (c) is a further distinguishing feature (see point X.(b)(iii) above). Section 6.4 on page 11 of document D5 discloses that a motion compensation block can have a size of 16x16, 16x8 or 8x16 pixels.

4.7 The board has not been convinced that Feature (b) improves coding efficiency and that Feature (a) and Feature (b) achieve a synergistic effect. The values of the weighting matrix inserted in the coded stream are not limited by claim 1. Thus there is no guarantee that, for a given output image quality, the additional bits needed for signalling these values in the coded stream are compensated for by a reduction in the number of bits necessary to signal the quantized coefficients. In any case, even if the board acknowledged that Feature (b) improved coding efficiency, the resulting coding efficiency obtained by Feature (a) and Feature (b) would merely be the sum of the respective impact of Feature (a) and Feature (b) on coding efficiency.

4.8 It is established case law that, if the features or sets of features of a claim are a mere aggregation of these features or sets of features which are not functionally interdependent, i.e. do not mutually influence each other to achieve a technical success over and above the sum of their respective individual effects, it is to be established whether each set of features is separately obvious in the light of the state of the art (Case Law, I.D.9.2.2).

4.9 Thus, in view of point 4.7, it is to be established whether each of Feature (a) and Feature (b) is separately obvious in the light of the state of the art.

4.10 Feature (a)

4.10.1 In view of point 3.5 above, the board has doubts whether Feature (a) results in less switching between orthogonal transformation sizes and thus whether it achieves the intended simplification of the coding apparatus. However, in the appellant's favour, the following assessment of inventive step is based on the assumption that the selection between a first and a second orthogonal transformation size is performed independently for the various motion compensation blocks within a macroblock, and thus that Feature (a) leads to less switching between orthogonal transformation sizes within a macroblock.

4.10.2 Hence the board formulates the first partial objective technical problem as being how to simplify the coding apparatus of document D5.

4.10.3 The board does not agree with the appellant that the person skilled in the art would not have thought of reducing the number of possible orthogonal transformation sizes. It is obvious to seek simplification in any part of an apparatus. The person skilled in the art of video coding would have sought to simplify any part of the coding apparatus disclosed in document D5, in particular any part related to the orthogonal transformation.

4.10.4 Depending on whether greater simplification or greater flexibility was desired, the person skilled in the art would have used only one of the possible orthogonal transform sizes - for example the 4x4 pixel size (point X.(c) above) - or several of them. The 4x4 and 8x8 pixel sizes are the only sizes representing common denominators among the orthogonal transform sizes used in document D5 (8x8, 8x4, 4x8, 4x4). The person skilled in the art would have correctly foreseen the trade-off between simplification and flexibility achieved by using these two sizes. Hence this solution merely represents one obvious solution among several.

4.10.5 In view of the above, Feature (a) does not contribute to an inventive step.

4.11 Feature (b)

4.11.1 In point 4.9 above, the board came to the conclusion that Feature (b) had to be assessed separately from Feature (a).

4.11.2 The board agrees that Feature (b) increases coding flexibility.

4.11.3 Thus the board formulates the second partial objective technical problem as being how to increase the flexibility of the coding apparatus.

4.11.4 Since the insertion of a weighting matrix in the coded stream is contemplated in the standard MPEG-2 (as recalled in document D7, column 16, lines 28 to 31), the board finds that Feature (b) is obvious in view of the common general knowledge of the person skilled in the art.

4.12 In view of the above, the subject-matter of claim 1 lacks inventive step within the meaning of Article 56 EPC 1973 because it is obvious in view of the disclosure of document D5 combined with the common general knowledge of the person skilled in the art.

5. Auxiliary request I, admittance (Article 13(2) RPBA 2020)

5.1 In the present case, the summons to oral proceedings was notified after the date on which RPBA 2020 entered into force, i.e. 1 January 2020 (Article 24(1) RPBA 2020). Thus, in accordance with Article 25(1) and (3) RPBA 2020, Article 13(2) RPBA 2020 applies to the question of whether to admit the appellant's auxiliary request I, which was filed after the summons to oral proceedings was notified, and it is therefore an amendment within the meaning of Article 13(2) RPBA 2020.

5.2 Article 13(2) RPBA 2020 implements the third level of the convergent approach applicable in appeal proceedings (see document CA/3/19, section VI, Explanatory remarks on Article 13(2), first paragraph, first sentence; see also Supplementary publication 2, OJ EPO 2020). Article 13(2) RPBA 2020 imposes the most stringent limitations on appeal submissions which are made at an advanced stage of the proceedings, namely after expiry of a period set by the board of appeal in a communication under Rule 100(2) EPC or, where no such communication is issued, after notification of a summons to oral proceedings (see document CA/3/19, section VI, Explanatory remarks on Article 13(2), first paragraph, second sentence). Where an amendment is made to a party's appeal case at this advanced stage of the proceedings, Article 13(2) RPBA 2020 provides that it will, in principle, no longer be taken into account unless the party concerned has shown compelling reasons why the circumstances are exceptional. If such circumstances are shown to exist, the board of appeal may, in exercising its discretion, decide to admit an amendment made to the appeal case at this advanced stage of the proceedings (see document CA/3/19, section VI, Explanatory remarks on Article 13(2), third paragraph, last sentence).

5.3 The Explanatory remarks on Article 13(2) RPBA 2020 also contain the following guidance: "At the third level of the convergent approach, the Board may also rely on criteria applicable at the second level of the convergent approach, i.e. as set out in proposed new paragraph 1 of Article 13." (Document CA/3/19, section VI, Explanatory remarks on Article 13(2), fourth paragraph). The board takes the view that, at the third level of the convergent approach, the boards of appeal are free to use or not use the criteria set out in Article 13(1) RPBA 2020 when deciding, in exercising their discretion in accordance with Article 13(2) RPBA 2020, whether to admit an amendment made at this stage of the proceedings (see also decisions T 989/15, point 16.2 of the Reasons, and T 954/17, point 3.10 of the Reasons).

5.4 The board accepts the appellant's argument that the board's comments in point 3.3.2 of its communication under Article 15(1) RPBA 2020 represent exceptional circumstances within the meaning of Article 13(2) RPBA 2020. However, it is still within the board's discretion to admit the request into the appeal proceedings.

5.5 The board finds it appropriate to rely on the criterion of "suitability" set out in Article 13(1) RPBA 2020. Therefore, the board exercises its discretion in view of the suitability of the amendment to resolve the issue of lack of inventive step raised by the board.

According to the appellant, the amendment to the patent application was only meant to clarify that a selection between a first and a second orthogonal transformation size was not made for only a single motion compensation block (see point XI. above).

Since the analysis of inventive step in relation to the main request is already made on this assumption (see point 4.10.1 above), the board finds that the amendment is not suitable to resolve the issue of lack of inventive step raised by the board.

5.6 In view of the above, the board exercised its discretion under Article 13(2) RPBA 2020 and decided not to admit auxiliary request I into the appeal proceedings.

6. Auxiliary request II - request for remittal

Since the board has decided on the allowability of the main request and the admittance of auxiliary request I, remitting the case to the department of first instance for further prosecution under Article 111(1), second sentence, EPC 1973 would have no meaningful purpose. Therefore, the appellant's auxiliary request II cannot be granted.

7. Since none of the appellant's requests is allowable, the appeal is to be dismissed.

Order

For these reasons it is decided that:

The appeal is dismissed.