T 1019/99 (Contrast enhancement/AGFA-GEVAERT) of 16.6.2004

Summary of Facts and Submissions

I. This appeal is against the decision of the opposition division to revoke European patent No. 0 527 525.

Opposition had been filed against the patent as a whole based on Article 100(a) EPC in conjunction with Articles 54 and 56 EPC having regard inter alia to the following documents:

D2: SID 86 Digest, 1986, pages 352-353, R.F. Bessler et al.: "Contrast Enhancement using Burt Pyramid Processing".

D13: GB-A-2 195 857

D16: MIT EECS Dept. Senior Thesis, May 1991, A.J. Kalb: "Noise Reduction in Video Images using Coring on QMF Pyramids".

The opposition division held that the subject-matter of claim 1 of the patent as granted or as amended according to the first to fourth auxiliary requests, did not involve an inventive step having regard to D2 and well known techniques in the art.

II. The proprietor (appellant) appealed the decision, and requested that the decision of the opposition division be set aside and that the patent be maintained as granted or on the basis of the first or second auxiliary request filed with the grounds of appeal, dated January 7, 2000.

The respondent (opponent) requested that the appeal be dismissed.

Both parties made an auxiliary request for oral proceedings.

III. Following a communication from the Board, the appellant submitted evidence in the form of five radiographic images and four conversion functions. The images resulted respectively from no applied enhancement, enhancement using the conversion function of the invention labelled "Mode 0", and enhancement by three additional conversion functions labelled "Mode 1", "Mode 6" and "Mode 4".

IV. Oral proceedings were held on 16 June 2004, at which the Board announced its decision to the parties.

V. Claim 1 of the main request reads as follows:

"A method of enhancing the contrast of an electronic representation of an original image represented by an array of pixel values by processing said image, said processing comprising the steps of

a) decomposing said original image into a sequence of multiple detail images at successively lower resolution levels and a residual image at a still lower resolution level, wherein

- the pixels of a detail image represent the amount of variation of pixel values within said original image at the resolution level of the detail image,

- resolution refers to the spatial extent of said variations,

- a residual image is an approximation of said original image with omission of all variations comprised said detail images,

b) modifying the pixel values of said detail images to yield pixel values of a set of modified detail images according to at least one non-linear monotonically increasing odd conversion function with a slope that gradually decreases with increasing argument values, and

c) computing a processed image by applying a reconstruction algorithm to the residual image and the modified detail images, the reconstruction algorithm being such that if it were applied to the residual images and the unmodified detail images, said original image or a close approximation thereof would be obtained."

Claim 1 of the first auxiliary request restricts claim 1 to a "radiographic" image.

Claim 1 of the second auxiliary request adds to claim 1 of the granted claim that the conversion function depends on the value of a pixel in the original image.

VI. The appellant argued as follows:

In the time interval between the publication of D2 in 1986 and the priority date of the patent in 1991, many thousands of researchers were working on the problem of improving the quality of digital imaging. Nevertheless, only the single document D2 proposed a contrast enhancing technique incorporating multi-scale decomposition. All other efforts employed single-scale techniques such as unsharp masking or adaptive histogram equalisation. The skilled person would have therefore considered D2 as an isolated document and would not have used it as a starting point for an invention. Even if the large number of researchers in the field had considered D2, the invention could not be obvious if they had taken five years to come up with it. Since the patent was published, there has been a shift in the industry towards using multi-scale decomposition for contrast enhancement.

The distinguishing features of claim 1 of the main request solved the problem of enhancing the contrast without creating artifacts. There was no hint in any of the prior art of solving this problem using the claimed sub-set of conversion functions.

In particular, D2 acknowledged, at page 352, column 1, penultimate paragraph, that the image enhancement would lead to artifacts. D2 did not attempt or need to remove them because the eye was tolerant to artifacts in video signals at thirty frames per second. The invention, on the other hand, aimed to remove artifacts completely. This would have led the skilled person away from using the teaching of D2 to solve the problem.

Even if the skilled person had tried to implement the conversion function described in D2, there were many other functions that did not fall under claim 1, such as those filed in response to the Board's communication.

The skilled person would not have considered the teaching of D13 because it concerned contrast enhancement using unsharp masking. This was a two-scale decomposition, which could not be combined with the teaching of multi-scale decomposition in D2. Furthermore, D13 only disclosed, in Figure 6, a piecewise linear conversion function that was asymmetric and did not have a slope that gradually decreased with increasing argument values. The skilled person would not have considered D16 because it concerned noise reduction. This involved attenuating low amplitude parts of the signal, whereas the patent was concerned with boosting these parts of the signal. The document did not mention or suggest any application to contrast enhancement.

Claim 16 covered functions with a constant or increasing slope at the lowest amplitude values, whereas according to claim 1 the slope was gradually decreasing over the whole range of argument values. Claim 16 was therefore an independent claim covering a different set of conversion functions.

Concerning the first auxiliary request, the skilled person would not have considered D2 for a radiographic application because D2 related to real-time video images at thirty frames per second in which aesthetic considerations were more important than the fidelity of the information, which was the overriding consideration in radiography. Furthermore, D2 implied tolerating artifacts that could not be allowed in a radiographic image where a correct diagnosis was critical.

In the second auxiliary request, the conversion function depended on the intensity value of the pixels in the original image. The adaptivity mentioned at the end of D2 depended on the information in the detail images, which was only a part of the original image. D2 also only mentioned adapting according to edge strength and not the intensity of the pixels.

VII. The respondent argued essentially as follows:

D2 was not an exotic document because it came from a well known institute and was presented at a well known conference. Since the document concerned contrast enhancement, the skilled person would have considered it. There could have been many reasons why it took five years to arrive at the patent after the publication of D2, including improved technology, dissemination of information, and company R&D policy.

If claim 1 were understood to include the functions defined in dependent claim 16, the subject-matter of claim 1 lacked novelty with respect to document D16.

The skilled person would have arrived at the definition given in claim 1 of the main request in an obvious manner by merely implementing the information contained in D2, namely the functional disclosure of the conversion function, without considering the problem of artifacts.

The skilled person would have also considered D2 when setting out to enhance the contrast of a radiographic image.

D2 also suggested, in the final section entitled "Future Work", using the adaptive conversion function of claim 1 of the second auxiliary request.

Reasons for the Decision

1. The appeal complies with the requirements referred to in Rule 65(1) EPC and is, therefore, admissible.

2. Closest prior art

2.1. The patent concerns the problem of image contrast enhancement in a radiographic imaging system in which there is a large difference in dynamic range between the sensor and the imaging device. This is solved by decomposing the image into multiple detail images at different resolution levels (multi- scale decomposition) and filtering some resolution levels with a non-linear conversion function.

2.2. Documents D13 and D16 do not relate to multi-scale contrast enhancement, nor do they disclose conversion functions identical to those of claim 1. In this context the Board does not consider the functions defined in claim 16 of the patent as granted to fall under the definition of claim 1, i.e. claim 16 is in fact an independent claim. D2, however, both concerns multi-scale enhancement and discloses a functional form of the conversion function that covers the claimed ones.

2.3. Nevertheless, the appellant is suggesting that D2 cannot be taken as a starting point to arrive at the invention because, on the one hand, it is an isolated document that no one had worked on in the five years prior to the patent, and, on the other hand, it mentions artifacts, the avoidance of which is a primary object of the invention.

2.4. The Board first notes that any document that is state of the art under Article 54(2) EPC may be a candidate for the closest prior art; the state of the art is everything made available to the public. The jurisprudence acknowledges, however, some cases where a document may not be a realistic starting point because it either relates to outdated technology, and/or is associated with such well known disadvantages that the skilled person would not even consider trying to improve on it. In the present case, the appellant is essentially offering an additional reason for not using D2, namely that it did not receive any attention after its publication.

2.5. Considering these various criteria, firstly, the Board does not judge that D2, published only five years before the priority date of the patent, in any way represents outdated technology, even in a fast moving area such as digital image processing. Secondly, it is true that D2 mentions artifacts, which are clearly undesirable in video images. However, the skilled person knows that image processing invariably results in artifacts in the image and is constantly trying to eliminate them or, at least, reduce their visibility to suit a particular application. The Board judges that the mere fact that D2 mentions artifacts does not represent such a well known disadvantage that the skilled person would not consider it. Moreover, D2 states that "Artifacts and distortions will tend to be dispersed globally over the entire image reducing their noticeability." Contrary to the appellant, the Board judges this to be a statement that would encourage the skilled person to consider the teaching of this document to reduce artifacts. Finally, concerning the status of D2 as an isolated document, the Board agrees with the respondent that there may be various unknown technical or economic reasons preventing an otherwise promising approach from being adopted rapidly after its early publication. In any case, a period of five years does not appear to be excessive, in particular, taking into account the time needed actually to implement and evaluate the teaching of D2.

2.6. The Board accordingly judges that D2 can be used as the closest prior art.

3. Objective technical problem

3.1. It is common ground that claim 1 of the main request differs from D2 in that the non-linear conversion function is specified to be monotonically increasing, odd and to have a slope that gradually decreases with increasing argument values, whereas the non-linear conversion function of D2 is chosen to "boost the low amplitude values and attenuate the high amplitude values within a given band" (see page 352, end of column 2).

3.2. The appellant argues that these features solve the problem of enhancing the contrast without creating artifacts, whereas the respondent considers that the problem is simply to implement the functional definition of the conversion function given in D2.

3.3. It is established case law that the objective technical problem to be used in the problem and solution approach is to be formulated so that it does not anticipate or contain pointers to the solution. This constrains the specificity of the formulation. However, there is also a constraint on the amount that the formulation can "back off" from this specificity, i.e. a constraint on the generality of the problem. The problem can be no more general than the disclosure of the prior art allows. Otherwise, a problem could be formulated so generally as to circumvent indications in a prior art document towards the claimed solution. Thus the correct procedure for formulating the problem is to choose a problem based on the technical effect of exactly those features distinguishing the claim from the prior art that is as specific as possible without containing elements or pointers to the solution.

3.4. In the present case, if D2 had disclosed a specific conversion function that did not overlap with the claimed function, the difference would have been the totality of the new function, and a general problem could possibly have been formulated such as that of avoiding artifacts. However, D2 discloses a functional form of the conversion function that actually covers the claimed function and thus already discloses some part of it. The only differences are the characteristics of the function that fulfil the functional definition. Furthermore, although the present patent specification mentions the object of reducing artifacts, it does not explain any surprising reduction in artifacts achieved by choosing the claimed characteristics of the conversion function over other possibilities. All this points to a more specific problem taking into account the complete disclosure of D2. The Board therefore judges that the direct effect of the claimed function is to find appropriate characteristics that meet the functional definition given in D2, as stated by the opposition division and respondent.

4. Inventive step (main request)

4.1. Starting from D2, the skilled person is thus faced with the task of implementing the functional disclosure of the non- linear conversion function, namely one that boosts the low amplitude values and attenuates the high amplitude values.

4.2. During the oral proceedings, the respondent drew a graph showing a starting point that was effectively a "non- modifying" function of a straight line passing through the origin and having a constant slope of one. In the grounds of appeal, the appellant had already objected to the opposition division taking this same approach essentially because D2 did not disclose this starting point, but only the above-mentioned functional form. The Board agrees that the analysis must start from what is disclosed in D2, but views the opposition division's and respondent's analysis rather as a visualisation of how the skilled person would attempt to put the abstract notion of the disclosed functional definition of the non- linear function in concrete terms.

4.3. The Board also agrees with the respondent that it follows from the functional requirement of boosting low amplitude values and attenuating high amplitude values, that the "non- modifying" function must be modified so that a point on the graph with a low amplitude value must be shifted up relative to a point with a higher amplitude value. If this process is applied to successive pairs of points without making any other modifications, the Board sees no other possibility than to arrive at a monotonically increasing function with a slope that decreases with increasing argument. The Board cannot imagine that the skilled person would consider any function that decreases with increasing amplitude values, since it would not start at the origin and would result in a reversal of the image.

The respondent also advances the argument that if the function were not monotonic, such as the appellant's "Mode 1" function accompanying the radiographic images, it would not be invertible, meaning that a given output value would have more than one corresponding input value. The Board is not fully convinced by this argument because the path set out above that the skilled person would follow to implement the function of D2 does not require an inversion of the function; an inversion is only described in connection with the Figure 4e embodiment of the invention. However, the Board equally sees no reason for the skilled person, using only the information in D2, to introduce any such non-monotonic element into the function.

4.4. It is common ground that in the present context, the term "gradually" should be interpreted as meaning smoothly. Again the Board judges that a gradual curve is an obvious possibility. It is true that a two-piece linear function, such as the appellant's "Mode 4" function accompanying the radiographic images, is a straightforward possible implementation. Again, however, there is no reason, following the teaching of D2 alone, to introduce an unsmooth portion into the function. Moreover, the Board judges that the skilled person would generally appreciate that a function with such unsmooth portions is likely to cause more distortion than a smooth function. Although, it is well known that piecewise linear functions containing straight sections are often used as approximations when smooth functions are desired because they are easier to implement, D13, at page 5, lines 34 to 35, implies that a smooth function is preferable to a discrete one when describing the virtually identical conversion function of Figure 6.

4.5. Finally, the Board judges that the skilled person would certainly consider the starting point of treating light-to- dark transitions the same as dark-to-light transitions, so that the function should be odd. This would be particularly so in an application where the fidelity of the information is more important than any perceived improvement.

4.6. The Board therefore judges that, although the appellant is correct in stating that D2 covers an infinite number of conversion functions, the skilled person would arrive at the claimed function (also covering an infinite number) in an obvious manner from only the information contained in D2. The main request is accordingly not allowable (Article 56 EPC).

5. Inventive step (first auxiliary request)

5.1. Claim 1 of the first auxiliary request limits the method of enhancing the contrast of an image in claim 1 of the main request to apply only to a "radiographic" image.

This restriction specifies the object claimed in claim 1 (radiographic image) so that it is no longer quite the same as that in the closest prior art (image in general).

5.2. A difference in the claimed object and the object disclosed in an otherwise similar prior art document typically causes difficulties applying the problem and solution approach. This is because an additional difference arises reflecting the change in object. In order to formulate the additional problem solved by this difference without containing elements or pointers to the solution, namely the new object, a form such as "find an alternative application for ... [the subject-matter of the prior art document]" is often the only possibility. This type of problem does not usually represent at all what a skilled person would do in real life and consequently leads to artificial reasoning. In these cases, another closest prior art document is usually chosen that relates to the same object as the claim. It is then a question of fact in the particular case whether the skilled person would combine the new starting point with the original document.

5.3. However, in the present case, the Board considers that a realistic problem can be derived from the additional difference of the restriction to a radiographic image. This is because D2 is a general teaching in the field of image processing as it refers to "many image analysis applications" in the opening line. Furthermore, the opening two paragraphs of D2 mention a specific problem in this general field, namely enhancing the contrast of an image with a discrepancy in dynamic range between the image sensor and the display device. Thus, the objective problem solved by the additional difference could be viewed as applying the contrast enhancement technique of D2 to a particular image analysis application where the image has such a discrepancy in dynamic range between the image sensor and the display device.

5.4. However, as explained in the introductory part of the description of the patent, it is well known that the primary problem encountered when reproducing radiographic images is caused by the discrepancy in dynamic range between the image sensor and the display device. Thus the Board judges that the skilled person would consider applying the contrast enhancement technique of D2 to radiographic images.

5.5. The Board does not agree with the appellant that the skilled person would not consider applying D2 to a radiographic application because D2 relates to real-time video images at thirty frames per second and because it implies tolerating artifacts which cannot be allowed in a radiographic image where a correct diagnosis is critical.

Firstly, radiographic systems also employ video processing systems, albeit high-end, and may also provide a moving image. Secondly, as mentioned in connection with the main request (see point 2.5), the skilled person would have known that artifacts were inevitable when applying non-linear conversion functions. Thus the question is rather how to control them to a degree that is acceptable in a radiographic image. In the Board's view the passage in D2 stating that the artifacts and distortions are dispersed globally over the entire image reducing their noticeability gives hope that the multi-scale decomposition technique might produce acceptable radiographic images, so that the skilled person would at least try it.

5.6. The Board accordingly judges that the subject-matter of claim 1 of the first auxiliary request does not add anything inventive.

6. Inventive step (second auxiliary request)

6.1. Claim 1 of the second auxiliary request adds to claim 1 of the granted claim that "said conversion function depends on the value of a pixel in said original image."

6.2. The parties agree that this feature means that the conversion function depends on the intensity value of the pixels in the original image.

6.3. The Board judges that this additional difference solves the additional problem of improving the contrast enhancement.

6.4. The Board agrees with the respondent that the skilled person would consider improving the contrast enhancement because D2 discloses, at page 353 in the first paragraph under the heading "Future Work", that identical processing over all areas is non-optimal. Moreover, the paragraph also states that this is because images often consist of extreme variations in background illumination and detail. The Board agrees with the respondent that in the light of this the skilled person would consider adapting the processing to the brightness of the image. Since the same paragraph also states that the problem with the non-adaptive scheme is that what gets done to an image sample in one area of the image gets done identically to an image sample in all areas of the image, the Board agrees that this would suggest to the skilled person that the adaptivity should vary on a pixel to pixel basis, and hence on the value of the pixel as claimed.

6.5. Although the appellant is correct in stating that other passages in D2 also mention that the adaptivity depends on the information in the detail images, which is only a part of the original image, and mentions adapting according to edge strength, the Board judges that D2 nevertheless suggests the claimed adaptivity for the reasons given above.

6.6. The subject-matter of claim 1 of the second auxiliary request accordingly lacks an inventive step.

7. Since claim 1 of all the requests lacks an inventive step, it is not necessary to consider further the allowability of independent claim 16 of the requests, and it follows that the appeal must be dismissed.

ORDER

For these reasons it is decided that:

The appeal is dismissed.