T 0741/01 () of 22.1.2003

Summary of Facts and Submissions

I. The appellants (opponents) lodged an appeal, received on 4 July 2001, against the decision of the opposition division, dispatched on 7 May 2001, to reject the opposition against the European patent No. 0 592 089. The fee for the appeal was paid on 4 July 2001. The statement setting out the grounds of appeal was received on 11 September 2001.

II. Opposition had been filed against the patent as a whole on the basis of Article 100(a) EPC, and in particular on the grounds that the subject-matter of the patent was not patentable within the terms of Articles 52 to 57. EPC because it did not involve an inventive step. To support their objections the opponents referred inter alia to the following documents:

(D8) Cell Calcium, vol. 13, 1989, pages 473 to 488, M. Gustafson et al. "A novel principle for quantitation of fast intracellular calcium changes using Fura-2 and a modified image processing system - applications in studies of neutrophil motility and phagocytosis";

(D11) Proceedings RMS, vol. 23, No. 5, September 1988, pages 289 to 297, D. Shotton "The Current Renaissance in Light Microscopy II. Blur-Free Optical Sectioning of Biological Specimens by Confocal Scanning Fluorescence Microscopy";

(D12) Trends in Biochem. Sci., vol. 11, November 1986, pages 450 to 455, R.Y. Tsien et al. "Fluorescence ratio imaging: a new window into intracellular ionic signaling";

(D13) DE-A-36 23 576

(D14) US-A-4 470 076

(D15) Journal of Cell Biology, vol. 109, September 1989, pages 1219 to 1227, T.R. Cheek et al. "Simultaneous Measurements of Cytosolic Calcium and Secretion in Single Bovine Adrenal Chromaffin Cells by Fluorescent Imaging of Fura-2 in Cocultured Cells".

III. On 22 January 2003 oral proceedings were conducted according to the auxiliary request of the respondents.

IV. At the oral proceedings the appellants requested that the decision under appeal be set aside and that the patent be revoked.

V. The respondents requested that the appeal be dismissed and that the patent be maintained as granted or, auxiliarily, on the basis of the requests 1 to 7 filed with the letter dated 19 December 2002.

VI. The wording of independent claim 1 of the main request reads as follows:

"A confocal microscope for scanning a sample (16) doped with a predetermined fluorescent indicator, comprising:

a scanner (9,10) for repeatedly scanning a laser beam (LB) in two dimensions across the sample, whereupon the sample fluoresces in first and second predetermined wavelengths;

first (22) and second (26) detectors for detecting light emitted by the sample in the first and second wavelengths and for generating a succession of frames of image data representing a sequence of related pairs of two-dimensional images of the sample, in the first and second wavelengths;

first (302) and second (304) frame memories for temporarily storing individual frames of image data for the respective first and second wavelengths;

a permanent data storage device (32);

transfer means (332,334) for transferring image data derived from the data generated by the first and second detectors to the permanent data storage device; and

a video display (31) for providing a repeatedly updated display of the scanned sample;

characterised in that the individual frames of image data for the respective wavelengths are of simultaneous two-dimensional images of the sample obtained simultaneously from the detectors;

in that the transfer means (332,334) are for transferring the image data in an alternating fashion from the detectors to the permanent data storage device;

by first (310,314) and second (312,316) averaging means for averaging a selected number of successive frames of image data representing the respective first and second wavelengths, to produce a sequence of related pairs of average image data for delivery to the ratio means;

by ratio means (326) for repeatedly computing and temporarily storing the ratio of the related pairs of average image data from the respective first and second averaging means; and

by video means (328,330) for receiving the sequence of ratios of average image data produced by the ratio means and providing a signal suitable for coupling to the video display (31), to provide a repeatedly updated display of the scanned sample that is substantially unaffected by any variations in the intensity of the scanning laser beam or in the local concentration in the sample of the fluorescent indicator."

The wording of independent claim 10 of the main request reads as follows:

"A method for scanning a sample (16) doped with a predetermined fluorescent indicator and providing a video display of the sample, comprising the steps of:

repeatedly scanning (9,10) a laser beam (LB) in two dimensions across the sample, whereupon the sample fluoresces in first and second predetermined wavelengths;

detecting (22,26) light emitted by the sample in the first and second wavelengths and generating a succession of frames of image data representing a sequence of related pairs of two-dimensional images of the sample, in the first and second wavelengths;

temporarily storing in first (302) and second (304) frame memories individual frames of image data for the respective first and second wavelengths;

transferring (332,334) image data derived from the data for the first and second wavelengths, generated in the step of detecting, to a permanent data storage device (32);

and displaying the scanned sample on a video display (31);

characterised in that the related pairs of two-dimensional images of the sample are of images detected simultaneously;

in that the image data are transferred to the permanent data storage device (32) in an alternating fashion between the first and second wavelengths;

by a step of averaging (310,316) a selected number of successive frames of image data representing the respective first and second wavelengths, to produce a sequence of related pairs of average image data;

by a step of repeatedly computing (326) and temporarily storing the ratio of the average image data for the respective first and second wavelengths, to produce a sequence of image data ratios;

and by a step of receiving (328,330) the sequence of ratios of average image data and providing a video signal suitable for coupling to a video display (31), to provide a repeatedly updated display of the sequence of ratios of average image data that is substantially unaffected by any variations in the intensity of the scanning laser beam or in the local concentration in the sample of the fluorescent indicator."

Claims 2 to 9 and 11 to 17 are dependent claims.

VII. The arguments of the appellants may be summarised as follows.

Document D11, which forms the closest prior art, discloses a confocal scanning fluorescence microscope and a method of scanning a fluorescently doped sample with the features of the preambles of respective claim 1 and claim 10. In particular Figure 3 and the Section "Dual wavelength imaging" on page 292 disclose a scanned beam confocal microscope including a laser light source which may have multi-line emission with which two labels can be excited simultaneously and a dual channel photomultiplier, furthermore this Section discloses that the two images are obtained in exact spatial register. The apparatus disclosed in D11 comprises a personal computer (PC). Therefore, since the detectors detect images at different wavelengths which, according to the caption of Figure 3, are digitized, stored and processed with dedicated image processing boards, the features "first and second frame memories" and "transfer means" are implicitly disclosed, where it is noted that the definition of "first" and "second" is a matter for the user. Document D11 also discloses on page 292, second paragraph, the advantage of digital image averaging and, on the same page, Section "Dual wavelength imaging", the enormous importance of the use of confocal scanning fluorescence microscopy for direct simultaneous fluorescence emission ratio imaging, whence document D11 anticipates averaging means and ratio means as defined in the independent claims. Finally it is commonplace that a video display is part of every PC, as is also shown in Figure 3 of D11. It follows that the only feature of the independent claims not disclosed in D11 is the feature "in that the transfer means are for transferring the image data in an alternating fashion from the detectors to the permanent data storage device". With respect to the respondents' argument that the apparatus disclosed in D11 would not be able of performing the data processing at video speed it should be noted that image readout at standard video rate is disclosed in D11 (page 291, left column, lines 5 and 6), and that, furthermore, such a feature is not defined in the independent claims.

The objective problem which may be defined by this difference is to provide an efficient and organized data transfer of image data, collected at the detectors, to a storage medium for further processing and to store the transferred data in an efficient and organized manner. The skilled person would find a solution of this problem in D15. This document is related to the similar application of ratio-imaging in microscopy. The skilled person learns from D15 a simultaneous visualisation (page 1220, left column, line 12); a ratio calculation at two wavelengths (same page, right column, line 26); displaying this ratio image on a video screen; and storage of the data on a video tape for subsequent processing. Furthermore D15 discloses an alternating transmission of TV frames of the two wavelengths. In this respect it is pointed out that it is irrelevant for the objective problem that in D15 the ratio of the two images from the different wavelengths is stored and not these individual images. The alternating transfer of images is furthermore disclosed in document D13, which addresses the similar problem of transmission and storage on a video recorder of stereoscopic partial images. This document discloses in claim 6 that the electronic partial images of the first and second cameras are transmitted or stored and displayed line by line and alternating. Therefore the skilled person learns from this document that pictures with different characteristics (e.g. wavelength, perspective...) should be transmitted and stored consecutively. The handling of a data stream of image data comprised of two or more partial images for image ratioing is furthermore disclosed in document D8. Finally document D14 equally discloses an alternating storage of colour signals (column 1, paragraph starting at line 63). Therefore by following the teachings of any of these documents D8, D13, D14 or D15 in solving the problem of efficient data transfer of the apparatus and method disclosed in D11 the skilled person would arrive at the subject-matter of independent claims 1 and 10 without an inventive step being involved.

VIII. The arguments of the respondents may be summarised as follows.

When considering document D11 as the closest prior art for the discussion of inventive step, it should be pointed out that this document is a research paper which suggests many ideas but fails to offer corresponding solutions. This document discloses a confocal scanning fluorescence microscope. According to page 292, left column, lines 22 and 23, by including in the apparatus first and second photodetectors, images of a sample fluorescing at first and second wavelengths may be collected. It is conceded that the confocal microscope shown in Figure 3 of D11 includes a Personal Computer (PC) in which data can be permanently stored, although such a PC is not capable of storing long data streams at video rate, and that this PC includes a video display. It is, however, not explicitly disclosed that this apparatus includes first and second frame memories for temporarily storing individual frames as defined in claims 1 and 10; nor is there a disclosure in D11 for transfer means within the definition of the independent claims, and in particular not that these are for transferring the image data in an alternating fashion to the storage device. Furthermore "averaging" and "ratio imaging" are mentioned in general terms in D11, but there is no functional relation between these computational steps as in the independent claims.

These differences reflect the technical problem addressed by the independent claims, namely to improve prior art confocal microscopy in enabling long data streams at high temporal resolution to be permanently recorded at video rates and provide image ratioing to show cellular Ca2+-concentrations in real time. Since such biological phenomena vary at a short time scale it is imperative that the data processing is at high speed. The apparatus and method defined in the independent claims enable such real time observations, which had been impossible with any of the prior art systems. In particular the claimed apparatus and method combine the instantaneous observation of the ratio-image at a video monitor and an economic long-term storage of the images, which facilitates the observation and marking of any interesting event on the video display and easy retrieval of these stored events. Furthermore in the claimed apparatus and method no pixel information is lost at all, except by the averaging of image frames. In this context it is pointed out that to sacrifice some temporal resolution in order to improve the signal-to-noise ratio in the ratio image by averaging is a non-obvious choice in a system where temporal resolution is an aim.

As to the cited prior art, documents D13 and D14 relate to different fields from that of the invention, and, moreover, use simple switching or analogue processing and are not concerned with digital processing, therefore they are inherently incapable of producing precise numerical data. In any case, neither D13 or D14 mentions ratiometric processing. With respect to D15 it is pointed out that this document is not concerned with and does not disclose a scanning confocal microscope. The illumination of this microscope is switched at approximately 30 Hz, which does not put high requirements on the synchronisation with the view signal. However, in the confocal scanning microscope of the invention the scanning mirror must oscillate in the order of 15 thousand per second, therefore the synchronisation used in D15 would not be useful for the scanning microscope of the patent in suit. Furthermore the images at the two wavelengths in the system shown in D15 are generated in sequence, and therefore 30 milliseconds apart, and not simultaneously as defined in the claims. Therefore a combination of the teaching of this document with D11 does not lead to the invention.

Reasons for the Decision

Inventive step

1.1. There is agreement amongst the parties that document D11 discloses the closest prior art. This document discloses in Figure 3 and its caption a confocal microscope including a scanner ("scanned beam confocal SOM") for repeatedly scanning a laser beam (argon ion laser) in two dimensions across a sample. The apparatus comprises first and second detectors (dual channel detection multipliers). The apparatus further comprises a scanning control, image digitization, image storage and subsequent image processing boards on an IBM PC AT compatible microcomputer. From the ability of the apparatus to detect image signals at two channels it is deduced that the two resulting images must also be storable in the microcomputer, therefore first and second parts of the memory for storing these image frames, to be assigned as "frame memories" must be included. A microcomputer also has means for permanently storing data (e.g. a hard disc), furthermore means for transferring image data from the dual channel photomultiplier to the storage device are equally implicit to the hardware of this microscope, as well as a video display. Therefore it appears that the scanned beam confocal microscope shown in Figure 3 of D11 includes the features of the preamble of claim 1. This applies correspondingly to the features of the preamble of method claim 10.

2.1. With respect to the features of the characterising portion of claim 1 the appellants have referred to Section "Dual wavelength imaging" on page 292 of D11. In particular the first phrase of this Section discloses that it is possible to convert a single wavelength scanned beam confocal fluorescence microscope into a multiparameter instrument capable of simultaneous imaging at two or more wavelengths. As a first application of such a technique it is proposed to simultaneously visualise two labels of a sample, for instance fluorescein and Texas red, by exciting these simultaneously by a multi-line argon laser. As a second application the use of confocal scanning fluorescence microscopy for direct simultaneous fluorescence emission ratio imaging is proposed, in the context of which reference is made to document D12.

2.2. Therefore D11 suggests that a single wavelength confocal scanning microscope such as the one shown in Figure 3 of this document may be modified in two different ways. In the context of the subject-matter of claims 1 and 10 the first modification is of lesser relevance, because these claims are related to ratio imaging, in which process the fluorescence of a sample stained with a single label (dye) and excited by a single wavelength laser source is measured at plural wavelengths. On the other hand, the first application in the cited Section suggests using multi-line excitation and plural labels, in which case the ratio imaging within the definition of claims 1 and 10 is not possible.

2.3. The second modification suggested in this Section "Dual wavelength imaging" is related to the process of ratio imaging as in claims 1 and 10 of the patent in suit. Document D11, however, does not disclose any further instrumental details in which way the apparatus should be modified apart from a reference to prior art documents, for instance document D12. Although according to the cited passage of D11 corresponding pixels in each image are measured "simultaneously", it appears that a simultaneous detection is, in fact, not carried out in the apparatus disclosed in D12. Rather, this document discloses a standard epifluorescence microscope equipped with a mechanism for frequent alternating between two excitation wavelengths and a single photodetector. It therefore appears that the confocal scanning microscope shown in Figure 3 of D11 and the microscope from D12 are fundamentally different (scanning versus standard epifluorescence microscope; excitation by laser versus excitation by a light source and two interference filters for providing two wavelengths; detection by a dual channel photomultiplier versus a single photomultiplier or a low-level television camera). Furthermore, since in D12 the images are collected alternatingly no teaching as to how the apparatus should be modified for simultaneous detection can be obtained from D12. Hence the reference made in D11 to document D12 can only be seen as a general reference to the technique of ratio imaging and the only suggestion obtainable from document D11 in this respect is that an application of confocal scanning fluorescence microscopy for direct simultaneous fluorescence emission ratio imaging would be desirable.

3.1. In the opinion of the board, this suggestion, to modify a prior art confocal scanning microscope such as the one shown in Figure 3 of D11, for providing direct simultaneous fluorescence emission ratio imaging and its related data storage can be seen as the objective problem.

3.2. Since in the context of ratio imaging document D11 suggests to use a single dye (Indo 1, see the cited Section), it would appear obvious to adapt the wavelength of the excitation laser source and the dual-channel detection photomultipliers (for instance, by selecting appropriate interference filters) of the system shown in Figure 3 in order to collect the images at the two required wavelengths. In this scanning confocal system the laser illuminates each voxel of the sample pointwise thereby exciting it to emit fluorescence radiation, which fluorescence signals at the two wavelengths are collected simultaneously. Hence, by adaptation of the microscope of Figure 3 of D11 for ratio imaging, this microscope would automatically include the first feature of the characterising portion of the independent claims (the individual frames of image data ...obtained simultaneously from the detectors).

4.1. With respect to the further features which are related to the data transfer, their processing and the visualisation, the appellants have referred to other passages of D11, and furthermore to the documents D8, D13, D14 and D15.

4.2.1. Claim 1 defines the feature that the transfer means are for transferring the image data in an alternating fashion from the detectors to the permanent data storage device. A similar requirement in forms of a process step is defined in claim 10. From the previous features in these claims this feature must be construed as implying that the image data are transferred in the form of image frames.

4.2.2. In the cited prior art, document D11 does not disclose any details concerning the image transfer and processing.

4.2.3. Documents D13 and D14 are concerned in transmission of TV-video signals in an analogue form. More particularly in document D13, claim 6, partial analogue TV frames are transmitted line by line, which is different from the alternating transfer of two complete frames as defined in the claims. Likewise in document D14, column 1, lines 31 to 37, the lines of two subsequent CCD half frame pictures are interpolated in order to form a full TV frame.

4.2.4. In document D15 (page 1220, right column, Section "Monitoring Fura-2 in Single Cells and Image Processing"), two TV frames are generated in an alternating way, but for the different reason that they are signals from subsequent and not from simultaneous images, furthermore these TV frames are not stored to a permanent storage device but are immediately used for computing the ratio image.

4.2.5. The gist of document D8 resides in the avoidance of digital processing and image storing, therefore this document does not suggest the claimed features related to the transfer of the image data in an alternating fashion.

4.3.1. As to the features "averaging means" in claim 1, respectively "step of averaging" in claim 10, the respondents have pointed out that these features must be considered together with the following features in these claims "ratio means", respectively "step of repeatedly computing and temporarily storing the ratio of the related pairs of average image data". The board agrees that these algorithm steps and their order are clearly linked by the wording of the independent claims, and that their occurrence in the prior art would similarly have to be in the claimed order.

4.3.2. In document D11, the only reference to digital image averaging (page 292, left column, lines 13 and 14) is a general reference and not in the context of ratio imaging. Therefore this document does not disclose or suggest the claimed features.

4.3.3. Document D15 discusses ratio imaging, albeit in the different instrumental set-up of a standard epifluorescence microscope, two alternating light sources and a single photodetector. The fluorescence signal excited at the two different alternating excitation wavelengths generates two TV frames, which are input to a video-rate image processor to give from each subsequent pair of frames a "live" ratio image, which is recursively filtered with a 200-ms time constant. It therefore appears that in the apparatus disclosed in D15 the averaging of the image signal is carried out after the ratioing.

4.3.4. Document D8 teaches to avoid digital image ratioing by the visualisation of fluorescent images excited at different wavelengths in different colours (hues) on a video display, the principle being shown in Figure 2b. As to the reference system shown in Figure 2a, this only discloses "background subtraction, ratio formation and calculation of [Ca2+], therefore no averaging step as defined in claims 1 and 10 is disclosed.

4.3.5. As to the further documents D13 or D14, these are not documents from the field of confocal scanning fluorescence microscopy and do not discuss ratio imaging.

4.4.1. Since the available prior art does not teach or suggest the discussed features concerning the image transfer or the image averaging-ratioing process step, it can also not disclose or suggest the last feature in the independent claims related to the receiving and displaying of the sequence of ratios of average image data, because the prior features in these claims are a requisite for this last feature.

4.4.2. Furthermore, although the above analysis of the features has been carried out by addressing the features concerning the image transfer and the averaging/ratioing separately, it is emphasised that these features cooperate to solving the technical problem of providing direct simultaneous fluorescence emission ratio imaging and its related data storage. Since there is no teaching for these features separately in the prior art documents, there is even less teaching of the claimed features in combination.

5. Therefore, in the opinion of the board there is no reason why the skilled person, when attempting to adapt the confocal scanning laser microscope in Figure 3 of D11 for ratio imaging, would have arrived at the subject-matter of the independent claims 1 and 10 of the main request in an obvious way.

6. Since the main request of the respondents is allowable, there is no need to address the auxiliary requests.

ORDER

For these reasons it is decided that:

The appeal is dismissed.