Encoding and decoding spatio-temporal information for super-resolution microscopy.

The SPLIT method Modulating the sample illumination and measuring the temporal dynamics of the fluorescence can alleviate the stringent condition of silencing all the fluorophores located on a specific portion of the DL-PSF. The N photons observed at each pixel can still originate from fluorophores located at any position within the DL-PSF but they could be emitted with different temporal dynamics according to the position of the generating fluorophore in the DL-PSF. The maximum achievable spatial resolution is ultimately determined by the ability to distinguish between different temporal dynamics. The key point here is that the issue of resolving spatial features is translated into the spectroscopy problem of resolving temporal dynamics components. The scheme of the method that we call SPLIT (Separation of Photons by LIfetime Tuning) is depicted in Fig. 1a . Suppose that within the DL-PSF of the microscope, we can distinguish two spatial components 1 and 2 characterized by different temporal dynamics. We make use of the phasor analysis of lifetime data 31 32 33 to represent the two different temporal dynamics as two vectors in the phasor plot. The total number of photons detected at one pixel is the sum of the photons originating in the two spatial components plus the uncorrelated background (BKGD) N = N 1 +N 2 +N BKGD , where only N 1 represents the ‘wanted’ part of all the photons. Following the rules of phasors, the vector P =( g , s ) associated with the intensity decay at one pixel can be expressed as the linear combination of the vectors P 1 =( g 1 , s 1 ) and P 2 =( g 2 , s 2 ) associated with the two components, with weights f 1 and f 2 given by the corresponding fractions of detected photons: P =( N 1 P 1 + N 2 P 2 )/ N = f 1 P 1 + f 2 P 2 . This is a linear system of equations in the unknowns f 1 and f 2 . We can write this system in the form P = M f , where f =( f 1 , f 2 ) is the vector of the fractional components and M ij is the matrix which describes the two different temporal dynamics in the phasor domain. For a given n × n matrix M , the solution of this system is given by f = M −1 P . Once we find f the images N i ( x , y ) ( i =1,…, n ) of the photons emitted in each of the n subdiffraction volumes and the image N BKGD ( x , y ) of the background can be calculated as N i ( x , y )= f i ( x , y ) N ( x , y ) and . As a result the original image N ( x , y ) has been split into n +1 images based on the assumption that we can observe and distinguish, within our observation volume, n different dynamics, associated with n linearly independent vectors in the n -dimensional phasor space. These dynamics, using the RESOLFT concept 6 , are seen as generalized reversible states of an ensemble of molecules, as they do not correspond necessarily to a specific state of the molecule but rather to a temporal fingerprint. The overall dynamics observed in the DL-PSF is described here using the linear combination properties of phasors: if we assume that there are only two components, the phasor will fall on the line connecting the phasors from pure components ( P 1 and P 2 ). If we take into account the uncorrelated background as a third component, the phasor P will fall in a triangle where the vertices are the phasors P 1 and P 2 and the phasor P BKGD of the uncorrelated background ( Fig. 1a ). To separate more than two dynamics components ( n >2) and the uncorrelated background, we extend the analysis to phasors obtained at multiple harmonic frequencies 34 35 . The image formation process in SPLIT is depicted schematically in Fig. 1b . Shortly, the temporal information of the signal at each pixel is used to generate the g and s images. These images are then processed to obtain the final SPLIT images.

The SPLIT method in time-resolved CW-STED We focus now on the specific case of SE-induced lifetime variations and on the CW-STED microscopy architecture, that is, a Gaussian excitation beam and a doughnut-shaped STED beam ( Fig. 2a ). However, the proposed approaches can be easily adapted to other configurations and other state transitions. The first ingredient is a model to describe the n dynamics components into which to split the measured intensity pixel-by-pixel, namely, we need the matrix M . For simplicity, we assume (i) a Gaussian profile of the conventional DL-PSF h ( x ′ ,y ′ ,z ′)=exp(−2 r 2 / w 2 )exp(−2 z ′ 2 / w z 2 ), with w and w z being the beam waists along the radial and axial direction, respectively; and r 2 = x ′ 2 + y ′ 2 the radial distance from the focal point ( x =0, y =0) (ii) a parabolic approximation for the doughnut-shaped STED beam I STED ( r )= I STED ( w ) r 2 / w 2 , with I STED ( w ) the STED beam intensity at position r = w ; (iii) a single exponential decay rate for the unperturbed fluorophores γ 0 =1/ τ 0 , where τ 0 is the unperturbed excited-state lifetime. Under these assumptions the spatial distribution of the decay rate is approximated by a parabolic function γ ( r 2 )= γ 0 + γ 0 k S r 2 / w 2 , where k S = I STED ( w )/ I SAT is the ratio between I STED ( w ) and the saturation value I SAT for which the probability of decay due to SE and spontaneous emission are equal (see Supplementary Note 1 ). Importantly, the value of k S determines the relative variation of decay rate values within the E-PSF of the CW-STED microscope ( Fig. 2a ). The time-dependent fluorescence signal F ( x , y , t ) at each pixel can be obtained by integrating the contribution of all the fluorophores located in the E-PSF centred in the pixel position ( x , y ) (see Supplementary Note 1 ) where C ( r 2 ) describes the concentration of the fluorophores in a concentric region of radius r around the pixel position and K is a constant that depends on the quantum yield of the fluorophore, the maximum of the excitation intensity and the detection efficiency. In contrast to deconvolution methods, the proposed method does not try to reassign photons to the original position, thereby the position at which the fluorophores are located within each concentric cylinder is not important. In other words, this approach does not need prior knowledge of the E-PSF of the CW-STED system, which makes this approach suitable also for non-expert users. The temporal dynamics of F ( x , y , t ) encodes nanoscale spatial information in the distribution of exponential decay components. We split the integral and calculate n dynamics components defined uniquely by the parameters γ 0 and k S (see Methods), from which the decoding matrix M is derived. We tested the proposed method on synthetic time-resolved CW-STED images obtained with known γ 0 and k S . Figure 2b shows the ability of the SPLIT method in separating the photons coming from the inner subdiffraction volume from those of the periphery and the uncorrelated background, whereas time gating is affected by an increasing fraction of background ( Fig. 2b,c ). Notably, the spatial features appearing on the background image are due to the approximation of the continuous distribution of dynamics to only two components (see Supplementary Fig. 1 ). The spatial resolution of the SPLIT image can be further increased using a higher number n of components ( Fig. 2d and Supplementary Fig. 2a ). In gated CW-STED microscopy this is done by increasing the time-delay T g ( Supplementary Fig. 2b ). For instance, using n =4 it is possible to get the same spatial resolution of CW-STED but at a STED beam intensity, which is 1 order of magnitude lower ( Supplementary Fig. 2c ). Note that the separation of dynamics obtained in the SPLIT method is conceptually different from time gating. The separation in SPLIT is based on the analysis of variations of the signal over the whole time range ( Fig. 1b ). For this reason, a SPLIT image could be obtained at increasing values of n even when T g is limited by the period T (the reciprocal of the repetition rate, typically in the order of 10 7 Hz) ( Supplementary Fig. 2d ). The SPLIT image exploits the additional spatial information potentially encoded in the g ( x , y ) and s ( x , y ) images (see Supplementary Note 2 ). This additional amount of spatial information is available on a STED image but not on a confocal image ( Supplementary Fig. 3 ). The improvement in spatial resolution in a SPLIT image at increasing values of n comes from the analysis of increasingly higher temporal frequencies in the signal (see Supplementary Note 2 ). Thus, it comes from a better sorting of photons as a function of dynamics/locations. However, when noise is taken into account, the larger the number n of components the higher will be the noise propagated to the final images, quantified as the condition number k cond of the matrix M to invert ( Fig. 2d and Supplementary Fig. 4 ; see Supplementary Note 3 ). For a given level of depletion and for a given level of noise, there is a finite number of values of n for which the noise in the final image is below a desired threshold.

Experimental determination of unknown decoding parameters

To decode the spatial information hidden in the gradients of dynamics induced by the STED beam, we need to know, according to our model, only the two parameters γ 0 and k S. The parameter τ 0 =1/ γ 0 is usually known for a specific fluorophore or can be easily measured from the sample with the very same instrumentation by setting the STED beam power to zero. The parameter k S = I STED ( w )/ I SAT is proportional to the STED beam power but its precise value depends on the optical configuration and on the properties of the sample. It is interesting that, using our analytical model of the SE-induced lifetime variations, we are able to estimate the value of k S from the same image F ( x , y , t ) by considering the average time-resolved decay of all the pixels of an image (see Supplementary Note 1 ), where B denotes the uncorrelated background. To validate the model, we imaged 40 nm fluorescent beads at several STED beam powers ( Fig. 3a ). The two-dimensional (2D) histogram of the values g ( x , y ) and s ( x , y ) associated with each pixel is represented in the phasor plot ( Fig. 3b ). The phasor of the confocal image (zero STED power) is centred to the position corresponding to a single exponential decay with τ 0 =4.5 ns. The same value τ 0 =1/ γ 0 is found by fitting the average photon-arrival time histogram to equation (3) with k S =0 ( Fig. 3c ). With the increasing of the STED beam power the phasor becomes elongated as different dynamics are sampled in the image. The precise value of k S at each STED power can be determined by fitting the average photon-arrival time histogram to equation (3) with τ 0 fixed ( τ 0 =4.5 ns). The good agreement with the model is confirmed by the linearity between k S and the STED beam power. The phasor associated with the theoretical decay expressed by equation (3) for τ 0 =4.5 ns and increasing the value of k S describes the expected trajectory of the average phasor of the image as a function of the STED power. To assess the validity of the method for the imaging of non-point-like structures, we also performed simulations using more convoluted structures similar to those found in cytoskeletal networks (see Supplementary Fig. 5 ). Also in this case, by using the values of k S obtained by fitting the average time-resolved STED decay of the image, we were able to separate the images of the super-resolved components and the background.

Methods

Calculation of phasors and their linear combination The phasor coordinates at a harmonic h corresponding to the time-resolved intensity I ( t ) are defined as 34 : where T is the period of excitation or a smaller value for which the function I ( t ) has already decayed to the uncorrelated background value. If the intensity at one point is due to the sum of two components plus the uncorrelated background, I ( t )= I 1 ( t )+ I 2 ( t )+ I BKGD , then its phasor can be expressed as a linear combination of the phasors of the two components and the phasor of the background (that is, a null vector): where the total number of photons N detected at one pixel is the sum of the photons originating in the two spatial components plus the uncorrelated background N = N 1 +N 2 +N BKGD . It can be seen that since the uncorrelated background is independent of t , its phasor coordinates are (0,0). The addition of uncorrelated background does not affect the value of the phase φ =tan −1 ( s / g ) but decreases the value of modulation m =( g 2 + s 2 ) 1/2 of a phasor. Separation of the intensity into n components The calculation of an arbitrary number of fractional components n was obtained by considering a matrix-vector representation and extending the phasor analysis to higher harmonics. If the intensity I ( t ) is sampled in N bin time windows, then the maximum number of harmonics we can use is N bin /2. P =( g , s , g (2) , s (2) ,…) is the n -element vector formed by the phasor coordinates derived from the intensity decay at one pixel. The last element of the vector P is g (( n +1)/2) (if n is odd) or s ( n /2) (if n is even). Provided that we know the temporal dynamics of the n components I j ( t ), we defined M ij as the n × n matrix whose column j is the vector with the phasor coordinates of the j th component up to g j (( n +1)/2) (if n is odd) or s j ( n /2) (if n is even): Then, provided that det M ≠0, the n -elements vector of the fractional components f =( f 1 ,…, f n ) was calculated by f = M −1 P .

Calculation of dynamics components in time-resolved CW-STED

In time-resolved CW-STED microscopy, the exact temporal dynamics of F ( x , y , t ) depends on the function C ( r 2 ), which acts as a pixel-dependent weight on the exponential decay components exp(− γ ( r 2 ) t ). To approximate the continuous distribution of decays in a discrete number n of components, we split the integral into n parts where I i ( t ) describes the average dynamics of the i th component The boundaries r i of the subdiffraction volumes were chosen in such a way that, for C ( r 2 )=constant, all the time-correlated photons were split in equal number among the n components (see Supplementary Note 1 ).

Simulations and data analysis

Simulations of time-resolved STED microscopy images of point-like particles were performed using custom-built software in ImageJ. The value of the intensity at pixel ( x , y ) and time t originating from N p point-like particles was set as: where , γ 0 =1/ τ 0 is the spontaneous decay rate and r i 2 is the square of the distance from the i th particle. Stacks consisted of 128 frames of 64 × 64 pixels. The parameter S is the maximum intensity signal from a particle expressed in counts detected at one pixel in one frame of the stack. The parameter B represents a uniform level of background expressed in counts detected at one pixel in one frame of the stack. The resulting ideal image is successively corrupted by Poisson noise. For all the simulations reported in Fig. 2b , Supplementary Fig. 1 , Supplementary Fig. 2 , Supplementary Fig. 3 and Supplementary Fig. 4 the confocal waist was set to the value w =167 nm (corresponding to a FWHM=200 nm), the pixel size to 5.2 nm and the time step to Δ t =0.097 ns. The other parameters were varied as indicated in the figures. The confocal image was obtained by adding all the frames of the confocal stack ( k S =0). The STED image was obtained by adding all the frames of the STED stack ( k S >0). The time-gated STED image was obtained by adding only those frames of the STED stack for which t ≥ T g . Simulations of convoluted structures similar to those found in cytoskeletal networks ( Supplementary Fig. 5 ) were performed using MATLAB (MathWorks). The cytoskeletal phantom was composed of 75 filaments with diameter of 30 nm. To each filament, we associated a value between 0 and 1 to simulate differences in the brightness of the structures. The maximum total number of photons detected from a single pixel position in one frame of the stack was set to S =120. A uniform level of background was set to the value B =0.5 counts per each pixel and per each frame of the stack. The phantom was convolved with a theoretical 3D ( x , y , t ) E-PSF of a CW-STED microscope 26 and the obtained image was corrupted by Poisson noise. In the example reported in Supplementary Fig. 5 , the stack consisted of 64 frames of 256 × 256 pixels, showing an area of 10 × 10 μm. The confocal FWHM was set to 235 nm. The total time period was set to T =12.5 ns so that Δ t =12.5/64 ns. The unperturbed decay rate was set to τ 0 =3.15 ns and relative variation of decay rate was set to the value of k S =12.7. Again, in this case, the STED image was obtained as the sum of all the frames in the stack. For the simulations of cytoskeletal structures the value of γ 0 was known, whereas the value of k S was determined by fitting equation (3) to the average time-resolved STED decay of the image ( Supplementary Fig. 5 ). The parameters γ 0 and k S relative to the experimental biological images were extracted from the full fields of view reported in Supplementary Fig. 6 . We extracted first the value of γ 0 by fitting equation (3) to the average confocal decay by fixing k S =0. Then we extracted the value of k S by fitting equation (3) to the average STED decay. By fitting the confocal decay to a single exponential decay, we obtain the value of τ 0 ( τ 0 =2.7 ns for the Alexa Fluor 488 sample; τ 0 =1.8 ns for the Oregon Green sample). Then we fix this parameter and estimate k S from the fitting of the STED decay ( k S =7.8 for the Alexa Fluor 488 sample; k S =4.9 for the Oregon Green sample). To test if the parameter k S varied across the sample, we performed the same analysis in smaller regions-of-interest of different size ( Supplementary Fig. 6c ). The values of k S extracted from regions-of-interest of size down to about 32 pixels were quite consistent between different regions and consistent with the k S value extracted from the whole image. We used the parameters γ 0 and k S to split the time-resolved STED image into the super-resolved components (1 and 2) and the background (BKGD). The SPLIT analysis was implemented writing a custom code in MATLAB. For each pixel ( x , y ) of the time-resolved STED image, the phasor coordinates g ( x , y ) and s ( x , y ) were calculated using a FFT algorithm. The phasor plots reported in Fig. 3 are the 2D histograms of these values and were obtained using Globals for Images (Laboratory for Fluorescence Dynamics). The values of γ 0 and k S were used to generate the expected theoretical decays of the n spatial components and the decoding matrix M . The condition number k cond of the matrix M was calculated in MATLAB. All the fitting procedures were performed in OriginPro (OriginLab) using an unweighted least squares procedure. Experiments All the time-resolved CW-STED experiments were performed on a home-built CW-STED microscope 25 29 . The excitation beam was provided by a supercontinuum source and the STED beam was provided by a CW visible fibre laser (VFL) emitting at 560 nm (VFL-P-1000-560, MPB Communication Inc.). We generated the supercontinuum source by pumping a photon-crystal-fibre (femtoWHITE-800, NKT Photonics) with a femtosecond mode-locked Ti:Sapphire laser of 150 fs pulse width, 80 MHz repetition rate (Chameleon, Vision II, Coherent). To obtain a doughnut-shaped diffraction pattern at the focus the STED beam passed through a polymeric mask imprinting 0–2 π helical phase-ramps (VPP-A1, RPC Photonics). The STED and the excitation beams were collinearly aligned using two dichroic mirrors (zt-488-RDC and z-560-sprdc, AHF analysentechnik), then deflected by two galvanometric scanning mirrors (6215HM40B, CTI-Cambridge) and directed towards the objective lens (HCX PL APO 100/1.40/0.70 Oil, Leica Microsystems) by the same set of scan and tube lenses as the ones used in a commercial scanning microscope (Leica TCS SP5, Leica Microsystems). The fluorescence light was collected by the same objective lens, de-scanned and passed through the dichroic mirrors as well as through a fluorescence band pass filter (ET Bandpass 525/50 nm, AHF analysentechnik) before being focused (focal length 60 mm, AC254-060-A-ML, Thorlabs) into a fibre pigtailed single photon avalanche diode (PDF Series, Micro Photon Devices). Photon-arrival times were detected at each pixel by a time-correlated-single-photon-counting-card (SPC-830, Becker & Hickl). Synchronization was obtained from the reference signal provided by the Ti:sapphire laser. All imaging operations were automated and managed by the software Imspector (Max Planck Innovation). For both the STED and excitation light, the average power P was measured at the back aperture of the objective lens. Due to losses in the objective lens, the power at the sample is actually lower by 15% and 12% at 488 nm and 560 nm, respectively. Samples of 40-nm diameter yellow–green fluorescent spheres (Yellow–Green, Invitrogen) were prepared as follows. The spheres were diluted in water by 1:3,000 (v/v). We dropped the dilute solution of fluorescent beads onto a poly- L -lysine (Sigma) coated glass coverslip, waited 10 min, washed it with water and dried the coverslip by blowing nitrogen onto it. Finally we mounted the coverslip with a special medium (Mounting Medium, Invitrogen) and we observed with the STED microscope. For immunofluorescence imaging, HeLa cells were cultured on glass coverslips (18-mm diameter) in Dulbecco’s modified Eagles medium (Invitrogen) supplemented with 10% foetal bovine serum (Invitrogen), 100 IU ml −1 penicillin and 100 μg ml −1 streptomycin (Invitrogen) at 37 °C in a humidified atmosphere containing 5% CO 2 for 24 h. Plated cells were rinsed with phosphate-buffered saline (PBS) (0.1 M, pH 7.4) and fixed by incubation in 4% formaldehyde in PBS for 15 min. Fixed cells were washed with PBS and permeabilized for 30 min at room temperature with 3% normal bovine serum albumin and 0.1% Triton X-100 in PBS. The cells were then incubated with the monoclonal mouse anti-α-tubulin antiserum (Sigma Aldrich) diluted in 3% bovine serum albumin 0.1% Triton/PBS (1:1,000) for 1 h at room temperature. Anti-α-tubulin antibody was revealed using Alexa Fluor 488 goat anti-mouse IgG (1:500, Molecular Probes) or Oregon Green 488 goat anti-mouse IgG (1:500, Molecular Probes). The coverslips were rinsed in PBS and then placed in an open-bath imaging chamber containing PBS and observed with the STED microscope.

Supplementary Material Supplementary Information Supplementary Figures 1-7, Supplementary Notes 1-3 and Supplementary References