Core 3: Biomedical Image Analysis
Segmented vesicle witih proteins
Within CISMM we are providing new methods for image analysis of biomedical data. We are developing automatic and semi-automatic methods to allow for quantitative analysis. Developed methods range from preprocessing algorithms (to facilitate subsequent quantitative analysis) to segmentation methods. Our design philosophy is to develop image analysis algorithms by focusing on a particular analysis problem. The developed methods generally have applicability beyond the original application.
- Image Preprocessing: (i) We have started working on the implementation of deconvolution methods for the new version of the Insight Toolkit (ITKv4) as part of a subcontract from the National Library of Medicine (NLM). (ii) We have also developed a novel method for the appearance normalization of H&E-stained histology slides, which is critical to allow for quantitative comparisons of image features for large-scale (possibly multi-site) histological studies.
- Model Extraction: We have made progress on a number of model extraction methods. We have (i) developed a method to quantify the orientation of meshes which can be used to evaluate mesh formation under flow. (ii) We have also developed a new method for the semi-automatic segmentation of synaptic vesicles, which makes use of robust segmentation method with applicability to the general segmentation of biomedical structures.
- Image Registration and Atlas-Building: We have developed a number of novel image registration methods, in particular we have worked on methods dealing with image time-series. We have developed (i) a method for the longitudinal atlas-building from diffusion tensor images, (ii) a generalization of least-squares line fitting to the space of images (termed geodesic regression) which allows for compact representation of image-time series by their initial momenta and hence for simplified subsequent statistical analysis, and (iii) have developed a registration method which can account for image appearance changes (as for example caused by a traumatic brain injury or a brain tumor) by jointly estimating a global space deformation and an overlaid geometric model change affecting image appearance.
Image Preprocessing
Deconvolution Methods
Our purpose is to provide high-quality reference deconvolution algorithms in open-source software to educate users in the life sciences about the applicability of particular deconvolution algorithms to their data. This project is part of a subcontract form the National Library of Medicine (NLM) to develop deconvolution methods for the new version of the Insight Toolkit (ITKv4). We will provide high-quality deconvolution algorithm implements as classes in the ITK library, documentation on the mathematical background and noise assumptions made by the algorithms, examples of how to use the software in other projects, and examples of real images processed with these algorithms.
We have focused so far on the implementation/adaption of deconvolution methods (Wiener filter, Landweber method, and Richardson-Lucy) as well as the implementation of two parametric point spread function (PSF) models. We implemented a modification of the Gibson-Lanni PSF model for widefield microscopes in ITK and started work on the Haeberle PSF model. The modification of the Gibson-Lanni PSF model involves a shearing transform that produces a better match to some measured point-spread functions that we have encountered. We are currently working towards the implementation of a parametric semi-blind deconvolution method.
As part of our fluorescence microscope image curation, we obtained example fluorescence images of 500nm fluorescent beads from one of our collaborators and verified the bead dimensions with additional images of the beads obtained with a scanning electron microscope. The resulting collecting of example images will be hosted on Kitware’s MIDAS system.
Appearance Normalization for Histology Slides
12 sample slides before appearance normalization
12 slides after appearance normalization.
In the context of skin cancer we have developed an automatic method to normalize the appearance of H&E-stained histology slides. This is a critical preprocessing step for subsequent quantitative image analysis to minimize the influence of differences in staining (due to different staining protocols, different ages of slides, etc.) on an analysis result. The method both estimates the stain vectors for hematoxylin and the eosin stains as well as overall stain intensities and transforms them into a pre-specified space for normalization. Prior information on the stains can be incorporates for increased robustness of the estimator. The method is fully automatic and we have used it successfully on hundreds of histology slides.
Image Registration and Atlas-Building
Longitudinal Atlas-Building for Diffusion Tensor Images
Principle of the longitudinal atlas building approach. We estimate individual growth trajectories (per subject) and then compute point-wise averages.
Existing atlas-building methods for diffusion-tensor images are not designed for longitudinal data. We propose a novel longitudinal atlas-building framework explicitly accounting for temporal dependencies of longitudinal MRI data. Subject-specific growth modeling, cross-sectional atlas-building and growth modeling in atlas space are combined with statistical longitudinal modeling, resulting in a longitudinal diffusion tensor atlas. The method captures changes in morphology, while modeling temporal changes and allowing to account for covariates. The component algorithms are based on large-displacement metric mapping formulations. To effectively account for measurements sparse in time, a continuous-discrete growth model is proposed. We applied the method to a longitudinal dataset of diffusion-tensor magnetic resonance brain images of developing macaque monkeys with time-points at ages 2 weeks, 3 months, and 6 months.
Geodesic Regression for Image Time Series
Principle of geodesic regression. The full image path is determined by a geodesic in image space which is fully described by an initial image and an initial momentum. This generalizes linear least squares, where the initial conditions of a line are its y-intercept and slope.
Registration of image-time series has so far been accomplished (i) by concatenating registrations between image pairs, (ii) by solving a joint estimation problem resulting in piecewise geodesic paths between image pairs, (iii) by kernel based local averaging or (iv) by augmenting the joint estimation with additional temporal irregularity penalties. We propose a generative model extending least squares linear regression to the space of images by using a second-order dynamic formulation for image registration. Unlike previous approaches, the formulation allows for a compact representation of an approximation to the full spatio-temporal trajectory through its initial values. The method also opens up possibilities to design image-based approximation algorithms. The resulting optimization problem is solved using an adjoint method. Key to the formulation is to be able to write the image registration problem in initial value form. In the scalar-valued case (for linear regression) this amount to recasting the least square estimation for the line model y=mx+c into the second order dynamical system form d^2/dt^2 y = 0, y(0)=c, dy/dt(0)=c, where the initial conditions are simply the y-intercept and the slope. For the image-case we have an initial image and its initial momentum. In the optimization all images along the geodesic exert forces which influence the initial condition for the geodesic. In the scalar-valued case this amounts to having a least squares solution which can be interpreted as a physical system under force and momentum balance.
Geometric Metamorphosis
Geometric Metamorphosis. An image is explained by a global deformation (via v) and a geometric model deformation (via v ^ au ). Corresponding structures in the source and target guide the estimation of v and v^ au addresses additional appearance diff erences at the pathology. To avoid faulty evaluation of image similarities, a suitable image composition method is required. Regions which carry no matchable information are set to 0 in the image composition model. For a shrinking geometric model (blue) this region is speci ed by I (1) (which already includes the background deformation) and for a growing geometric model (red) by T2.
Standard image registration methods do not account for changes in image appearance. Hence, metamorphosis approaches have been developed which jointly estimate a space deformation and a change in image appearance to construct a spatio-temporal trajectory smoothly transforming a source to a target image. For standard metamorphosis, geometric changes are not explicitly modeled. We propose a geometric metamorphosis formulation, which explains changes in image appearance by a global deformation, a deformation of a geometric model, and an image composition model. This work is motivated by the clinical challenge of predicting the long-term eff ects of traumatic brain injuries based on time-series images. This work is also applicable to the quanti cation of tumor progression (e.g., estimating its in ltrating and displacing components) and predicting chronic blood perfusion changes after stroke.
Model Extraction: Fibrin Mesh Orientation
In collaboration with Alisa Wolberg from Pathology and Laboratory Medicine, Resource graduate student Cory Quammen developed software to perform statistical analysis of fiber orientations in 3D confocal images of fibrin meshes. The core algorithm is a multiscale vesselness approach developed by Frangi et al. (1998) for segmenting blood vessels in medical images. This approach is robust to the kind of brightness changes present in the confocal images, producing better segmentations than intensity-based thresholding. At each voxel, the algorithm computes a measure indicating the likelihood that the voxel is on a fiber as well as the fiber orientation. Thresholding the fiber likelihood produces a segmentation of the entire fibrin mesh. To avoid directional bias caused by noncubic voxels in the 3D confocal images, the software first resamples the confocal images to produce cubic voxels. In addition, the program thins the segmentation so that thick fibers several voxels across will not bias the statistical analysis of the orientations of local fiber segments.
The angle formed between the short segment of fiber represented by the voxel is computed for all the voxels in the skeletonized fibrin mesh. In meshes formed under flow, the reference orientation is the flow direction. A 1D histogram of the angles between each fiber segment and the reference orientation is then computed. Because only one orientation produces a 0-degree angle (the reference orientation) and many orientations produce a 90-degree angle (all directions perpendicular to the reference orientation), the histogram of fiber segment orientations can be difficult to interpret. The histogram is therefore divided bin-wise by the expected histogram from a mesh consisting of uniformly distributed orientations. The resulting plot shows the over- and underrepresentation of fiber orientations in comparison to a mesh with no preferred orientation. Using this approach, we have obtained preliminary evidence that fibers in meshes formed under flow tend to orient themselves with the flow direction while meshes formed under stasis show no preferred orientation.