ImageJ User & Developer Conference 2008
bUnwarpJ: Consistent and Elastic Registration in ImageJ. Ignacio Arganda-Carreras1,2 Biocomputing Unit, National Centre for Biotechnology (CSIC). Madrid, Spain 2 Escuela Politécnica Superior. Universidad Autónoma de Madrid. Madrid, Spain 1
1
Outline
User or developer? What can this plugin offer me?
As user: Main features. Parameters. Results.
As developer: Flow chart. Optimizer. Future work? Extensions?
2
bUnwarpJ for users (if you dare…)
3
Consistent and elastic registration
4
The plugin: main features Source and target images Speed vs. Accuracy Deformation precision
Energy function weights
Stopping criterion Output info and save option
5
Image and deformation representation: cubic B-splines • Multi-resolution approach
• Spline deformation • Spline interpolation • Vectorial splines
6
So, what is the deformation precision?
Multi-resolution: from “Very coarse” to “Super Fine”. Meaning: from 20 x 20 = 1x1 intervals of B-spline coefficients to 24 x 24 = 16x16 intervals. Basically, more B-spline coefficients, more details.
Deformation pyramid Image pyramid
7
Energy function E =w i E img w E w d E dvg wr E rot w c E cons Images’ Similarity
Landmarks
Regularization
Deformation Consistency
Image similarity: MSE - Mean Square Error (grayscale pixel value) Landmarks: geometric error between landmark points. Regularization: divergence and curl of the deformations. Consistency: geometric distances between the pixel coordinates after applying both transformations (direct and inverse). 8
Weights: similarity and landmarks
Similarity: difference between pixel values. Weight: 1.0 usually enough. Landmarks: distance between manual landmark points. Weight: 1.0 (if any). 9
Weights: regularization
The regularization weights penalize the divergence and curl of the vector field. Meaning: we penalize vector fields with many points like this:
Result: we force the deformation to be smooth. Weights: 0.1 and 0.1 when there’s no prior knowledge about the deformation shape. 10
Weights: consistency
Similarity-Consistency 2160,00
35
2140,00
30
2120,00
25
2100,00 2080,00
20
2060,00 15
2040,00 2020,00
10
2000,00
Consistency Error
How invertible are the deformations? Weights: 10-30 are usually stable values. Advice: play around!
Similarity Error
Similarity Error Consistency Error
5
1980,00 1960,00 0
20
40
60
80
0 100
Consistency Weight
11
The toolbar
Landmarks
Masks
Input/Output Menu
12
Results information (1) Basic
“Verbose” option checked
(2 stacks: from source to target and vice versa)
13
Results information (2)
If the “Verbose” option is checked, then every step of the optimization process is displayed. The “Stop Threshold” is the difference between these steps that forces the program to end. The optimal error values are displayed at the end of the process.
14
Other relevant features for users
Since version 2.0 (August 29th, 2008), bUnwarpJ is fully compatible with ImageJ macro language. Example: run("bUnwarpJ", "source_image=A target_image=B registration=Accurate initial_deformation=[Very Coarse] final_deformation=[Very Fine] divergency_weight=0.1 curl_weight=0.1 landmark_weight=1 image_weight=1 consistency_weight=10 stop_threshold=0.01 verbose save_transformations");
bUnwarpJ can be called from command line (no GUI). Color images are processed in grayscale and the resulting deformations are applied to the RGB channels. No, there is no such a thing as bUnwarpJ 3D (yet). 15
bUnwarpJ for developers (if you dare…)
16
Flow chart No
Image A
Image pyramids
Next pyramids level
Optimize coefficients for this level
Yes Stop?
Show Results
Image B
Optimal deformations
The initial deformations are the affine transformations between landmarks if they exist or the Identity if they don’t. 17
Optimization Gradient
Gradient descend
• Trust region methods • Levenberg-Marquardt like + BFGS
f(x), x, dx
Optimizer
+Hessian
New x
18
Optimizer steps
It starts at the lowest resolution of both pyramids. It increases first the deformation detail. When the level optimum is found, it moves up in the other pyramid. 19
Results Source Only Elastic
Direct
Target
Inverse
Elastic + Consistent 20
Results (2)
Especially useful for serial images of broken, torn or folded tissue. Example: TEM sections of Lamina tissue from Drosophila Melanogaster.
Images by courtesy of Marta Rivera-Alba
21
SIFT and MOPS plugins support
Automatic landmarks introduced thanks to Stephan Saalfeld's plugin. Don't miss next talk ;-)
22
Future work (any volunteer?)
Extension to 3D images:
Complexity. Visualization? Open source alternatives: Elastix, ITK?
Change similarity metric, mutual information? Detailed manual.
23
Questions?
24
