bioimage-analysis-fundamentals.md 13 KB
Newer Older
Christian Tischer's avatar
Christian Tischer committed
1 2
# Bioimage analysis fundamentals

Christian Tischer's avatar
Christian Tischer committed
3
## Pixel values, coordinates, and data types
Christian Tischer's avatar
Christian Tischer committed
4 5 6 7 8 9 10 11 12 13

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
    image -> pixel [label="  has many"];
    pixel -> value;
    pixel -> indices;
    pixel -> coordinates;
    indices -> calibration;
    calibration -> coordinates;
Christian Tischer's avatar
Christian Tischer committed
14
    image -> calibration [label="  can have"];
Christian Tischer's avatar
Christian Tischer committed
15 16 17 18 19 20
    pixel -> voxel [label="  3D"];   
  }
'/>

### Activity

Christian Tischer's avatar
Christian Tischer committed
21
* Open image: xy_8bit__nuclei_noisy_different_intensity.tif
Christian Tischer's avatar
Christian Tischer committed
22 23 24
	* Explore different ways to inspect pixel values and indices
	* Add image calibration
	* Check where the calibration is visible
Christian Tischer's avatar
Christian Tischer committed
25
	
Christian Tischer's avatar
Christian Tischer committed
26 27
### Formative assessment

Christian Tischer's avatar
Christian Tischer committed
28
True or false?
Christian Tischer's avatar
Christian Tischer committed
29

Christian Tischer's avatar
Christian Tischer committed
30 31 32 33 34
* Pixel coordinates are always integer values.
* Changing the image calibration changes the pixel values.
* Pixel coordinates depend on image calibration.
* Pixel indices are always positive integer values.
* The lowest pixel index of a 2D image always is `[1,1]`.
Christian Tischer's avatar
Christian Tischer committed
35

Christian Tischer's avatar
Christian Tischer committed
36 37 38 39 40 41 42
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
43 44 45 46 47
## Image display

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
48 49 50 51
    LUT -> color;
    LUT -> brightness;
    min -> LUT;
    max -> LUT;
Christian Tischer's avatar
Christian Tischer committed
52
    value -> LUT;
Christian Tischer's avatar
Christian Tischer committed
53 54 55 56
  }
'/>

```
Christian Tischer's avatar
Christian Tischer committed
57 58
brightness = ( value - min ) / ( max - min )

Christian Tischer's avatar
Christian Tischer committed
59
0 <= brightness <= 1 
Christian Tischer's avatar
Christian Tischer committed
60 61

contrast = max - min 
Christian Tischer's avatar
Christian Tischer committed
62 63 64 65
``` 

### Activity

Christian Tischer's avatar
Christian Tischer committed
66
* Open image: xy_8bit__nuclei_noisy_different_intensity.tif
Christian Tischer's avatar
Christian Tischer committed
67
* Explore different LUTs and LUT settings
Christian Tischer's avatar
Christian Tischer committed
68
	* Appreciate that LUT settings do not affect image content.
Christian Tischer's avatar
Christian Tischer committed
69 70


Christian Tischer's avatar
Christian Tischer committed
71 72
### Formative Assessment

Christian Tischer's avatar
Christian Tischer committed
73
Fill in the blanks, using those words: decrease, larger_than, increase, smaller_than 
Christian Tischer's avatar
Christian Tischer committed
74

Christian Tischer's avatar
Christian Tischer committed
75 76 77 78
* Pixels with values _____ `max` will appear saturated. 
* Decreasing `max` while keeping `min` constant will _____ the contrast.
* Decreasing both `max` and `min` will _____ the overall brightness.
* Pixels with values _____ the `min` will appear black, when using a grayscale LUT.
Christian Tischer's avatar
Christian Tischer committed
79 80 81 82 83 84 85 86


&nbsp;

&nbsp;

&nbsp;

Christian Tischer's avatar
Christian Tischer committed
87

88

Christian Tischer's avatar
Christian Tischer committed
89 90 91 92 93
## Image math and pixel data types

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
94
    "data type" -> "pixel values" [label="  restricts"];
95 96
    "image math" -> "pixel values" [label="  changes"];
    "N-bit unsigned integer" -> "0, 1, ..., 2^N-1";
Christian Tischer's avatar
Christian Tischer committed
97 98
    "data type" -> float -> "..., -1031.0, ..., 10.5, ...";
    "data type" -> "...";
Christian Tischer's avatar
Christian Tischer committed
99
    "data type" -> "N-bit unsigned integer";
Christian Tischer's avatar
Christian Tischer committed
100 101 102 103 104
  }
'/>

### Motivation

Christian Tischer's avatar
Christian Tischer committed
105
It sometimes is necessary to change the numeric content of images. It is important to understand how to do this properly in order to avoid uncontrolled artifacts.
Christian Tischer's avatar
Christian Tischer committed
106 107 108 109 110 111 112 113 114 115 116 117 118

What are good reasons to change the pixel values in an image?

1. For intensity measurements, the image background (e.g. camera based offset) should be subtracted from all pixels.
2. For threshold based image segmentation (object detection), it helps to first filter noise in the image.
3. For intensity measurements, it helps to filter noise in the image.
4. The image appears to dark, multiplication of all pixels by a constant number is a means to make it brighter.
5. For uneven illumination (e.g. occuring in wide-field microscopy with large camera chips), one should do flat-field correction, which makes the intensity values even across the image.
6. Our microscope was broken. We took images on a replacement microscope. The pixel values were consistently higher than on our usual microscope. We multiplied the pixels on all images from the replacement microscope by a constant factor to make them comparable to our usual data.


### Activity: Pixel based background subtraction

Christian Tischer's avatar
Christian Tischer committed
119
* Open image: xy_8bit__nuclei_noisy_different_intensity.tif
120
* Appreciate the significant background intensity
Christian Tischer's avatar
Christian Tischer committed
121 122
* Measure pixel values at `[ 28, 35 ]` and `[ 28, 39 ]`
* Measure the image background intensity in this region:
123 124 125
    * upper left corner at `[ 20, 35 ]`
    * width = 10
    * height = 10
Christian Tischer's avatar
Christian Tischer committed
126
* Subtract the measured background intensity from each pixel.
Christian Tischer's avatar
Christian Tischer committed
127
* Measure the pixel values again. 
Christian Tischer's avatar
Christian Tischer committed
128
* Observe that the resuls are incorrect.
Christian Tischer's avatar
Christian Tischer committed
129 130 131

Repeat above activity, but:

Christian Tischer's avatar
Christian Tischer committed
132
* After opening the image, convert its data type to floating point.
Christian Tischer's avatar
Christian Tischer committed
133

Christian Tischer's avatar
Christian Tischer committed
134
### Activity: Explore the limitations of `float` data type
Christian Tischer's avatar
Christian Tischer committed
135

Christian Tischer's avatar
Christian Tischer committed
136 137 138 139
* Create an empty image
* Set all pixel values to 1000000000.0
* Add 1.0 to all pixel values
* Be shocked...
Christian Tischer's avatar
Christian Tischer committed
140 141 142 143 144

...it turns out that from 16777216 on you cannot represent all integers anymore within a float. 

### Formative Assessment

Christian Tischer's avatar
Christian Tischer committed
145
True or false?
Christian Tischer's avatar
Christian Tischer committed
146

Christian Tischer's avatar
Christian Tischer committed
147 148 149 150 151
* Subtracting 100 from 50 in a 8-bit image will result in -50.
* Adding 1 to 255 in a 8-bit image will result in 256.
* Subtracting 10.1 from 10.0 in a float image will result in -0.1
* Adding 1.0 to 255.0 in a float image will result in 256.0
* Adding 1000.0 to 1000000000.0 in a float image will result in 1000001000.0
Christian Tischer's avatar
Christian Tischer committed
152 153 154

### Learn more

Christian Tischer's avatar
Christian Tischer committed
155
* [Limitations of float](https://randomascii.wordpress.com/2012/02/13/dont-store-that-in-a-float/)
Christian Tischer's avatar
Christian Tischer committed
156

Christian Tischer's avatar
Christian Tischer committed
157 158 159 160 161 162
&nbsp;

&nbsp;

&nbsp;

163
## Pixel data type conversions
Christian Tischer's avatar
Christian Tischer committed
164 165 166 167

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
168 169
    "data type conversion" -> "values" [label="  can change"];
    "data type conversion" -> "value range" [label="  changes"];
Christian Tischer's avatar
Christian Tischer committed
170 171 172 173 174 175 176
  }
'/>

### Motivation

What are good reasons to change the pixel data type of an image?

Christian Tischer's avatar
Christian Tischer committed
177 178
* TODO
* TODO
Christian Tischer's avatar
Christian Tischer committed
179 180 181

### Activity: 16-bit to 8-bit conversion

Christian Tischer's avatar
Christian Tischer committed
182
* Open image: xy_16bit__two_values.tif
Christian Tischer's avatar
Christian Tischer committed
183
* Convert to 8-bit
Christian Tischer's avatar
Christian Tischer committed
184
* Understand the mathematics underlying the conversion from 16-bit to 8-bit.
Christian Tischer's avatar
Christian Tischer committed
185 186 187

### Activity: 16-bit to float conversion

Christian Tischer's avatar
Christian Tischer committed
188
* Open image: xy_16bit__two_values.tif
Christian Tischer's avatar
Christian Tischer committed
189
* Convert to float
Christian Tischer's avatar
Christian Tischer committed
190 191 192

### Formative Assessment

Christian Tischer's avatar
Christian Tischer committed
193
True or false? Discuss with your neighbor!
Christian Tischer's avatar
Christian Tischer committed
194 195 196 197

1. Changing pixel data type never changes pixel values.
2. Converting from 16-bit unsigned integer to float never changes the pixel values.
3. Changing from float to 16-bit unsigned integer never changes the pixel values.
Christian Tischer's avatar
Christian Tischer committed
198
4. There is only one correct way to convert from 16-bit to 8-bit.
Christian Tischer's avatar
Christian Tischer committed
199

Christian Tischer's avatar
Christian Tischer committed
200 201 202 203 204 205
&nbsp;

&nbsp;

&nbsp;

Christian Tischer's avatar
Christian Tischer committed
206 207
## Thresholding

Christian Tischer's avatar
Christian Tischer committed
208
In order to find objects in a image, the first step often is to determine whether a pixel is part of an object (foreground) or of the image background. In fluorescence microscopy this often can be achieved by thresholding.
Christian Tischer's avatar
Christian Tischer committed
209 210 211 212

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
213
    "intensity image" -> threshold;
Christian Tischer's avatar
Christian Tischer committed
214
    threshold -> "binary image";
Christian Tischer's avatar
Christian Tischer committed
215 216 217 218 219
    "binary image" -> "background value";
    "binary image" -> "foreground value";
    "background value" -> "0";
    "foreground value" -> "1";
    "foreground value" -> "255";
Christian Tischer's avatar
Christian Tischer committed
220 221
    "pixel value" -> ">= threshold" -> foreground;
    "pixel value" -> "< threshold" -> background;
Christian Tischer's avatar
Christian Tischer committed
222 223 224
 }
'/>

Christian Tischer's avatar
Christian Tischer committed
225
### Activity: Threshold an image
Christian Tischer's avatar
Christian Tischer committed
226

Christian Tischer's avatar
Christian Tischer committed
227
* Open image: xy_8bit__two_cells.tif
Christian Tischer's avatar
Christian Tischer committed
228
* Convert the image to a binary image by means of thresholding.
Christian Tischer's avatar
Christian Tischer committed
229

Christian Tischer's avatar
Christian Tischer committed
230
### Formative assessment
Christian Tischer's avatar
Christian Tischer committed
231

Christian Tischer's avatar
Christian Tischer committed
232
True or false? Discuss with your neighbour!
Christian Tischer's avatar
Christian Tischer committed
233

Christian Tischer's avatar
Christian Tischer committed
234 235 236 237
* For each image there is only one correct threshold value.
* The result of thresholding is a binary image.
* A binary image can have three values: `-1, 0, +1`
* Values below the threshold are always set to `1`.
Christian Tischer's avatar
Christian Tischer committed
238

Christian Tischer's avatar
Christian Tischer committed
239 240 241 242 243 244 245
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
246 247 248 249 250
## Connected components analysis

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
251 252
    "intensity image" -> "connected component analysis" -> "label image";
    connectivity -> "connected component analysis"; 
Christian Tischer's avatar
Christian Tischer committed
253 254 255 256 257 258
  }
'/>


### Activity: 2D connected components analysis

Christian Tischer's avatar
Christian Tischer committed
259
* Open image: xy_8bit_binary__nuclei.tif
Christian Tischer's avatar
Christian Tischer committed
260 261 262
* Perform connected components analysis
* Explore multi-color LUTs for object labelling
* Explore removing and joining labels
Christian Tischer's avatar
Christian Tischer committed
263 264 265 266 267 268


### Activity: 3D connected components analysis

Repeat above activity but use a 3D image:

Christian Tischer's avatar
Christian Tischer committed
269
* Open image: xyz_8bit_binary__spots.tif
Christian Tischer's avatar
Christian Tischer committed
270 271 272

### Formative assessment

Christian Tischer's avatar
Christian Tischer committed
273
Fill in the blanks, using these words: less, more, 8, 255, 4, more.
Christian Tischer's avatar
Christian Tischer committed
274

Christian Tischer's avatar
Christian Tischer committed
275 276 277 278 279 280 281
* For a given input image there is only one correct connectivity.
* In 3D, pixels have _____ neighbors than in 2D.
* 8-connected connectivity results in _____ objects than 4-connected connectivity.
* In 3D, pixels have ____ non-diagonal neighbors.
* In 2D, pixels have ____ non-diagonal neighbors.
* A 8-bit label image can maximally have _____ objects.
* The maximum value in a label image is equal to or _____ than the number of objects.
Christian Tischer's avatar
Christian Tischer committed
282 283


Christian Tischer's avatar
Christian Tischer committed
284 285 286 287 288 289 290
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
291 292 293 294
## Shape measurements

<img src='https://g.gravizo.com/svg?
 digraph G {
Christian Tischer's avatar
Christian Tischer committed
295 296 297 298 299
	shift [fontcolor=white,color=white];
	"label image" -> shape_analysis -> table;
	table -> object_rows;
	table -> feature_columns;
	table -> visualisation;  
Christian Tischer's avatar
Christian Tischer committed
300 301 302 303 304 305
}
'/>


### Activity: Measure object shape parameters

Christian Tischer's avatar
Christian Tischer committed
306
* Open image: xy_8bit_labels__four_objects.tif
Christian Tischer's avatar
Christian Tischer committed
307
* Perform shape measurements and discuss their meanings.
Christian Tischer's avatar
Christian Tischer committed
308 309
* Explore results visualisation
	* Color objects by their measurement values.
Christian Tischer's avatar
Christian Tischer committed
310 311
* Add a calibration to the image and check which shape measurements are affected.
* Draw a test image to understand the shape measurements even better.
Christian Tischer's avatar
Christian Tischer committed
312 313 314

### Formative assessment

Christian Tischer's avatar
Christian Tischer committed
315
True or false? Discuss with your neighbour!
Christian Tischer's avatar
Christian Tischer committed
316

Christian Tischer's avatar
Christian Tischer committed
317 318 319 320 321
* Circularity is independent of image calibration.
* Area is independent of image calibration.
* Perimeter can strongly depend on spatial sampling.
* Volume can strongly depend on spatial sampling.
* Drawing test images to check how certain shape parameters behave is a good idea.
Christian Tischer's avatar
Christian Tischer committed
322 323 324

### Learn more

Christian Tischer's avatar
Christian Tischer committed
325
* Especially surface and perimeter measurements are affected by sampling and resolution (see for example: https://en.wikipedia.org/wiki/Coastline_paradox).
Christian Tischer's avatar
Christian Tischer committed
326

Christian Tischer's avatar
Christian Tischer committed
327 328 329 330 331 332 333
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
334 335 336 337 338
## Object shape measurement workflow

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
339 340 341
    "intensity image" -> "binary image" [label="  threshold"];
    "binary image" -> "label image" [label="  connected components"];
    "label image" -> table [label="  measure_shape"];
Christian Tischer's avatar
Christian Tischer committed
342 343 344 345 346
}
'/>

### Activity: Segment objects and measure shapes

Christian Tischer's avatar
Christian Tischer committed
347
* Open image: xy_8bit__two_cells.tif
Christian Tischer's avatar
Christian Tischer committed
348 349
* Segment the cells and measure their shapes.
	* Devise code to automate the workflow.
Christian Tischer's avatar
Christian Tischer committed
350 351 352

### Formative assessment

Christian Tischer's avatar
Christian Tischer committed
353
Fill in below blanks, using these words: equal_to, larger_than, smaller_than, binary, connected_component_analysis, thresholding
Christian Tischer's avatar
Christian Tischer committed
354

Christian Tischer's avatar
Christian Tischer committed
355 356 357 358 359
* A label image is the result of _____ .
* The number of pixels in a binary image is typically _____ the number of connected components. 
* The number of distinct values in a label image is _____ the number of objects (minus one).
* Converting an intensity image to a _____ image can be achieved by _____ .
* The number of connected components can be _____ the maximal label.
Christian Tischer's avatar
Christian Tischer committed
360

Christian Tischer's avatar
Christian Tischer committed
361 362 363 364 365 366 367
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
368
## Intensity measurements
Christian Tischer's avatar
Christian Tischer committed
369

Christian Tischer's avatar
Christian Tischer committed
370
### Activity: Measure intensities in image regions
Christian Tischer's avatar
Christian Tischer committed
371

Christian Tischer's avatar
Christian Tischer committed
372
* Open image: xy_float__h2b_bg_corr.tif
Christian Tischer's avatar
Christian Tischer committed
373 374 375 376 377 378 379
* Measure for both nuclei:
	* Maximum intensity
	* Average intensity
	* Median intensity
	* Sum intensity
* Discuss the interpretation!
* Discuss where to measure!
Christian Tischer's avatar
Christian Tischer committed
380 381


Christian Tischer's avatar
Christian Tischer committed
382 383 384
### Activity: Intensity measurements without pixel based background correction

#### Motivation
Christian Tischer's avatar
Christian Tischer committed
385

Christian Tischer's avatar
Christian Tischer committed
386
There are several good reasons not to subtract the background from each pixel in an image: 
Christian Tischer's avatar
Christian Tischer committed
387

Christian Tischer's avatar
Christian Tischer committed
388 389
* It is a bit tricky to do it right, because one has to convert to float to accomodate floting point and negative values.
* If one has really big image data (TB) one would need (at least) another TB storage for the background corrected version of the image.
Christian Tischer's avatar
Christian Tischer committed
390

Christian Tischer's avatar
Christian Tischer committed
391
#### Workflow
Christian Tischer's avatar
Christian Tischer committed
392

Christian Tischer's avatar
Christian Tischer committed
393
* Open image: xy_calibrated_8bit__two_nuclei_high_background.tif
Christian Tischer's avatar
Christian Tischer committed
394 395 396 397 398 399 400 401 402 403
* Measure for both nuclei and a background region:
	* Maximum intensity
	* Average intensity
	* Median intensity
	* Sum intensity
* Discuss how to correct the intensities for the background
	* Appreciate that you also need the region areas for this task
	* Measure the region areas
		* Watch out: the image is calibrated!
		* Use the area for the correction.
Christian Tischer's avatar
Christian Tischer committed
404

Christian Tischer's avatar
Christian Tischer committed
405
### Formative assessment: Intensity measurements
Christian Tischer's avatar
Christian Tischer committed
406

Christian Tischer's avatar
Christian Tischer committed
407
Fill in the blanks, using these words: integrated, mean, number_of_pixels, decrease, increase, sum
Christian Tischer's avatar
Christian Tischer committed
408

Christian Tischer's avatar
Christian Tischer committed
409 410 411 412
* Average intensity is just another word for _____ intensity.
* The _____ intensity is equal to the mean intensity times the _____ in the measured region.
* In an 8-bit image, increasing the size of the measurement region can only _____ the sum intensity.
* In a float image, increasing the size of the measurement region can _____ the sum intensity. 
Christian Tischer's avatar
Christian Tischer committed
413

Christian Tischer's avatar
Christian Tischer committed
414 415 416 417 418 419 420
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
421 422 423 424 425 426
## Convolution filters

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
    "intensity image" -> "convolution" -> "filtered image";
Christian Tischer's avatar
Christian Tischer committed
427 428
    "small image" -> size;
    "small image" -> "pixel values";
Christian Tischer's avatar
Christian Tischer committed
429
    "kernel" -> "small image" [label="  is"]; 
Christian Tischer's avatar
Christian Tischer committed
430 431 432 433 434 435
    "kernel" -> "convolution";
}
'/>

### Activity: Explore convolution filters

Christian Tischer's avatar
Christian Tischer committed
436
* Open image: xy_8bit__nuclei_noisy_different_intensity.tif
Christian Tischer's avatar
Christian Tischer committed
437 438 439 440 441 442
* Try the result of different convolution filters, e.g.
	* https://en.wikipedia.org/wiki/Kernel_(image_processing)
	* Mean filter
	* Gaussian blur
	* Edge detection
* Appreciate that the results are (slightly) wrong within the 8-bit range of the input image.
Christian Tischer's avatar
Christian Tischer committed
443 444 445

### Activity: Use mean filter to facilitate image segmentation

Christian Tischer's avatar
Christian Tischer committed
446
* Open image: xy_8bit__nuclei_noisy_different_intensity.tif
Christian Tischer's avatar
Christian Tischer committed
447 448 449
* Appreciate that you cannot readily threshold the image
* Apply a mean filter
* Threshold the filtered image
Christian Tischer's avatar
Christian Tischer committed
450 451 452

### Formative assessment

Christian Tischer's avatar
Christian Tischer committed
453 454
* Draw the kernel of a 3x3 mean filter.
* Draw three different kernels that enhance edges.
Christian Tischer's avatar
Christian Tischer committed
455 456 457
 
### Learn more

Christian Tischer's avatar
Christian Tischer committed
458
* https://en.wikipedia.org/wiki/Kernel_(image_processing)
Christian Tischer's avatar
Christian Tischer committed
459

Christian Tischer's avatar
Christian Tischer committed
460

Christian Tischer's avatar
Christian Tischer committed
461 462 463 464 465 466 467
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
468
## Typical image analysis workflow
Christian Tischer's avatar
Christian Tischer committed
469

Christian Tischer's avatar
Christian Tischer committed
470
![image](/uploads/b4bdce17515908f40d858b35d5e9256e/image.png)
Christian Tischer's avatar
Christian Tischer committed
471

Christian Tischer's avatar
Christian Tischer committed
472 473 474 475 476 477 478
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
479 480