bioimage-analysis-fundamentals.md 12 KB
Newer Older
Christian Tischer's avatar
Christian Tischer committed
1 2
# Bioimage analysis fundamentals

Christian Tischer's avatar
Christian Tischer committed
3
## Pixel values, coordinates, and data types
Christian Tischer's avatar
Christian Tischer committed
4 5 6 7 8 9 10 11 12 13

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
    image -> pixel [label="  has many"];
    pixel -> value;
    pixel -> indices;
    pixel -> coordinates;
    indices -> calibration;
    calibration -> coordinates;
Christian Tischer's avatar
Christian Tischer committed
14
    image -> calibration [label="  can have"];
Christian Tischer's avatar
Christian Tischer committed
15 16 17 18 19 20
    pixel -> voxel [label="  3D"];   
  }
'/>

### Activity

Christian Tischer's avatar
Christian Tischer committed
21
* Open image: xy_8bit__nuclei_noisy_different_intensity.tif
Christian Tischer's avatar
Christian Tischer committed
22 23 24
	* Explore different ways to inspect pixel values and indices
	* Add image calibration
	* Check where the calibration is visible
Christian Tischer's avatar
Christian Tischer committed
25
	
Christian Tischer's avatar
Christian Tischer committed
26 27
### Formative assessment

Christian Tischer's avatar
Christian Tischer committed
28
True or false?
Christian Tischer's avatar
Christian Tischer committed
29

Christian Tischer's avatar
Christian Tischer committed
30 31 32 33 34
* Pixel coordinates are always integer values.
* Changing the image calibration changes the pixel values.
* Pixel coordinates depend on image calibration.
* Pixel indices are always positive integer values.
* The lowest pixel index of a 2D image always is `[1,1]`.
Christian Tischer's avatar
Christian Tischer committed
35
* When looking at a 2D image, the lowest pixel index is in the lower left corner.
Christian Tischer's avatar
Christian Tischer committed
36

Christian Tischer's avatar
Christian Tischer committed
37 38 39 40 41 42
&nbsp;

&nbsp;

&nbsp;

Christian Tischer's avatar
Christian Tischer committed
43
<div style="page-break-after: always;"></div>
Christian Tischer's avatar
Christian Tischer committed
44

Christian Tischer's avatar
Christian Tischer committed
45 46 47 48 49
## Image display

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
50 51 52 53
    LUT -> color;
    LUT -> brightness;
    min -> LUT;
    max -> LUT;
Christian Tischer's avatar
Christian Tischer committed
54
    value -> LUT;
Christian Tischer's avatar
Christian Tischer committed
55 56 57 58
  }
'/>

```
Christian Tischer's avatar
Christian Tischer committed
59 60
brightness = ( value - min ) / ( max - min )

Christian Tischer's avatar
Christian Tischer committed
61
0 <= brightness <= 1 
Christian Tischer's avatar
Christian Tischer committed
62 63

contrast = max - min 
Christian Tischer's avatar
Christian Tischer committed
64 65 66 67
``` 

### Activity

Christian Tischer's avatar
Christian Tischer committed
68
* Open image: xy_8bit__nuclei_noisy_different_intensity.tif
Christian Tischer's avatar
Christian Tischer committed
69
* Explore different LUTs and LUT settings
Christian Tischer's avatar
Christian Tischer committed
70
	* Appreciate that LUT settings do not affect image content.
Christian Tischer's avatar
Christian Tischer committed
71 72


Christian Tischer's avatar
Christian Tischer committed
73 74
### Formative Assessment

Christian Tischer's avatar
Christian Tischer committed
75
Fill in the blanks, using those words: decrease, larger than, increase, smaller than 
Christian Tischer's avatar
Christian Tischer committed
76

Christian Tischer's avatar
Christian Tischer committed
77 78 79 80
1. Pixels with values _____ `max` will appear saturated. 
2. Decreasing `max` while keeping `min` constant will _____ the contrast.
3. Decreasing both `max` and `min` will _____ the overall brightness.
4. Pixels with values _____ the `min` will appear black, when using a grayscale LUT.
Christian Tischer's avatar
Christian Tischer committed
81 82 83 84 85 86 87 88


&nbsp;

&nbsp;

&nbsp;

Christian Tischer's avatar
Christian Tischer committed
89

90

Christian Tischer's avatar
Christian Tischer committed
91 92 93 94 95
## Image math and pixel data types

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
96
    "data type" -> "pixel values" [label="  restricts"];
97 98
    "image math" -> "pixel values" [label="  changes"];
    "N-bit unsigned integer" -> "0, 1, ..., 2^N-1";
Christian Tischer's avatar
Christian Tischer committed
99 100
    "data type" -> float -> "..., -1031.0, ..., 10.5, ...";
    "data type" -> "...";
Christian Tischer's avatar
Christian Tischer committed
101
    "data type" -> "N-bit unsigned integer";
Christian Tischer's avatar
Christian Tischer committed
102 103 104 105 106
  }
'/>

### Activity: Pixel based background subtraction

Christian Tischer's avatar
Christian Tischer committed
107
* Open image: xy_8bit__nuclei_noisy_different_intensity.tif
108
* Appreciate the significant background intensity
Christian Tischer's avatar
Christian Tischer committed
109 110
* Measure pixel values at `[ 28, 35 ]` and `[ 28, 39 ]`
* Measure the image background intensity in this region:
111 112 113
    * upper left corner at `[ 20, 35 ]`
    * width = 10
    * height = 10
Christian Tischer's avatar
Christian Tischer committed
114
* Subtract the measured background intensity from each pixel.
Christian Tischer's avatar
Christian Tischer committed
115
* Measure the pixel values again. 
Christian Tischer's avatar
Christian Tischer committed
116
* Observe that the resuls are incorrect.
Christian Tischer's avatar
Christian Tischer committed
117 118 119

Repeat above activity, but:

Christian Tischer's avatar
Christian Tischer committed
120
* After opening the image, convert its data type to floating point.
Christian Tischer's avatar
Christian Tischer committed
121

Christian Tischer's avatar
Christian Tischer committed
122
### Activity: Explore the limitations of `float` data type
Christian Tischer's avatar
Christian Tischer committed
123

Christian Tischer's avatar
Christian Tischer committed
124 125 126 127
* Create an empty image
* Set all pixel values to 1000000000.0
* Add 1.0 to all pixel values
* Be shocked...
Christian Tischer's avatar
Christian Tischer committed
128 129 130 131 132

...it turns out that from 16777216 on you cannot represent all integers anymore within a float. 

### Formative Assessment

Christian Tischer's avatar
Christian Tischer committed
133
True or false?
Christian Tischer's avatar
Christian Tischer committed
134

Christian Tischer's avatar
Christian Tischer committed
135 136 137 138 139
* Subtracting 100 from 50 in a 8-bit image will result in -50.
* Adding 1 to 255 in a 8-bit image will result in 256.
* Subtracting 10.1 from 10.0 in a float image will result in -0.1
* Adding 1.0 to 255.0 in a float image will result in 256.0
* Adding 1000.0 to 1000000000.0 in a float image will result in 1000001000.0
Christian Tischer's avatar
Christian Tischer committed
140 141 142

### Learn more

Christian Tischer's avatar
Christian Tischer committed
143
* [Limitations of float](https://randomascii.wordpress.com/2012/02/13/dont-store-that-in-a-float/)
Christian Tischer's avatar
Christian Tischer committed
144

Christian Tischer's avatar
Christian Tischer committed
145 146 147 148 149 150
&nbsp;

&nbsp;

&nbsp;

151
## Pixel data type conversions
Christian Tischer's avatar
Christian Tischer committed
152 153 154 155

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
156 157
    "data type conversion" -> "values" [label="  can change"];
    "data type conversion" -> "value range" [label="  changes"];
Christian Tischer's avatar
Christian Tischer committed
158 159 160 161 162
  }
'/>

### Activity: 16-bit to 8-bit conversion

Christian Tischer's avatar
Christian Tischer committed
163
* Open image: xy_16bit__two_values.tif
Christian Tischer's avatar
Christian Tischer committed
164
* Convert to 8-bit
Christian Tischer's avatar
Christian Tischer committed
165
* Understand the mathematics underlying the conversion from 16-bit to 8-bit.
Christian Tischer's avatar
Christian Tischer committed
166 167 168

### Activity: 16-bit to float conversion

Christian Tischer's avatar
Christian Tischer committed
169
* Open image: xy_16bit__two_values.tif
Christian Tischer's avatar
Christian Tischer committed
170
* Convert to float
Christian Tischer's avatar
Christian Tischer committed
171 172 173

### Formative Assessment

Christian Tischer's avatar
Christian Tischer committed
174
True or false? Discuss with your neighbor!
Christian Tischer's avatar
Christian Tischer committed
175 176 177 178

1. Changing pixel data type never changes pixel values.
2. Converting from 16-bit unsigned integer to float never changes the pixel values.
3. Changing from float to 16-bit unsigned integer never changes the pixel values.
Christian Tischer's avatar
Christian Tischer committed
179
4. There is only one correct way to convert from 16-bit to 8-bit.
Christian Tischer's avatar
Christian Tischer committed
180

Christian Tischer's avatar
Christian Tischer committed
181 182 183 184 185 186
&nbsp;

&nbsp;

&nbsp;

Christian Tischer's avatar
Christian Tischer committed
187 188
## Thresholding

Christian Tischer's avatar
Christian Tischer committed
189
In order to find objects in a image, the first step often is to determine whether a pixel is part of an object (foreground) or of the image background. In fluorescence microscopy this often can be achieved by thresholding.
Christian Tischer's avatar
Christian Tischer committed
190 191 192

<img src='https://g.gravizo.com/svg?
 digraph G {
Christian Tischer's avatar
Christian Tischer committed
193 194 195 196 197 198 199 200 201 202 203
	shift [fontcolor=white,color=white];
	"intensity image" -> threshold;
	threshold -> "binary image";
	"binary image" -> "mask" [label="  aka"];
	"binary image" -> "background value";
	"binary image" -> "foreground value";
	"background value" -> "0";
	"foreground value" -> "1";
	"foreground value" -> "255";
	"pixel value" -> ">= threshold" -> foreground;
	"pixel value" -> "< threshold" -> background;
Christian Tischer's avatar
Christian Tischer committed
204 205 206
 }
'/>

Christian Tischer's avatar
Christian Tischer committed
207
### Activity: Threshold an image
Christian Tischer's avatar
Christian Tischer committed
208

Christian Tischer's avatar
Christian Tischer committed
209
* Open image: xy_8bit__two_cells.tif
Christian Tischer's avatar
Christian Tischer committed
210
* Convert the image to a binary image by means of thresholding.
Christian Tischer's avatar
Christian Tischer committed
211

Christian Tischer's avatar
Christian Tischer committed
212
### Formative assessment
Christian Tischer's avatar
Christian Tischer committed
213

Christian Tischer's avatar
Christian Tischer committed
214
True or false? Discuss with your neighbour!
Christian Tischer's avatar
Christian Tischer committed
215

Christian Tischer's avatar
Christian Tischer committed
216 217 218 219
* For each image there is only one correct threshold value.
* The result of thresholding is a binary image.
* A binary image can have three values: `-1, 0, +1`
* Values below the threshold are always set to `1`.
Christian Tischer's avatar
Christian Tischer committed
220

Christian Tischer's avatar
Christian Tischer committed
221 222 223 224 225 226 227
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
228 229 230 231 232
## Connected components analysis

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
233 234
    "intensity image" -> "connected component analysis" -> "label image";
    connectivity -> "connected component analysis"; 
Christian Tischer's avatar
Christian Tischer committed
235 236 237 238 239 240
  }
'/>


### Activity: 2D connected components analysis

Christian Tischer's avatar
Christian Tischer committed
241
* Open image: xy_8bit_binary__nuclei.tif
Christian Tischer's avatar
Christian Tischer committed
242 243 244
* Perform connected components analysis
* Explore multi-color LUTs for object labelling
* Explore removing and joining labels
Christian Tischer's avatar
Christian Tischer committed
245 246 247 248 249 250


### Activity: 3D connected components analysis

Repeat above activity but use a 3D image:

Christian Tischer's avatar
Christian Tischer committed
251
* Open image: xyz_8bit_binary__spots.tif
Christian Tischer's avatar
Christian Tischer committed
252 253 254

### Formative assessment

Christian Tischer's avatar
Christian Tischer committed
255
Fill in the blanks, using these words: less, more, 8, 255, 4, more.
Christian Tischer's avatar
Christian Tischer committed
256

Christian Tischer's avatar
Christian Tischer committed
257 258 259 260 261 262
1. In 3D, pixels have _____ neighbors than in 2D.
2. 8-connected connectivity results in _____ objects than 4-connected connectivity.
3. In 3D, pixels have ____ non-diagonal neighbors.
4. In 2D, pixels have ____ non-diagonal neighbors.
5. A 8-bit label image can maximally have _____ objects.
6. The maximum value in a label image is equal to or _____ than the number of objects.
Christian Tischer's avatar
Christian Tischer committed
263 264


Christian Tischer's avatar
Christian Tischer committed
265 266 267 268 269 270 271
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
272 273 274 275
## Shape measurements

<img src='https://g.gravizo.com/svg?
 digraph G {
Christian Tischer's avatar
Christian Tischer committed
276 277 278 279 280
	shift [fontcolor=white,color=white];
	"label image" -> shape_analysis -> table;
	table -> object_rows;
	table -> feature_columns;
	table -> visualisation;  
Christian Tischer's avatar
Christian Tischer committed
281 282 283 284 285 286
}
'/>


### Activity: Measure object shape parameters

Christian Tischer's avatar
Christian Tischer committed
287
* Open image: xy_8bit_labels__four_objects.tif
Christian Tischer's avatar
Christian Tischer committed
288
* Perform shape measurements and discuss their meanings.
Christian Tischer's avatar
Christian Tischer committed
289 290
* Explore results visualisation
	* Color objects by their measurement values.
Christian Tischer's avatar
Christian Tischer committed
291 292
* Add a calibration to the image and check which shape measurements are affected.
* Draw a test image to understand the shape measurements even better.
Christian Tischer's avatar
Christian Tischer committed
293 294 295

### Formative assessment

Christian Tischer's avatar
Christian Tischer committed
296
True or false? Discuss with your neighbour!
Christian Tischer's avatar
Christian Tischer committed
297

Christian Tischer's avatar
Christian Tischer committed
298 299 300 301 302
* Circularity is independent of image calibration.
* Area is independent of image calibration.
* Perimeter can strongly depend on spatial sampling.
* Volume can strongly depend on spatial sampling.
* Drawing test images to check how certain shape parameters behave is a good idea.
Christian Tischer's avatar
Christian Tischer committed
303 304 305

### Learn more

Christian Tischer's avatar
Christian Tischer committed
306 307 308 309 310
* Especially surface and perimeter measurements are affected by sampling and resolution, see for example:
	* https://en.wikipedia.org/wiki/Coastline_paradox).
* Results visualisation:
	* https://imagej.net/MorphoLibJ#Grayscale_morphological_filters: **Label visualization in 3D viewer**

Christian Tischer's avatar
Christian Tischer committed
311

Christian Tischer's avatar
Christian Tischer committed
312 313 314 315 316 317 318
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
319 320 321 322 323
## Object shape measurement workflow

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
324 325 326
    "intensity image" -> "binary image" [label="  threshold"];
    "binary image" -> "label image" [label="  connected components"];
    "label image" -> table [label="  measure_shape"];
Christian Tischer's avatar
Christian Tischer committed
327 328 329 330 331
}
'/>

### Activity: Segment objects and measure shapes

Christian Tischer's avatar
Christian Tischer committed
332
* Open image: xy_8bit__two_cells.tif
Christian Tischer's avatar
Christian Tischer committed
333 334
* Segment the cells and measure their shapes.
	* Devise code to automate the workflow.
Christian Tischer's avatar
Christian Tischer committed
335 336 337

### Formative assessment

Christian Tischer's avatar
Christian Tischer committed
338
Fill in below blanks, using these words: equal_to, larger_than, smaller_than, binary, connected_component_analysis, thresholding
Christian Tischer's avatar
Christian Tischer committed
339

Christian Tischer's avatar
Christian Tischer committed
340 341 342 343 344
1. A label image is the result of _____ .
2. The number of pixels in a binary image is typically _____ the number of connected components. 
3. The number of distinct values in a label image is _____ the number of objects (minus one).
4. Converting an intensity image to a _____ image can be achieved by _____ .
5. The number of connected components can be _____ the maximal label.
Christian Tischer's avatar
Christian Tischer committed
345

Christian Tischer's avatar
Christian Tischer committed
346 347 348 349 350 351 352
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
353
## Intensity measurements
Christian Tischer's avatar
Christian Tischer committed
354

Christian Tischer's avatar
Christian Tischer committed
355
### Activity: Measure intensities in image regions
Christian Tischer's avatar
Christian Tischer committed
356

Christian Tischer's avatar
Christian Tischer committed
357
* Open image: xy_float__h2b_bg_corr.tif
Christian Tischer's avatar
Christian Tischer committed
358 359 360 361 362 363 364
* Measure for both nuclei:
	* Maximum intensity
	* Average intensity
	* Median intensity
	* Sum intensity
* Discuss the interpretation!
* Discuss where to measure!
Christian Tischer's avatar
Christian Tischer committed
365 366


Christian Tischer's avatar
Christian Tischer committed
367 368 369
### Activity: Intensity measurements without pixel based background correction

#### Motivation
Christian Tischer's avatar
Christian Tischer committed
370

Christian Tischer's avatar
Christian Tischer committed
371
There are several good reasons not to subtract the background from each pixel in an image: 
Christian Tischer's avatar
Christian Tischer committed
372

Christian Tischer's avatar
Christian Tischer committed
373 374
* It is a bit tricky to do it right, because one has to convert to float to accomodate floting point and negative values.
* If one has really big image data (TB) one would need (at least) another TB storage for the background corrected version of the image.
Christian Tischer's avatar
Christian Tischer committed
375

Christian Tischer's avatar
Christian Tischer committed
376
#### Workflow
Christian Tischer's avatar
Christian Tischer committed
377

Christian Tischer's avatar
Christian Tischer committed
378
* Open image: xy_calibrated_8bit__two_nuclei_high_background.tif
Christian Tischer's avatar
Christian Tischer committed
379 380 381 382 383 384 385 386 387 388
* Measure for both nuclei and a background region:
	* Maximum intensity
	* Average intensity
	* Median intensity
	* Sum intensity
* Discuss how to correct the intensities for the background
	* Appreciate that you also need the region areas for this task
	* Measure the region areas
		* Watch out: the image is calibrated!
		* Use the area for the correction.
Christian Tischer's avatar
Christian Tischer committed
389

Christian Tischer's avatar
Christian Tischer committed
390
### Formative assessment: Intensity measurements
Christian Tischer's avatar
Christian Tischer committed
391

Christian Tischer's avatar
Christian Tischer committed
392
Fill in the blanks, using these words: integrated, mean, number_of_pixels, decrease, increase, sum
Christian Tischer's avatar
Christian Tischer committed
393

Christian Tischer's avatar
Christian Tischer committed
394 395 396 397
1. Average intensity is just another word for _____ intensity.
2. The _____ intensity is equal to the mean intensity times the _____ in the measured region.
3. In an 8-bit image, increasing the size of the measurement region can only _____ the sum intensity.
4. In a float image, increasing the size of the measurement region can _____ the sum intensity. 
Christian Tischer's avatar
Christian Tischer committed
398

Christian Tischer's avatar
Christian Tischer committed
399 400 401 402 403 404 405
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
406 407 408 409 410 411
## Convolution filters

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
    "intensity image" -> "convolution" -> "filtered image";
Christian Tischer's avatar
Christian Tischer committed
412 413
    "small image" -> size;
    "small image" -> "pixel values";
Christian Tischer's avatar
Christian Tischer committed
414
    "kernel" -> "small image" [label="  is"]; 
Christian Tischer's avatar
Christian Tischer committed
415 416 417 418 419 420
    "kernel" -> "convolution";
}
'/>

### Activity: Explore convolution filters

Christian Tischer's avatar
Christian Tischer committed
421
* Open image: xy_8bit__nuclei_noisy_different_intensity.tif
Christian Tischer's avatar
Christian Tischer committed
422 423 424 425 426 427
* Try the result of different convolution filters, e.g.
	* https://en.wikipedia.org/wiki/Kernel_(image_processing)
	* Mean filter
	* Gaussian blur
	* Edge detection
* Appreciate that the results are (slightly) wrong within the 8-bit range of the input image.
Christian Tischer's avatar
Christian Tischer committed
428 429 430

### Activity: Use mean filter to facilitate image segmentation

Christian Tischer's avatar
Christian Tischer committed
431
* Open image: xy_8bit__nuclei_noisy_different_intensity.tif
Christian Tischer's avatar
Christian Tischer committed
432 433 434
* Appreciate that you cannot readily threshold the image
* Apply a mean filter
* Threshold the filtered image
Christian Tischer's avatar
Christian Tischer committed
435 436 437

### Formative assessment

Christian Tischer's avatar
Christian Tischer committed
438 439
* Draw the kernel of a 3x3 mean filter.
* Draw three different kernels that enhance edges.
Christian Tischer's avatar
Christian Tischer committed
440 441 442
 
### Learn more

Christian Tischer's avatar
Christian Tischer committed
443
* https://en.wikipedia.org/wiki/Kernel_(image_processing)
Christian Tischer's avatar
Christian Tischer committed
444

Christian Tischer's avatar
Christian Tischer committed
445

Christian Tischer's avatar
Christian Tischer committed
446 447 448 449 450 451 452
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
453
## Typical image analysis workflow
Christian Tischer's avatar
Christian Tischer committed
454

Christian Tischer's avatar
Christian Tischer committed
455
![image](/uploads/b4bdce17515908f40d858b35d5e9256e/image.png)
Christian Tischer's avatar
Christian Tischer committed
456

Christian Tischer's avatar
Christian Tischer committed
457 458 459 460 461 462 463
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
464 465