bioimage-analysis-fundamentals.md 12.1 KB
Newer Older
Christian Tischer's avatar
Christian Tischer committed
1 2
# Bioimage analysis fundamentals

Christian Tischer's avatar
Christian Tischer committed
3
## Pixel values, coordinates, and data types
Christian Tischer's avatar
Christian Tischer committed
4 5 6 7 8 9 10 11 12 13

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
    image -> pixel [label="  has many"];
    pixel -> value;
    pixel -> indices;
    pixel -> coordinates;
    indices -> calibration;
    calibration -> coordinates;
Christian Tischer's avatar
Christian Tischer committed
14
    image -> calibration [label="  can have"];
Christian Tischer's avatar
Christian Tischer committed
15 16 17 18 19 20
    pixel -> voxel [label="  3D"];   
  }
'/>

### Activity

Christian Tischer's avatar
Christian Tischer committed
21
* Open image: xy_8bit__nuclei_noisy_different_intensity.tif
Christian Tischer's avatar
Christian Tischer committed
22 23 24
	* Explore different ways to inspect pixel values and indices
	* Add image calibration
	* Check where the calibration is visible
Christian Tischer's avatar
Christian Tischer committed
25
	
Christian Tischer's avatar
Christian Tischer committed
26 27
### Formative assessment

Christian Tischer's avatar
Christian Tischer committed
28
True or false?
Christian Tischer's avatar
Christian Tischer committed
29

Christian Tischer's avatar
Christian Tischer committed
30 31 32 33 34
* Pixel coordinates are always integer values.
* Changing the image calibration changes the pixel values.
* Pixel coordinates depend on image calibration.
* Pixel indices are always positive integer values.
* The lowest pixel index of a 2D image always is `[1,1]`.
Christian Tischer's avatar
Christian Tischer committed
35
* When looking at a 2D image, the lowest pixel index is in the lower left corner.
Christian Tischer's avatar
Christian Tischer committed
36

Christian Tischer's avatar
Christian Tischer committed
37 38 39 40 41 42
&nbsp;

&nbsp;

&nbsp;

Christian Tischer's avatar
Christian Tischer committed
43
<div style="page-break-after: always;"></div>
Christian Tischer's avatar
Christian Tischer committed
44

Christian Tischer's avatar
Christian Tischer committed
45 46 47 48 49
## Image display

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
50 51 52 53
    LUT -> color;
    LUT -> brightness;
    min -> LUT;
    max -> LUT;
Christian Tischer's avatar
Christian Tischer committed
54
    value -> LUT;
Christian Tischer's avatar
Christian Tischer committed
55 56 57 58
  }
'/>

```
Christian Tischer's avatar
Christian Tischer committed
59 60
brightness = ( value - min ) / ( max - min )

Christian Tischer's avatar
Christian Tischer committed
61
0 <= brightness <= 1 
Christian Tischer's avatar
Christian Tischer committed
62 63

contrast = max - min 
Christian Tischer's avatar
Christian Tischer committed
64 65 66 67
``` 

### Activity

Christian Tischer's avatar
Christian Tischer committed
68
* Open image: xy_8bit__nuclei_noisy_different_intensity.tif
Christian Tischer's avatar
Christian Tischer committed
69
* Explore different LUTs and LUT settings
Christian Tischer's avatar
Christian Tischer committed
70
	* Appreciate that LUT settings do not affect image content.
Christian Tischer's avatar
Christian Tischer committed
71 72


Christian Tischer's avatar
Christian Tischer committed
73 74
### Formative Assessment

Christian Tischer's avatar
Christian Tischer committed
75
Fill in the blanks, using those words: decrease, larger_than, increase, smaller_than 
Christian Tischer's avatar
Christian Tischer committed
76

Christian Tischer's avatar
Christian Tischer committed
77 78 79 80
* Pixels with values _____ `max` will appear saturated. 
* Decreasing `max` while keeping `min` constant will _____ the contrast.
* Decreasing both `max` and `min` will _____ the overall brightness.
* Pixels with values _____ the `min` will appear black, when using a grayscale LUT.
Christian Tischer's avatar
Christian Tischer committed
81 82 83 84 85 86 87 88


&nbsp;

&nbsp;

&nbsp;

Christian Tischer's avatar
Christian Tischer committed
89

90

Christian Tischer's avatar
Christian Tischer committed
91 92 93 94 95
## Image math and pixel data types

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
96
    "data type" -> "pixel values" [label="  restricts"];
97 98
    "image math" -> "pixel values" [label="  changes"];
    "N-bit unsigned integer" -> "0, 1, ..., 2^N-1";
Christian Tischer's avatar
Christian Tischer committed
99 100
    "data type" -> float -> "..., -1031.0, ..., 10.5, ...";
    "data type" -> "...";
Christian Tischer's avatar
Christian Tischer committed
101
    "data type" -> "N-bit unsigned integer";
Christian Tischer's avatar
Christian Tischer committed
102 103 104 105 106
  }
'/>

### Activity: Pixel based background subtraction

Christian Tischer's avatar
Christian Tischer committed
107
* Open image: xy_8bit__nuclei_noisy_different_intensity.tif
108
* Appreciate the significant background intensity
Christian Tischer's avatar
Christian Tischer committed
109 110
* Measure pixel values at `[ 28, 35 ]` and `[ 28, 39 ]`
* Measure the image background intensity in this region:
111 112 113
    * upper left corner at `[ 20, 35 ]`
    * width = 10
    * height = 10
Christian Tischer's avatar
Christian Tischer committed
114
* Subtract the measured background intensity from each pixel.
Christian Tischer's avatar
Christian Tischer committed
115
* Measure the pixel values again. 
Christian Tischer's avatar
Christian Tischer committed
116
* Observe that the resuls are incorrect.
Christian Tischer's avatar
Christian Tischer committed
117 118 119

Repeat above activity, but:

Christian Tischer's avatar
Christian Tischer committed
120
* After opening the image, convert its data type to floating point.
Christian Tischer's avatar
Christian Tischer committed
121

Christian Tischer's avatar
Christian Tischer committed
122
### Activity: Explore the limitations of `float` data type
Christian Tischer's avatar
Christian Tischer committed
123

Christian Tischer's avatar
Christian Tischer committed
124 125 126 127
* Create an empty image
* Set all pixel values to 1000000000.0
* Add 1.0 to all pixel values
* Be shocked...
Christian Tischer's avatar
Christian Tischer committed
128 129 130 131 132

...it turns out that from 16777216 on you cannot represent all integers anymore within a float. 

### Formative Assessment

Christian Tischer's avatar
Christian Tischer committed
133
True or false?
Christian Tischer's avatar
Christian Tischer committed
134

Christian Tischer's avatar
Christian Tischer committed
135 136 137 138 139
* Subtracting 100 from 50 in a 8-bit image will result in -50.
* Adding 1 to 255 in a 8-bit image will result in 256.
* Subtracting 10.1 from 10.0 in a float image will result in -0.1
* Adding 1.0 to 255.0 in a float image will result in 256.0
* Adding 1000.0 to 1000000000.0 in a float image will result in 1000001000.0
Christian Tischer's avatar
Christian Tischer committed
140 141 142

### Learn more

Christian Tischer's avatar
Christian Tischer committed
143
* [Limitations of float](https://randomascii.wordpress.com/2012/02/13/dont-store-that-in-a-float/)
Christian Tischer's avatar
Christian Tischer committed
144

Christian Tischer's avatar
Christian Tischer committed
145 146 147 148 149 150
&nbsp;

&nbsp;

&nbsp;

151
## Pixel data type conversions
Christian Tischer's avatar
Christian Tischer committed
152 153 154 155

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
156 157
    "data type conversion" -> "values" [label="  can change"];
    "data type conversion" -> "value range" [label="  changes"];
Christian Tischer's avatar
Christian Tischer committed
158 159 160 161 162
  }
'/>

### Activity: 16-bit to 8-bit conversion

Christian Tischer's avatar
Christian Tischer committed
163
* Open image: xy_16bit__two_values.tif
Christian Tischer's avatar
Christian Tischer committed
164
* Convert to 8-bit
Christian Tischer's avatar
Christian Tischer committed
165
* Understand the mathematics underlying the conversion from 16-bit to 8-bit.
Christian Tischer's avatar
Christian Tischer committed
166 167 168

### Activity: 16-bit to float conversion

Christian Tischer's avatar
Christian Tischer committed
169
* Open image: xy_16bit__two_values.tif
Christian Tischer's avatar
Christian Tischer committed
170
* Convert to float
Christian Tischer's avatar
Christian Tischer committed
171 172 173

### Formative Assessment

Christian Tischer's avatar
Christian Tischer committed
174
True or false? Discuss with your neighbor!
Christian Tischer's avatar
Christian Tischer committed
175 176 177 178

1. Changing pixel data type never changes pixel values.
2. Converting from 16-bit unsigned integer to float never changes the pixel values.
3. Changing from float to 16-bit unsigned integer never changes the pixel values.
Christian Tischer's avatar
Christian Tischer committed
179
4. There is only one correct way to convert from 16-bit to 8-bit.
Christian Tischer's avatar
Christian Tischer committed
180

Christian Tischer's avatar
Christian Tischer committed
181 182 183 184 185 186
&nbsp;

&nbsp;

&nbsp;

Christian Tischer's avatar
Christian Tischer committed
187 188
## Thresholding

Christian Tischer's avatar
Christian Tischer committed
189
In order to find objects in a image, the first step often is to determine whether a pixel is part of an object (foreground) or of the image background. In fluorescence microscopy this often can be achieved by thresholding.
Christian Tischer's avatar
Christian Tischer committed
190 191 192

<img src='https://g.gravizo.com/svg?
 digraph G {
Christian Tischer's avatar
Christian Tischer committed
193 194 195 196 197 198 199 200 201 202 203
	shift [fontcolor=white,color=white];
	"intensity image" -> threshold;
	threshold -> "binary image";
	"binary image" -> "mask" [label="  aka"];
	"binary image" -> "background value";
	"binary image" -> "foreground value";
	"background value" -> "0";
	"foreground value" -> "1";
	"foreground value" -> "255";
	"pixel value" -> ">= threshold" -> foreground;
	"pixel value" -> "< threshold" -> background;
Christian Tischer's avatar
Christian Tischer committed
204 205 206
 }
'/>

Christian Tischer's avatar
Christian Tischer committed
207
### Activity: Threshold an image
Christian Tischer's avatar
Christian Tischer committed
208

Christian Tischer's avatar
Christian Tischer committed
209
* Open image: xy_8bit__two_cells.tif
Christian Tischer's avatar
Christian Tischer committed
210
* Convert the image to a binary image by means of thresholding.
Christian Tischer's avatar
Christian Tischer committed
211

Christian Tischer's avatar
Christian Tischer committed
212
### Formative assessment
Christian Tischer's avatar
Christian Tischer committed
213

Christian Tischer's avatar
Christian Tischer committed
214
True or false? Discuss with your neighbour!
Christian Tischer's avatar
Christian Tischer committed
215

Christian Tischer's avatar
Christian Tischer committed
216 217 218 219
* For each image there is only one correct threshold value.
* The result of thresholding is a binary image.
* A binary image can have three values: `-1, 0, +1`
* Values below the threshold are always set to `1`.
Christian Tischer's avatar
Christian Tischer committed
220

Christian Tischer's avatar
Christian Tischer committed
221 222 223 224 225 226 227
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
228 229 230 231 232
## Connected components analysis

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
233 234
    "intensity image" -> "connected component analysis" -> "label image";
    connectivity -> "connected component analysis"; 
Christian Tischer's avatar
Christian Tischer committed
235 236 237 238 239 240
  }
'/>


### Activity: 2D connected components analysis

Christian Tischer's avatar
Christian Tischer committed
241
* Open image: xy_8bit_binary__nuclei.tif
Christian Tischer's avatar
Christian Tischer committed
242 243 244
* Perform connected components analysis
* Explore multi-color LUTs for object labelling
* Explore removing and joining labels
Christian Tischer's avatar
Christian Tischer committed
245 246 247 248 249 250


### Activity: 3D connected components analysis

Repeat above activity but use a 3D image:

Christian Tischer's avatar
Christian Tischer committed
251
* Open image: xyz_8bit_binary__spots.tif
Christian Tischer's avatar
Christian Tischer committed
252 253 254

### Formative assessment

Christian Tischer's avatar
Christian Tischer committed
255
Fill in the blanks, using these words: less, more, 8, 255, 4, more.
Christian Tischer's avatar
Christian Tischer committed
256

Christian Tischer's avatar
Christian Tischer committed
257 258 259 260 261 262 263
* For a given input image there is only one correct connectivity.
* In 3D, pixels have _____ neighbors than in 2D.
* 8-connected connectivity results in _____ objects than 4-connected connectivity.
* In 3D, pixels have ____ non-diagonal neighbors.
* In 2D, pixels have ____ non-diagonal neighbors.
* A 8-bit label image can maximally have _____ objects.
* The maximum value in a label image is equal to or _____ than the number of objects.
Christian Tischer's avatar
Christian Tischer committed
264 265


Christian Tischer's avatar
Christian Tischer committed
266 267 268 269 270 271 272
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
273 274 275 276
## Shape measurements

<img src='https://g.gravizo.com/svg?
 digraph G {
Christian Tischer's avatar
Christian Tischer committed
277 278 279 280 281
	shift [fontcolor=white,color=white];
	"label image" -> shape_analysis -> table;
	table -> object_rows;
	table -> feature_columns;
	table -> visualisation;  
Christian Tischer's avatar
Christian Tischer committed
282 283 284 285 286 287
}
'/>


### Activity: Measure object shape parameters

Christian Tischer's avatar
Christian Tischer committed
288
* Open image: xy_8bit_labels__four_objects.tif
Christian Tischer's avatar
Christian Tischer committed
289
* Perform shape measurements and discuss their meanings.
Christian Tischer's avatar
Christian Tischer committed
290 291
* Explore results visualisation
	* Color objects by their measurement values.
Christian Tischer's avatar
Christian Tischer committed
292 293
* Add a calibration to the image and check which shape measurements are affected.
* Draw a test image to understand the shape measurements even better.
Christian Tischer's avatar
Christian Tischer committed
294 295 296

### Formative assessment

Christian Tischer's avatar
Christian Tischer committed
297
True or false? Discuss with your neighbour!
Christian Tischer's avatar
Christian Tischer committed
298

Christian Tischer's avatar
Christian Tischer committed
299 300 301 302 303
* Circularity is independent of image calibration.
* Area is independent of image calibration.
* Perimeter can strongly depend on spatial sampling.
* Volume can strongly depend on spatial sampling.
* Drawing test images to check how certain shape parameters behave is a good idea.
Christian Tischer's avatar
Christian Tischer committed
304 305 306

### Learn more

Christian Tischer's avatar
Christian Tischer committed
307 308 309 310 311
* Especially surface and perimeter measurements are affected by sampling and resolution, see for example:
	* https://en.wikipedia.org/wiki/Coastline_paradox).
* Results visualisation:
	* https://imagej.net/MorphoLibJ#Grayscale_morphological_filters: **Label visualization in 3D viewer**

Christian Tischer's avatar
Christian Tischer committed
312

Christian Tischer's avatar
Christian Tischer committed
313 314 315 316 317 318 319
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
320 321 322 323 324
## Object shape measurement workflow

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
325 326 327
    "intensity image" -> "binary image" [label="  threshold"];
    "binary image" -> "label image" [label="  connected components"];
    "label image" -> table [label="  measure_shape"];
Christian Tischer's avatar
Christian Tischer committed
328 329 330 331 332
}
'/>

### Activity: Segment objects and measure shapes

Christian Tischer's avatar
Christian Tischer committed
333
* Open image: xy_8bit__two_cells.tif
Christian Tischer's avatar
Christian Tischer committed
334 335
* Segment the cells and measure their shapes.
	* Devise code to automate the workflow.
Christian Tischer's avatar
Christian Tischer committed
336 337 338

### Formative assessment

Christian Tischer's avatar
Christian Tischer committed
339
Fill in below blanks, using these words: equal_to, larger_than, smaller_than, binary, connected_component_analysis, thresholding
Christian Tischer's avatar
Christian Tischer committed
340

Christian Tischer's avatar
Christian Tischer committed
341 342 343 344 345
* A label image is the result of _____ .
* The number of pixels in a binary image is typically _____ the number of connected components. 
* The number of distinct values in a label image is _____ the number of objects (minus one).
* Converting an intensity image to a _____ image can be achieved by _____ .
* The number of connected components can be _____ the maximal label.
Christian Tischer's avatar
Christian Tischer committed
346

Christian Tischer's avatar
Christian Tischer committed
347 348 349 350 351 352 353
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
354
## Intensity measurements
Christian Tischer's avatar
Christian Tischer committed
355

Christian Tischer's avatar
Christian Tischer committed
356
### Activity: Measure intensities in image regions
Christian Tischer's avatar
Christian Tischer committed
357

Christian Tischer's avatar
Christian Tischer committed
358
* Open image: xy_float__h2b_bg_corr.tif
Christian Tischer's avatar
Christian Tischer committed
359 360 361 362 363 364 365
* Measure for both nuclei:
	* Maximum intensity
	* Average intensity
	* Median intensity
	* Sum intensity
* Discuss the interpretation!
* Discuss where to measure!
Christian Tischer's avatar
Christian Tischer committed
366 367


Christian Tischer's avatar
Christian Tischer committed
368 369 370
### Activity: Intensity measurements without pixel based background correction

#### Motivation
Christian Tischer's avatar
Christian Tischer committed
371

Christian Tischer's avatar
Christian Tischer committed
372
There are several good reasons not to subtract the background from each pixel in an image: 
Christian Tischer's avatar
Christian Tischer committed
373

Christian Tischer's avatar
Christian Tischer committed
374 375
* It is a bit tricky to do it right, because one has to convert to float to accomodate floting point and negative values.
* If one has really big image data (TB) one would need (at least) another TB storage for the background corrected version of the image.
Christian Tischer's avatar
Christian Tischer committed
376

Christian Tischer's avatar
Christian Tischer committed
377
#### Workflow
Christian Tischer's avatar
Christian Tischer committed
378

Christian Tischer's avatar
Christian Tischer committed
379
* Open image: xy_calibrated_8bit__two_nuclei_high_background.tif
Christian Tischer's avatar
Christian Tischer committed
380 381 382 383 384 385 386 387 388 389
* Measure for both nuclei and a background region:
	* Maximum intensity
	* Average intensity
	* Median intensity
	* Sum intensity
* Discuss how to correct the intensities for the background
	* Appreciate that you also need the region areas for this task
	* Measure the region areas
		* Watch out: the image is calibrated!
		* Use the area for the correction.
Christian Tischer's avatar
Christian Tischer committed
390

Christian Tischer's avatar
Christian Tischer committed
391
### Formative assessment: Intensity measurements
Christian Tischer's avatar
Christian Tischer committed
392

Christian Tischer's avatar
Christian Tischer committed
393
Fill in the blanks, using these words: integrated, mean, number_of_pixels, decrease, increase, sum
Christian Tischer's avatar
Christian Tischer committed
394

Christian Tischer's avatar
Christian Tischer committed
395 396 397 398
* Average intensity is just another word for _____ intensity.
* The _____ intensity is equal to the mean intensity times the _____ in the measured region.
* In an 8-bit image, increasing the size of the measurement region can only _____ the sum intensity.
* In a float image, increasing the size of the measurement region can _____ the sum intensity. 
Christian Tischer's avatar
Christian Tischer committed
399

Christian Tischer's avatar
Christian Tischer committed
400 401 402 403 404 405 406
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
407 408 409 410 411 412
## Convolution filters

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
    "intensity image" -> "convolution" -> "filtered image";
Christian Tischer's avatar
Christian Tischer committed
413 414
    "small image" -> size;
    "small image" -> "pixel values";
Christian Tischer's avatar
Christian Tischer committed
415
    "kernel" -> "small image" [label="  is"]; 
Christian Tischer's avatar
Christian Tischer committed
416 417 418 419 420 421
    "kernel" -> "convolution";
}
'/>

### Activity: Explore convolution filters

Christian Tischer's avatar
Christian Tischer committed
422
* Open image: xy_8bit__nuclei_noisy_different_intensity.tif
Christian Tischer's avatar
Christian Tischer committed
423 424 425 426 427 428
* Try the result of different convolution filters, e.g.
	* https://en.wikipedia.org/wiki/Kernel_(image_processing)
	* Mean filter
	* Gaussian blur
	* Edge detection
* Appreciate that the results are (slightly) wrong within the 8-bit range of the input image.
Christian Tischer's avatar
Christian Tischer committed
429 430 431

### Activity: Use mean filter to facilitate image segmentation

Christian Tischer's avatar
Christian Tischer committed
432
* Open image: xy_8bit__nuclei_noisy_different_intensity.tif
Christian Tischer's avatar
Christian Tischer committed
433 434 435
* Appreciate that you cannot readily threshold the image
* Apply a mean filter
* Threshold the filtered image
Christian Tischer's avatar
Christian Tischer committed
436 437 438

### Formative assessment

Christian Tischer's avatar
Christian Tischer committed
439 440
* Draw the kernel of a 3x3 mean filter.
* Draw three different kernels that enhance edges.
Christian Tischer's avatar
Christian Tischer committed
441 442 443
 
### Learn more

Christian Tischer's avatar
Christian Tischer committed
444
* https://en.wikipedia.org/wiki/Kernel_(image_processing)
Christian Tischer's avatar
Christian Tischer committed
445

Christian Tischer's avatar
Christian Tischer committed
446

Christian Tischer's avatar
Christian Tischer committed
447 448 449 450 451 452 453
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
454
## Typical image analysis workflow
Christian Tischer's avatar
Christian Tischer committed
455

Christian Tischer's avatar
Christian Tischer committed
456
![image](/uploads/b4bdce17515908f40d858b35d5e9256e/image.png)
Christian Tischer's avatar
Christian Tischer committed
457

Christian Tischer's avatar
Christian Tischer committed
458 459 460 461 462 463 464
&nbsp;

&nbsp;

&nbsp;


Christian Tischer's avatar
Christian Tischer committed
465 466