bioimage-analysis-fundamentals.md 13.5 KB
Newer Older
Christian Tischer's avatar
Course  
Christian Tischer committed
1 2
# Bioimage analysis fundamentals

Christian Tischer's avatar
Christian Tischer committed
3
## Pixel values, coordinates, and data types
Christian Tischer's avatar
Course  
Christian Tischer committed
4 5 6 7 8 9 10 11 12 13

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
    image -> pixel [label="  has many"];
    pixel -> value;
    pixel -> indices;
    pixel -> coordinates;
    indices -> calibration;
    calibration -> coordinates;
Christian Tischer's avatar
Christian Tischer committed
14
    image -> calibration [label="  can have"];
Christian Tischer's avatar
Course  
Christian Tischer committed
15 16 17 18 19 20
    pixel -> voxel [label="  3D"];   
  }
'/>

### Activity

Christian Tischer's avatar
Christian Tischer committed
21
- Open image: `xy_8bit__nuclei_noisy_different_intensity.tif`
Christian Tischer's avatar
Christian Tischer committed
22
	- Explore different ways to inspect pixel values and indices
Christian Tischer's avatar
Course  
Christian Tischer committed
23
	- Add image calibration
Christian Tischer's avatar
Christian Tischer committed
24 25
	- Check where the calibration is visible
	
Christian Tischer's avatar
Course  
Christian Tischer committed
26 27
### Formative assessment

Christian Tischer's avatar
Christian Tischer committed
28
True or false?
Christian Tischer's avatar
Course  
Christian Tischer committed
29 30 31 32 33 34 35 36 37 38 39 40

- Pixel coordinates are always integer values.
- Changing the image calibration changes the pixel values.
- Pixel coordinates depend on image calibration.
- Pixel indices are always positive integer values.
- The lowest pixel index of a 2D image always is `[1,1]`.

## Image display

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
41 42 43 44
    LUT -> color;
    LUT -> brightness;
    min -> LUT;
    max -> LUT;
Christian Tischer's avatar
Christian Tischer committed
45
    "pixel value" -> LUT;
Christian Tischer's avatar
Course  
Christian Tischer committed
46 47 48 49 50 51 52 53 54 55 56
  }
'/>

```
brightness( pixel_value ) = ( pixel_value - LUT_min ) / ( LUT_max - LUT_min )
0 <= brightness <= 1 
contrast = LUT_max - LUT_min 
``` 

### Activity

Christian Tischer's avatar
Christian Tischer committed
57
- Open image: `xy_8bit__nuclei_noisy_different_intensity.tif` 
Christian Tischer's avatar
Course  
Christian Tischer committed
58 59 60 61
- Change LUT settings
	- Appreciate that LUT settings do not affect image content.


Christian Tischer's avatar
Christian Tischer committed
62 63
### Formative Assessment

Christian Tischer's avatar
Christian Tischer committed
64 65 66 67 68 69 70 71 72
Fill in the blanks:

decrease, larger_than, increase, smaller_than 

- Pixels with values _____ the LUT_max will appear saturated. 
- Decreasing the LUT_max while keeping the LUT_min constant will _____ the contrast.
- Decreasing both LUT_max and LUT_min will _____ the overall brightness.
- Pixels with values _____ the LUT_min will appear black, when using a grayscale LUT.

Christian Tischer's avatar
Christian Tischer committed
73 74 75

## Image math and pixel data types

Christian Tischer's avatar
Christian Tischer committed
76 77 78 79 80 81 82 83 84 85 86
<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
    "data type" -> image;
    image -> "pixel values";
    "image math" -> "pixel values" [label="  changes"];
    "image math" -> "data type" [label="  doesn't change"];
   }
'/>


Christian Tischer's avatar
Christian Tischer committed
87 88 89
<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
90 91 92 93 94
    "data type" -> image;
    image -> "pixel values";
    "image math" -> "pixel values" [label="  changes"];
    "image math" -> "data type" [label="  doesn't change"];
    "data type" -> "8-bit unsigned integer";
Christian Tischer's avatar
Christian Tischer committed
95 96 97 98 99 100
    "8-bit unsigned integer" -> "0 to 255";
    "16-bit unsigned integer" -> "0 to 65535";
    "N bit unsigned integer" -> "0 to 2^N - 1";
    "pixel data type" -> "16-bit unsigned integer";
    "pixel data type" -> "32-bit float";
    "pixel data type" -> "N-bit unsigned integer";
Christian Tischer's avatar
Christian Tischer committed
101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145
  }
'/>

### Motivation

It sometimes is necessary to change numberic content of images. It is important to understand how to do this properly in order to avoid uncontrolled artifacts.

What are good reasons to change the pixel values in an image?

1. For intensity measurements, the image background (e.g. camera based offset) should be subtracted from all pixels.
2. For threshold based image segmentation (object detection), it helps to first filter noise in the image.
3. For intensity measurements, it helps to filter noise in the image.
4. The image appears to dark, multiplication of all pixels by a constant number is a means to make it brighter.
5. For uneven illumination (e.g. occuring in wide-field microscopy with large camera chips), one should do flat-field correction, which makes the intensity values even across the image.
6. Our microscope was broken. We took images on a replacement microscope. The pixel values were consistently higher than on our usual microscope. We multiplied the pixels on all images from the replacement microscope by a constant factor to make them comparable to our usual data.


### Activity: Pixel based background subtraction

- Open image: `xy_8bit__nuclei_noisy_different_intensity.tif`
- Appreciate the significant background intensity
- Measure pixel value at `[ 28, 35 ]` and `[ 28, 39 ]`
- Measure background intensity in below region:
        - upper left corner at `[ 20, 35 ]`
        - width = 10
        - height = 10
- Subtract the measured background intensity from each pixel
- Measure pixel values again at above coordinates ( `[ 28, 35 ]` and `[ 28, 39 ]` )
- Discuss how the pixel values changed during background subtraction

Repeat above activity, but:

- After opening the image, convert its pixel data type to floating point

### Activity: Limitations of float

- Create an empty image
- Set all pixel values to 1000000000.0
- Add 1.0 to all pixel values
- Be shocked :-)

...it turns out that from 16777216 on you cannot represent all integers anymore within a float. 

### Formative Assessment

Christian Tischer's avatar
Christian Tischer committed
146
True or false?
Christian Tischer's avatar
Christian Tischer committed
147

Christian Tischer's avatar
Christian Tischer committed
148 149 150 151 152
- Subtracting 100 from 50 in a 8-bit image will result in -50.
- Adding 1 to 255 in a 8-bit image will result in 256.
- Subtracting 10.1 from 10.0 in a float image will result in -0.1
- Adding 1.0 to 255.0 in a float image will result in 256.0
- Adding 1000.0 to 1000000000.0 in a float image will result in 1000001000.0
Christian Tischer's avatar
Christian Tischer committed
153 154 155 156 157 158 159 160 161 162

### Learn more

- [Limitations of float](https://randomascii.wordpress.com/2012/02/13/dont-store-that-in-a-float/)

## Pixel data type conversions

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
163 164
    "pixel_type_conversion" -> "pixel_values" [label="  can change"];
    "pixel_type_conversion" -> pixel_value_range [label="  changes"];
Christian Tischer's avatar
Christian Tischer committed
165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188
  }
'/>

### Motivation

What are good reasons to change the pixel data type of an image?

- a
- b
- c
- d

### Activity: 16-bit to 8-bit conversion

- Open image: `xy_16bit__two_values.tif`
- Convert to 8-bit

### Activity: 16-bit to float conversion

- Open image: `xy_16bit__two_values.tif`
- Convert to float

### Formative Assessment

Christian Tischer's avatar
Christian Tischer committed
189 190 191 192 193
True or false?

1. Changing pixel data type never changes pixel values.
2. Converting from 16-bit unsigned integer to float never changes the pixel values.
3. Changing from float to 16-bit unsigned integer never changes the pixel values.
Christian Tischer's avatar
Christian Tischer committed
194 195 196 197 198 199 200 201


## Image segmentation overview


<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
202 203
    "intensity image" -> "binary image" -> label_image;
    "binary image" <- background_value;
Christian Tischer's avatar
Christian Tischer committed
204 205 206 207 208
    _0_ -> background_value;
    _1_ -> foreground_value;
    _255_ -> foreground_value;
    foreground_value -> binary_image;
    object_indices -> label_image;
Christian Tischer's avatar
Christian Tischer committed
209 210 211 212 213 214
  }
'/>


## Thresholding

Christian Tischer's avatar
Christian Tischer committed
215
In order to find objects in a image, the first step often is to determine whether a pixel is part of an object (foreground) or of the image background. In fluorescence microscopy this often can be achieved by thresholding.
Christian Tischer's avatar
Christian Tischer committed
216 217 218 219

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
220
    "intensity image" -> threshold;
Christian Tischer's avatar
Christian Tischer committed
221 222 223 224 225 226 227 228 229 230 231 232 233 234
    threshold -> binary_image;
    pixel_value -> larger_equal_threshold -> foreground;
    pixel_value -> smaller_threshold -> background;
 }
'/>


## Activity: Threshold an image

- Open image: `xy_8bit__two_cells.tif`
- Convert the image to a binary image by means of thresholding.

## Formative assessment

Christian Tischer's avatar
Christian Tischer committed
235
True or false? Discuss with your neighbor!
Christian Tischer's avatar
Christian Tischer committed
236 237 238 239

- For each image there is only one correct threshold value.
- The result of thresholding is a binary image.
- A binary image can have three values: -1,0,+1
Christian Tischer's avatar
Christian Tischer committed
240
- Values below the threshold are always set to 1.
Christian Tischer's avatar
Christian Tischer committed
241 242 243 244 245 246

## Connected components analysis

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
247
    "intensity image" -> connected_component_analysis -> label_image;
Christian Tischer's avatar
Christian Tischer committed
248
    connectivity -> connected_component_analysis; 
Christian Tischer's avatar
Christian Tischer committed
249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268
  }
'/>


### Activity: 2D connected components analysis

- Open image: `xy_8bit_binary__nuclei.tif`
- Perform connected components analysis
- Explore multi-color LUTs for object labelling
- Explore removing and joining labels


### Activity: 3D connected components analysis

Repeat above activity but use a 3D image:

- Open image: `xyz_8bit_binary__spots.tif`

### Formative assessment

Christian Tischer's avatar
Christian Tischer committed
269 270 271
Fill in the blanks:

less, more, 8, 255, 4, more.
Christian Tischer's avatar
Christian Tischer committed
272 273

- For a given input image there is only one correct connectivity.
Christian Tischer's avatar
Christian Tischer committed
274 275 276 277 278 279
- In 3D, pixels have _____ neighbors than in 2D.
- 8-connected connectivity results in _____ objects than 4-connected connectivity.
- In 3D, pixels have ____ non-diagonal neighbors.
- In 2D, pixels have ____ non-diagonal neighbors.
- A 8-bit label image can maximally have _____ objects.
- The maximum value in a label image is equal to or _____ than the number of objects.
Christian Tischer's avatar
Christian Tischer committed
280 281 282 283 284 285 286


## Shape measurements

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
287
    "label image" -> shape_analysis -> table;
Christian Tischer's avatar
Christian Tischer committed
288 289
    object_rows -> table;
    feature_columns -> table;  
Christian Tischer's avatar
Christian Tischer committed
290 291 292 293 294 295 296 297 298 299 300 301 302 303 304
}
'/>


### Activity: Measure object shape parameters

- Open image: `xy_8bit_labels__four_objects.tif`
- Perform shape measurements and discuss their meanings.
- Color objects by their measurement values.
- Add a calibration to the image and check which shape measurements are affected.
- Draw a test image to understand the shape measurements even better.


### Formative assessment

Christian Tischer's avatar
Christian Tischer committed
305
Which statements are true? Discuss with your neighbor!
Christian Tischer's avatar
Christian Tischer committed
306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321

- Circularity is independent of image calibration.
- Area is independent of image calibration.
- Perimeter can strongly depend on spatial sampling.
- Volume can strongly depend on spatial sampling.
- Drawing test images to check how certain shape parameters behave is a good idea.

### Learn more

- Especially surface and perimeter measurements are affected by sampling and resolution (see for example: https://en.wikipedia.org/wiki/Coastline_paradox).

## Object shape measurement workflow

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
Christian Tischer's avatar
Christian Tischer committed
322 323 324
    "intensity image" -> "binary image" [label="  threshold"];
    "binary image" -> "label image" [label="  connected components"];
    "label image" -> table [label="  measure_shape"];
Christian Tischer's avatar
Christian Tischer committed
325 326 327 328 329 330 331 332 333 334 335
}
'/>

### Activity: Segment objects and measure shapes

- Open image: `xy_8bit__two_cells.tif`
- Segment the cells and measure their shapes.
	- Devise code to automate the workflow.

### Formative assessment

Christian Tischer's avatar
Christian Tischer committed
336
Fill in below blanks, using these words:
Christian Tischer's avatar
Christian Tischer committed
337

Christian Tischer's avatar
Christian Tischer committed
338
equal_to, larger_than, smaller_than, binary, connected_component_analysis, thresholding
Christian Tischer's avatar
Christian Tischer committed
339

Christian Tischer's avatar
Christian Tischer committed
340 341 342 343 344
- A label image is the result of _____ .
- The number of pixels in a binary image is typically _____ the number of connected components. 
- The number of distinct values in a label image is _____ the number of objects (minus one).
- Converting an intensity image to a _____ image can be achieved by _____ .
- The number of connected components can be _____ the maximal label.
Christian Tischer's avatar
Christian Tischer committed
345

Christian Tischer's avatar
Christian Tischer committed
346
## Intensity measurements
Christian Tischer's avatar
Christian Tischer committed
347

Christian Tischer's avatar
Christian Tischer committed
348
### Activity: Measure DNA intensity
Christian Tischer's avatar
Christian Tischer committed
349

Christian Tischer's avatar
Christian Tischer committed
350 351 352 353 354 355 356
- Open image: `xy_float__h2b_bg_corr.tif`
- Measure for both nuclei:
	- Maximum intensity
	- Average intensity
	- Sum intensity
- Discuss the interpretation!
- Discuss where to measure!
Christian Tischer's avatar
Christian Tischer committed
357 358


Christian Tischer's avatar
Christian Tischer committed
359 360 361
### Activity: Intensity measurements without pixel based background correction

#### Motivation
Christian Tischer's avatar
Christian Tischer committed
362

Christian Tischer's avatar
Christian Tischer committed
363
There are several good reasons not to subtract the background from each pixel in an image: 
Christian Tischer's avatar
Christian Tischer committed
364

Christian Tischer's avatar
Christian Tischer committed
365 366
- It is a bit tricky to do it right, because one has to convert to float to accomodate floting point and negative values.
- If one has really big image data (TB) one would need (at least) another TB storage for the background corrected version of the image.
Christian Tischer's avatar
Christian Tischer committed
367

Christian Tischer's avatar
Christian Tischer committed
368
#### Workflow
Christian Tischer's avatar
Christian Tischer committed
369

Christian Tischer's avatar
Christian Tischer committed
370 371 372 373 374 375 376 377 378 379
- Open image: `xy_calibrated_8bit__two_nuclei_high_background.tif`
- Measure for both nuclei and a background region:
	- Maximum intensity
	- Average intensity
	- Median intensity
	- Sum intensity
- Discuss how to correct the intensities for the background
- Measure the region areas in pixel units
	- Watch out: the image is calibrated!
	- Use the area for the correction.
Christian Tischer's avatar
Christian Tischer committed
380

Christian Tischer's avatar
Christian Tischer committed
381
### Formative assessment: Intensity measurements
Christian Tischer's avatar
Christian Tischer committed
382

Christian Tischer's avatar
Christian Tischer committed
383 384 385 386 387 388 389 390 391
Fill in below blanks with those words:

integrated, mean, number_of_pixels, decrease, increase, sum

- Average intensity is just another word for _____ intensity.
- The _____ intensity is equal to the mean intensity times the _____ in the measured region.
- In an 8-bit image, increasing the size of the measurement region can only _____ the sum intensity.
- In a float image, increasing the size of the measurement region can _____ the sum intensity. 

Christian Tischer's avatar
Christian Tischer committed
392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420
## Convolution filters

<img src='https://g.gravizo.com/svg?
 digraph G {
    shift [fontcolor=white,color=white];
    "intensity image" -> "convolution" -> "filtered image";
    "size" -> "small image";
    "pixel values" -> "small image";
    "small image" -> "kernel"; 
    "kernel" -> "convolution";
}
'/>

### Activity: Explore convolution filters

- Open image: `xy_8bit__nuclei_noisy_different_intensity.tif` 
- Try the result of different convolution filters, e.g.
	- https://en.wikipedia.org/wiki/Kernel_(image_processing)
	- Mean filter
	- Gaussian blur
	- Edge detection
- Appreciate that the results are (slightly) wrong within the 8-bit range of the input image.

### Activity: Use mean filter to facilitate image segmentation

- Open image: `xy_8bit__nuclei_noisy_different_intensity.tif` 
- Appreciate that you cannot readily threshold the image
- Apply a mean filter
- Threshold the filtered image
Christian Tischer's avatar
Christian Tischer committed
421 422 423

### Formative assessment

Christian Tischer's avatar
Christian Tischer committed
424 425 426 427 428 429
- Draw the kernel of a 3x3 mean filter.
- Draw three different kernels that enhance edges.
 
### Learn more

- https://en.wikipedia.org/wiki/Kernel_(image_processing)
Christian Tischer's avatar
Christian Tischer committed
430 431 432 433

### Formative assessment

Which statements are true?
Christian Tischer's avatar
Course  
Christian Tischer committed
434

Christian Tischer's avatar
Christian Tischer committed
435
## Typical image analysis workflow
Christian Tischer's avatar
Course  
Christian Tischer committed
436

Christian Tischer's avatar
Christian Tischer committed
437
![image](/uploads/b4bdce17515908f40d858b35d5e9256e/image.png)
Christian Tischer's avatar
Course  
Christian Tischer committed
438

Christian Tischer's avatar
Christian Tischer committed
439 440
## Recap

Christian Tischer's avatar
Christian Tischer committed
441
(Work in pairs of two)
Christian Tischer's avatar
Christian Tischer committed
442 443 444 445 446 447 448 449 450 451 452

- Take one A4 paper
- Draw a typical workflow: From intensity image to objects shape table.
- Write down what you remember (max. 3 facts) about:
	- Intensity measurements
	- Object shape measurements
	- Label image
	- Pixel data types