Skip to content

Changes

Page history
separtated files authored by Antonio Politi's avatar Antonio Politi
Hide whitespace changes
Inline Side-by-side
AdaptiveFeedback.md 0 → 100644
View page @ 4bfe47c9
# Adaptive feedback microscopy/online image analysis
With the adaptive feedback microscopy capability of MyPiC the user can combine online image analysis and image acquisition to perform:
* Detection of rare events and start a different imaging protocol
* Automated FCS and imaging experiments
* Time-lapse FCS measurements
* Cell tracking
* Fluorescence recovery after photobleaching (FRAP) experiments
Images are acquired by the microscope and then processed by an image analysis program that is monitoring the arrival of new images to be analyzed. After the image has been analyzed commands are sent to the microscope in order to start a different pipeline or update the XYZ position for tracking an object in space and time. To start a different pipeline upon event detection the settings for the [Trigger1 and/or Trigger2](#trigger) must be defined.
The communication between the image analysis software is through the windows registry, thus any program that can read and write to the windows registry can be used. An example of an application using ImageJ for online image analysis can be found in
## Windows registry
MyPiC communicates with external programs by reading and writing in the Windows Registry
```
HKEY_CURRENT_USER\Software\VB and VBA Program Settings\OnlineImageAnalysis\macro
```
This occurs for each imaging job when **Online img. analysis** in the Method of [Processing and Tracking](#taskopt) is set. After the acquisition of the corresponding job the key [**CodeMic**](#codemic) will be set to `wait`, [**CodeOia**](#codeoia) to `newImage` and [**filePath**](#filepath) to the name of image to analyze. MyPiC will continuously monitor **CodeMic** until the value changes from `wait` to one of the other possible values. If **CodeMic** is `focus`, `trigger1`, `trigger2` coordinates from keys [**X, Y, Z**](#xyz) and [**FCSX, FCSY, FCSZ**](#fcsxyz) are read. Via the registry the user can also dynamically set bleaching or acquisition [ROIs](#rois) for an automated FRAP experiment. The full list of online image analysis keys can be obtained by clicking on the **Show OIA Keys** button of the main Macro window. Here we provide a list of the most important commands.
<div align='center'>
<img src="./images/windowsRegistryCommands.jpg" width = "600px">
</div>
<a name="codemic"></a>
#### Key **CodeMic**
Contains the commands for the microscope
Value | Meaning
----------- | ------
`wait` | do not progress in imaging
`error` | the online image analsys pipeline has encountered an error
`nothing` | Nothing has been found
`trigger1` | Start **Trigger1** pipeline using values given in **X**, **Y**, **Z** registry entries
`trigger2` | Start **Trigger2** imaging using values given in **X**, **Y**, **Z** registry entries
`focus` | Update XYZ coordinate of current imaging position. This can be used to perform object based tracking
`setFcsPos` | Read position(s) for FCS measurement from **fcsX**, **fcsY**, **fcsZ**. <br/> Start an FCS experiment if next task in pipeline is a FCS job
`setRoi` | Read ROIs and update ROIs for subsequent task(s) in pipeline
<a name="codeoia"></a>
#### Key **CodeOia**
Contains the commands for the image analysis program.
Value | Meaning
------ | ------
`nothing` | Nothing to do
`newImage` | A new image with path **filePath** needs to be analysed
<a name="filepath"></a>
#### Key **filePath**
Contains the full path of image that needs to be analysed
Value | Meaning
------ | ------
`path/to/image` | Path of image to analyse
<a name="xyz"></a>
#### Keys **X**, **Y**, **Z**
Specifies stage positions. Coordinates with respect to upper left corner of the image (in pixels) and first slice (0,0,0). Several points are separated by `;`
Value | Meaning
----------- | ------
`X1;X2;X3;...` | X positions with respect to left corner in pixels
`Y1;Y2;Y3;...` | Y positions with respect to upper corner in pixels
`Z1;Z2;Z3;...` | Z positions with respect to bottom slice in pixels. If this key is left empty current focus position is used.
<a name="fcsxyz"></a>
#### Keys **fcsX**, **fcsY**, **fcsZ**
Specifies FCS scanner positions. Coordinates with respect to upper left corner of the image (in pixels) and first slice (0,0,0). Several points are separated by `;`
Value | Meaning
----------- | ------
`X1;X2;X3;...` | X positions with respect to left corner in pixels
`Y1;Y2;Y3;...` | Y positions with respect to upper corner in pixels
`Z1;Z2;Z3;...` | Z positions with respect to bottom slice in pixels. If this key is left empty current focus position is used
<a name="rois"></a>
#### Keys **roiAim**, **roiType**, **roiX**, **roiY**
A ROI is defined by 4 different keys.
Value | Meaning
----------- | ------
`roiAim` | This can take the values `nothing`, `bleach`, `acquisition`, `acquisitionBleach`
`roiType` | This can take the values `circle`, `rectangle`, `polyline`, `ellipse`
`roiX` | The X coordinates of the ROI (in pixels) separated by a comma.
`roiY`| The Y coordinates of the ROI (in pixels) separated by a comma.
For a circle specify 2 points: the center of the circle and one point on the circle.
For a rectangle specify 2 points: the upper left and lower right corners.
For a polyline specify at least 3 points.
For an ellipse specify 3 points: the center of the ellipse and a point on axis1 and axis2, respectively.
## <a name= trigger></a> Trigger1 and Trigger2 pipelines
The pipelines **Trigger1** or **Trigger2** are executed when the command ```trigger1``` or ```trigger2```, respectively, is received through the windows registry [**CodeMic**](#codemic). A typical workflow is a multi-position low-resolution imaging using the **Default** pipeline. When objects of interest are detected using online image analysis then the **Trigger1/2** pipeline is started with position(s) specified using the registry keys [**X, Y, Z**](#xyz). If no stage coordinates are specified imaging is performed at the current stage and focus position of the **Default** pipeline.
Multiple imaging/stage positions can be imaged sequentially without returning to the **Default** pipeline.
1. Several positions are specified at once in the registry [**X, Y, Z**](#xyz) from the analysis of a single image from the **Default** pipeline.
2. The start of the **Trigger1/2** pipeline is delayed. The pipeline starts when a certain number of stage positions are specified within a time interval. In this interval the **Default** pipeline continues.
When all repetitions of all positions are finished the imaging continues with the **Default** pipeline. The setup of the pipelines is similar as for the default pipeline. Within a **Trigger1/2** pipeline additional online image analysis steps can be specified.
Trigger1/2 pipelines | Explanations of the buttons
:------------|:---------------------
<img src="./images/PipCon_trigger1.PNG" width = "400px"> | 1. Set thd order of tasks and Z-offset <br/> 2. Specify whether further process task <br/> 3. Set repetitions <br/> 4. Starts pipeline after **max. waiting (sec)** interval has elapsed. Interval is <br/> 5. Start pipeline when at least **max # Pts** stage positions have been specified. <br/> 6. If **Keep parent position** is unchecked then the stage position <br/>of the **Default** pipeline from which the trigger has been released is deleted. <br/> This option is useful to avoid triggering repeated false positives from the same position.
## <a name= adaptiveFeedback></a>Using adaptive feedback
With the adaptive feedback microscopy one can perform complex experiments. We provide here an example for high-throughout F(C)CS and imaging using the ImageJ/FiJi macro [Automated FCS](https://git.embl.de/politi/adaptive_feedback_mic_fiji)
### Adaptive feedback automated FC(C)S of single cells
The workflow can be used to obtain F(C)CS data in high-throuhput. The FCS measurements and images are linked so that the data can be used for FCS-Calibrated imaging.
For an illustration of the parameters of the image analysis program refer to [https://git.embl.de/politi/adaptive_feedback_mic_fiji.](https://git.embl.de/politi/adaptive_feedback_mic_fiji)
For FCS analysis software and FCS calibrated imaging please refer to [https://git.embl.de/politi/FCSanalyze.](https://git.embl.de/politi/FCSAnalyze/)
Cells express a fluorescent protein (mEGFP) and their DNA has been stained with SiR-DNA (far red dye). Following imaging and FCS jobs are used in MyPiC
* AF: XZ scan with reflection mode on.
* LR: Low resolution image to detect cells expressing the protein at the correct level.
* HR: High-resolution image is used to automatically place FCS points. The image can also be used be used for FCS-calibration analysis.
* fcsPOI: FCS settings to measure the fluorescent protein.
A two step workflow is used to find cell of interest and start FCS measurements
* Default pipeline:
* task1: A XZ scan to find the reflection of the glass
* task2: A low resolution imaging of the fluorescent protein (Channel 1) and DNA (Channel 2). At the end of the image MyPiC waits for a command from **Automated FCS** (process **Online image Analysis**).
* Trigger 1:
* task1: A XZ scan to find the reflection of the glass
* task2: A high resolution imaging of the fluorescent protein (Channel 1) and DNA (Channel 3). At the end of the image MyPiC waits for a command from **Automated FCS** (process **Online image Analysis**).
* task3: FCS measurements
**JobSetter Imaging** | **JobSetter FCS**| **Default pipeline task 1** | **Default pipeline task 2**
:--- | :--- | :--- | :---
<img src = './images/fcscalibrated/jobsetter1.png' width = "200px" > | <img src = './images/fcscalibrated/jobsetter2.png' width = "200px" > | <img src = './images/fcscalibrated/default1.png' width = "200px" > | <img src = './images/fcscalibrated/default2.png' width = "200px" >
**Trigger1 pipeline task1** | **Trigger1 pipeline task2** | **Trigger1 pipeline task3**
<img src = './images/fcscalibrated/trigger1_1.png' width = "200px" >|<img src = './images/fcscalibrated/trigger1_2.png' width = "200px" > | <img src = './images/fcscalibrated/trigger1_3.png' width = "200px" >
The FiJi plugin [**Automated FCS**](https://git.embl.de/politi/adaptive_feedback_mic_fiji) is used analyze the image Default pipeline task2 and Trigger1 pipeline task 2. Following parameters are used
<div align = "center" > <img src = './images/fcscalibrated/automatedfcs2.png' width = "400px" ></div>
The **Automated FCS** plugins performs:
* Job1: Cell detection using DNA channel. Cells where the fluorescent protein is within a certain range are selected. The coordinates of cells that fulfill all criteria are passed to MyPiC to start the Trigger1 pipeline.
* Job2: detects cells using the DNA stain. Chooses the cell closest to the center where the fluorescent protein is within a certain range. Determines the coordinates of the FCS measurements and pass their values to MyPiC. In task 3 of Trigger1 MyPiC will start FCS measurements at these positions.
**JOB1: Output generated for every processed image** | **JOB1: Output generated when objects of interest are found**
:--- | :---
<img src = './images/fcscalibrated/example_output.png' width = "400px" > | <img src = './images/fcscalibrated/example_output2.png' width = "400px" >
Segmentation of the DNA channel (red) returned 17 ROIs. <br/>Only for ROI 10 and 13 is the intensity of the fluorescent protein (green) within the specified range. | Outlines for the two cells picked are shown in green.<br/> The coordinates are passed to MyPiC to execute the Trigger1 pipeline for each position.
**JOB2: Output generated for every processed image**| **JOB2: Output generated when objects of interest are found**
<img src = './images/fcscalibrated/example_output3.png' width = "400px" > | <img src = './images/fcscalibrated/example_output4.png' width = "400px" >
Segmenation of the DNA channel (red) returned 2 ROIs. <br/> For ROI 1 and 2 the intensity of the fluorescent protein (green) is within the specified range. | Cell closest to center of image is picked (green outline). <br/>The red outline shows a cell that has not been picked because too far from the center of the image. <br/> Coordinates for the two FCS points are passed to MyPiC.
\ No newline at end of file