Updates for TREC pipelines

Changes supporting use of pipelines in trec-adaptive-feedback project:

• generally, accessors' and models' IDs can be set with respective argument
• test client methods now handle response success check and variable parsing
• implemented functional threshold segmentation and (in base.roiset) classification models; threshold is relative to dataset full range (0.0 to 1.0)
• consequently, more tests cover base package without dependency on ilastik
• RoiSet returned by pipelines get IDs and are registered in server session, accesible at respective endpoints
• Pydantic models only used by FastAPI now; underlying model, accessor, and pipeline functions should interact with python objects and parameter dictionaries only
• RoiSet filters also include minimal and diagonal height, width dimension; are applied before time-intensity pixel array operations to increase performance
• performance improvement when passing patch stacks to ilastik pixel classifier
• RoiSet.classify_by can optionally filter by additional classification columns, e.g. to implement intensity gating

model_server/base/roiset.py

@@ -1125,32 +1125,23 @@ class RoiSetWithDerivedChannels(RoiSet):
         :return: None
         """
 
-        raw_acc = self.get_patches_acc(channels=channels, expanded=False, pad_to=None)  # all channels
+        acc_in = self.get_patches_acc(channels=channels, expanded=False, pad_to=None).get_channels(channels)
         if derived_channel_functions is not None:
-            mono_data = [raw_acc.get_mono(c).data for c in range(0, raw_acc.chroma)]
             for fcn in derived_channel_functions:
-                der = fcn(raw_acc) # returns patch stack
-                if der.shape != mono_data[0].shape or der.dtype not in ['uint8', 'uint16']:
-                    raise DerivedChannelError(
-                        f'Error processing derived channel {der} with shape {der.shape_dict} and dtype {der.dtype}'
-                    )
+                der = fcn(acc_in) # returns patch stack
                 self.accs_derived.append(der)
 
             # combine channels
-            data_derived = [acc.data for acc in self.accs_derived]
-            input_acc = PatchStack(
-                np.concatenate(
-                    [*mono_data, *data_derived],
-                    axis=raw_acc._ga('C')
-                )
-            )
+            acc_app = acc_in
+            for acc_der in self.accs_derived:
+                acc_app = acc_app.append_channels(acc_der)
 
         else:
-            input_acc = raw_acc
+            acc_app = acc_in
 
         # do this on a patch basis, i.e. only one object per frame
         obmap_patches = object_classification_model.label_patch_stack(
-            input_acc,
+            acc_app,
             self.get_patch_masks_acc(expanded=False, pad_to=None)
         )