Smart Microscopy connects bioimage analysis with computer-controlled microscopy to generate automated and adaptive imaging workflows to enable quantitative, statistically meaningful results in complex biological systems. This approach brings new challenges, such as accessing and controlling instrument hardware via suitable software, designing good workflows or handling large data sets during operation.
On 13th October 2022, Euro-BioImaging and its Industry Board organized a one-day workshop with facility staff from the Euro-BioImaging Nodes and industry representatives to discuss recent developments in Smart Microscopy, the user and facility needs when building adaptive workflows with a wide range of instruments.
The possibilities to combine different imaging technologies, to trigger imaging modalities based on the online image analysis results and to synchronize image acquisition with external devices for sample manipulation including liquid handling or cell manipulation in theory enable the automation of complex workflows. Such workflows enable researchers of image acquisition in a non-biased, more reproducible and faster way than manual imaging, often enabling experiments which are otherwise hardly feasible. In practice the diversity of formats and functions, implemented in the control and analysis software often limits flexibility of the designed workflows. Currently the huge potential of Smart Microscopy can only be utilized by a expert imaging scientists and is difficult to implement in a service environment or within individual labs, often requiring bespoke workflows for each user project.
The workshop participants concluded that the first steps towards standardization and making smart microscopy more accessible would be to
Build a consensus on what the field should be called and what is included
Agree on a standard ontology for Smart Microscopy and workflow components to be used and adopted across different instrument control and analysis software and define a list of core workflow components that should be accessible in each software
Identify the top community needs of microscope users and facilities for smart microscopy (such as example workflows, common data and metadata formats and technical requirements for software interfaces)
Explore the possibilities for joint training activities in the area of Smart microscopy.
We would like to approach these topics through a number of posts in this forum with the aim to collect community input for further discussion with instrument and software producers.
With this post we would like to trigger developing of standardized nomenclature in the area of the “Smart microscopy”. We would like to begin with definition of Smart Microscopy and related terms, which are often used as synonyms.
We are putting forward the following definitions for discussion:
Automated microscopy - the set of techniques to acquire many images on the microscope automatically, ac opposed to each image capture triggered by user. Automated microscope includes pre-programmed workflows e.g.(high-throughput imaging) and as well as feedback-microscopy based applications or workflow syncronising microscopes with external devices
Feedback microscopy (or Adaptive feedback microscopy) - technology to run automated image acquisitions controlled by online image analysis. Adaptive feedback microscopy might be used for different applications, such as i) high resolution imaging of target phenotypes, rare and transient events; tracking moving objects or photomanipulation of the automatically defined ROIs.
Smart microscopy - set of technologies to set up and run complex microscopy workflows including, but not limited to feedback microscopy.
I particularly appreciated back at the workshop, the introduction by Aliaksandr of the 4 “classical” feedback microscopy applications (screenshot of the youtube recording below).
I think these applications could be good benchmark/reference scenario to test/illustrate further developments.
I mention during the discussion, that various packages/libraries/API were readily available for microscopy control via external applications. Such as micro/pycro-manager, python-microscope…
They already allow substantial functionalities, but there is no “consortium” or common vocabulary to depict similar operations or components.
I was thinking, some kind of top-most “abstract”, standardized and application-independent reference model defining a minimum set of operations and components for automated microscopy could help standardizing these functionalities and would allow some degree of interoperability between implementations of this model.
If the concept of abstraction sounds unfamiliar, I would recommend the paragraph Implementation > Abstract Base classes of the python microscope paper, where this is introduced.
A good starting point for this model, would be those readily available abstract classes of the previously mentioned packages.
Talking about “common vocabulary”, this is in other words defining an ontology of terms describing automated microscopy operations and components. There is maybe the opportunity to define a new ontology, inspired by the current work on metadata, and ontologies available in other communities (ex: gene and species ontology).
Interesting topic for discussion. It may be worth posting a link to this thread in the confocal microscopy list server (CONFOCALMICROSCOPY@lists.umn dot edu for those not familiar). Guessing there will be a decent amount of interest there.
