# (Belatedly) Announcing pymmcore-plus: an ecosystem of pure-python tools for running experiments with micro-manager core

**URL:** <https://forum.microlist.org/t/belatedly-announcing-pymmcore-plus-an-ecosystem-of-pure-python-tools-for-running-experiments-with-micro-manager-core/2268>\
**Category:** Announcements\
**Tags:** micro-manager, python, napari, software\
**Created:** [September 8, 2023, 2:38pm UTC](https://forum.microlist.org/t/belatedly-announcing-pymmcore-plus-an-ecosystem-of-pure-python-tools-for-running-experiments-with-micro-manager-core/2268 "2023-09-08T14:38:08Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![talley](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.microlist.org/talley/32/3_2.png) [@talley](https://forum.microlist.org/u/talley)\
**Post date:** [September 8, 2023, 2:38pm UTC](https://forum.microlist.org/t/belatedly-announcing-pymmcore-plus-an-ecosystem-of-pure-python-tools-for-running-experiments-with-micro-manager-core/2268/1 "2023-09-08T14:38:08Z")

</div>

**NOTE: [cross-posting](https://forum.image.sc/t/belatedly-announcing-pymmcore-plus-an-ecosystem-of-pure-python-tools-for-running-experiments-with-micro-manager-core/86093) this here from the image.sc forum as it may be of interest to this community as well**

Hi All 👋

We’ve been working on the pymmcore-plus/napari-micromanager ecosystem for a couple years now, but never really made any sort of public announcements. Thanks to the efforts of many people (notably @Federico_Gasparoli and Ian Hunt-Isaak, with great input from many others) I think the toolset is now pretty feature rich and relatively stable with a good amount of testing behind it.

Since many may still be unaware of the libraries, I’d like to give a brief overview and welcome you to give it a try if you want to drive microscopes in a pure python/C++ environment while integrating micro-manager’s device adapters (without requiring Java or any cross-process communication).

## pymmcore(-plus)

When we started on napari-micromanager, we wanted to directly communicate with the C++ core of micromanager, so we began with [pymmcore](https://github.com/micro-manager/pymmcore): the official python bindings for MMCore. If you’ve used it in the past, you may have felt that it lacked some developer niceties (such as type hinting and IDE autocompletion). We upstreamed type stubs that now give you a nice developer UX (this is the case regardless of whether you use `pymmcore` directly or `pymmcore-plus`):

 ![Screen Shot 2023-09-08 at 8.31.26 AM](https://us1.discourse-cdn.com/flex020/uploads/microlist/original/2X/7/75ec08f4b3513bc59b78e651bb2a5b553a7e2e1a.png)

 ![Screen Shot 2023-09-08 at 8.30.09 AM](https://us1.discourse-cdn.com/flex020/uploads/microlist/original/2X/2/2ead5f6bcdaf7f3ebae208e659f1e8af8a73c7a0.png)

However, there were still many things that were limiting us in an event-loop-based environment like napari/Qt, and we also wanted to be able to execute multi-dimensional experiments similar to the micro-manager application (the logic of which was all implemented in originally in clojure and later in Java, rather than C++), so we started building [`pymmcore-plus`](https://pymmcore-plus.github.io/pymmcore-plus/): a library of utilities on top of `pymmcore` that adds all of the stuff we found we needed. The main object there is `pymmcore_plus.CMMCorePlus`, a subclass of `pymmcore.CMMCore`. That library has reached a nice level of maturity, and now includes:

- A full-featured and customizeable multi-dimensional [acquisition engine](https://pymmcore-plus.github.io/pymmcore-plus/guides/mda_engine/) implemented in pure python
- More flexible [event](https://pymmcore-plus.github.io/pymmcore-plus/api/events/) handling and core-monitoring capabilities (particularly useful if you want to build GUIs on top of it)
- extended [core documentation](https://pymmcore-plus.github.io/pymmcore-plus/api/cmmcoreplus/), so you don’t have to browse the Java docs or C++ doxygen docs when working in a python world.
- generally more pythonic feel and object oriented APIs (e.g. implementing a proper `MutableMapping` interface for [pymmcore’s Configuration objects](https://pymmcore-plus.github.io/pymmcore-plus/api/configuration/#pymmcore_plus.Configuration), etc…).

> note: we have chosen _not_ to rename all of the `camelCase` names to their `snake_case` equivalents. Similar to other python C-extensions, like Qt, we find that it often gets even more confusing to remember which “regime” you’re working in, so we just stick with the MMCore API.

### Acquisition engine and useq-schema

In particular, I’d like to draw your attention to the [multidimensional acquisition engine](https://pymmcore-plus.github.io/pymmcore-plus/guides/mda_engine/) included in pymmcore-plus, which offers both high level and low level control and execution of a sequence of events.

In order to better define exactly _what_ an experiment is (before worrying about exactly how to execute it), we built [useq-schema](https://pymmcore-plus.github.io/useq-schema/): a set of types to declaratively define a sequence of events to be executed on a microscope. In YAML, an example experiment might look like this (but [there are _many_ ways](https://pymmcore-plus.github.io/useq-schema/schema/sequence/) to define it):

```yaml
channels:
   - config: DAPI
     exposure: 1.0
grid_plan:
  columns: 2
  rows: 2
stage_positions:
   - x: 1.0
     y: 1.0
     z: 1.0
time_plan:
  interval: '0:00:00.100000'
  loops: 2
z_plan:
  range: 3.0
  step: 1.0

```

pymmcore-plus implements support for that specification, which means that you can define, load, and run a full experiment from python code or a YAML/JSON file:

```python
from pymmcore_plus import CMMCorePlus
import useq

mmc = CMMCorePlus.instance()
mmc.loadSystemConfiguration()

# create a sequence, here directly in python
mda_sequence = useq.MDASequence(
    time_plan={"interval": 2, "loops": 10},
    z_plan={"range": 4, "step": 0.5},
    channels=[
        {"config": "DAPI", "exposure": 50},
        {"config": "FITC", "exposure": 20},
    ]
)

# Run it!
mmc.run_mda(mda_sequence)

```

At runtime, that `MDASequence` object behaves like an iterable of [`event`](https://pymmcore-plus.github.io/useq-schema/schema/event/) objects that the pymmcore-plus acquisition engine consumes and executes.

> side note, _any_ acquisition scheme, (such as [ImSwitch](https://github.com/ImSwitch/ImSwitch), [python-microscope](https://github.com/python-microscope/microscope), or the java half of micro-manager etc…) could conceivable adopt the `useq-schema` specification) which would make it easier to adopt/swap different engines.

### engine customization

If the high-level interface isn’t flexible enough for your needs, you can fully [customize the acquisition engine](https://pymmcore-plus.github.io/pymmcore-plus/guides/custom_engine/) to your needs to, for example:

- write and integrate python drivers for hardware for which a [micro-manager device adapter](https://micro-manager.org/Device_Support) does not exist (without losing the ability to drive devices that micro-manager _does_ control)
- generally execute any arbitrary python code in the same process as the C++ engine itself, with direct access to images and data as it comes from the engine.
- you could even pick and choose what parts of the hardware you want micromanager to drive, for example to swap in a [high-performance camera driver](https://github.com/acquire-project/acquire-python)

> technically, the MDA Runner doesn’t even need to be controlling micro-manager’s core. It passes an iterable of useq-schema `MDAEvents` to an `MDAEngine` that knows how to execute each event. Our reference MDAEngine drives the micro-manager MMCore object, but you can customize as well if desired.

### event customization

There are a number of different ways to [generate/modify events in realtime](https://pymmcore-plus.github.io/pymmcore-plus/guides/event_driven_acquisition/), in response to acquisition data, for event-driven acquisition (aka “smart microscopy”). You can use python generators, `queue.Queue` objects, or any `Iterable[MDAEvent]` pattern that you like. Many thanks to Willi Stepp and Kyle Douglass for fruitful discussions on that topic.

## pymmcore-widgets

Doing this all in a pure-python environment of course comes at the cost of losing [MMStudio](https://github.com/micro-manager/micro-manager) (micro-manager’s Java/ImageJ GUI). So we’ve been building a set of Qt widgets to control `pymmcore-plus`, and we’ve made a lot of progress towards parity with the controls available in the Java GUI, including:

- device property browser
- a full hardware configuration wizard
- configuration group editors/selectors
- multi-dimensional acquisition window
- most of the other little buttons like snap/live/stage/roi-control, etc…

 ![8da3910469d893976dec76605cfb63b905dc1324](https://us1.discourse-cdn.com/flex020/uploads/microlist/original/2X/8/8da3910469d893976dec76605cfb63b905dc1324.jpeg)

These widgets generally stay in sync with the state of the underlying C++ core object thanks to the event/signals system in pymmcore-plus. You can pick and arrange these widgets however you choose to make your own custom pure-python micro-manager GUI.

> Notably, the MDA widgets output `useq-schema` objects: so even if you don’t care about micro-manager, you could conceivably take advantage of the MDA acquisition GUI if your downstream processing implements support for useq-schema.

## napari-micromanager

The highest level project (the thing that started this all) is [napari-micromanager](https://github.com/pymmcore-plus/napari-micromanager). It is essentially little more than an “opinionated” arrangement of the widgets in pymmcore-widgets, with the events from the pymmcore-plus acquisition engine connected to the napari viewer:

 ![33db9378707472dc386ddd31d6ef01e17c4aed15](https://us1.discourse-cdn.com/flex020/uploads/microlist/original/2X/a/ac46c141dcd8730b35acbae01c11b6662c0cb401.jpeg)

Admittedly, this is the project that has received the least amount of direct attention, as we’ve been focusing on building the lower level parts in a composable way that should be useful beyond a napari context. But it works in many ways! 🙂 and we welcome more people to give it a try if they already use micro-manager on their scopes (you can use the same config file).

## features not yet implemented

One of the main things you’ll find lacking at the moment is any sort of file IO/writer specification in pymmcore-plus itself. While napari-micromanager does have basic support for writing the experiment out to zarr and/or tiff, this is an area that could use more attention in the future. You can, always connect directly to the [`frameReady`](https://pymmcore-plus.github.io/pymmcore-plus/api/events/#pymmcore_plus.mda.events._protocol.PMDASignaler.frameReady) events coming of the engine to write the data to disk or otherwise process the data as best suits your needs. This is something we do plan to add, but given the huge variety of experimental needs and existing solutions, we really don’t want to recreate the wheel or impose a specific set of constraints.

## feedback 🙏

If you are looking to drive microscopes in a pure-python environment, and want to be able to integrate micro-managers rich set of device adapters, we hope you’ll give it a try. As always feedback is greatly appreciated. If you find bugs, have questions, or would like to request features, feel free to [open an issue](https://github.com/pymmcore-plus/pymmcore-plus/issues)