Folder structure in Omero

Software
1

Dear all
We have recently set up an Omero server. Everything seems to work nicely except for one thing: It seems that we can only have 2 layers of folders inside the Omero dabase, Projects and Datasets.
How can we have more folders inside folders? or is it possible to automatically tag the images with their parent folder? Since this is the structure we currently have, and files inside the folders have the same name exactly, there is no way we can use Omero if we cannot have more than 2 folder layers and if we cannot point to a higher level folder to upload. Any idea?
Cheers
Sylvie

2

you will likely have better luck with OMERO questions if you post at the image.sc forum, and tag it with “omero”. There are multiple people from the OMERO team who keep a close eye on that forum.

1 Like
3

Hi Silvie,
No direct help, but as a side note, it seems a bit dangerous to have multiple files with exactly the same name that require the parent folder for identification. Keeping track of data in a speaker that way if impossible or requires a lot of manual annotation.

My solution, which I suggest to all my users, is to add the date (format YearMonthDay) at the start of all image names. For existing files, on the small scale both MacOS and Windows have the ability to batch rename files in the folder view (Finder for Mac, File Explorer for Windows). Renaming large numbers of folders will likely need a more sophisticated solution, but I can’t imagine there isn’t something out there for that already.

Reorganization is tough and can be frustrating, but it will be better at the end! Good luck!!

4

I have but no one has answered so I thought I would try my luck.

5

Hi Wendy

Well this is the way our data comes out of the light sheet microscope. But in any case even for other microscopes, it is very common that users have folders inside folders. How do you deal with this in Omero? is it really the case that there are only 2 layers of folder or am I missing something?

6

I have but no one has answered so I thought I would try my luck.

ah I see, sorry…

What I do know is that the Project > Datasets > Images structure is pretty much at the core of the OMERO model, so it does seem like you’ll need to devise a workaround. Can you give an example of the folder structure you’re trying to add? If your current structure has a ton of individual tiff files representing different TCZ of a single experiment (e.g. timelapse of a single stage position), one possibility is to convert that to a single OME dataset prior to upload (yes, that requires some additional coding/preprocessing, but given the omero model, you may not have a choice). Then your OMERO dataset could have a number of images, where the “images” are actually 5D experiments that correspond to full folders on your computer prior to omero upload. Just a thought…

7

example of folder structure:
microscope_3/filament_project/treated_samples/drug_5/250_ug
so we have 4 layers of folders and in the last one (250_ug) we have images.
We have not used Omero before but now I have it on the server. I am trying to convince our users to use it but the first one bumped into this problem. How can they upload their images and know what is what if the folder structure is not preserved?

and the image in the 250_ug folder is already a 5D image

8

How can they upload their images and know what is what if the folder structure is not preserved?

The short answer is that, if you want to use omero, you’ll have to get used to the different organizational structure. OMERO is a database, and it was never meant to behave like your local hard drive with a folder-tree, so you’re going to have to do a little renaming to get stuff in there, and detach yourself a little bit from the current practice of using folders to organize everything. You can take advantage of the rich metadata and annotations in OMERO, so you don’t necessarily need “microscope 3” in the filename or project structure… since you will be able to filter using metadata too. Drugs and treatment conditions can also be annotations that can be filtered/organized independently of dataset structure.

one (of many) possible omero structure in this case would be:

Filemant Project
├── drug 5
│   ├── 20180801_000_ug
│   ├── 20180801_150_ug
│   ├── 20190901_pos1_000_ug
│   ├── 20190901_pos2_000_ug
│   ├── 20190901_pos1_150_ug
│   ├── 20190901_pos2_150_ug
│   ├── 20190901_pos1_250_ug
│   └── 20190901_pos1_250_ug
├── drug 6
│   ├── 20180801_000_ug
│   ├── 20180801_150_ug
│   ├── 20190901_pos1_250_ug
│   └── 20190901_pos1_250_ug
└── etc...
    └── images...

or

Some other Project
├── 20180801
│   ├── drugA_000_ug
│   ├── drugA_100_ug
│   ├── drugA_2500_ug
│   ├── drugB_000_ug
│   ├── drugB_100_ug
│   └── drugB_2500_ug
├── 20181003

…

1 Like
9

I see… thanks for the answer. I will test a bit myself and see if I can convince our users.

10

Yes there are software tools which can even do renaming automatically for you. Here’s an article about how to. https://sirv.com/help/resources/image-naming-and-batch-renaming/ A long time ago in a different life I had to do this with high content screening data (we’re talking well over a decade ago when the hardware was er ‘at an early stage’ shall we say). I used the predecessor of the Batch rename tool, it was well worth it.

1 Like
11

Adding my 5 cents to @A.P_Wheeler: I’ve used Bulk Rename Utility for years. It supports RegEx and has a lot of other useful features like search and replace, numbering and it works recursively on folders etc. They also have a portable version (no need to install) that is particularly useful for renaming on the fly.

3 Likes
12

Agreed, big fan of Bulk Rename Utility!

13

Cross-linking for future reference: https://forum.image.sc/t/folder-structure-in-omero/30535/4