Prof. Dr Onofre Martorell - Prof. Dr Michael Möller

Image Reconstruction - More than just Putting Pictures Together

In a dark room with black walls and ceilings, an array of small green LEDs illuminates a tiny biological specimen. The pattern of the array changes. And changes again. Each time it changes, a picture is taken. Prof. Dr Onofre Martorell from the University of the Balearic Islands Mallorca explains: “By utilising artificial intelligence – or, more precisely – machine learning algorithms, we are able to automatically combine the data from the various images taken with different illumination patterns and produce one super-resolution image revealing the specimen’s fine structures that would otherwise be invisible”. A Mathematician by training, Martorell specialises in algorithmic merging of images. In 2022, during his PhD with Prof Dr Antoni Buades at the University of the Balearic Islands Mallorca, he was looking for an internship abroad. A colleague in Mallorca introduced him to Prof. Dr Michael Möller, Professor of Computer Vision in the Department of Computer Science and Electrical Engineering at the University of Siegen. Martorell and Möller got along extremely well right from the beginning. Not only did they both earn their doctoral degrees in applied mathematics, making them theoretically oriented, they also share a passion for image processing and reconstruction, where they algorithmically combine and improve images taken with consumer-grade cameras or reconstruct images that cannot be acquired directly. Their collaboration has stayed active following Prof Martorell’s original stay and when Möller received funding for the research unit Learning to Sense (L2S), selected in a special call as one of only eight AI-related research units by the German research foundation, the opportunity opened to amend their research focus by an experimental component. Enter Prof Ivo Ihrke’s group working on Computational Sensorics & Communications Engineering at University of Siegen’s Center for Sensor Systems (ZESS).

The LEDs change their pattern again, and now, on a computer screen attached to the experimental setup, appears an abstract picture in tremendous resolution showing intricate details. Onofre’s collaboration partner from the research unit, Dr John Meshreki, takes a closer look at what he explains to be a mushroom section. Meshreki, a physicist by training, currently works as part of the research unit Learning to Sense at Prof Ihrke’s group and is the main researcher behind the Fourier Ptychography Microscopy setup (or FPM in short) being used here, providing Möller and Martorell with real-world images to work on. The FPM is an advanced microscopy technique which is special as it does not alter the specimen’s properties – unlike traditional staining methods, such as fluorescence microscopy. Instead, it images the specimen using the phase of light, allowing for the study of biological samples in their natural state. This has tremendous potential for the healthcare industry, for example, in digital pathology. With the help of this computational imaging technique, medical doctors can potentially detect even small-sized tumours or, in the long run, study cancer cells – without the need to alter the specimen or expensive microscope lenses.

When Prof Martorell isn’t working with his colleagues from Siegen, he currently works as an Assistant Professor, teaching Bachelors’ courses and conducting research in his main area of expertise, which lies in merging images taken at different exposure times, a technique called bracketing. The resulting image possesses distinct details and colours over the complete range at the same time, from the darkest to the brightest parts of the scene. Thus, it is called high dynamic range (HDR). The presence of moving objects in the scene and small dots called noise that usually appear in dark images make this task very challenging. Continuous improvements are highly desirable as both mobile phone cameras and image processing software like Adobe Photoshop make extensive use of this technique.

Despite high-quality images being ubiquitous, the process from the initial acquisition to a polished-looking picture we take for granted to see on our screens is long and has a lot of math behind it. Möller and Martorell are both working on further improving this process and publishing their results together. The FPM project is currently working towards a publication in a high-impact journal and a joint technical work, Kissing to Find a Match: Efficient Low-Rank Permutation Representation, co-authored by Martorell and Möller among others, was accepted at the Neural Information Processing Systems (NeurIPS) conference 2023, the most prestigious international machine learning conference.

Dr Martorell's research stay at the University of Siegen in the first half of 2025 is supported by our research alumni grant ‘Return for Research’ with funds from the Alumni Association Electrical Engineering and Computer Science of the University of Siegen.

This article is written by Onofre Martorell and Jan Söhlke.