AI-driven development of customized tumor sensors

Hyperspectral imaging (HSI) is an advanced imaging technique that captures light across the magnetic spectrum in a two-dimensional space. Unlike conventional RGB-cameras, which cover only three bands (red, green and blue) of visible light, hyperspectral imagers capture hundreds of narrow, contiguous bands ranging from ultraviolet to infrared and thus beyond our visual range. The additional spectral signature makes HSI a highly sought-after technology for identification of various kinds of material.

While hyperspectral imaging (HSI) has well-established applications in fields such as remote sensing, agriculture and environmental monitoring, there has been a surge of research interest from the medical field as well. This interest is not only motivated by HSI’s rich spectral information, but also by its harmless nature, as opposed to for example state-of-the-art fluorescent microscopy, which might cause some side effects. The belief is that HSI holds great potential to advance surgical guidance for gastroenterology and neuro-oncology but also has a value for pathology, and many other subfields of medicine.

Despite great promise, the adoption of HSI in real-world clinical systems has been hampered by a trade-off between image acquisition speed on the one hand, and spatial and spectral resolution on the other hand. That is, capturing the full range of spectral bands at high resolution can take several seconds to minutes per image using the Imec's SNAPSCAN devices, while Imec's faster SNAPSHOT devices enable real-time acquisitions, but at lower spatial and spectral resolution. Medical applications such as surgical guidance for tumor resection in neuro-oncology, however, require high resolution images at near-real-time speeds.

The goal of this PhD is to use deep learning (DL) methods to get insights to simplify and customize the imec snapshot sensor, hereby improving image resolution and enabling real-time image acquisition. Decreasing the number of bands in a snapshot sensor towards only relevant hyperspectral bands increases the spatial resolution without any decrease in speed.

You would start with adopting existing DL methods to handle and learn from both the spatial and the spectral domain of the hyperspectral images and this for several relevant medical problems such as tumor segmentation or quantification of oxygen saturation. Additionally, you should create a general approach for interpretability of different hyperspectral bands for these medical tasks and link this with biological meaningful processes. Based on these insights, a general framework can be developed to identify the most relevant bands related to a certain medical task, considering domain knowledge of sensor design.

Together, the findings of this PhD research will be of direct relevance to ongoing imec projects in the medical domain, including but not limited to developing next-gen tumor detection sensors  with hyperspectral imaging.

We offer a challenging, stimulating and pleasant research environment, where PhD students can engage in international research on artificial intelligence with a close link to the underlying hardware. A PhD student working on this topic will be part of the AI & Algorithms department, but will also collaborate closely with imec hardware, sensor development and university teams (such as the HSI team) to produce novel solutions together.

Our ideal candidate for this position has the following qualifications:  

• You have a Master’s degree in Computer Science, Informatics, Physics, Engineering, Electronics or a related field. 
• You have experience with deep learning, ideally in the visual domain
• You have strong python skills and familiarity with deep learning libraries such as PyTorch
• Physics knowledge (sensor knowledge) is considered a plus
• You are able to plan and carry out your tasks in an independent way. 
• You have strong analytical skills and the ability to think critically about research results
• You are a responsible, communicative and flexible person. 
• You are a team player. 
• You are fluent in English (speaking and writing).

References

[1] BARBERIO, Manuel, et al. Intraoperative bowel perfusion quantification with hyperspectral imaging: a guidance tool for precision colorectal surgery. Surgical Endoscopy, 2022, 36.11: 8520-8532.

[2] LEON, Raquel, et al. Hyperspectral imaging benchmark based on machine learning for intraoperative brain tumour detection. NPJ Precision Oncology, 2023, 7.1: 119.

[3] MANNI, Francesca, et al. Hyperspectral imaging for glioblastoma surgery: improving tumor identification using a deep spectral-spatial approach. Sensors, 2020, 20.23: 6955.