DeepNeo: Deep Learning for neointimal tissue characterization using optical coherence tomography

Valentin Koch^1,2,3, Olle Holmberg^1,2,4, Edna Blum⁵, Ece Sancar^1,2, Alp Aytekin⁵, Masaru Seguchi⁵, Erion Xhepa⁵, Jens Wiebe⁵, Salvatore Cassese⁵, Sebastian Kufner⁵, Thorsten Kessler^5,6, Hendrik Sager^5,6, Felix Voll⁵, Tobias Rheude⁵, Tobias Lenz⁵, Adnan Kastrati^5,6, Heribert Schunkert^5,6, Julia A. Schnabel^1,2,7, Michael Joner^5,6, Carsten Marr¹, Philipp Nicol⁵

1. Helmholtz Munich - German Research Center for Environmental Health, Munich, Germany 2. School of Computation and Information Technology, Technical University of Munich, Munich, Germany
3. Munich School for Data Science, Munich, Germany
4. Helsing GmbH, Munich, Germany
5. German Heart Centre Munich, Technical University of Munich, Munich, Germany
6. German Center for Cardiovascular Research, Partner Site Munich Heart Alliance, Munich, Germany
7. School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK

Introduction

We present DeepNeo, a deep learning-based algorithm, to automate the process of segmenting and characterizing neointimal tissue of stented patients (tissue that grows over the stent) in optical coherence tomography (OCT) images. OCT is a tool that provides high-resolution imaging of stented segments after PCI (percutaneous coronary intervention, a non-surgical procedure that improves blood flow to the heart, e.g. by implanting a stent).

Demo

We provide a tool for researchers to quickly analyze and download results,

click on the image for a video demonstration:

The user-friendly interface is designed with several features to facilitate accurate and efficient analysis, including an upload mask (a), which allows users to upload OCT pullback images (DICOM or .zip), a visual representation of the current OCT frame with segmentation and neointima prediction (b), a schematic view of quadrants (c) (top row represents quadrant I, bottom row quadrant IV) and neointima and lumen (d) that provides a visual representation of the tissue characteristics, including a slider that enables users to move through the pullback. In addition, the interface includes a pullback analysis (e) that provides a detailed analysis of the OCT images and a manual correction feature (f) to correct beginning and end of the stent if necessary. The webtool also allows users to download a detailed analysis of their results and provides an information tab (g) for additional guidance. Users are required to accept the research-only use on the welcome page (h) before accessing the tool.

Usage

To run the DeepNeo tool, follow these steps:

Step 1: Clone the Repository

First, clone this repository to your local machine.

git clone https://github.com/ValentinKoch/DeepNeo.git

Step 2: Navigate to the App Folder

cd app

Step 3: Create a Conda Environment

conda create --name DeepNeo python=3.10

Step 4: Activate the Environment and Install Requirements

conda activate DeepNeo
pip install -r requirements.txt

Step 5: Update Configuration File

Before running the application, update the configuration file to match your system's paths. Locate the config.py file in the app folder and make the necessary adjustments to the folder paths.

Step 6: Run the Application

Within the app folder, start the DeepNeo web application by executing:

python run_gradio.py

Step 7: Access the Web Application

After running the script, a local server will start, and a URL will be generated. Open this URL in your web browser to access the DeepNeo web application.

Models & Contact

We are very happy to provide models, please request them at Zenodo and contact Valentin Koch (valentin.koch@helmholtz-munich.de), Carsten Marr (carsten.marr@helmholtz-munich.de), or Michael Joner (joner@dhm.mhn.de).