AI in Radiology and Radiomics 101: Resources and Roadmap

I have a lot of colleagues who want to learn AI to improve their radiology practice. They frequently ask me how to learn radiomics, machine learning, and deep learning for radiology. I decided to compile resources to help other radiologists and draw a roadmap.

There are four steps. First, learning programming. Second, learn essentials of machine learning and deep learning. Third, dive into details of your area of focus, such as LLMs, segmentation, classification, hardware, or deployment. Finally, develop your own project.

Step 1: Learn Programming

As a radiologist, most of the time you don’t need to code everything from scratch, but I still recommend learning programming, because even if you don’t code from scratch, you should be able to use frameworks.

Python is the first choice in my opinion, because most AI frameworks have a Python implementation, so you don’t need to code low-level functions on hardware. Additionally, it is easy to use and works on almost every computer, small or big, without a hassle.

There are a lot of books, courses, and tutorials to learn Python, but I recommend DataCamp’s Python course for people with non-technical background, because the lectures are well-planned and you can apply your knowledge immediately with an interactive coding interface. In short, you can learn by doing.

Step 2: Learn Machine Learning and Deep Learning

Before using machine learning and deep learning in Radiology, you should learn the general principles. Again, there are a lot of resources to learn them, but I recommend the resources below. I completed both the books and the course, but you can choose one and stick with it as your main reference.

Books

I recommend these two books, because they start with the basic principles, and the mathematical background, then go into the details. You will also learn about popular frameworks like scikit-learn, PyTorch and Keras. Both books are written by top-tier practitioners and writers, which is why I recommend them so strongly.

Courses

If you prefer courses over books, you can go with the Deep Learning Specialization offered by deeplearning.ai.

Coursera Deep Learning Specialization

Step 3: Learn AI in Radiology

So far, we’ve learned the general principles of AI. Now we will learn how we to apply this knowledge to Radiology.

Learn Radiomics

Learn Medical Imaging Frameworks & Tools

MONAI

NVIDIA and King’s College London started Project MONAI. It is a Python framework for healthcare imaging. It has a lot of specific implementations and demos for Radiology and Pathology.

NVIDIA Holoscan

The NVIDIA Holoscan platform is a domain-specific AI computing platform designed specifically for the development of medical devices. It has both hardware and software solutions mainly focused on real-time analysis and sensor processing.

NVIDIA TAO Toolkit

NVIDIA TAO Toolkit is a no-code model training platform. It is not specialized in healthcare, but you can create highly accurate, customized models faster and more easilyu using the TAO Toolkit. You use configurations instead of code. It is not very easy to use at first, but once you get familiar with it, you will want to use it for fast prototyping.

Step 4: Next Steps

You’ve come a long way. It is time to find a problem and try to solve it. I guarantee this is the best way to learn. Reading books and watching tutorials can get you to a certain level, but to really learn details and make the knowledge stick, the best way is to build your own project. How? Keep reading.

Datasets & Models

Collecting and annotating data is exhausting, because working with healthcare data is highly regulated. Should we quit? No, use open-source datasets. Pick a dataset and play with it like a toy. Train your own model using it. Ask yourself: what can I do with this dataset? Once I saw a study predicting patients’ age by looking at their retinas using the diabetic retinopathy dataset. You can do the same.

Additionally, download and test other people’s models. Do they work well? You can train a smaller or bigger version of them. You can fine-tune them to achieve a different task. How did they implement it? What did they use?

Radiology Datasets for AI Model Training – Curated lists of radiology datasets prepared by me. I am planning to update it regularly.
Ultrasound Open Access Datasets – NIDUSAI – A collection of open access ultrasound imaging data sets.
Kaggle
HuggingFace – You can find both datasets and models.

Papers

Following new papers is important, because I believe at some point in your journey you will want to publish your own AI paper. Also, reading a peer-reviewed article is a quick way to learn something, because you learn from others’ experiences. I strongly recommend reading papers that come with the dataset and source code.

Radiology AI Papers with Code – A curated list of radiology / medical-imaging AI papers that ship public code, grouped by imaging modality.
RSNA Radiology AI Journal

Certifications

I don’t rely on certifications, but some of our doctor colleagues give them more importance than they deserve. So here are some of the best options worth taking a look at.

My Final Tips

Don’t rush. Take your time. It is a long journey, and you shouldn’t try to learn everything in a single day.
Reading books and watching tutorials are good, looking at others code is better, but building your own project is the best.
Share your work with others and get feedback. You can use GitHub.
You can ask me anything whenever you have questions, on social media or via email.

AI in Radiology and Radiomics 101: Resources and Roadmap

Step 1: Learn Programming