NVIDIA has announced that National University Health  System (NUHS) has built an AI production platform based on an NVIDIA DGX A100 system to  become the first healthcare group in Singapore to have real-time streaming capabilities to  deliver better patient care and treatment, collaborate on biomedical research and transform  how illnesses are managed and treated. 

At the heart of NUHS’ newly-launched Endeavour AI platform, NVIDIA DGX A100 will run AI  tools that make real-time predictions on diagnosis, progression of diseases, readmissions, risk  of falls, and others. 

The new system will be integrated with NUHS’ Discovery AI training platform to form a  complete training and inference system as part of the group’s digital transformation. 

“There are many demands on healthcare these days and we are undertaking a digital  transformation throughout the cluster. In the centre of our digital transformation is the use of  AI. Advances in healthcare require great compute resources, and NVIDIA DGX A100 delivers  easy access to performance needed to aid in a world class hospital,” said Dr Ngiam Kee Yuan,  group chief technology officer of NUHS and deputy chief medical informatics officer of National  University Hospital (NUH). 

NUHS is one of three public healthcare clusters in Singapore and an integrated academic health  system and regional health system that delivers value-driven, innovative and sustainable  healthcare. Its network covers 19 hospitals, polyclinics, specialist centres, medical centre, and  academic health science institutions. 

CPU limitations 

Discovery AI, which runs on NVIDIA GPUs, is used for training models using large data sets while  CPUs are used for inferencing.  

“As the number, volume and speed at which we are running inferencing increase, GPUs  become necessary. Otherwise, we need to expend a lot more CPUs to run the same inferencing  at that speed,” said Ngiam. 

For example, one AI tool would run about 100 to 200 inferences per second. For every patient  who turns up at its hospitals and polytechnics, every time a doctor clicks, saves or free texts, or  when new lab test results are out, an AI tool will be running in the background. All the data gets  processed by the AI tools. This is done hundreds of times per second throughout the whole  cluster at a large volume. If only CPUs are used, NUHS will run out of processing speed very  quickly.  

“That’s why we strategised, planned and built in NVIDIA DGX A100 from Day One when we  deployed Endeavour AI. This is because we will be using it for high speed and large volume  inference processed by our AI tools,” said Ngiam. 

NVIDIA DGX A100 is the universal system for all AI infrastructure, from analytics to training to  inference. It packs five petaFLOPS of AI performance into a 6U form factor, replacing legacy  infrastructure silos with one platform for every AI workload.

Streaming data, real-time output 

Endeavour AI is a software and hardware stack that features streaming data as well as AI tools  running micro services off a Kubernetes backbone to process all the streaming data and  produce outputs on a real-time basis. 

With a capacity to handle up to 150 projects, Endeavour AI will start off with dozens of projects  initially before scaling up. 

Among the first projects are those that impact the whole cluster, ranging from predictions on  how a patient with a certain condition will fare when admitted to a hospital to analysing  magnetic resonance imaging (MRI) images. The projects will involve everything from structured  medical data to text-based medical data that form the basis for generating chatbots that are  conversational in nature. 

NUHS produces between 20 and 30 GB of structured data and text daily, or between 1,800 and  2,500 messages per second for one hospital. This translates to about 10,000 to 15,000  messages per second at peak for the entire cluster. The AI tools need to run quickly in the  background to absorb all the data on a day-to-day basis. 

“We do not want to build one project at a time. We are building a platform that enables  multiple projects to run at a time. We have multiple uses for GPUs, largely in training at this  point, but certainly we are well underway in operationalising the production use cases. Without  the GPU, we cannot do a lot of these things,” said Ngiam.  

Operationalising AI, optimising healthcare 

With Endeavour AI, an inferencing platform for streaming data powered by NVIDIA A100 GPUs  on X86 server, NUHS has become the first healthcare group in Singapore to achieve stream  capability by operationalising AI tools in real time for the entire cluster.

Patients interacting with AI-powered chatbots will experience improvement in appointment  making, reduction in waiting time, and enjoy optimised care due to some of the work on  patient trajectories. 

Radiologists and clinicians benefit from improved accuracy and speed of processing of images,  X rays and scans, thanks to the stream capability. 

In day-to-day hospital operations, a number of predictions is done automatically without even  needing to click a button. Data that streams out of the electronic medical record system are  processed and the doctor is alerted if a patient meets a certain set of weights for high risk.  

“These are tangible realities and outcomes that we expected when we Endeavour AI went live,”  said Ngiam. 

Looking towards for compute power 

Even though the new NVIDIA DGX A100 has just gone live, NUHS is already looking forward to  the next generation of the system to tackle the expected growth in datasets and speed needed  to process those data in the next few years. 

“We have invested in programmes that look at genomics. When this genomic data hits us, we  are not talking about gigabytes but one terabyte of data per day. The amount of compute  required to run genomic type processing going forward is going to be exponentially larger. Until  then, our next step is to look at how to optimally use our GPUs for the next few years,” said  Ngiam.  

“NVIDIA DGX A100 lets NUHS consolidate training, inference and analytics into a unified AI  infrastructure. It will provide the computing power to help the hospital group achieve  operational and scientific breakthroughs in the healthcare sector, benefitting clinicians and 

patients in Singapore,” said Dennis Ang, director of enterprise business for the SEA and ANZ  Region at NVIDIA. 

Mark Ko

Mark Ko

Besides tech, I love chicken rice. Point me in the right direction and I'll go and try it. :)
The Role of Digital Gig Workers in Machine Learning and Artificial Intelligence Previous post UiPath Academic Alliance Reaches More Than 500 Higher Education Institutions in Asia Pacific as Demand Climbs for Automation Skills
Siemens Partners with Hyundai Motor Company and Kia Corporation for Digital Mobility Transformation Next post Siemens Partners with Hyundai Motor Company and Kia Corporation for Digital Mobility Transformation

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: