Reading the Screen Abiomed’s Cloud System for Extracting Data from Medical Device Displays Using OCR

Medical devices such as intravascular blood pumps generate continuous streams of critical patient data, but that data is typically only visible on a screen attached to the device itself. A divisional patent from Abiomed – the maker of the Impella heart pump – describes a cloud-based architecture that captures images of a medical device’s graphical display, strips away the visual interface elements using image masking, and extracts the underlying numerical data using optical character recognition, enabling remote monitoring by medical professionals anywhere on the network.

The Problem

Life-support and monitoring devices used in cardiac care, critical care, and surgical settings operate continuously and generate large amounts of patient data: pressure readings, flow rates, pump speeds, temperatures, voltages, and biometric conditions. This data is presented on the device’s own screen as waveforms, plots, and numerical readouts. Medical professionals must be physically present at the bedside to observe this information in real time.

In clinical environments where a patient may require monitoring across multiple locations, or where specialist oversight needs to be available remotely, the inability to access medical device data without physical proximity is a significant limitation. The problem is compounded by the fact that medical device displays are complex graphical compositions: they include not just the data values but also labels, axes, graphical interface elements, status indicators, and background graphics. Simply photographing the screen and transmitting the image does not efficiently extract the underlying numerical data, which is what clinicians actually need.

What This Invention Does

The system captures video from the medical device’s display through a data module connected to the device. At regular intervals, the data module captures still images from the video stream and transmits them to a router and then to a server over a data network.

At the server, the image is processed to isolate the data-containing portions from the surrounding interface graphics. This is achieved using a mask selected based on the specific graphical user interface (GUI) of the medical device model. The mask is applied to the first image, blanking out the GUI elements such as labels, borders, axes, and static interface graphics and leaving only the second set of image portions – those that contain the numerical values and meaningful data signals. Optical character recognition is then applied to this cleaned second image to extract the underlying numerical data values.

The system can be configured with masks for different medical device models, and the extracted data can be transmitted to client devices such as tablets, laptops, or smartphones used by medical professionals at any location on the network. The architecture supports scalability to an indefinite number of users and client devices, and can route video streaming, data uploads, and notifications to specified destinations based on configuration by a portal.

Key Features

Image masking by GUI type. The system selects an image mask based on the specific GUI of the medical device, allowing the same OCR-based extraction architecture to work across different device models by using device-specific masks to isolate the data-containing portions of each model’s unique display layout.

Optical character recognition on cleaned images. By applying the mask before OCR processing, the system avoids confusing graphical interface elements for data values, substantially improving extraction accuracy compared to applying OCR to the raw device screen image.

Video stream capture with configurable intervals. The data module captures images from the continuous video stream at configurable time intervals, enabling near-real-time data extraction with an update frequency appropriate to the clinical application.

Cloud-based scalability. The server architecture supports distribution of extracted data to an indefinite number of client devices and users, enabling multi-site and multi-user monitoring from a single device data stream.

Data validity verification. The system includes a step for determining the validity of extracted data after OCR processing, allowing erroneously extracted values to be flagged or discarded before distribution.

Broad sensor data support. Beyond the display-captured data, the system can also incorporate sensor data from pressure sensors, temperature sensors, flow rate sensors, voltage sensors, current sensors, optical sensors, and audio sensors connected to the medical device.

Who Is Behind It?

Abiomed, Inc. is the applicant, a US company best known as the manufacturer of the Impella percutaneous ventricular assist devices used in interventional cardiology and cardiac surgery. The named inventor is Alessandro Simone Agnello. This divisional was filed on 26 February 2026, derived from AU 2024200248, which was itself a divisional of AU 2018290312. The priority claim traces to US Provisional Application 62/523,890, filed on 23 June 2017. Spruson and Ferguson in Sydney are the Australian patent attorneys.

Why It Matters

Remote monitoring of cardiac assist devices and other life-support equipment has clear patient safety benefits, particularly in settings where specialist cardiologists or perfusionists cannot always be physically present at the bedside. The OCR-based image extraction approach described in this patent is a pragmatic solution to a fundamental integration challenge: most medical devices were not designed with API-level data output in mind, and retrofitting direct data connections is often impractical given regulatory and certification constraints.

By treating the device screen as the interface and applying intelligent image processing to extract structured data from it, the system can work with existing deployed devices without requiring hardware modifications or firmware changes. The masking step that isolates data regions from GUI elements is the key innovation that makes OCR reliable enough for clinical use – a direct OCR pass on a typical medical device screen would produce too many false extractions from labels and graphical elements to be useful.

---

AU 2026201435 was published in the Australian Official Journal of Patents on 19 March 2026 and is open for public inspection. Patent applications represent inventions that are sought to be protected and do not necessarily reflect commercially available products.

Related Concepts

The Impella is a family of percutaneous ventricular assist devices used to support cardiac output in patients with acute myocardial infarction or undergoing high-risk cardiac intervention. These devices generate continuous haemodynamic data that clinicians must interpret in real time.

Optical character recognition (OCR) converts images of text into machine-readable data. Applying OCR to medical device screens – after masking out non-data graphical elements – is a pragmatic approach to extracting structured patient data from devices that lack modern API-level data output, enabling cloud-based remote monitoring without hardware modification.