Masked Generative Distillation for Robust Multi-Lead-to-Single-Lead ECG Classification on IoMT Devices.

Introduction

Cardiovascular diseases (CVDs) remain a major cause of morbidity and mortality worldwide [1]. Timely detection of cardiac abnormalities is therefore important for effective diagnosis and long‐term management [2]. Electrocardiography (ECG) is one of the most widely used non‐invasive tools to assess cardiac electrical activity, providing diagnostically relevant morphological information such as P‐waves, QRS complexes and T‐waves [3].

In clinical practice, ECG analysis typically relies on the standard 12‐lead configuration, which captures the heart's electrical propagation from multiple spatial perspectives and supports comprehensive assessment across a broad range of cardiac conditions [4]. However, continuous 12‐lead monitoring is impractical outside controlled clinical settings. As wearable and portable monitoring technologies continue to expand, single‐lead ECG systems have become increasingly important for ambulatory and remote healthcare applications because of their convenience, low hardware complexity and suitability for long‐term monitoring [5].

The trade‐off, however, is a substantial loss of spatial information, which can reduce diagnostic performance relative to full multi‐lead systems [6].

At the same time, the rapid growth of the Internet of Medical Things (IoMT) is reshaping healthcare delivery by enabling continuous physiological monitoring, real‐time data exchange and remote patient management. Despite these advantages, IoMT devices operate under strict constraints in memory, computation, energy consumption and latency. These limitations make it difficult to deploy high‐capacity deep learning models directly on wearable and embedded platforms [7].

Recent edge‐oriented ECG architectures have likewise emphasized the need for computationally efficient models for real‐time deployment [8]. Consequently, two coupled challenges arise for ECG intelligence in wearable settings: first, the diagnostic information available from single‐lead signals is inherently limited compared with multi‐lead recordings; second, the models deployed on such devices must remain lightweight enough for practical edge inference.

This convergence motivates the central question addressed in this work: how can diagnostically useful knowledge from multi‐lead ECG systems be transferred to single‐lead models while preserving computational efficiency and improving robustness under noisy wearable conditions? To investigate this question, this study evaluates a teacher–student framework in which a high‐capacity 12‐lead model transfers knowledge to a lightweight single‐lead model using masked generative distillation (MGD). The framework is assessed on the PTB‐XL dataset [9], with particular emphasis on performance under synthetic signal degradation, interpretability alignment and deployment‐oriented efficiency.

Deep learning has substantially advanced automated ECG analysis over the past decade. Residual architectures have demonstrated strong performance for ECG classification [4]. Multi‐scale convolutional models have likewise achieved competitive results [10].

Large‐scale benchmark datasets such as PTB‐XL have further supported systematic development and evaluation of modern ECG models [11]. However, high‐capacity ECG models can be difficult to deploy on resource‐constrained hardware. Chauhan et al. addressed this issue through a low‐resource knowledge‐distillation framework [12], while Sepahvand et al. investigated lightweight reduced‐lead distillation for efficient ECG classification [13].

In parallel, knowledge distillation (KD) has been explored as a means of transferring information from multi‐lead ECG models to reduced‐lead students. Sepahvand and Abdali‐Mohammadi used teacher–student distillation to reduce 12‐lead ECG classification to a single lead [6], while Qin et al. proposed multi‐view knowledge transfer for multi‐label single‐lead ECG classification [14]. While these approaches improve efficiency, most existing ECG distillation methods focus primarily on output matching or direct feature alignment.

As a result, they do not fully address robustness under signal corruption, nor do they explicitly examine whether the student preserves diagnostically meaningful attention patterns under noisy conditions. MGD, originally developed for computer vision, offers an alternative distillation strategy in which the student reconstructs the teacher feature representation from a masked version of its own internal features rather than relying on direct feature imitation [15]. However, its application to ECG time‐series classification remains limited.

Masked generative sistillation (MGD) is an existing feature‐distillation method originally developed for computer‐vision tasks [15]; therefore, the novelty of this work does not lie in the MGD algorithm itself, but in the design and evaluation of an ECG‐specific teacher–student pipeline for multi‐lead‐to‐single‐lead classification. Specifically, the present study makes four original contributions. First, it adapts MGD to 1D ECG representations through linear temporal feature alignment, a lightweight 1D reconstruction generator and a curriculum‐based temporal masking schedule suited to ECG feature maps.

Second, it integrates this feature‐reconstruction mechanism into a 12‐lead‐teacher‐to‐single‐lead‐student framework and evaluates its interaction with conventional output‐level KD and corruption‐aware training through a controlled five‐model ablation study. Third, it introduces a joint robustness‐and‐interpretability evaluation protocol spanning multiple synthetic wearable‐relevant noise types and SNR levels, together with Grad‐CAM‐based teacher–student attention comparison and SSIM analysis. Fourth, it evaluates deployment‐oriented feasibility through simulation‐based profiling of parameter count, computational complexity, latency and memory requirements, together with post‐training INT8 quantization.

These contributions collectively establish an ECG‐specific pipeline for investigating whether masked feature reconstruction can improve the robustness of lightweight single‐lead models under resource‐constrained wearable conditions.

The remainder of this paper is organized as follows. Section 2 reviews the relevant literature on ECG classification, knowledge distillation, lightweight medical AI and robustness analysis. Section 3 describes the dataset, preprocessing, model architectures, distillation framework, ablation design and evaluation methodology.

Section 4 presents the experimental results and analysis. Section 5 concludes the paper and outlines future research directions.

Literature Review

Deep Learning for ECG Classification and the Lead‐Reduction Problem

Automated electrocardiogram (ECG) analysis has advanced substantially with the adoption of deep learning methods. Earlier approaches typically relied on handcrafted features combined with conventional classifiers, but their performance was often limited by poor generalizability across heterogeneous patient populations and varying signal quality conditions [10]. The introduction of deep neural networks, particularly convolutional neural networks (CNNs), enabled end‐to‐end learning directly from raw ECG waveforms and reduced reliance on manual feature engineering.

A range of deep architectures, including residual CNNs and multi‐scale convolutional models, have demonstrated strong performance in ECG classification tasks by learning discriminative temporal and morphological representations from the signal [4, 10]. Large benchmark datasets such as PTB‐XL have further accelerated progress by providing sufficiently large and diverse 12‐lead ECG collections for training and evaluating modern models in both multi‐class and multi‐label settings [11]. These developments have established deep learning as a leading approach for automated ECG interpretation.

Despite these advances, two practical limitations remain. First, many of the highest‐performing models are computationally demanding, with substantial memory and inference requirements that hinder deployment on low‐power embedded systems [7, 12, 13]. Second, much of the reported performance has been achieved using standard 12‐lead ECG input, which provides multiple spatial perspectives of cardiac electrical activity and supports more comprehensive diagnostic assessment [3, 4].

In contrast, wearable and ambulatory ECG devices commonly operate using reduced‐lead or single‐lead configurations because of hardware simplicity, comfort and suitability for long‐term monitoring [5]. This reduction in input dimensionality improves practicality but removes diagnostically useful spatial information, often leading to measurable performance degradation relative to multi‐lead systems [6, 14].

The lead‐reduction problem is therefore central to wearable ECG intelligence: although single‐lead devices are attractive for real‐world monitoring, they create an information bottleneck that can limit classification accuracy. This challenge has motivated research into methods that can transfer knowledge from richer multi‐lead representations to more deployable reduced‐lead models.

Knowledge Distillation for Efficient and Reduced‐Lead ECG Models

Knowledge distillation (KD) was originally introduced as a model‐compression strategy in which a smaller student network learns from the softened output distribution of a larger teacher model [16]. By transferring class‐similarity information encoded in soft targets, KD can improve the generalization ability of compact models beyond what is achievable with hard‐label supervision alone. Subsequent extensions moved beyond output‐level supervision to feature‐level distillation, where the student is encouraged to align intermediate representations with those of the teacher [6, 17].

These approaches aim to retain more of the teacher's internal structure while reducing model size.

In ECG analysis, KD has attracted growing attention as a means of improving the efficiency of diagnostic models while preserving acceptable performance under wearable or edge constraints [7, 12, 14]. Prior studies have shown that teacher–student transfer can partially compensate for the information loss associated with lead reduction. For example, multi‐view knowledge transfer frameworks have demonstrated that information from multi‐lead ECG systems can be transferred to single‐lead students through combinations of distillation and contrastive objectives [14].

Other approaches have employed structured feature‐transfer mechanisms, such as decomposition‐based modules, to bridge the capacity and representation gap between large 12‐lead teachers and lightweight reduced‐lead students [13]. Distillation has also been used in low‐resource heart‐monitoring architectures to achieve substantial parameter reduction while maintaining useful classification capability [12], and lightweight arrhythmia classification systems have leveraged KD to reach compression ratios suitable for microcontroller‐class deployment [7].

Beyond single‐teacher distillation, multi‐teacher knowledge distillation has also been explored as a means of improving generalization and robustness to corrupted inputs. Amirkhani et al. proposed a multi‐teacher knowledge distillation framework for semantic segmentation in which several teacher networks, each trained under a distinct robustness‐oriented augmentation scenario, are aggregated through a score‐weight mechanism that selects the most reliable pseudo‐label based on both clean and corrupted‐data performance [18]. Their experiments showed that this multi‐teacher aggregation improved robustness to synthetic image corruption by 9% to 32.18% relative to a single‐teacher baseline [18], demonstrating that distributing knowledge across multiple differently trained teachers can substantially strengthen student robustness beyond what a single teacher–student pair achieves.

Although this approach was developed for image‐based semantic segmentation rather than physiological time‐series classification, it highlights multi‐teacher distillation as a promising direction for extending corruption‐robust reduced‐lead ECG modelling, a direction noted as future work in Section 5.1.

These studies show that KD is a promising strategy for efficient ECG modelling, particularly when reduced‐lead deployment is the goal. However, most existing ECG distillation frameworks still rely primarily on output matching or direct feature imitation. While effective for compression, these mechanisms may be insufficient when the teacher–student capacity gap is large or when the student must operate under substantial signal corruption.

In such cases, strict feature alignment can over‐constrain the student and may limit its ability to learn flexible, noise‐tolerant representations. In addition, prior ECG distillation studies have focused mainly on classification accuracy and compression, while robustness under controlled signal degradation and interpretability alignment between teacher and student remain comparatively less explored.

Masked Generative Distillation and Its Relevance to ECG

Masked generative distillation (MGD) was proposed as an alternative feature‐distillation paradigm in which the student reconstructs the teacher feature representation from a masked version of its own features rather than directly replicating the teacher representation [15]. Instead of enforcing one‐to‐one feature matching, MGD introduces stochastic masking into the student representation and uses a lightweight generator to recover the corresponding teacher feature map. This design discourages trivial copying and shifts the distillation process from strict imitation to structured reconstruction.

Compared with conventional feature distillation, MGD offers several potential advantages. First, the masking operation acts as a regularizer, encouraging the student to infer missing information from partial context and thereby promoting more generalizable representations [15]. Second, reconstruction‐based supervision may be more tolerant of architectural mismatch between teacher and student than direct feature alignment.

Third, because the student must recover informative teacher structure from incomplete representations, MGD may be better suited to learning features that remain useful under perturbation or partial signal degradation.

To date, MGD has been studied primarily in computer vision tasks such as image classification, object detection and segmentation, where it has shown benefits in student performance and architectural flexibility [15]. However, its application to biomedical time‐series analysis, and particularly to ECG classification, has received limited attention. This creates a relevant opportunity for reduced‐lead ECG modelling, where the student must learn from a weaker input source and is expected to operate in noisy wearable environments.

In this context, a distillation method based on reconstructing informative teacher structure may offer advantages over conventional output‐ or feature‐matching approaches.

Deployment, Robustness and Interpretability in Wearable ECG AI

For wearable and Internet of Medical Things (IoMT) applications, classification accuracy alone is not sufficient. Models intended for long‐term ambulatory ECG monitoring must also satisfy constraints related to memory usage, computational cost, inference latency and energy consumption [7, 12]. Compression techniques such as pruning, quantization and distillation have therefore become central to deployment‐oriented ECG research.

Recent work in TinyML and embedded inference has shown that highly compact neural networks can process ECG signals directly on low‐power hardware, supporting on‐device intelligence without continuous cloud dependence [19]. Nevertheless, the trade‐off between efficiency and diagnostic performance remains a persistent challenge, particularly for multi‐class and multi‐label ECG classification on reduced‐lead input [7].

Robustness is an equally important consideration. In wearable settings, ECG recordings are frequently corrupted by baseline wander, motion artefacts, muscle noise and electrode disturbances. Such perturbations can substantially degrade model reliability, yet explicit robustness evaluation remains limited in much of the ECG distillation literature.

Models that appear competitive on clean benchmark data may perform poorly under realistic signal corruption, which reduces their practical value for real‐world monitoring.

Interpretability is also increasingly important in medical AI. Methods such as gradient‐weighted class activation mapping (Grad‐CAM) have been used to visualize the temporal regions of ECG signals that contribute most strongly to model predictions [11]. These tools can help determine whether a model attends to diagnostically meaningful waveform structures.

However, prior studies have rarely examined interpretability in a teacher–student setting, and quantitative assessment of alignment between teacher and student attention patterns remains limited.

The literature shows strong progress in three related areas: high‐performing deep learning for ECG classification, efficient reduced‐lead modelling through distillation, and deployment‐oriented model compression. However, an important gap remains at their intersection. Existing ECG distillation studies have not adequately examined whether a robustness‐oriented generative distillation strategy can support multi‐lead‐to‐single‐lead transfer under controlled signal degradation, while also preserving interpretable attention behaviour and maintaining deployment‐relevant efficiency.

Addressing this gap motivates the present study, which investigates masked generative distillation within a controlled ablation framework for robust and deployment‐oriented single‐lead ECG classification.

Methodology

Overview of the Methodology

This section presents the methodological framework used to evaluate whether diagnostically useful knowledge from multi‐lead ECG can be transferred to a lightweight single‐lead model while preserving robustness under wearable‐relevant signal degradation. The study combines ECG preprocessing, teacher–student model design, controlled distillation training, robustness testing under synthetic noise, interpretability analysis and deployment‐oriented profiling.

The experimental pipeline begins with PTB‐XL data preparation, including label mapping, filtering, normalization and patient‐wise splitting. A 12‐lead 1D ResNet‐18 is trained as the teacher model, and a compact single‐lead CNN is used as the student architecture. To assess the contributions of different training mechanisms, five model variants are evaluated through a controlled ablation design spanning supervised learning, conventional knowledge distillation (KD), masked generative distillation (MGD) and noise augmentation.

Model performance is then assessed along four dimensions. First, classification capability is measured on clean ECG data. Second, robustness is evaluated under synthetic wearable‐noise conditions, including baseline wander, muscle artefact and electrode motion contamination at multiple signal‐to‐noise ratios.

Third, teacher–student interpretability alignment is examined using Grad‐CAM and structural similarity index (SSIM) analysis. Finally, deployment‐oriented feasibility is assessed through profiling of parameter count, computational cost, latency, memory footprint and post‐training INT8 quantization. Together, these components provide a structured framework for evaluating accuracy, robustness, interpretability and efficiency in reduced‐lead ECG classification.

Dataset Selection Criteria

An appropriate dataset for this study must satisfy four requirements: sufficient scale for training deep neural networks, clinically meaningful diagnostic annotations, standard 12‐lead recordings for teacher‐model training and patient‐level metadata to support leakage‐free data splitting. PTB‐XL was selected because it meets these requirements while also providing a widely used benchmark for ECG classification research.

PTB‐XL contains more than 21,000 clinical 12‐lead ECG recordings from nearly 19,000 patients, making it substantially larger than many commonly used ECG datasets. This scale is advantageous for training and evaluating high‐capacity deep learning models, particularly in multi‐label classification settings. In contrast, smaller datasets such as MIT‐BIH are valuable for arrhythmia research but provide more limited sample size, label structure and patient diversity for the present multi‐lead distillation framework.

PTB‐XL is particularly suitable for this work because it includes standard 12‐lead recordings and detailed SCP‐code annotations that can be aggregated into the five diagnostic superclasses commonly used in prior studies (NORM, MI, STTC, CD and HYP). In addition, the availability of patient identifiers enables patient‐wise splitting, which reduces the risk of information leakage between training and evaluation sets. These characteristics make PTB‐XL an appropriate and reproducible benchmark for studying multi‐lead‐to‐single‐lead ECG knowledge transfer under controlled experimental conditions.

Dataset Description

The experiments were conducted on the PTB‐XL dataset, a large publicly available collection of clinical 12‐lead electrocardiograms (ECGs) comprising more than 21,000 recordings from 18,885 patients [9]. In this study, the 100 Hz version of PTB‐XL was used, such that each record consists of 10 s of signal with 1000 samples per lead. This yields an input tensor of length 1000 for each of the 12 ECG leads.

PTB‐XL provides expert annotations using SCP (Standard Communication Protocol) statement codes. Following the commonly used diagnostic‐superclass formulation in the PTB‐XL literature, only diagnostic statements were retained and mapped to five superclass labels: normal ECG (NORM), myocardial infarction (MI), ST/T change (STTC), conduction disturbance (CD) and hypertrophy (HYP). Because a single recording may contain more than one diagnostic superclass, the task was formulated as a multi‐label classification problem.

Each ECG record was therefore represented by a 5‐dimensional binary target vector indicating the presence or absence of each superclass.

Records without valid diagnostic superclass assignments, that is, records for which none of the SCP codes mapped to any of the five target superclasses, were excluded, while records with multiple superclass labels were retained without modification as part of the multi‐label formulation. The final dataset after preprocessing contained 21,388 ECG recordings from 18,617 unique patients. Class counts for the five superclasses were as follows: NORM = 9514, MI = 5469, STTC = 5,235, CD = 4898 and HYP = 2649.

Because PTB‐XL is inherently imbalanced at the superclass level, this class distribution was taken into account through class‐weighted loss functions during training.

To prevent information leakage, all experiments used a fixed patient‐wise split, ensuring that no patient appeared in more than one subset. The data were divided into training, validation and test sets in a 70%/10%/20% ratio using multi‐label stratification at the patient level, with random_state = 42 for reproducibility. The resulting split sizes were Ntrain = 14,991, Nval = 2124 and Ntest = 4273 recordings, respectively.

This design ensures that model performance reflects generalization to unseen patients rather than memorization of correlated recordings from the same individual.

ECG Signal Preprocessing

All ECG recordings were preprocessed before model training to reduce signal artefacts and standardize input scale across samples. Because the experiments used the 100 Hz PTB‐XL recordings, each 10‐s ECG record contained 1000 time samples per lead.

First, each lead was filtered independently using a fourth‐order Butterworth bandpass filter with cutoff frequencies of 0.5 and 40 Hz in order to suppress low‐frequency baseline drift and high‐frequency electrical noise. The filter was applied using zero‐phase forward–backward filtering (scipy.signal.filtfilt), which processes the signal in both the forward and reverse directions so that the phase responses cancel, preserving waveform morphology without introducing phase distortion. This frequency range was selected because it is commonly used in ECG preprocessing and retains the principal morphological components relevant for diagnostic analysis, including P‐waves, QRS complexes and T‐waves.

After filtering, each lead was normalized using z‐score normalization. For a signal x, the normalized signal xnorm was computed as (1)xnorm=x−μσ,where μ and σ denote the mean and standard deviation of that lead, respectively. Normalization was performed on a per‐record, per‐lead basis over the full 10‐s signal, resulting in approximately zero‐mean, unit‐variance inputs and improving optimization stability during training.

The processed recordings were then arranged in channel‐first format for PyTorch. For the teacher model, the final input tensor had shape (N,12,1000), where N denotes the number of ECG recordings, 12 is the number of leads and 1000 is the temporal dimension.

For the student model, only Lead II was retained in order to simulate a reduced‐lead wearable ECG setting. Lead II was selected because it is commonly used in single‐lead monitoring systems and generally provides diagnostically informative rhythm and waveform morphology. The resulting student input tensor therefore had shape (N,1,1000).

This configuration defines a controlled lead‐reduction setting in which the student must learn from a substantially weaker input representation than the 12‐lead teacher.

Teacher Model Architecture

The teacher model was implemented as a 1D adaptation of ResNet‐18 to process full 12‐lead ECG recordings. ResNet‐18 was selected as the teacher architecture because it provides sufficient representational capacity for multi‐label ECG classification while remaining structurally well suited to feature‐based distillation.

The network begins with a 1D convolutional stem that maps the 12‐channel ECG input to 64 feature channels using a kernel size of 7 and stride of 2, followed by batch normalization, ReLU activation and max pooling. This is followed by four residual stages arranged in the standard ResNet‐18 configuration of [2, 2, 2, 2] residual blocks. Each block contains two 1D convolutional layers with batch normalization and ReLU activation, together with identity or projection shortcuts where required.

Across stages, the channel dimension increases from 64 to 128, 256 and 512, while the temporal resolution is progressively reduced by strided downsampling.

After the final residual stage, global average pooling is applied over the temporal dimension to obtain a compact 512‐dimensional representation, which is passed to a fully connected layer producing five superclass logits for multi‐label classification. In addition to the classifier output, the feature map from the final residual stage is retained before global pooling and used as the teacher representation for feature‐level distillation. This tensor serves as the reconstruction target in the MGD framework.

Figure 1 illustrates the overall teacher architecture.

Student Model Architecture

The student model is a compact 1D convolutional neural network designed for reduced‐lead ECG classification under resource‐constrained deployment settings. A lightweight backbone was chosen to ensure consistency across all ablation variants (M1–M5) while remaining small enough for deployment‐oriented profiling.

The network receives a single‐lead ECG input of shape (N,1,1000), corresponding to a 10‐s recording sampled at 100 Hz, and processes it through four sequential 1D convolutional blocks, each comprising a convolution (without bias), batch normalization and ReLU activation. Blocks 1–3 are each followed by max‐pooling with kernel size 2 and stride 2, while Block 4 omits pooling. The convolutional layers use progressively smaller kernels of sizes 7, 5, 3 and 3, with strides of 2, 1, 1 and 1, and output channel dimensions of 32, 64, 128 and 128 respectively.

The kernel sizes were chosen considering the temporal scale of ECG morphology. The first layer's kernel of 7 corresponds to approximately 70 ms at 100 Hz, which lies within the temporal scale of a typical QRS complex and allows the network to capture local cardiac morphology at an early stage. Kernel sizes were then reduced in later blocks (5, 3, 3), since pooling at each stage expands the effective receptive field even as the nominal kernel shrinks; by the final block, the effective receptive field reaches approximately 850 ms, comparable to a full typical cardiac cycle, so later layers can rely on smaller kernels while still integrating whole‐beat context.

The stride configuration was selected to balance computational reduction with preservation of ECG morphology. A stride of 2 is used only in the first convolution to reduce the initial temporal resolution and computational load at an early stage, while the subsequent convolutional layers use stride 1 to avoid excessive temporal decimation of diagnostically relevant waveform features. Further downsampling is performed progressively through the max‐pooling layers, yielding a final temporal length of 62 samples while maintaining sufficient resolution for feature extraction and Grad‐CAM analysis.

Padding values of 3, 2, 1 and 1 are used to preserve feature alignment throughout the network. Together, the three pooling operations reduce the temporal dimension from 1000 to 62. The complete layer‐by‐layer specification is provided in Table 1.

The feature map exiting Block 4 has shape (N,128,62) and serves as the student‐side representation for feature‐level supervision in the MGD variants, as well as the target layer for Grad‐CAM visualization. Global average pooling is then applied over the temporal dimension, collapsing the feature map into a compact (N,128) representation. This vector is passed to a fully connected layer producing five superclass logits for multi‐label classification.

The resulting architecture contains approximately 85,000 trainable parameters, approximately 45 times fewer than the teacher network, while maintaining sufficient representational capacity for effective knowledge transfer and robustness evaluation.

Knowledge Distillation Training

Conventional knowledge distillation (KD) was used as the output‐level teacher–student baseline for the reduced‐lead student model. During student training, the pretrained 12‐lead teacher remained frozen and processed the full ECG input to produce a set of five logits, one for each diagnostic superclass. A single teacher checkpoint trained with random seed 42 was used as the fixed teacher for all distillation‐based student runs (M2‐M5), irrespective of the student‐training seed.

Because the task is multi‐label rather than mutually exclusive multi‐class classification, the teacher logits were converted to softened probabilities using a temperature‐scaled sigmoid function rather than softmax.

Let zt and zs denote the teacher and student logits, respectively, and let T denote the distillation temperature. The softened teacher and student probabilities are defined as (2)pt=σztT,(3)ps=σzsT,where σ(.) is the sigmoid function. The student was trained using a weighted combination of the supervised task loss and the distillation loss: (4)LKDtotal=αLtask+(1−α)Ldistill.Here, Ltask denotes weighted binary cross‐entropy with respect to the ground‐truth superclass labels, and Ldistill denotes binary cross‐entropy between the softened teacher and student probabilities: (5)Ldistill=BCE(ps,pt)The hyperparameter α controls the trade‐off between direct supervision from the annotated labels and indirect supervision from the teacher model.

This formulation serves as the conventional KD baseline in the ablation study, against which the masked generative distillation variants are compared. Figure 2 illustrates the corresponding teacher–student training framework.

Masked Generative Distillation

Masked generative distillation (MGD) extends conventional knowledge distillation by introducing feature‐level supervision in addition to output‐level teacher guidance. Whereas standard KD encourages the student to match the teacher's softened output probabilities, MGD further requires the student to reconstruct a high‐level teacher feature representation from a partially masked version of its own internal feature map. This provides an additional supervisory signal that is intended to improve representational robustness, particularly when the student is trained on reduced‐lead ECG input.

Let Ft∈RN×Ct×Tt denote the feature map extracted from the final residual stage of the frozen teacher network, and let Fs∈RN×Cs×Ts denote the feature map produced by the final convolutional block of the student network before global average pooling. In the implemented architectures, the teacher feature map has shape (N,512,32), whereas the student feature map has shape (N,128,62). Because the two representations have different temporal resolutions, the student feature map is first linearly resampled along the temporal dimension from 62 to 32 time steps, yielding an aligned representation F¯s∈RN×128×32.

This temporal alignment is performed before masking and does not alter the student channel dimension. A binary temporal mask M∈{0,1}1×32 is then applied to the aligned student feature map, with broadcasting across batch and channel dimensions, yielding (6)F∼s=F¯s⊙M,where ⊙ denotes element‐wise multiplication with broadcasting across channels. The masking ratio was not fixed; it followed a curriculum schedule during training, with exact values provided in Table 2, Section 3.15.

The masked student representation is then passed through a lightweight generator network G(·) comprising three sequential 1D convolutional layers. The first is a 1×1 convolution (no bias) that projects the student feature channels (128) to the teacher feature dimension (512), performing a linear channel‐wise remapping without spatial mixing. This is followed by two 3×1 convolutional layers (kernel size 3, padding 1, 512 output channels, no bias), each followed by a ReLU activation, which introduce local temporal context to reconstruct the teacher feature map.

No batch normalization is applied within the generator.

The 1×1 layer performs a lightweight channel projection from the student's 128 feature channels to the teacher's 512 channels without mixing information across time steps, while the two 3×1 layers introduce a small local receptive field so that masked temporal positions can be reconstructed using information from their immediate neighbours rather than in isolation. This keeps the generator small enough that the reconstruction task remains informative about the student's own features rather than being absorbed by the generator itself.

The generator architecture was therefore selected according to three design requirements: (i) matching the channel dimensionality of the student and teacher feature spaces, (ii) preserving the temporal resolution required for point‐wise feature reconstruction and (iii) introducing only limited local temporal modelling so that the generator remains substantially simpler than the student and teacher networks. The 1×1 projection satisfies the first requirement without altering temporal resolution, while the two 3×1 convolutions provide a limited receptive field for reconstructing masked temporal features. A deeper or wider generator was deliberately avoided because excessive generator capacity could reduce the need for the student representation itself to encode teacher‐relevant information.

The generated feature representation is therefore given by (7)F^t=G(F∼s).Feature‐level supervision is imposed by minimizing the mean squared error between the generated representation and the corresponding teacher feature map: (8)LMGD=1NCtTtF^t−Ft22.

This reconstruction objective discourages trivial feature imitation and instead requires the student to infer missing information from the unmasked temporal context, thereby encouraging the learning of more structured and potentially more noise‐tolerant representations. The complete MGD training objective combines supervised classification loss, output‐level distillation loss and feature‐reconstruction loss: (9)L=αLtask+βLdistill+γLMGD,where Ltask is the weighted binary cross‐entropy loss with respect to the ground‐truth superclass labels, Ldistill is the output‐level distillation loss defined in Equation (5) and LMGD is the feature‐reconstruction term defined in Equation (8). The coefficients α, β and γ control the relative contribution of the three losses during training, with α+β+γ=1.

The exact loss weights and masking schedule are provided in Table 2, Section 3.15.

Under this formulation, the teacher network remains fixed and serves as a stable multi‐lead reference, while the student is trained to match both the teacher's predictions and its high‐level feature structure. Figure 3 illustrates the full MGD‐based student training framework.

Noise Augmentation Strategy

Noise augmentation was introduced in the augmentation‐enabled student variants to evaluate the independent and combined effects of corruption‐aware training and distillation. The objective was to expose the student network to synthetic but wearable‐relevant signal degradation during training, thereby encouraging more robust feature learning under noisy input conditions.

Three common ECG artefact types were considered: baseline wander, muscle artefact and electrode motion noise. Baseline wander represents low‐frequency signal drift typically associated with respiration or body movement. Muscle artefact introduces higher‐frequency interference caused by skeletal muscle activity, while electrode motion noise reflects transient disturbances arising from changes in electrode‐skin contact.

These noise sources were selected because they are representative of the types of corruption commonly encountered in ambulatory and wearable ECG monitoring.

During training, noise was applied online to the student's single‐lead input with a fixed augmentation probability, while the teacher network always received the corresponding clean 12‐lead ECG signal. For augmented samples, one noise type was randomly selected and added to the student input at a signal‐to‐noise ratio (SNR) drawn from a predefined range. This asymmetric training configuration provides a stable clean reference through the teacher while encouraging the student to map corrupted single‐lead input toward teacher‐aligned representations.

Noise augmentation was used in the M3 and M5 ablation variants only. In M3, it was combined with conventional output‐level KD to isolate the effect of augmentation in the absence of feature reconstruction. In M5, it was combined with both KD and MGD to evaluate whether corruption‐aware training and masked feature reconstruction provide complementary robustness benefits.

The exact augmentation probability, SNR range and noise settings used in the experiments are reported in Table 2, Section 3.15.

The noise used in this study is synthetically generated rather than obtained from real ambulatory recordings. Accordingly, we do not claim that the generated corruptions reproduce the full non‐stationary, device‐specific and patient‐dependent characteristics of real‐world wearable ECG artefacts. The controlled synthetic‐noise protocol was instead adopted as a reproducible stress‐testing framework that permits independent manipulation of artefact type and SNR severity across all ablation variants.

Similar robustness studies in other CNN application domains have used controlled synthetic corruptions to quantify model sensitivity and evaluate corruption‐aware training strategies [20]. The robustness results reported here should therefore be interpreted as relative comparisons among the evaluated training strategies under controlled corruption, rather than as direct estimates of performance in real‐world wearable conditions. Validation using naturally corrupted ambulatory ECG recordings is required to assess robustness under real deployment conditions and is identified as future work in Section 5.1.

Ablation Study Design

To disentangle the effects of output‐level distillation, masked feature reconstruction and corruption‐aware training, a controlled five‐model ablation study was designed. The proposed framework involves three components: (i) knowledge distillation from the teacher's softened output logits, (ii) masked generative reconstruction of teacher feature representations and (iii) noise augmentation applied to the student's single‐lead input during training. The ablation study was constructed to evaluate the individual and combined contributions of these components under a shared experimental setting.

Five student variants were trained and evaluated. All variants used the same Student CNN backbone, and all were trained under a controlled protocol in which the intended supervisory and augmentation components were varied systematically. The exact optimization settings are reported in Section 3.15.

M1 (supervised only): The student was trained using weighted binary cross‐entropy with ground‐truth labels only, without teacher supervision. This variant establishes the single‐lead supervised baseline.M2 (supervised + KD): Conventional output‐level knowledge distillation was added to the supervised loss. The student received temperature‐scaled soft targets from the frozen teacher in addition to the ground‐truth labels.

This variant evaluates the contribution of output‐level teacher guidance.M3 (supervised + KD + noise augmentation): Noise augmentation was added to M2, with corrupted single‐lead input presented to the student during training while the teacher continued to receive the corresponding clean 12‐lead signal. This variant evaluates the contribution of corruption‐aware training in the absence of feature‐level reconstruction.M4 (supervised + KD + MGD): The student was trained with the combined task, KD and MGD feature‐reconstruction losses, but without noise augmentation. Both teacher and student received clean input signals.

This variant evaluates the contribution of masked generative reconstruction when explicit corruption‐aware training is absent.M5 (supervised + KD + MGD + noise augmentation): All three components were enabled, combining supervised learning, output‐level distillation, masked feature reconstruction and noise augmentation. This is the full proposed method.

This design supports five principal comparisons. The transition from M1 to M2 tests the effect of conventional output‐level KD. The comparison of M2 and M3 tests the contribution of noise augmentation when feature reconstruction is not used.

The comparison of M2 and M4 tests the contribution of masked generative reconstruction under clean‐input training. The comparison of M2 and M5 provides the primary clean‐test benchmark contrast, directly evaluating the full proposed method against the conventional KD baseline on unmodified test data. Finally, the comparison of M3 and M5 tests whether MGD provides additional robustness benefits beyond those obtained from noise augmentation alone.

These comparisons provide a structured basis for attributing observed performance differences to specific components of the proposed training framework. Because the MGD variants introduce an additional loss term and use the corresponding loss weighting and training schedule reported in Table 2, Section 3.15, the comparisons involving M4 and M5 assess the contribution of the MGD‐enabled training configuration as implemented rather than the MGD term in strict isolation.

Evaluation Metrics

Model evaluation was conducted along four dimensions: clean‐data classification performance, robustness under synthetic corruption, interpretability alignment and deployment‐oriented efficiency. Because the PTB‐XL superclass task is multi‐label and class‐imbalanced, macro‐averaged classification metrics were prioritized so that each superclass contributed equally to the overall score.

Classification Metrics

The primary classification metric was macro F1‐score. For threshold‐dependent metrics, sigmoid outputs were converted to binary predictions using a fixed threshold of 0.50, as reported in Table 2, Section 3.15. The primary robustness endpoint was mean macro F1 degradation from the clean‐test baseline, averaged across all 15 noisy test conditions (three noise types × five SNR levels from 30 to 0 dB).

The primary stress‐test condition was the 0 dB SNR setting, representing the most severe signal corruption evaluated. These definitions establish a pre‐specified evaluation hierarchy that enables unambiguous comparison across ablation variants without selective reporting. The following metrics were used: Macro F1‐score: The F1‐score was computed independently for each superclass and then averaged without class weighting.

This was the primary metric because it balances precision and recall while remaining sensitive to minority‐class performance.Macro precision: Precision was computed per superclass and then macro‐averaged across the five labels.Macro recall: Recall was computed per superclass and then macro‐averaged across the five labels.Exact‐match accuracy: Also referred to as subset accuracy, this measures the proportion of ECG records for which the full predicted label set matches the ground‐truth label set exactly.Hamming accuracy: This metric reflects the proportion of individual label decisions classified correctly across all samples and classes.Macro AUROC: The area under the receiver operating characteristic curve was computed independently for each superclass in a one‐versus‐rest manner and then averaged without class weighting. This metric provides a threshold‐independent measure of class separability.

Interpretability Metric

To evaluate whether the student preserved teacher‐aligned attention behaviour, gradient‐weighted class activation mapping (Grad‐CAM) was applied to both models and the resulting 1D heatmaps were compared using the structural similarity index (SSIM). Structural similarity index (SSIM): SSIM was computed between the teacher and student Grad‐CAM heatmaps to quantify their structural similarity. Higher SSIM values indicate greater alignment between the temporal attention patterns of the two models, whereas lower values indicate greater divergence in the regions of the ECG signal emphasised for prediction.

Deployment‐Oriented Metrics

To assess computational efficiency and deployment potential under simulated resource‐constrained conditions, the following profiling metrics were used: Parameter count: The total number of trainable model parameters.MACs: The number of multiply‐accumulate operations required for a single forward pass, used as a proxy for computational complexity.Inference latency: The execution time of one forward pass, reported in milliseconds.Flash memory: The estimated non‐volatile storage required to store model weights.Peak SRAM: The estimated peak volatile memory required during inference to store inputs, intermediate activations and outputs.

Together, these metrics provide a structured basis for evaluating predictive performance on clean and corrupted ECG signals, alignment between teacher and student attention behaviour and deployment‐oriented efficiency under simulated edge‐device constraints.

Robustness Evaluation

Robustness was evaluated by applying controlled synthetic noise corruption to the held‐out test set and measuring the resulting degradation in classification performance. Each ECG record was corrupted independently using the three wearable‐relevant artefact types introduced in Section 3.9: baseline wander, muscle artefact and electrode motion noise. For each artefact type, corrupted test sets were generated at multiple signal‐to‐noise ratio (SNR) levels of 30, 20, 10, 5 and 0 dB, where lower SNR values correspond to more severe signal degradation.

Noise types were evaluated separately so that their effects could be analysed individually. For each model, performance was measured on the clean test set and on each corrupted test condition using macro F1‐score as the primary classification metric. The primary robustness endpoint was mean macro F1 degradation from the clean‐test baseline, averaged across all 15 noisy conditions (three noise types × five SNR levels).

The 0 dB SNR condition was additionally designated as the primary severe‐noise stress test, as it represents the most challenging corruption level evaluated and provides the sharpest contrast between ablation variants.

In total, robustness was assessed across all combinations of three noise types and five SNR levels (15 conditions), providing a structured basis for comparing clean‐condition performance against corruption‐induced degradation in reduced‐lead ECG classification.

Interpretability Analysis

Interpretability was analysed using gradient‐weighted class activation mapping (Grad‐CAM) to examine whether the student model preserved teacher‐aligned attention patterns under both clean and corrupted input conditions. For each model, Grad‐CAM was computed with respect to the final convolutional layer, yielding a 1D saliency map over the temporal ECG signal that highlights the regions most influential for prediction.

To enable direct teacher–student comparison, Grad‐CAM heatmaps were generated for corresponding ECG samples using the predicted class of the teacher model as the target class. The teacher's dominant predicted class was used to ensure that teacher and student heatmaps were compared with respect to the same decision target in a consistent one‐class‐at‐a‐time interpretability analysis. Specifically, for each sample, the teacher's output logits were used to determine the predicted superclass via argmax, and this class index was used as the Grad‐CAM target for all three models (teacher, M2 (KD) and M5 (MGD + noise)), ensuring that the heatmaps reflect attention toward the same classification decision.

All Grad‐CAM and SSIM analyses were performed using the seed‐42 checkpoints of the teacher, M2 and M5. The resulting 1D heatmaps were then normalized to the range [0,1] and compared using the SSIM, which provides a quantitative measure of structural agreement between the teacher and student attention profiles. SSIM was computed on a per‐sample basis and then aggregated across a randomly sampled evaluation subset of 200 test records (random_state = 42).

A fixed subset of 200 test records was used to keep Grad‐CAM generation and bootstrap‐based SSIM estimation computationally tractable while preserving reproducibility.

Higher SSIM values indicate greater structural similarity between the temporal regions emphasized by the teacher and student models. In this study, interpretability analysis was used to assess whether the full M5 training framework (KD + MGD + noise augmentation) retains greater teacher‐aligned attention under signal degradation than the conventional KD baseline, rather than to isolate the effect of MGD alone.

Deployment‐Aware Performance Evaluation

Beyond classification performance, the proposed models were also assessed with respect to deployment‐oriented efficiency under simulated resource‐constrained conditions. The aim of this analysis was to determine whether the distilled single‐lead student models provide a favourable trade‐off between predictive performance and computational cost relative to the larger 12‐lead teacher.

Deployment‐aware evaluation focused on five metrics: parameter count, multiply‐accumulate operations (MACs), inference latency, flash memory footprint and peak SRAM usage. Parameter count and MACs were used to quantify model size and computational complexity. Inference latency was measured as the execution time of a single forward pass on CPU hardware, while flash memory and peak SRAM were estimated from model weights and intermediate activation requirements during inference.

Together, these metrics provide a practical proxy for the computational and memory constraints typically associated with wearable and IoMT‐class devices.

Inference latency was measured on the host CPU using single‐sample forward passes (batch size = 1) with FP32 precision. All models were evaluated in torch.no_grad() mode on an AMD Ryzen 7 7435HS CPU. Prior to timing, 20 warm‐up forward passes were performed to allow the runtime to reach a steady state and eliminate cold‐start overheads.

Latency was then recorded over 200 consecutive forward passes using Python's time.perf_counter() wall‐clock timer, and the mean latency across all runs is reported.

Post‐training static quantization was additionally applied to convert model weights from 32‐bit floating‐point representation to 8‐bit integer (INT8) representation using PyTorch's qnnpack backend. The quantized models were re‐evaluated to determine whether reduced numerical precision resulted in measurable changes in classification performance while lowering memory requirements. This analysis was performed primarily on the student models, with the teacher retained as a reference for relative complexity and compression.

Latency measurements were obtained only for the FP32 models; inference latency was not re‐measured for the INT8 variants. Flash‐memory requirements were estimated analytically from parameter counts, assuming 4 bytes per parameter for FP32 and 1 byte per parameter for INT8. Peak SRAM was estimated separately from the maximum inference‐time activation footprint for batch size 1 under the corresponding numerical precision, rather than through direct hardware profiling.

Accordingly, the deployment study should be interpreted as simulation‐based profiling rather than direct validation on a physical embedded device. Nevertheless, it provides a structured basis for assessing whether the proposed student architectures are plausibly compatible with resource‐constrained wearable ECG inference. The exact profiling setup and implementation details are reported in Section 3.15.

Prior work has demonstrated that quantized compact CNNs can be validated for real‐time inference on low‐cost embedded hardware using hardware‐in‐the‐loop testing, achieving practical frame rates on devices such as the NVIDIA Jetson Nano [21]. Given the compact size and computational complexity of the proposed student models, real‐time execution on a cost‐effective embedded platform and potentially on a suitably capable microcontroller, appears feasible; however, this cannot be established from CPU‐based profiling alone and requires direct hardware‐in‐the‐loop validation on the target device.

Implementation Details

All five ablation models (M1–M5) used the same Student CNN backbone and were trained under a controlled experimental protocol designed to vary only the intended supervision and augmentation components. Specifically, the presence or absence of output‐level KD, MGD feature‐reconstruction loss and noise augmentation was varied systematically across ablation settings, while the remaining optimization choices were kept as consistent as possible. The exact training hyperparameters, masking settings and reproducibility controls are summarized in Table 2.

The experimental pipeline was implemented in Python 3.10 using PyTorch 2.x. ECG records were parsed with the wfdb library, signal preprocessing and synthetic noise generation were implemented with SciPy, and evaluation metrics were computed using scikit‐learn. The model complexity in terms of MACs was profiled using THOP, and post‐training quantization was performed using torch.ao.quantization with the qnnpack backend.

Training was repeated across multiple random seeds as reported in Table 2 to reduce seed‐specific bias and improve the reliability of the reported results.

Statistical Analysis

To quantify uncertainty arising from training stochasticity, the teacher model and all five student variants (M1–M5) were each trained under five random seeds (42, 123, 777, 2024 and 31415). The five independently trained teacher models were used to characterize variability in teacher performance. For student distillation, however, a single fixed teacher checkpoint corresponding to seed 42 was used for all M2–M5 training runs, while the student random seed was varied across the five runs.

Consequently, the reported across‐seed uncertainty for the student models reflects student‐side training stochasticity conditional on a fixed teacher checkpoint. Results are reported as mean ± standard deviation across the five runs. As the primary seed‐level uncertainty estimate, a 95 % t‐interval was computed from the five seed‐level macro F1 scores using a t‐distribution with four degrees of freedom (t0.975,4=2.776).

This approach captures run‐to‐run variability arising from random initialization and stochastic optimization on a fixed held‐out test set.

The primary classification metric was macro F1‐score, and the primary robustness endpoint was mean macro F1 degradation from the clean baseline across all 15 noise conditions (three noise types × five SNR levels). The 0 dB condition was additionally treated as the primary severe‐noise stress test. For key model comparisons, paired bootstrap resampling was used to quantify uncertainty in the observed metric differences.

Because all models are evaluated on the same held‐out test records, a paired procedure is appropriate: for each bootstrap replicate (n=1000), the same subset of test records was resampled with replacement for both models simultaneously, the metric of interest was recomputed for each replicate and the difference (M5 minus comparator) was recorded. The 2.5th and 97.5th percentiles of the resulting distribution define the 95 % bootstrap CI for the difference. To quantify uncertainty in the aggregate robustness endpoint across corruption conditions, an additional paired bootstrap was performed over the 15 condition‐level degradation values using 10,000 resamples.

This analysis quantifies variability across the evaluated corruption conditions and is distinct from the record‐level bootstrap applied to the 0 dB test predictions. Record‐level paired bootstrap comparisons were performed for three clean‐data metrics (macro F1, Hamming accuracy and exact‐match accuracy) for M2 versus M5, and for macro F1 at 0 dB SNR for M2 versus M5 and M3 versus M5 across each noise type. The additional condition‐level bootstrap was applied to the aggregate mean macro F1 degradation across the 15 noisy test conditions.

For the record‐level bootstrap, a confidence interval excluding zero was interpreted as evidence that the observed model difference is not attributable to test‐record sampling variability alone. The condition‐level bootstrap was used to characterize the consistency of aggregate degradation differences across the 15 evaluated corruption conditions and should not be interpreted as patient‐level inference or as evidence of generalization to unseen corruption domains. Record‐level paired bootstrap comparisons were performed using ensemble predictions obtained by averaging the sigmoid output probabilities across all five seed checkpoints (42, 123, 777, 2024, 31415) prior to thresholding and metric computation.

This ensemble‐based macro F1 can differ numerically from the mean of the five individually computed per‐seed macro F1 scores reported in Tables 3 and 5, which are presented in Sections 4.2 and 4.3 respectively.

Results and Analysis

Overview of Experimental Evaluation

This section reports the results of the controlled ablation study for multi‐lead‐to‐single‐lead ECG classification. The objective is to determine whether masked generative distillation (MGD) improves the robustness of a lightweight single‐lead student model, and whether any observed gains can be distinguished from the effects of conventional knowledge distillation (KD) and noise augmentation.

The experimental analysis is organized along four dimensions. First, baseline classification performance is evaluated on the clean PTB‐XL test set. Second, robustness is assessed under synthetic wearable‐relevant noise at multiple signal‐to‐noise ratio (SNR) levels.

Third, teacher–student interpretability alignment is examined using Grad‐CAM visualization and SSIM analysis. Fourth, deployment‐oriented efficiency is evaluated through simulation‐based profiling of model size, computational cost, latency, memory footprint and post‐training quantization behaviour.

The evaluated models include the 12‐lead teacher and five single‐lead student variants from the ablation study: Teacher model (12‐lead ResNet‐18): High‐capacity reference model trained on full 12‐lead ECG signals.M1 (supervised only): Student trained using ground‐truth labels only; establishes the reduced‐lead supervised baseline.M2 (supervised + KD): Student trained with conventional output‐level knowledge distillation.M3 (supervised + KD + noise augmentation): M2 with corruption‐aware training applied to the student input.M4 (supervised + KD + MGD): Student trained with the combined task, KD and masked generative reconstruction losses, but without noise augmentation.M5 (supervised + KD + MGD + noise augmentation): Full proposed method, combining supervised learning, KD, MGD and noise augmentation.

The teacher provides a reference for classification performance, while the student variants share the same single‐lead backbone and differ only in the supervised, distillation and augmentation components defined in the ablation design. These comparisons provide a structured basis for assessing how output‐level teacher supervision, masked feature reconstruction and corruption‐aware training affect performance, robustness, interpretability and efficiency in reduced‐lead ECG classification.

Baseline Classification Performance on Clean ECG Data

Table 3 summarizes the clean‐test classification results for the teacher and all five ablation variants. Teacher and student results are reported as mean ± standard deviation across five independent training runs (random seeds 42, 123, 777, 2024 and 31415). A 95% t‐interval computed from the five seed‐level macro F1 scores is also reported to characterize run‐to‐run variability arising from random weight initialization and stochastic optimization.

Across the five student variants, macro F1‐scores lie within a narrow range (0.638–0.645), and the corresponding confidence intervals overlap substantially. The student variants therefore achieve closely similar clean‐condition performance in terms of the primary metric. At the same time, the full proposed method shows small reductions in Hamming accuracy and exact‐match accuracy relative to the conventional KD baseline, indicating minor trade‐offs in some thresholded secondary metrics.

Overall, these results show that the addition of KD, MGD and noise augmentation does not materially improve clean‐condition classification performance, and that the principal value of the proposed framework lies instead in robustness under signal corruption, which is examined in the following subsections.

Table 4 reports per‐class F1‐scores. Hypertrophy (HYP) remains the most challenging superclass for all student models, with F1 values between 0.415 and 0.431, consistent with the difficulty of inferring voltage‐amplitude‐related criteria from a single lead. Among the student variants, M5 achieves the numerically highest HYP F1 (0.431) and CD F1 (0.684) among the student variants, although these differences are small and were not subjected to separate class‐wise statistical testing.

M3 shows the smallest across‐seed standard deviations (≤0.006 across all classes), indicating particularly stable training, while M1 and M2 show the highest variance, especially for the MI and HYP classes. Student‐model macro AUROC values (0.856–0.859) are notably higher than the corresponding macro F1‐scores. This difference is expected in a thresholded multi‐label setting, where AUROC reflects threshold‐independent class separability, whereas F1 depends directly on the fixed sigmoid decision threshold and is more sensitive to class imbalance.

In this study, the same thresholding rule was used for all models, enabling direct comparison under matched evaluation conditions. The 12‐lead teacher achieves a macro F1‐score of 0.715 and a macro AUROC of 0.907, providing a reference for reduced‐lead classification. Overall, the clean‐test results are broadly consistent with prior PTB‐XL reports for ResNet‐based teacher models and lightweight single‐lead student models.

The principal question, therefore, is not whether MGD improves clean‐condition performance, but whether it improves robustness under synthetic wearable‐relevant noise, which is analysed next.

For external context, the clean‐test macro F1‐score achieved by the full proposed method (0.6449) can be compared with previously reported single‐lead results on the PTB‐XL five‐superclass task. Qin et al. benchmarked several single‐lead distillation and knowledge‐transfer methods using Lead I, reporting F1‐scores from 59.8% for attention‐based KD to 62.6% for their MVKT‐ECG framework [14]. The proposed method therefore exceeds the reported MVKT‐ECG F1‐score by approximately 1.9 percentage points, although this literature‐based comparison should be interpreted cautiously because of differences in lead selection (Lead II versus Lead I) and evaluation protocol.

More recently, Sepahvand et al. proposed a multi‐teacher decomposed feature‐distillation framework that transfers information from 12‐lead teachers to a single‐lead student using Tucker decomposition [13], reporting 96.48% accuracy on the Chapman ECG dataset. Because that study uses a different dataset, classification formulation and evaluation metric, its numerical result is not directly comparable with the present PTB‐XL macro F1‐score; nevertheless, it provides a relevant recent methodological reference for sophisticated multi‐lead‐to‐single‐lead knowledge transfer. These comparisons position the proposed method within recent reduced‐lead distillation research while highlighting its additional emphasis on robustness under controlled signal corruption.

Paired bootstrap resampling of the held‐out test records confirms that the macro F1 difference between M5 and M2 on clean data (+0.003) is not distinguishable from test‐set sampling variability, with a 95 % bootstrap CI of [−0.001,+0.010] that includes zero. For Hamming accuracy, the difference is −0.009 (95 % CI [−0.012,−0.007]), and for exact‐match accuracy it is −0.019 (95 % CI [−0.027,−0.012]); both CIs exclude zero, indicating that M5 yields marginally lower label‐level and set‐level accuracy than M2 on clean data. These results are consistent with the study's primary claim that the proposed framework targets robustness under signal degradation, and the bootstrap analysis does not demonstrate a clear clean‐condition macro F1 advantage from adding MGD and noise augmentation to conventional KD.

Ablation Analysis: Assessing Component Contributions

To determine whether the observed robustness gains arise from MGD, from noise augmentation, or from their combination, this subsection analyses pairwise differences across the ablation variants. Table 5 summarizes macro F1‐scores at 0 dB SNR, corresponding to the most severe corruption condition, for all five student models across the three evaluated noise types. The ablation results support five main observations.

First, the transition from M1 to M2 shows that conventional output‐level KD provides only a marginal clean‐condition improvement (+0.004 macro F1) and little robustness benefit under severe corruption. Averaged across all 15 noisy test conditions, the mean macro F1 degradation from the clean baseline is 0.084 for M1 and 0.098 for M2, indicating that conventional output‐level KD does not improve aggregate robustness over purely supervised training under the evaluated conditions.

Second, the comparison of M2 and M4 assesses the effect of introducing the MGD‐based training configuration in the absence of noise augmentation. Relative to M2, M4 improves macro F1 at 0 dB by +0.049 under baseline wander, +0.056 under muscle artefact and +0.025 under electrode motion noise. These gains suggest that feature reconstruction contributes to robustness even when the student is trained only on clean input signals.

Third, the comparison of M2 and M3 shows that noise augmentation alone provides a substantial robustness benefit. Across all 15 noisy test conditions, the mean macro F1 degradation from the clean baseline decreases from 0.098 in M2 to 0.036 in M3, indicating that corruption‐aware training is an important component of robust reduced‐lead performance.

Fourth, the comparison of M2 and M5 at 0 dB SNR directly evaluates whether the full proposed method outperforms the conventional KD baseline under severe signal degradation. Under baseline wander, M5 achieves a macro F1 of 0.623 compared with 0.405 for M2, an absolute improvement of +0.218. Under muscle artefact, M5 retains 0.605 against M2's 0.259 (+0.346).

Under electrode motion noise, M5 achieves 0.590 compared with M2's 0.345 (+0.245). Paired bootstrap resampling of the 0 dB test records confirms that these differences are not attributable to test‐set sampling variability. The observed macro F1 differences (M5 minus M2) are +0.233 (95 % CI [+0.221,+0.244]) under baseline wander, +0.391 (95 % CI [+0.377,+0.405]) under muscle artefact and +0.254 (95 % CI [+0.242,+0.264]) under electrode motion noise; all three confidence intervals exclude zero by a wide margin.

Fifth, the comparison of M3 and M5 provides the most direct assessment of the MGD‐enabled training configuration when noise augmentation is already present. The paired bootstrap differences (M5 minus M3) at 0 dB are +0.085 (95 % CI [+0.075,+0.095]) under baseline wander, +0.159 (95 % CI [+0.147,+0.171]) under muscle artefact and +0.060 (95 % CI [+0.051,+0.069]) under electrode motion noise; all three confidence intervals exclude zero, showing that the MGD‐enabled training configuration achieves a consistent robustness advantage over augmentation‐only training under the evaluated severe‐noise conditions.

Averaged across all 15 noisy test conditions, the mean macro F1 degradation from the clean baseline is 0.098 for M2 and 0.012 for M5, a reduction of 87 %. The difference in aggregate degradation between M2 and M5 is 0.086 (95 % bootstrap CI [0.036, 0.143]), with the bootstrap interval remaining above zero across resampling of the evaluated corruption conditions. The comparison of M3 and M5 across the same 15 conditions yields mean degradation values of 0.036 and 0.012, respectively; the difference of 0.024 (95 % bootstrap CI [0.005, 0.048]) likewise excludes zero, indicating that the MGD‐enabled training configuration provides an additional robustness advantage over noise augmentation alone under the evaluated corruption conditions.

Both confidence intervals were obtained via a paired bootstrap procedure (10,000 resamples over the 15 noisy conditions). Taken together with the overlapping clean‐condition results reported in Section 4.2, these findings support the interpretation that the primary advantage of the full proposed method lies in improved robustness under signal degradation rather than in improved clean‐data classification performance.

Robustness Evaluation Under Noisy Conditions

Wearable ECG monitoring is frequently affected by signal degradation arising from motion‐related artefacts, muscle interference and instability at the electrode‐skin interface. To assess robustness under such wearable‐relevant corruption, synthetic noise was added to the test signals using three artefact types: baseline wander, muscle artefact and electrode motion noise. Each noise type was evaluated separately across signal‐to‐noise ratio (SNR) levels from 30 to 0 dB, where lower SNR values correspond to more severe corruption.

The primary robustness endpoint is mean macro F1 degradation from the clean‐test baseline across all 15 conditions, and the 0 dB condition serves as the primary stress test. The resulting robustness trends for the student models are shown in Figure 4.

Across all three noise families, both KD‐ and MGD‐based students remain close to their clean‐test baselines under mild corruption (30 and 20 dB). However, their behaviour diverges as the noise level increases. The conventional KD model (M2) exhibits a progressively steeper decline in macro F1‐score, whereas the full MGD‐based model with noise augmentation (M5) retains substantially stronger performance under severe corruption.

At the most challenging condition (0 dB), the robustness advantage of M5 is pronounced. Under muscle artefact noise, M2 falls to a macro F1 of 0.259, while M5 retains 0.605, corresponding to an absolute difference of +0.346. Under electrode motion noise, M2 falls to 0.345 and M5 retains 0.590 (+0.245).

Under baseline wander, M2 falls to 0.405 while M5 maintains 0.623 (+0.218). Expressed relative to their respective clean‐condition baselines, M5 retains 96.5%, 93.9% and 91.5% of its clean macro F1 under baseline wander, muscle artefact and electrode motion noise, respectively, compared with 63.2%, 40.4% and 53.8% for M2.

Figure 5 complements the SNR curves by summarizing mean macro F1 degradation from the clean baseline per noise type, averaged across all SNR levels. The overall pattern is consistent with the component‐level analysis in Section 4.3: M2 shows the largest mean degradation among the distillation‐based variants, M5 shows the smallest and M3 occupies an intermediate position that reflects the benefit of noise augmentation without masked generative reconstruction.

These results are consistent with the component‐level interpretation established in Section 4.3. Noise augmentation alone substantially improves robustness (M3 vs. M2), while the gap between M3 and M5 supports an additional robustness advantage for the MGD‐enabled training configuration when corruption‐aware training is already present.

M4 also performs better than M2 under severe corruption, providing complementary evidence that the MGD‐enabled configuration is beneficial even without explicit noise augmentation.

Interpretability Alignment Between Teacher and Student Models

In addition to classification performance, it is important to assess whether the student model preserves attention patterns that are structurally aligned with those of the teacher. To this end, interpretability analysis was performed using Grad‐CAM, which produces a 1D heatmap indicating the temporal regions of the ECG signal that contribute most strongly to a model's prediction. By comparing teacher and student Grad‐CAM profiles, it is possible to examine whether the reduced‐lead student emphasizes similar waveform regions to the full 12‐lead teacher.

Figure 6 presents visual comparisons for two representative ECG examples under both clean input conditions and severe corruption (0 dB muscle artefact noise).

The qualitative examples in Figure 6 show that, under clean conditions, both the student models attend to temporal regions that broadly overlap with those highlighted by the teacher. Under severe noise, however, the two students behave differently. The KD student's (M2) attention becomes more diffuse and less clearly aligned with the teacher heatmap, whereas the MGD‐based student (M5) retains more localized activation patterns that remain temporally closer to the teacher's response.

These visual observations are consistent with the hypothesis that the full MGD‐plus‐noise training framework helps retain greater teacher‐aligned attention under input degradation than conventional KD.

To quantify this effect, the SSIM was computed between teacher and student Grad‐CAM heatmaps over the evaluation subset. Higher SSIM values indicate stronger structural similarity between the temporal attention patterns of the two models. Table 6 reports the aggregate SSIM scores under clean, moderate‐noise and severe‐noise conditions.

Under clean conditions, both students show broadly comparable alignment with the teacher (M2: 0.101, 95% CI [0.078, 0.126]; M5: 0.110, 95% CI [0.083, 0.135]). As noise severity increases, the two students diverge consistently across all three noise types. Under muscle artefact at 10 dB, M2's mean SSIM decreases to 0.068 ([0.045, 0.089]) while M5 retains 0.102 ([0.075, 0.128]).

At 0 dB, the gap widens further: M2's mean SSIM falls to 0.017 ([−0.001, 0.033]), with its confidence interval including zero, indicating that structural alignment is no longer reliably distinguishable from the absence of alignment at this corruption level. M5 retains a mean SSIM of 0.081 ([0.057, 0.104]), whose confidence interval remains entirely above zero.

The absolute SSIM values are modest even under clean conditions (M2: 0.101; M5: 0.110), indicating that neither student reproduces the teacher's Grad‐CAM structure closely in an absolute sense. This limited agreement is plausible given the substantial mismatch between the models: the teacher operates on 12‐lead input with a deeper architecture, whereas the student receives only Lead II and uses a much smaller convolutional backbone. Their Grad‐CAM representations therefore arise from different input information and feature hierarchies, and high point‐wise structural similarity should not be assumed.

Accordingly, the SSIM analysis should be interpreted as a measure of relative teacher–student attention similarity rather than evidence of strong absolute attention preservation. Under increasing corruption, M5 consistently retains higher SSIM than M2 across the evaluated noise conditions, indicating that the full MGD‐plus‐noise training framework reduces the loss of teacher‐aligned attention relative to conventional KD. At 0 dB, for example, M5 yields higher SSIM for baseline wander (0.077 vs.

0.035), muscle artefact (0.081 vs. 0.017) and electrode motion (0.052 vs. 0.020).

These results therefore support greater relative preservation of teacher‐aligned temporal attention under severe corruption, while the low absolute SSIM values indicate that substantial teacher–student attention differences remain.

The qualitative and quantitative evidence suggests that M5 retains greater teacher‐aligned attention similarity than M2 as signal quality deteriorates; however, the modest absolute SSIM values indicate that this alignment remains limited. This finding complements the robustness results in Section 4.4 by indicating that the improved noisy‐condition classification performance of M5 is accompanied by more stable attention to teacher‐aligned temporal waveform regions.

Deployment Feasibility Analysis

A central objective of this study is to determine whether the reduced‐lead student models offer a practically favourable efficiency profile relative to the larger 12‐lead teacher. To this end, the models were analysed using simulation‐based deployment profiling, with the results summarized in Table 7.

The student architectures are substantially more compact than the teacher network. Relative to the 12‐lead ResNet‐18 teacher (3.85M parameters, 176M MACs), both student models contain approximately 85K parameters and require only 9M MACs, corresponding to a compression ratio of approximately 45×. This reduction indicates a large decrease in both model size and computational complexity, which is desirable for resource‐constrained wearable inference.

Latency and memory estimates show the same trend. Under the evaluated CPU‐based profiling setup, inference latency decreases from 7.6 ms for the teacher to 1.0–1.1 ms for the student models. Estimated peak SRAM usage for both students is 0.065 MB in FP32 and 0.016 MB in INT8, while FP32 flash‐memory requirements decrease from 14.69 MB for the teacher to 0.33 MB for the student models.

Together, these results indicate that the distilled single‐lead architectures are substantially more compatible with constrained inference settings than the full 12‐lead teacher.

Post‐training static quantization to INT8 further reduces storage requirements while causing only minimal performance change. The macro F1 decrease after quantization is 0.0001 for M2 and 0.0011 for M5, indicating that both student models are largely insensitive to reduced numerical precision under the evaluated setup. This makes INT8 deployment particularly attractive for the reduced‐lead models, as it improves memory efficiency without materially altering classification performance.

From a real‐time deployment perspective, the student requires approximately 9 million MACs per 10‐s ECG window, compared with 176 million MACs for the teacher. For non‐overlapping 10‐s windows, this corresponds to an average computational requirement of approximately 0.9 million MACs/s; even if inference were updated once per second using a sliding 10‐s window, the corresponding requirement would be approximately 9 million MACs/s. Together with the approximately 85K parameters and the estimated INT8 weight footprint of about 0.08 MB, these characteristics indicate that the student is a realistic candidate for evaluation on cost‐effective embedded hardware.

Nevertheless, the reported host‐CPU latency cannot be extrapolated directly to a microcontroller, because execution speed also depends on processor architecture, memory bandwidth, inference libraries and hardware acceleration. Accordingly, real‐time microcontroller feasibility remains to be established through direct hardware‐in‐the‐loop implementation. The HIL study of Khosravian et al. [21], which demonstrates real‐time execution of a quantized CNN on embedded NVIDIA hardware, provides a relevant methodological precedent for such future validation.

Overall, the deployment‐oriented profiling results show that the student models achieve substantial reductions in size, complexity, latency and memory requirements relative to the teacher, while remaining stable after INT8 quantization. These findings should be interpreted as simulation‐based evidence of deployment potential rather than direct validation on a physical embedded device, but they nevertheless support the practicality of the proposed single‐lead architectures for resource‐constrained wearable ECG inference.

Summary of Findings

Taken together, the experimental results show that the main advantage of the proposed framework lies in improved robustness under signal corruption rather than higher clean‐condition classification performance. On the clean PTB‐XL test set, all student variants achieve closely similar macro F1‐scores, indicating that the addition of KD, masked generative reconstruction and noise augmentation does not materially alter baseline reduced‐lead performance under matched evaluation conditions.

The ablation analysis shows that robustness gains arise from both corruption‐aware training and masked generative reconstruction, with the strongest performance obtained when these components are combined. In particular, the full proposed method (M5) consistently exhibits the lowest degradation under synthetic wearable‐relevant noise, outperforming both the conventional KD baseline and the augmentation‐only variant under severe corruption. These findings support the interpretation that MGD provides complementary benefit beyond noise augmentation alone.

Interpretability analysis further suggests that M5 (MGD + noise) retains greater teacher‐aligned attention similarity than M2 under signal degradation. Although the absolute SSIM values remain modest, both the qualitative Grad‐CAM examples and the quantitative alignment scores are consistent with more stable teacher‐aligned attention under noisy conditions.

From a deployment‐oriented perspective, the student architectures achieve large reductions in parameter count, computational complexity, latency and memory footprint relative to the 12‐lead teacher, while showing only minimal performance change after INT8 quantization. Accordingly, the results support the use of reduced‐lead distilled models as a practical direction for resource‐constrained ECG inference, while also indicating that the evidence should be interpreted as simulation‐based deployment profiling rather than direct embedded‐device validation.

Conclusions and Future Work

This study investigated whether diagnostically useful knowledge from a high‐capacity multi‐lead ECG model can be transferred to a lightweight single‐lead model while preserving robustness under wearable‐relevant signal corruption. To address this problem, a teacher–student framework was evaluated on the PTB‐XL dataset using controlled ablation across supervised learning, conventional KD, MGD and noise augmentation.

The experimental results show that the main benefit of the proposed framework does not lie in higher clean‐condition classification performance. On the clean test set, the student variants achieved closely similar macro F1 performance, although the full proposed method showed small reductions in Hamming accuracy and exact‐match accuracy relative to the conventional KD baseline. Instead, the primary advantage emerges under synthetic noise corruption.

Across the ablation study, the strongest robustness was obtained when MGD was combined with noise augmentation, indicating that masked feature reconstruction and corruption‐aware training provide complementary benefits. Under severe noise conditions, the full proposed method consistently preserved higher macro F1‐scores than the conventional KD baseline while remaining broadly comparable in the primary clean‐condition metric.

M5 retained greater teacher‐aligned attention similarity than the conventional KD baseline when the input signal was degraded. Although the absolute SSIM values remained modest, both the qualitative Grad‐CAM examples and the quantitative alignment scores were consistent with more stable teacher‐aligned attention under noisy conditions.

From a deployment‐oriented perspective, simulation‐based profiling showed that the lightweight single‐lead student models achieved substantial reductions in parameter count, computational complexity, latency and memory footprint relative to the 12‐lead teacher, with only minimal performance change after INT8 quantization. These findings support the use of reduced‐lead distilled models as a practical direction for resource‐constrained ECG inference under the evaluated conditions.

Overall, the results indicate that masked generative distillation is a promising component of robust reduced‐lead ECG modelling, particularly when combined with noise augmentation. More broadly, the findings show that teacher–student transfer can narrow part of the gap between diagnostically rich multi‐lead ECG systems and deployment‐oriented single‐lead wearable models under synthetic wearable‐relevant noise conditions.

Limitations and Future Work

Several limitations should be considered when interpreting the present findings. First, the experiments were conducted on a single benchmark dataset, PTB‐XL. Although PTB‐XL is large and widely used, additional validation on other ECG datasets and diagnostic settings is required to assess the generalisability of the proposed framework across different populations, recording conditions and label definitions.

Accordingly, the present experiments establish within‐dataset generalization to unseen PTB‐XL patients but do not establish cross‐dataset or unseen‐domain generalization.

Second, robustness was evaluated using synthetic wearable‐relevant noise rather than real ambulatory ECG recordings. While this controlled design is useful for isolating the effects of baseline wander, muscle artefact and electrode motion, it does not fully capture the variability of real‐world acquisition conditions. Future work should therefore examine the proposed method on externally collected wearable ECG data and under broader forms of signal corruption and domain shift.

Related work in autonomous perception has shown that synthetic domain expansion using GAN‐based generation and environmental modelling, together with augmentation strategies, can improve robustness and cross‐domain generalization to adverse conditions not adequately represented in the training data [22]. Although the application domain differs from ECG, analogous domain‐generalization strategies could be explored by incorporating rare or naturally occurring ECG artefacts, recordings from heterogeneous wearable devices and broader acquisition conditions alongside the controlled synthetic noise considered here.

Third, the deployment analysis was based on simulation‐oriented profiling rather than execution on a physical embedded platform. Although the results indicate large reductions in size, complexity, latency and memory requirements, further work is needed to validate these gains on real IoMT or microcontroller‐class hardware, including measurements of runtime, power consumption and end‐to‐end system behaviour.

Fourth, the present study considered a single reduced‐lead configuration based on Lead II. Additional work is needed to determine how the proposed framework behaves under alternative lead selections, multi‐sensor wearable configurations or more flexible teacher–student transfer settings. Related extensions may also include multi‐teacher distillation, alternative robustness‐enhancement strategies and more advanced explainability methods to better characterize the physiological relevance of the learned representations.

These limitations define a clear direction for future research: to validate the framework beyond PTB‐XL, under real wearable conditions, on physical low‐power hardware and across broader reduced‐lead and interpretability settings.

Author Contributions

Anandu Shivakumar: Conceptualization, data curation, formal analysis, methodology, writing – original draft. Obada Al‐Khatib: Methodology, project administration, resources, supervision, writing – original draft. Ghalia Nassreddine: Formal Analysis, investigation, software, validation.

Abeer ElKhouly: Data Curation, resources, visualization, validation, writing – review & editing.

Funding

The authors have nothing to report.

Conflicts of Interest

The authors declare no conflicts of interest.