Paroxysmal Atrial Fibrillation Events Detection from Dynamic ECG Recordings: The 4th China Physiological Signal Challenge 2021

Invitation Letter (in English)        Invitation Letter (in Chinese)

Welcome to join CPSC 2021 Wechat Group for free discussion and information sharing. Please add the conference wechat and remark with "Please invite me into the Challenge Wechat Group"

[2021-4-1]The training set in phase II has already been updated. Please download from
( ).

Data Download
The 4th China Physiological Signal Challenge 2021 (CPSC 2021) aims to encourage the development of algorithms for searching the paroxysmal atrial fibrillation (AF) events from dynamic ECG recordings.

ECG signal provides an important role in non-invasively monitoring and clinical diagnosis for cardiovascular disease (CVD). AF is the most frequent arrhythmia, but PAF often remains unrecognized[1, 2]. Early screening and early detection of paroxysmal AF are particularly important. It is of great value for AF surgery options, drug intervention, and the diagnosis and treatment of various clinical complications [3].

Although accurate detection of paroxysmal AF is very important, there is currently no algorithm that can efficiently measure the onsets and offsets of AF events in dynamic or wearable ECGs [4]. Previous AF detection algorithms usually focus on the classification of AF rhythm, such as entropy feature-based [5, 6] or machine learning-based methods [7, 8], without the location of onsets and offsets of AF events. Thus, the clinical significance for the personalized treatment and management of AF patients is limited. In clinical applications, other abnormal rhythms can significantly influence the accurate identification of AF rhythm. In this year’s challenge, we focus on the detection of paroxysmal AF events from dynamic ECGs. A new dynamic ECG database containing episodes with totally or partly AF rhythm, or non-AF rhythm was constructed, to encourage the development of the more efficient and robust algorithms for paroxysmal AF event detection.

Challenge Data
Data are recorded from 12-lead Holter or 3-lead wearable ECG monitoring devices. Challenge data provides variable-length ECG fragments extracted from lead I and lead II of the long-term source ECG data, each sampled at 200 Hz. To avoid ambiguity in annotation, AF event is limited to be no less than 5 heart beats.
The training set in the 1st stage consists of 716 records, extracted from the Holter records from 12 AF patients and 42 non-AF patients (usually including other abnormal and normal rhythms).
The training set in the 2nd stage consists of 707 records from 37 AF patients (18 PAF patients) and 14 non-AF patients.
The test set comprises data from the same source as the training set as well as different data source. We ensure that at least one test subset was collected by a different ECG monitoring system compared with the training set. Same as in previous years, we are not planning to release the test set at any point.
All data is provided in WFDB format and the annotations are standardized according to PhysioBank Annotations. The annotation includes the beat annotations (R peak location and beat type), the rhythm annotations (rhythm change flag and rhythm type) and the diagnosis of the global rhythm. Please refer to the example code entry of the challenge for specific data load functions. Note that the flag of atrial fibrillation and atrial flutter (‘AFIB’ and ‘AFL’) in annotated information are seemed as the same type when scoring the method.
Please download the training data from Training Set I and Training Set II.

For this year’s Challenge, we developed a new scoring metric that awards the correct detection of paroxysmal AF events. The scoring metric includes two steps:
  • the first step is to classify the rhythm types: non-AF rhythm (N), persistent AF rhythm (AFf) and paroxysmal AF rhythm (AFp).
  • the second step is to locate the onset and offset for any AF episode prediction.
The participants are only required to provide the final onset and offset locations for AF episodes. If the current ECG record is classified as AFf, the provided onset and offset locations should be the first and last record points. If the ECG record is classified as N, the answer should be an empty list.

Figure 1. The scoring matrix for the global rhythm classification.
A scoring matrix (as shown in Figure 1) is designed to firstly reward the correct answers and penalize the misdiagnosis for the three rhythm types. Let Ur be the score for this step evaluation.

Figure 2. Grading instance for onset and offset detections of AF episodes.

Then the detection of onsets and offsets of AF episode will be scored (only reward), and the reward is based on the consistence between the detected onsets (or offsets) and the annotated answers. Figure 2 gives the detailed explanation. Let Ue be the scores for evaluating the detection of onsets and offsets. We reward score Ue with +1 if the detected onset (or offset) is within ±1 beat of the annotated position, and score Ue with +0.5 if within ±2 beats. Note that a paroxysmal AF record may contain multiple AF episodes and the onset and offset points will be separately scored (rewarded). A weight is assigned to Ue, where Ma and Mr are the amount of AF episodes from annotated answers and the predictions, for each record respectively. Over-estimation of AF episodes may cause lower scores.
The final score U is the sum of Ur and Ue, and is defined as:

where N is the amount of the test records. The score is calculated for each record and then averaged for the entire test set.
For example, for an ECG record with only one paroxysmal AF episode, if the algorithm classified the record correctly, the first score Ur=1. Then the predicted onset and offset of AF episode will be checked. If the both onset and offset are within ±1 beat of the annotated position, the second score Ue=2. Thus, the final score U=3.

Sample Submission
A simple example algorithm is provided and may be used as a template for your own submission. Python, MATLAB implementations are available. Similarly, to last year’s Challenge, teams must submit both the code for their models and the code for training their models (if you are using supervised learning methods).
Note that the format of the answer file must follow the entry code. Please save the answer file for each record and name one after the record name. If your development environment is Python, please save the result as a ‘.json’ file. The format is as {‘predict_endpoints’: [[s0, e0], [s1, e1], …, [sm-1, em-2]]. If your development environment is MATLAB, please save the variable, ‘predict_endpoints’, as a ‘.mat’ file.

Preparation and submission instructions
1. Create a private GitHub or Gitlab repository for your code. We recommend cloning our example code and replacing it with your code. Add cpsc-helper as a collaborator to your repository.
2. Add your algorithm code to your repository. Like the example code, your code must be in the root directory of the master branch.
3. Do not include extra files that are not required to create and run your code, such as the training data.
4. Follow the instructions for the programming language of your submission.
5. Register through We will clone your repository using the HTTPS URL that ends in .git. On GitHub, you can get this URL by clicking on “Clone or download” and copying and pasting the URL, e.g., Please see here for an example.
6. We will put the scores for successful entries on the leaderboard. The leaderboard will publicly show your team name, run time, and score. Note that the limitation of averaged code-run-time should not exceed 1 minute pretesting record. If the code running time exceeds the limit, a timeout error will be feedback.

Python-specific instructions
1. Using our Sample Python entry ( as a template, format your code in the following way. Consider downloading this repository, replacing our code with your code, and adding the updated files to your repository.
2. requirements.txt: Add Python packages to be installed with pip. Specify the versions of these packages that you are using on your machine. Remove unnecessary packages, such as Matplotlib, that your submitted code does not need.
3. AUTHORS.txt, LICENSE.txt, Update as appropriate. Please include your authors.
4. Update this script to load and run your trained model. While testing, we will firstly run (python TEST_SET_PATH RESULT_SAVE_PATH) to generate the prediction results for next-step scoring.
5. Do not change this script. It is a script containing the functions for the baseline method.
6. Do not change this script. It is a sample script for reference and help participants test their code in local environment. Another standardized score will be used in the back-end server.

MATLAB-specific instructions
1. Confirm that your MATLAB code compiles and runs in MATLAB R2020B or R2021A.
2. Using our sample MATLAB entry ( as a template, format your code in the following way. Consider downloading this repository, replacing our code with your code, and adding the updated files to your repository.
3. AUTHORS.txt, LICENSE.txt, Update as appropriate. Please include your authors. Unfortunately, our submission system will be unable to read your README file to change how we run your code.
4. comp_cosEn.m: Do not edit this script. It extracts extract features from the ECG recordings.
5. qrs_detect.m: Do not edit this script. It extracts the position of the R peak.
6. challenge.m: Update this script to load and run your model weights and any parameters from files in your submission. It takes the input sample path, and returns predict endpoints.
7. Result.m: Update this script to load and run your model weights and any parameters from files in your submission. It takes the input sample name, sample path, save path, and returns predict endpoints and saves to a specific path with a file name as the result of submission.
8. Add your code to the root/base directory of the master branch of your repository.
9. We will download your code, compile it using the MATLAB compiler: Outputting each result (mcc -m challenge.m –a .) and saving to a specific path with a file name as the result of submission (mcc -m Result.m -a .), and run them on backend server.

Awards and Rules
The winner will be selected on the basis of the obtained final U on the hidden test data. The first three for each Event challenging will receive certificates and generous bonuses:
• First prize: Certificate plus bonus of RMB 15,000
• Second prize: Certificate plus bonus of RMB 10,000
• Third prize: Certificate plus bonus of RMB 5,000

We welcome all the individual or research group around the world to attend the challenge. To be eligible for the award, please do the following:
(1) Register the team before June 1, 2021.
(2) Submit at least one open-source entry that can be scored before August 1, 2021.
(3) Attend ICBEB 2021 (November 15-18, 2021) in Suzhou, China and present your work there.

Please do not submit analysis of this year’s Challenge data to other Conferences or Journals until after ICBEB 2021 has taken place, so the competitors are able to discuss the results in a single forum.

Important Dates
20 March, 2021 -- Challenge open
June 1, 2021 -- Deadline for team registration to join the challenge
August 1, 2021 -- Deadline for submitting the first challenge open-source entry
October 25, 2021 -- Deadline for submitting the challenge entries
November 1, 2021 -- Confirm for submitting the best entry for final scoring
November 15-18, 2021 – Announcement of the winners of CPSC 2021 in ICBEB 2021

Any questions or problems about the Challenge, please feel free to contact

Challenge Chair:
Chengyu Liu

Challenge Committee:
Xingyao Wang
Caiyun Ma
Xiangyu Zhang
Hongxiang Gao
Songsheng Zhu

International Advisory Chair:
Gari D. Clifford

International Advisory Co-chairs:
Jianqing Li
Aiguo Song
Minglong Chen
Kang-Ping Lin
Yuqiu Zhong

International Advisory Committee:
Eddie Ng Yin Kwee
Yi Peng
Zhengtao Cao
Xianzheng Sha
Xingming Guo
Shoushui Wer
Chang Cui
Alistair Johnson

Hosted by:
School of Instrument Science and Engineering, Southeast University, China
The State Key Laboratory of Bioelectronics, Southeast University, China
School of Biomedical Engineering and Information, Nanjing Medical University, China

Supported by:
ICBEB Organizing Committee

Awards sponsored by:
Nalong Technology Co., Ltd.

[1] G. F. Michaud and W. G. Stevenson, "Atrial Fibrillation," New England Journal of Medicine, vol. 384, no. 4, pp. 353-361, 2021.
[2] C. Ma, S. Wei, T. Chen, J. Zhong, Z. Liu, and C. Liu, "Integration of results from convolutional neural network in a support vector machine for the detection of atrial fibrillation," IEEE Transactions on Instrumentation and Measurement, 2020.
[3] R. R. De With et al., "Temporal patterns and short-term progression of paroxysmal atrial fibrillation: data from RACE V," EP Europace, vol. 22, no. 8, pp. 1162-1172, 2020.
[4] H. Baumgartner et al., "2020 ESC Guidelines for the management of adult congenital heart disease: The Task Force for the management of adult congenital heart disease of the European Society of Cardiology (ESC). Endorsed by: Association for European Paediatric and Congenital Cardiology (AEPC), International Society for Adult Congenital Heart Disease (ISACHD)," European heart journal, vol. 42, no. 6, pp. 563-645, 2021.
[5] C. Liu et al., "A comparison of entropy approaches for AF discrimination," Physiological measurement, vol. 39, no. 7, p. 074002, 2018.
[6] L. Zhao, C. Liu, S. Wei, Q. Shen, F. Zhou, and J. Li, "A new entropy-based atrial fibrillation detection method for scanning wearable ecg recordings," Entropy, vol. 20, no. 12, p. 904, 2018.
[7] V. Kalidas and L. S. Tamil, "Detection of atrial fibrillation using discrete-state Markov models and Random Forests," Computers in biology and medicine, vol. 113, p. 103386, 2019. [8] X. Zhang, J. Li, Z. Cai, L. Zhang, Z. Chen, and C. Liu, "Over-fitting suppression training strategies for deep learning-based atrial fibrillation detection," Medical & Biological Engineering & Computing, vol. 59, no. 1, pp. 165-173, 2021.