The China Physiological Signal Challenge 2018
The 7th International Conference on Biomedical Engineering and Biotechnology (ICBEB 2018)
October 17th - 20th, 2018, Nanjing, China
• 中文版     • English
f
The China Physiological Signal Challenge 2018: Automatic identification of the rhythm/morphology abnormalities in 12-lead ECGs

Invitation Letter (in English)        Invitation Letter (in Chinese)

Good news! The Scores of some entries come out as shown below. More will be coming soon…

Notes:
Please note the scores are based on 450 samples randomly picking from the test set (50 samples from each type).
The reasons for this evaluation are from two aspects:
1) to reduce the evaluation burden and give timely feedback to the challenge teams so that they can optimize the model as soon as possible;
2) to prevent challengers frequently testing the parameter effects on the final test database.
In the late stage of the challenge (before deadline), Each team will be asked to confirm the final code version they want to be evaluated, and will run the code on the whole test database, which will generate the final challenge results and ranking.

1）降低评估工作负荷，提供给竞赛团队更为及时的得分反馈以改善模型；
2）阻止竞赛团队通过不断提交评估程序（仅仅做参数的修改）来不断测试在整个test集上的结果。

 Score List for Matlab Code No. Register Email Team Members F11 F12 F13 F14 F15 F16 F17 F18 F19 F1 Faf Fblock Fpc Fst CPSC0129 zytboy@sdu.edu.cn Yatao Zhang, Zhenguo Ma 0.721 0.887 0.813 0.876 0.778 0.707 0.694 0.729 0.525 0.748 0.887 0.817 0.701 0.650 CPSC0161 897675693@qq.com Wendan Wang, Runze Wang 0.540 0.869 0.816 0.800 0.806 0.256 0.712 0.619 0.563 0.665 0.869 0.808 0.516 0.598 CPSC0160 vadim.gliner@gmail.com Gliner Vadim 0.585 0.864 0.595 0.805 0.746 0.591 0.789 0.593 0.406 0.664 0.864 0.720 0.701 0.516 CPSC0119 mostayad@mail.uc.edu Ahmed Mostayed, Junye Luo, Xingliang Shu, and William G. Wee 0.590 0.924 0.819 0.882 0.807 0.404 0.512 0.598 0.387 0.658 0.924 0.833 0.451 0.521 CPSC0122 vadim.gliner@gmail.com Gliner Vadim 0.571 0.860 0.568 0.675 0.650 0.587 0.547 0.577 0.300 0.593 0.860 0.631 0.566 0.471 CPSC0168 liangyongbo001@gmail.com Yongbo Liang, Qunfeng Tang, Bangwen Yu, Shiyong Li 0.496 0.794 0.478 0.727 0.752 0.429 0.596 0.539 0.300 0.568 0.794 0.657 0.517 0.457 CPSC0084 rajeshr@cukerala.ac.in Rajesh R. 0.557 0.642 0.516 0.706 0.709 0.370 0.526 0.587 0.361 0.553 0.642 0.649 0.455 0.506 CPSC0155 liangyongbo001@gmail.com Yongbo Liang, Qunfeng Tang, Bangwen Yu, Shiyong Li 0.409 0.812 0.364 0.641 0.710 0.452 0.589 0.467 0.271 0.524 0.812 0.586 0.525 0.398 CPSC0158 13261993531@163.com Yanhui Dang, Baojuan Shan 0.489 0.704 0.580 0.276 0.559 0.066 0.139 0.595 0.328 0.415 0.704 0.510 0.105 0.510 CPSC0116 liangyongbo001@gmail.com Yongbo Liang, Qunfeng Tang, Bangwen Yu, Shiyong Li 0.197 0.299 0.198 0.244 0.403 0.195 0.292 0.436 0.033 0.255 0.299 0.295 0.247 0.292 CPSC0112 13261993531@163.com Yanhui Dang, Baojuan Shan 0.202 0.303 0.130 0.333 0.115 0.036 0.000 0.000 0.184 0.145 0.303 0.199 0.019 0.109 CPSC0138 zxy317@mail.ustc.edu.cn Xiaoyu Zhang, Yu Wu, Yufei Zhang 0.171 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.359 0.059 0.000 0.000 0.000 0.280 CPSC0110 r06b48001@ntu.edu.tw Chih-Han Huang 0.200 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.022 0.000 0.000 0.000 0.000

 Score List for Python Code No. Register Email Team Members F11 F12 F13 F14 F15 F16 F17 F18 F19 F1 Faf Fblock Fpc Fst CPSC0154 897789661@qq.com Kedong Mao, Juewei Lin, Shenjia Zhao, Fei Lu, Ping Cao, Qing Pan, Luping Fang 0.702 0.891 0.825 0.903 0.857 0.774 0.860 0.784 0.648 0.805 0.891 0.861 0.816 0.726 CPSC0072 sdnjly@126.com Runnan He, Yang Liu, Kuanquan Wang, Na Zhao, Yongfeng Yuan, Qince Li, Henggui Zhang 0.679 0.889 0.758 0.889 0.822 0.752 0.788 0.627 0.769 0.775 0.889 0.824 0.771 0.701 CPSC0142 sdnjly@126.com Runnan He, Yang Liu, Kuanquan Wang, Na Zhao, Yongfeng Yuan, Qince Li, Henggui Zhang 0.695 0.902 0.804 0.851 0.855 0.692 0.837 0.727 0.571 0.770 0.902 0.838 0.762 0.667 CPSC0170 dksrobin@ibms.sinica.edu.tw Tsai-Min Chen, Chih-Han Huang, Edward S. C. Shih, Ming-Jing Hwang 0.689 0.883 0.840 0.884 0.857 0.702 0.796 0.750 0.475 0.764 0.883 0.860 0.749 0.645 CPSC0139 stfeifzu@163.com Tengfei Shen, Yang Chen, Minjun Chen 0.712 0.919 0.788 0.875 0.783 0.792 0.796 0.680 0.508 0.761 0.919 0.813 0.794 0.617 CPSC0148 zhongdiliu@icloud.com Zhongdi Liu, Xiang-Ao Meng, Jiajia Cui, Zhibei Huang 0.672 0.919 0.842 0.851 0.841 0.645 0.774 0.733 0.484 0.751 0.919 0.844 0.710 0.638 CPSC0164 wenjiecai@aliyun.com Wenjie Cai, Jing Ma, Li Yang, Danqin Hu, Yanan Liu 0.691 0.904 0.826 0.805 0.816 0.700 0.713 0.729 0.516 0.744 0.904 0.816 0.706 0.646 CPSC0172 343117063@qq.com Yangyang Yan 0.655 0.905 0.774 0.829 0.818 0.705 0.667 0.624 0.592 0.730 0.905 0.807 0.684 0.611 CPSC0150 y.sun16@imperial.ac.uk Yingnan Sun, Frank Lo, Benny Lo 0.667 0.864 0.812 0.822 0.800 0.700 0.667 0.673 0.492 0.722 0.864 0.811 0.683 0.605 CPSC0159 1252490845@qq.com Bo Wu, Yanwen Sun, Rui Li, Jingxiao Xu 0.629 0.869 0.857 0.917 0.829 0.359 0.617 0.679 0.576 0.704 0.869 0.864 0.491 0.640 CPSC0074 617207490@qq.com Wei Guo, Xixuan Liu, Qi Shu 0.611 0.855 0.822 0.860 0.857 0.414 0.744 0.577 0.545 0.698 0.855 0.848 0.578 0.565 CPSC0086 985759542@qq.com Yongchao Wang, Yunqiu Xu, Bin Xiao, Weisheng Li, Xu Ma 0.619 0.937 0.845 0.876 0.832 0.434 0.500 0.635 0.548 0.692 0.937 0.849 0.462 0.602 CPSC0125 z14louse@gmail.com Jiannshu Li, Yaoxian Ma, Guangyi Li 0.607 0.869 0.873 0.894 0.792 0.343 0.588 0.635 0.625 0.692 0.869 0.852 0.477 0.631 CPSC0156 1252490845@qq.com Bo Wu, Yanwen Sun, Rui Li, Jingxiao Xu 0.565 0.837 0.851 0.948 0.800 0.358 0.590 0.627 0.531 0.679 0.837 0.863 0.483 0.593 CPSC0153 rex92hbr@163.com Borui Hou, Pu Wang, Siyu Shao, Ruqiang Yan 0.603 0.873 0.808 0.857 0.847 0.306 0.660 0.684 0.393 0.670 0.873 0.838 0.509 0.584 CPSC0144 lihuyang@sia.cn Jiayu Huo, Huyang Li 0.619 0.924 0.840 0.876 0.831 0.297 0.513 0.559 0.554 0.668 0.924 0.847 0.408 0.557 CPSC0145 897789661@qq.com Kedong Mao, Juewei Lin, Shenjia Zhao, Fei Lu, Ping Cao, Qing Pan, Luping Fang 0.576 0.833 0.827 0.860 0.815 0.328 0.788 0.557 0.406 0.666 0.833 0.832 0.618 0.497 CPSC0046 18071410137@189.cn Yue Yu, Bin Wang, Fan Yang 0.642 0.881 0.791 0.917 0.721 0.438 0.571 0.566 0.452 0.664 0.881 0.803 0.512 0.526 CPSC0132 wenjiecai@aliyun.com Wenjie Cai, Jing Ma, Li Yang, Danqin Hu, Yanan Liu 0.580 0.873 0.738 0.833 0.867 0.389 0.659 0.605 0.387 0.659 0.873 0.819 0.535 0.534 CPSC0163 lihuyang@sia.cn Jiayu Huo, Huyang Li 0.574 0.826 0.857 0.857 0.796 0.198 0.589 0.637 0.545 0.653 0.826 0.835 0.409 0.603 CPSC0149 15821893571@163.com Yun Luo, Shu Jiang 0.586 0.807 0.827 0.821 0.828 0.265 0.557 0.579 0.562 0.648 0.807 0.825 0.407 0.573 CPSC0166 360932990@qq.com Song Huapeng 0.581 0.867 0.838 0.869 0.852 0.182 0.486 0.631 0.508 0.646 0.867 0.853 0.357 0.588 CPSC0147 rex92hbr@163.com Borui Hou, Pu Wang, Siyu Shao, Ruqiang Yan 0.545 0.821 0.804 0.800 0.872 0.253 0.604 0.574 0.387 0.629 0.821 0.829 0.446 0.506 CPSC0094 qinyang@seu.edu.cn Yang Qin, Lingfei Mo 0.593 0.681 0.716 0.725 0.837 0.161 0.703 0.647 0.500 0.618 0.681 0.770 0.484 0.588 CPSC0162 qinyang@seu.edu.cn Yang Qin, Lingfei Mo 0.522 0.864 0.891 0.872 0.868 0.265 0.367 0.581 0.264 0.611 0.864 0.877 0.325 0.486 CPSC0128 15821893571@163.com Yun Luo, Shu Jiang 0.597 0.793 0.812 0.835 0.851 0.214 0.547 0.564 0.000 0.579 0.793 0.834 0.391 0.400 CPSC0105 1252490845@qq.com Bo Wu, Yanwen Sun, Rui Li, Jingxiao Xu 0.453 0.676 0.800 0.863 0.738 0.176 0.312 0.556 0.361 0.548 0.676 0.795 0.248 0.485 CPSC0096 gagh86@gmail.com Jeong-Hwan Kim, Kyeong-Seop Kim 0.517 0.618 0.506 0.812 0.729 0.100 0.462 0.513 0.154 0.490 0.618 0.692 0.292 0.400 CPSC0077 393444150@qq.com Yuxiang Li, Haibin Zhang, Zhe Chen, Yu Cheng 0.431 0.763 0.775 0.837 0.794 0.000 0.297 0.491 0.000 0.487 0.763 0.799 0.191 0.386 CPSC0083 yuejw1112@163.com Jingwei Yue, Haochen He, Yang Liu, Chunyong Li, Jiguo Xue, Chi Zhang 0.281 0.677 0.674 0.682 0.679 0.140 0.180 0.321 0.172 0.423 0.677 0.678 0.161 0.271 CPSC0171 gagh86@gmail.com Jeong-Hwan Kim, Kyeong-Seop Kim 0.454 0.243 0.368 0.627 0.647 0.118 0.437 0.329 0.194 0.379 0.243 0.535 0.279 0.272 CPSC0075 troianll@163.com Leilei Zhou, Zuoheng Zhang 0.476 0.600 0.457 0.000 0.698 0.174 0.423 0.442 0.082 0.372 0.600 0.488 0.348 0.302 CPSC0085 dksrobin@ibms.sinica.edu.tw Tsai-Min Chen, Chih-Han Huang, Edward S. C. Shih, Ming-Jing Hwang 0.361 0.721 0.629 0.522 0.812 0.074 0.070 0.049 0.000 0.360 0.721 0.681 0.072 0.032 CPSC0165 769229487@qq.com Renjie Zhu, Dongru Xie, Ke Lan 0.183 0.148 0.163 0.066 0.131 0.071 0.079 0.040 0.032 0.102 0.148 0.122 0.075 0.037 CPSC0167 dksrobin@ibms.sinica.edu.tw Tsai-Min Chen, Chih-Han Huang, Edward S. C. Shih, Ming-Jing Hwang 0.423 0.000 0.021 0.052 0.080 0.042 0.000 0.000 0.000 0.069 0.000 0.051 0.021 0.000

The following sample entries are for your reference. Please read the README files in each sample entry and submit the standard matlab/python code entry according to the requirement. The team is encouraged to submit up to 5 entries from now on, which can significantly reduce the evaluation works for algorithm evaluation.
sample_entry_matlab
sample_entry_python
validation_set

Q1: How can I register to attend this challenge?
R1: Please click http://cpsc.papersub.com/ to complete the registration.

Q2: How can we register team information to join the challenge?
R2: We suggest that the team leader, or one of the team representatives registers an account on behalf of the team in the system.

Q4: How should I submit the challenge entries with the provided sample entry ‘CSPC2018_challenge’?
R4: You can add the classification codes in the ‘CSPC2018_challenge.m', and submit the updated version of 'CSPC2018_challenge.m'. At this moment, please focus on the methods you need to develop, and submit the ‘CSPC2018_challenge.m’ with developed codes. Please do not only upload the example code without any essential content.

Q5: Any requirements for input and output of the code to be submitted?
R5: Test data and training data are exactly in the same format. The input of the submitted code is the data path, and the output is the classification result of all the data.

Q6: How should I name the main function?

Q7: If we use the python environment, can the test system support commonly used packages, such as keras, wfdb, numpy?
R7: Yes, It can.

Introduction
The China Physiological Signal Challenge (CPSC) 2018 is the 1st China Physiological Signal Challenge, which will be held during the 7th International Conference on Biomedical Engineering and Biotechnology (ICBEB 2018) in Nanjing, China. The CSPC aims to provide a platform for the open-source data and algorithms for the physiological signal analysis, and thus to promote the open-source research pattern for the cardiovascular disease detection and prediction in China.
The CPSC2018 aims to encourage the development of algorithms to identify the rhythm/morphology abnormalities from 12-lead ECGs, lasting several seconds to tens of seconds. The 12-lead ECGs used in CPSC 2018 include one normal type and eight abnormal types, which are detailed as:
(1) Atrial fibrillation (AF)
(2) First-degree atrioventricular block (I-AVB)
(3) Left bundle brunch block (LBBB)
(4) Right bundle brunch block (RBBB)
(5) Premature atrial contraction (PAC)
(6) Premature ventricular contraction (PVC)
(7) ST-segment depression (STD)
(8) ST-segment elevated (STE)

Challenge Data
The challenge ECG recordings were collected from 11 hospitals. The training set contains 6,877 (female: 3178; male: 3699) 12 leads ECG recordings lasting from 6 s to just 60 s (see Table 1) and the test set contains 2,954 ECG recordings with the similar lengths. The test set is unavailable to the public and will remain private for the purpose of scoring.
ECG recordings were sampled as 500 Hz. All data are provided in MATLAB format (each recording is a .mat file containing the ECG data, as well as the patient sex and age information). More details of the training set can be seen in Table 1. Figure 1 shows an example 12-lead ECG waveforms.
There is a REFERENCE.csv file for the training data to provide the labels for each recording. The majority of the recordings only have one labeling (denoted as First label). However, a few recordings have up to three labels (denoted as First label, Second label and Third label respectively). For example, the recording of A0043 has two labels: First label 5 and Second label 2, indicating that this recording is identified as both RBBB and AF. If your code gives either of these two judgments, it is right for classification.

Since the size of the whole training set is large, we seperate the training set as three sub-sets. You can download them respectively and imerge them in one training set file. For detailed training data, please download from Training Set 1, Training Set 2, and Training Set 3.

Table 1: Data profile for the training set according to the ‘Frist label’ annotations.
 Type #recording Time length (s) Mean SD Min Median Max Normal 918 15.43 7.61 10.00 13.00 60.00 Atrial fibrillation (AF) 1098 15.01 8.39 9.00 11.00 60.00 First-degree atrioventricular block (I-AVB) 704 14.32 7.21 10.00 11.27 60.00 Left bundle branch block (LBBB) 207 14.92 8.09 9.00 12.00 60.00 Right bundle branch block (RBBB) 1695 14.42 7.60 10.00 11.19 60.00 Premature atrial contraction (PAC) 556 19.46 12.36 9.00 14.00 60.00 Premature ventricular contraction (PVC) 672 20.21 12.85 6.00 15.00 60.00 ST-segment depression (STD) 825 15.13 6.82 8.00 12.78 60.00 ST-segment elevated (STE) 202 17.15 10.72 10.00 11.89 60.00 Total 6877 15.79 9.04 6.00 12.00 60.00

Figure 1. Example of the 12-lead ECG waveforms (Left bundle brunch block, from A0011)

Scoring
The scoring for CPSC2018 uses a F1 measure, which is an average of the nice F1 values from each classification type. The counting rules for the numbers of the variables are defined in Table 2 as below:
Table 2: Counting rules for the numbers of the variables.
 Predicted Normal AF I-AVB LBBB RBBB PAC PVC STD STE Total Reference Normal N11 N12 N13 N14 N15 N16 N17 N18 N19 N1x AF N21 N22 N23 N24 N25 N26 N27 N28 N29 N2x I-AVB N31 N32 N33 N34 N35 N36 N37 N38 N39 N3x LBBB N41 N42 N43 N44 N45 N46 N47 N48 N49 N4x RBBB N51 N52 N53 N54 N55 N56 N57 N58 N59 N5x PAC N61 N62 N63 N64 N65 N66 N67 N68 N69 N6x PVC N71 N72 N73 N74 N75 N76 N77 N78 N79 N7x STD N81 N82 N83 N84 N85 N86 N87 N88 N89 N8x STE N91 N92 N93 N94 N95 N96 N97 N98 N99 N9x Total NX1 NX2 NX3 NX4 NX5 NX6 NX7 NX8 NX9

For each of the nine types, ${F}_{1}$ is defined as:
Normal: $F sub 1n equals 2 times Nn divided by the quantity sum of the N plus sum of the n$                       AF: $F sub 1n equals 2 times Nn divided by the quantity sum of the N plus sum of the n$                       I-AVF: $F sub 1n equals 2 times Nn divided by the quantity sum of the N plus sum of the n$                       LBBB: $F sub 1n equals 2 times Nn divided by the quantity sum of the N plus sum of the n$

RBBB: $F sub 1n equals 2 times Nn divided by the quantity sum of the N plus sum of the n$             PAC: $F sub 1n equals 2 times Nn divided by the quantity sum of the N plus sum of the n$             PVC: $F sub 1n equals 2 times Nn divided by the quantity sum of the N plus sum of the n$            STD: $F sub 1n equals 2 times Nn divided by the quantity sum of the N plus sum of the n$            STE: $F sub 1n equals 2 times Nn divided by the quantity sum of the N plus sum of the n$

The final challenge score is defined as follows:

$F sub 1n equals 2 times Nn divided by the quantity sum of the N plus sum of the n$

In addition, we also calculate the F1 measures for each of the four sub-abnormal types, i.e., the AF, block, premature contraction and ST-segment change, as follows:

AF: $F sub 1n equals 2 times Nn divided by the quantity sum of the N plus sum of the n$                                                                         Block: $F sub 1n equals 2 times Nn divided by the quantity sum of the N plus sum of the n$

Premature contraction: $F sub 1n equals 2 times Nn divided by the quantity sum of the N plus sum of the n$                    ST-segment change: $F sub 1n equals 2 times Nn divided by the quantity sum of the N plus sum of the n$

Participating in the challenge
To participate in the challenge, you will need to submit a software written by Matlab or Python that is able to run on the test data and output the final classification result without user interaction in our test environment. One sample entry ‘CSPC2018_challenge.m’ (written in MATLAB) is available to help you get started. The team must submit the first challenge open-source entry before June 1, 2018. The deadline for submitting the challenge entries is October 1, 2018.

Awards and Rules
The winner will be selected on the basis of the obtained final F1 score on the hidden test data. The first three will receive certificates and generous bonuses:
• First prize: Certificate plus bonus of RMB 30,000
• Second prize: Certificate plus bonus of RMB 15,000
• Third prize: Certificate plus bonus of RMB 5,000

In addition, we also provide the certificates and bonuses for the winners with the highest scores of FAF, FBlock, FPC and FST respectively, for each of the four sub-abnormal types classification:
• AF classification winner: Certificate plus bonus of RMB 2,500
• Block classification winner: Certificate plus bonus of RMB 2,500
• Premature contraction classification winner: Certificate plus bonus of RMB 2,500
• ST-segment change classification winner: Certificate plus bonus of RMB 2,500

We welcome all the individual or research group in the world to attend the challenge. To be eligible for the award, please do the following:
(Please be noted that paper submission is not mandatory and will not affect the challenge score. The rule of No. (2) has been modified as below.)
(1). Submit at least one open-source entry that can be scored before June 1, 2018.
(2). Submit a conference paper about your work to ICBEB 2018 before August 31. Please select "China Physiological Signal Challenge" as the topic of your submitted paper, so it can be identified easily by the review committee.
(3). Attend ICBEB 2018 (October 17-20, 2018) in Nanjing, China and present your work there.

Please do not submit analysis of this year’s Challenge data to other Conferences or Journals until after ICBEB 2018 has taken place, so the competitors are able to discuss the results in a single forum.

Important Dates
January 15, 2018 -- Challenge open
April 1, 2018 -- Deadline for team registration to join the challenge
June 1, 2018 -- Deadline for submitting the first challenge open-source entry
October 1, 2018 -- Deadline for submitting the challenge entries
October 17-20, 2018 – Announcement of the winners of CPSC 2018 in ICBEB 2018

Any questions or problems about the Chanllenge, please feel free to contact chinachallenge_cspc@icbeb.org.

Challenge Chair:
Prof. Chengyu Liu, Southeast University, China

Challenge Committee:
Dr. Feifei Liu, Southeast University, China
Dr. Xingyao Wang, Southeast University, China
Dr. Xiangyu Zhang, Southeast University, China
Dr. Lina Zhao, Southeast University & Shandong University, China
Prof. Xiaoling Wu, Nanjing Medical University, China

Prof. Gari D. Clifford, Emory University & Georgia Institute of Technology, USA

Prof. Aiguo Song, Southeast University, China
Prof. Jianqing Li, Nanjing Medical University, China
Mr. Yingjia Yao, Lenovo Group, China
Prof. Zhongze Gu, Southeast University, China

Prof. Eddie Ng Yin Kwee, Nanyang Technological University, Singapore
Prof. Amit Shah, Emory University, USA
Prof. Li Zhang, University of Northumbria, UK
Prof. Zhengbo Zhang, The 301 Military Hospital, China
Prof. Shoushui Wei, Shandong University, China
Prof. Hong Tang, Dalian University of Technology, China
Prof. Yuan Zhang, University of Jinan, China

Hosted by:
School of Instrument Science and Engineering, Southeast University, China
School of Biomedical Engineering and Information, Nanjing Medical University, China

Organized by:
ICBEB Organizing Committee
Chinese Institute of Electronics, Biomedical Electronics Branch
China Cognitive Society, Neuro-education Branch
Jiangsu Instrument and Control Society