The China Physiological Signal Challenge 2018
The 7th International Conference on Biomedical Engineering and Biotechnology (ICBEB 2018)
October 17th - 20th, 2018, Nanjing, China
• 中文版     • English
f
The China Physiological Signal Challenge 2018: Automatic identification of the rhythm/morphology abnormalities in 12-lead ECGs

Invitation Letter (in English)        Invitation Letter (in Chinese)

Good news! The Scores of some entries come out as shown below. More will be coming soon…

Notes:
Please note the scores are based on 450 samples randomly picking from the test set (50 samples from each type).
The reasons for this evaluation are from two aspects:
1) to reduce the evaluation burden and give timely feedback to the challenge teams so that they can optimize the model as soon as possible;
2) to prevent challengers frequently testing the parameter effects on the final test database.
In the late stage of the challenge (before deadline), Each team will be asked to confirm the final code version they want to be evaluated, and will run the code on the whole test database, which will generate the final challenge results and ranking.

注:
请注意评分是基于test集随机选取的450个样本得出的,每类选取了50个record。
这样做的目的有两个:
1)降低评估工作负荷,提供给竞赛团队更为及时的得分反馈以改善模型;
2)阻止竞赛团队通过不断提交评估程序(仅仅做参数的修改)来不断测试在整个test集上的结果。
在竞赛末期(deadline之前),我们会向各参赛团队确认每个团队需要最终评估的程序版本,并在整个test集上运行此最终版本,据此得出最终的竞赛成绩和排名。

Score List for Matlab
Code No. Register EmailTeam Members F11F12F13F14F15F16F17F18F19F1FafFblockFpcFst
CPSC0129zytboy@sdu.edu.cnYatao Zhang, Zhenguo Ma0.7210.8870.8130.8760.7780.7070.6940.7290.5250.7480.8870.8170.7010.650
CPSC0161897675693@qq.comWendan Wang, Runze Wang0.5400.8690.8160.8000.8060.2560.7120.6190.5630.6650.8690.8080.5160.598
CPSC0160vadim.gliner@gmail.comGliner Vadim0.5850.8640.5950.8050.7460.5910.7890.5930.4060.6640.8640.7200.7010.516
CPSC0119mostayad@mail.uc.eduAhmed Mostayed, Junye Luo, Xingliang Shu, and William G. Wee0.5900.9240.8190.8820.8070.4040.5120.5980.3870.6580.9240.8330.4510.521
CPSC0122vadim.gliner@gmail.comGliner Vadim0.5710.8600.5680.6750.6500.5870.5470.5770.3000.5930.8600.6310.5660.471
CPSC0168liangyongbo001@gmail.comYongbo Liang, Qunfeng Tang, Bangwen Yu, Shiyong Li0.4960.7940.4780.7270.7520.4290.5960.5390.3000.5680.7940.6570.5170.457
CPSC0084rajeshr@cukerala.ac.inRajesh R.0.5570.6420.5160.7060.7090.3700.5260.5870.3610.5530.6420.6490.4550.506
CPSC0155liangyongbo001@gmail.comYongbo Liang, Qunfeng Tang, Bangwen Yu, Shiyong Li0.4090.8120.3640.6410.7100.4520.5890.4670.2710.5240.8120.5860.5250.398
CPSC015813261993531@163.comYanhui Dang, Baojuan Shan0.4890.7040.5800.2760.5590.0660.1390.5950.3280.4150.7040.5100.1050.510
CPSC0116liangyongbo001@gmail.comYongbo Liang, Qunfeng Tang, Bangwen Yu, Shiyong Li0.1970.2990.1980.2440.4030.1950.2920.4360.0330.2550.2990.2950.2470.292
CPSC011213261993531@163.comYanhui Dang, Baojuan Shan0.2020.3030.1300.3330.1150.0360.0000.0000.1840.1450.3030.1990.0190.109
CPSC0138zxy317@mail.ustc.edu.cnXiaoyu Zhang, Yu Wu, Yufei Zhang0.1710.0000.0000.0000.0000.0000.0000.0000.3590.0590.0000.0000.0000.280
CPSC0110r06b48001@ntu.edu.twChih-Han Huang0.2000.0000.0000.0000.0000.0000.0000.0000.0000.0220.0000.0000.0000.000

Score List for Python
Code No. Register EmailTeam Members F11F12F13F14F15F16F17F18F19F1FafFblockFpcFst
CPSC0154897789661@qq.comKedong Mao, Juewei Lin, Shenjia Zhao, Fei Lu, Ping Cao, Qing Pan, Luping Fang0.7020.8910.8250.9030.8570.7740.8600.7840.6480.8050.8910.8610.8160.726
CPSC0072sdnjly@126.comRunnan He, Yang Liu, Kuanquan Wang, Na Zhao, Yongfeng Yuan, Qince Li, Henggui Zhang0.6790.8890.7580.8890.8220.7520.7880.6270.7690.7750.8890.8240.7710.701
CPSC0142sdnjly@126.comRunnan He, Yang Liu, Kuanquan Wang, Na Zhao, Yongfeng Yuan, Qince Li, Henggui Zhang0.6950.9020.8040.8510.8550.6920.8370.7270.5710.7700.9020.8380.7620.667
CPSC0170dksrobin@ibms.sinica.edu.twTsai-Min Chen, Chih-Han Huang, Edward S. C. Shih, Ming-Jing Hwang0.6890.8830.8400.8840.8570.7020.7960.7500.4750.7640.8830.8600.7490.645
CPSC0139stfeifzu@163.comTengfei Shen, Yang Chen, Minjun Chen0.7120.9190.7880.8750.7830.7920.7960.6800.5080.7610.9190.8130.7940.617
CPSC0148zhongdiliu@icloud.comZhongdi Liu, Xiang-Ao Meng, Jiajia Cui, Zhibei Huang0.6720.9190.8420.8510.8410.6450.7740.7330.4840.7510.9190.8440.7100.638
CPSC0164wenjiecai@aliyun.comWenjie Cai, Jing Ma, Li Yang, Danqin Hu, Yanan Liu0.6910.9040.8260.8050.8160.7000.7130.7290.5160.7440.9040.8160.7060.646
CPSC0172343117063@qq.comYangyang Yan0.6550.9050.7740.8290.8180.7050.6670.6240.5920.7300.9050.8070.6840.611
CPSC0150y.sun16@imperial.ac.ukYingnan Sun, Frank Lo, Benny Lo0.6670.8640.8120.8220.8000.7000.6670.6730.4920.7220.8640.8110.6830.605
CPSC01591252490845@qq.comBo Wu, Yanwen Sun, Rui Li, Jingxiao Xu0.6290.8690.8570.9170.8290.3590.6170.6790.5760.7040.8690.8640.4910.640
CPSC0074617207490@qq.comWei Guo, Xixuan Liu, Qi Shu0.6110.8550.8220.8600.8570.4140.7440.5770.5450.6980.8550.8480.5780.565
CPSC0086985759542@qq.comYongchao Wang, Yunqiu Xu, Bin Xiao, Weisheng Li, Xu Ma0.6190.9370.8450.8760.8320.4340.5000.6350.5480.6920.9370.8490.4620.602
CPSC0125z14louse@gmail.comJiannshu Li, Yaoxian Ma, Guangyi Li0.6070.8690.8730.8940.7920.3430.5880.6350.6250.6920.8690.8520.4770.631
CPSC01561252490845@qq.comBo Wu, Yanwen Sun, Rui Li, Jingxiao Xu0.5650.8370.8510.9480.8000.3580.5900.6270.5310.6790.8370.8630.4830.593
CPSC0153rex92hbr@163.comBorui Hou, Pu Wang, Siyu Shao, Ruqiang Yan0.6030.8730.8080.8570.8470.3060.6600.6840.3930.6700.8730.8380.5090.584
CPSC0144lihuyang@sia.cnJiayu Huo, Huyang Li0.6190.9240.8400.8760.8310.2970.5130.5590.5540.6680.9240.8470.4080.557
CPSC0145897789661@qq.comKedong Mao, Juewei Lin, Shenjia Zhao, Fei Lu, Ping Cao, Qing Pan, Luping Fang0.5760.8330.8270.8600.8150.3280.7880.5570.4060.6660.8330.8320.6180.497
CPSC004618071410137@189.cnYue Yu, Bin Wang, Fan Yang0.6420.8810.7910.9170.7210.4380.5710.5660.4520.6640.8810.8030.5120.526
CPSC0132wenjiecai@aliyun.comWenjie Cai, Jing Ma, Li Yang, Danqin Hu, Yanan Liu0.5800.8730.7380.8330.8670.3890.6590.6050.3870.6590.8730.8190.5350.534
CPSC0163lihuyang@sia.cnJiayu Huo, Huyang Li0.5740.8260.8570.8570.7960.1980.5890.6370.5450.6530.8260.8350.4090.603
CPSC014915821893571@163.comYun Luo, Shu Jiang0.5860.8070.8270.8210.8280.2650.5570.5790.5620.6480.8070.8250.4070.573
CPSC0166360932990@qq.comSong Huapeng0.5810.8670.8380.8690.8520.1820.4860.6310.5080.6460.8670.8530.3570.588
CPSC0147rex92hbr@163.comBorui Hou, Pu Wang, Siyu Shao, Ruqiang Yan0.5450.8210.8040.8000.8720.2530.6040.5740.3870.6290.8210.8290.4460.506
CPSC0094qinyang@seu.edu.cnYang Qin, Lingfei Mo0.5930.6810.7160.7250.8370.1610.7030.6470.5000.6180.6810.7700.4840.588
CPSC0162qinyang@seu.edu.cnYang Qin, Lingfei Mo0.5220.8640.8910.8720.8680.2650.3670.5810.2640.6110.8640.8770.3250.486
CPSC012815821893571@163.comYun Luo, Shu Jiang0.5970.7930.8120.8350.8510.2140.5470.5640.0000.5790.7930.8340.3910.400
CPSC01051252490845@qq.comBo Wu, Yanwen Sun, Rui Li, Jingxiao Xu0.4530.6760.8000.8630.7380.1760.3120.5560.3610.5480.6760.7950.2480.485
CPSC0096gagh86@gmail.comJeong-Hwan Kim, Kyeong-Seop Kim0.5170.6180.5060.8120.7290.1000.4620.5130.1540.4900.6180.6920.2920.400
CPSC0077393444150@qq.comYuxiang Li, Haibin Zhang, Zhe Chen, Yu Cheng0.4310.7630.7750.8370.7940.0000.2970.4910.0000.4870.7630.7990.1910.386
CPSC0083yuejw1112@163.comJingwei Yue, Haochen He, Yang Liu, Chunyong Li, Jiguo Xue, Chi Zhang0.2810.6770.6740.6820.6790.1400.1800.3210.1720.4230.6770.6780.1610.271
CPSC0171gagh86@gmail.comJeong-Hwan Kim, Kyeong-Seop Kim0.4540.2430.3680.6270.6470.1180.4370.3290.1940.3790.2430.5350.2790.272
CPSC0075troianll@163.comLeilei Zhou, Zuoheng Zhang0.4760.6000.4570.0000.6980.1740.4230.4420.0820.3720.6000.4880.3480.302
CPSC0085dksrobin@ibms.sinica.edu.twTsai-Min Chen, Chih-Han Huang, Edward S. C. Shih, Ming-Jing Hwang0.3610.7210.6290.5220.8120.0740.0700.0490.0000.3600.7210.6810.0720.032
CPSC0165769229487@qq.comRenjie Zhu, Dongru Xie, Ke Lan0.1830.1480.1630.0660.1310.0710.0790.0400.0320.1020.1480.1220.0750.037
CPSC0167dksrobin@ibms.sinica.edu.twTsai-Min Chen, Chih-Han Huang, Edward S. C. Shih, Ming-Jing Hwang0.4230.0000.0210.0520.0800.0420.0000.0000.0000.0690.0000.0510.0210.000


The following sample entries are for your reference. Please read the README files in each sample entry and submit the standard matlab/python code entry according to the requirement. The team is encouraged to submit up to 5 entries from now on, which can significantly reduce the evaluation works for algorithm evaluation.
sample_entry_matlab
sample_entry_python
validation_set

Frequently Asked Questions (FAQs)
Q1: How can I register to attend this challenge?
R1: Please click http://cpsc.papersub.com/ to complete the registration.

Q2: How can we register team information to join the challenge?
R2: We suggest that the team leader, or one of the team representatives registers an account on behalf of the team in the system.

Q3: The sample entry, ‘CSPC2018_challenge.m’, is unable to be downloaded.
R3: Right click, save link as, and then download is finished.

Q4: How should I submit the challenge entries with the provided sample entry ‘CSPC2018_challenge’?
R4: You can add the classification codes in the ‘CSPC2018_challenge.m', and submit the updated version of 'CSPC2018_challenge.m'. At this moment, please focus on the methods you need to develop, and submit the ‘CSPC2018_challenge.m’ with developed codes. Please do not only upload the example code without any essential content.

Q5: Any requirements for input and output of the code to be submitted?
R5: Test data and training data are exactly in the same format. The input of the submitted code is the data path, and the output is the classification result of all the data.

Q6: How should I name the main function?
R6: Please name it CSPC2018_challenge.

Q7: If we use the python environment, can the test system support commonly used packages, such as keras, wfdb, numpy?
R7: Yes, It can.

Introduction
The China Physiological Signal Challenge (CPSC) 2018 is the 1st China Physiological Signal Challenge, which will be held during the 7th International Conference on Biomedical Engineering and Biotechnology (ICBEB 2018) in Nanjing, China. The CSPC aims to provide a platform for the open-source data and algorithms for the physiological signal analysis, and thus to promote the open-source research pattern for the cardiovascular disease detection and prediction in China.
The CPSC2018 aims to encourage the development of algorithms to identify the rhythm/morphology abnormalities from 12-lead ECGs, lasting several seconds to tens of seconds. The 12-lead ECGs used in CPSC 2018 include one normal type and eight abnormal types, which are detailed as:
(1) Atrial fibrillation (AF)
(2) First-degree atrioventricular block (I-AVB)
(3) Left bundle brunch block (LBBB)
(4) Right bundle brunch block (RBBB)
(5) Premature atrial contraction (PAC)
(6) Premature ventricular contraction (PVC)
(7) ST-segment depression (STD)
(8) ST-segment elevated (STE)

Challenge Data
The challenge ECG recordings were collected from 11 hospitals. The training set contains 6,877 (female: 3178; male: 3699) 12 leads ECG recordings lasting from 6 s to just 60 s (see Table 1) and the test set contains 2,954 ECG recordings with the similar lengths. The test set is unavailable to the public and will remain private for the purpose of scoring.
ECG recordings were sampled as 500 Hz. All data are provided in MATLAB format (each recording is a .mat file containing the ECG data, as well as the patient sex and age information). More details of the training set can be seen in Table 1. Figure 1 shows an example 12-lead ECG waveforms.
There is a REFERENCE.csv file for the training data to provide the labels for each recording. The majority of the recordings only have one labeling (denoted as First label). However, a few recordings have up to three labels (denoted as First label, Second label and Third label respectively). For example, the recording of A0043 has two labels: First label 5 and Second label 2, indicating that this recording is identified as both RBBB and AF. If your code gives either of these two judgments, it is right for classification.

Since the size of the whole training set is large, we seperate the training set as three sub-sets. You can download them respectively and imerge them in one training set file. For detailed training data, please download from Training Set 1, Training Set 2, and Training Set 3.

Table 1: Data profile for the training set according to the ‘Frist label’ annotations.
Type#recordingTime length (s)
MeanSDMinMedianMax
Normal91815.437.6110.0013.0060.00
Atrial fibrillation (AF)109815.018.399.0011.0060.00
First-degree atrioventricular block (I-AVB)70414.327.2110.0011.2760.00
Left bundle branch block (LBBB)20714.928.099.0012.0060.00
Right bundle branch block (RBBB)169514.427.6010.0011.1960.00
Premature atrial contraction (PAC)55619.4612.369.0014.0060.00
Premature ventricular contraction (PVC)67220.2112.856.0015.0060.00
ST-segment depression (STD)82515.136.828.0012.7860.00
ST-segment elevated (STE)20217.1510.7210.0011.8960.00
Total687715.799.046.0012.0060.00



Figure 1. Example of the 12-lead ECG waveforms (Left bundle brunch block, from A0011)

Scoring
The scoring for CPSC2018 uses a F1 measure, which is an average of the nice F1 values from each classification type. The counting rules for the numbers of the variables are defined in Table 2 as below:
Table 2: Counting rules for the numbers of the variables.
Predicted
NormalAFI-AVBLBBBRBBBPACPVCSTDSTETotal
ReferenceNormalN11N12N13N14N15N16N17N18N19N1x
AFN21N22N23N24N25N26N27N28N29N2x
I-AVBN31N32N33N34N35N36N37N38N39N3x
LBBBN41N42N43N44N45N46N47N48N49N4x
RBBBN51N52N53N54N55N56N57N58N59N5x
PACN61N62N63N64N65N66N67N68N69N6x
PVCN71N72N73N74N75N76N77N78N79N7x
STDN81N82N83N84N85N86N87N88N89N8x
STE N91N92N93N94N95N96N97N98N99N9x
TotalNX1NX2NX3NX4NX5NX6NX7NX8NX9

For each of the nine types, F 1 is defined as:
Normal: F11=2 × N11N1x + Nx1                       AF: F12=2 × N22N2x + Nx2                       I-AVF: F13=2 × N33N3x + Nx3                       LBBB: F14=2 × N44N4x + Nx4

RBBB: F15=2 × N55N5x + Nx5             PAC: F16=2 × N66N6x + Nx6             PVC: F17=2 × N77N7x + Nx7            STD: F18=2 × N88N8x + Nx8            STE: F19=2 × N99N9x + Nx9

The final challenge score is defined as follows:

F1=F11  +  F12   +  F13   +  F14   +  F15   +  F16   +  F17   +  F18   +  F199

In addition, we also calculate the F1 measures for each of the four sub-abnormal types, i.e., the AF, block, premature contraction and ST-segment change, as follows:

AF: FAF=2 × N22N2x + Nx2                                                                         Block: FBlock=2 × (N33   +  N44   +  N55   )N3x + Nx3 + N4x + Nx4 + N5x + Nx5

Premature contraction: FPC=2 × (N66   + N77   )N6x + Nx6 + N7x + Nx7                    ST-segment change: FST=2 × (N88   +  N99   )N8x + Nx8 + N9x + Nx9

Participating in the challenge
To participate in the challenge, you will need to submit a software written by Matlab or Python that is able to run on the test data and output the final classification result without user interaction in our test environment. One sample entry ‘CSPC2018_challenge.m’ (written in MATLAB) is available to help you get started. The team must submit the first challenge open-source entry before June 1, 2018. The deadline for submitting the challenge entries is October 1, 2018.
Please use the following link to upload your codes, http://cpsc.papersub.com/ .

Awards and Rules
The winner will be selected on the basis of the obtained final F1 score on the hidden test data. The first three will receive certificates and generous bonuses:
• First prize: Certificate plus bonus of RMB 30,000
• Second prize: Certificate plus bonus of RMB 15,000
• Third prize: Certificate plus bonus of RMB 5,000

In addition, we also provide the certificates and bonuses for the winners with the highest scores of FAF, FBlock, FPC and FST respectively, for each of the four sub-abnormal types classification:
• AF classification winner: Certificate plus bonus of RMB 2,500
• Block classification winner: Certificate plus bonus of RMB 2,500
• Premature contraction classification winner: Certificate plus bonus of RMB 2,500
• ST-segment change classification winner: Certificate plus bonus of RMB 2,500

We welcome all the individual or research group in the world to attend the challenge. To be eligible for the award, please do the following:
(Please be noted that paper submission is not mandatory and will not affect the challenge score. The rule of No. (2) has been modified as below.)
(1). Submit at least one open-source entry that can be scored before June 1, 2018.
(2). Submit a conference paper about your work to ICBEB 2018 before August 31. Please select "China Physiological Signal Challenge" as the topic of your submitted paper, so it can be identified easily by the review committee.
(3). Attend ICBEB 2018 (October 17-20, 2018) in Nanjing, China and present your work there.

Please do not submit analysis of this year’s Challenge data to other Conferences or Journals until after ICBEB 2018 has taken place, so the competitors are able to discuss the results in a single forum.

Important Dates
January 15, 2018 -- Challenge open
April 1, 2018 -- Deadline for team registration to join the challenge
June 1, 2018 -- Deadline for submitting the first challenge open-source entry
October 1, 2018 -- Deadline for submitting the challenge entries
October 17-20, 2018 – Announcement of the winners of CPSC 2018 in ICBEB 2018

Any questions or problems about the Chanllenge, please feel free to contact chinachallenge_cspc@icbeb.org.

Challenge Chair:
Prof. Chengyu Liu, Southeast University, China

Challenge Committee:
Dr. Feifei Liu, Southeast University, China
Dr. Xingyao Wang, Southeast University, China
Dr. Xiangyu Zhang, Southeast University, China
Dr. Lina Zhao, Southeast University & Shandong University, China
Prof. Xiaoling Wu, Nanjing Medical University, China

International Advisory Chair:
Prof. Gari D. Clifford, Emory University & Georgia Institute of Technology, USA

International Advisory Co-chairs:
Prof. Aiguo Song, Southeast University, China
Prof. Jianqing Li, Nanjing Medical University, China
Mr. Yingjia Yao, Lenovo Group, China
Prof. Zhongze Gu, Southeast University, China

International Advisory Committee:
Prof. Eddie Ng Yin Kwee, Nanyang Technological University, Singapore
Prof. Amit Shah, Emory University, USA
Prof. Li Zhang, University of Northumbria, UK
Prof. Zhengbo Zhang, The 301 Military Hospital, China
Prof. Shoushui Wei, Shandong University, China
Prof. Hong Tang, Dalian University of Technology, China
Prof. Yuan Zhang, University of Jinan, China

Hosted by:
School of Instrument Science and Engineering, Southeast University, China
School of Biomedical Engineering and Information, Nanjing Medical University, China

Organized by:
ICBEB Organizing Committee
Chinese Institute of Electronics, Biomedical Electronics Branch
China Cognitive Society, Neuro-education Branch
Jiangsu Instrument and Control Society

Awards sponsored by:
Lenovo Group


Sponsors:

                              
                        
The 7th International Conference on Biomedical Engineering and Biotechnology
Conference Secretary General: Ms. Linda Li   Assistants: Ms. Annie Zhu, Ms. Cassie Cheng
Email: icbeb@icbeb.org / icbeb@academicconf.com   Tel: +86-27-87051286  Cell phone: +86-13018020541
Address: No. 1, Optical valley avenue, East Lake High-Tech Development Zone, Wuhan, Hubei, China