Data Exploration for Passive Data
Raw data: Questionnaire & lab register for PASSIVE survey lates.xlsx
File: PassiveData.ipynb
- Data has 3045 rows and 170 columns
- No presence of parasite eggs in stool for all 3045 patients
- Data set has two values for ELISA test (binary positive/negative and continuous ELISA test value):
| Positive | Negative |
|---|
| count | 74 | 2901 |
| mean | 0.625946 | 0.399461 |
| std | 0.123276 | 0.219057 |
| min | 0.5 | 0.024 |
| 25% | 0.5725 | 0.22 |
| 50% | 0.62 | 0.3 |
| 75% | 0.63 | 0.58 |
| max | 1.47 | 1.47 |
- This data set is missing significantly less symptom durations:
| Symptom | Presence % | Missing Duration % |
|---|
| Cough | 96.815496% | 0.03391% |
| Expectoration | 80.302035% | 0.040883% |
| Chest Pain | 81.28693% | 0% |
| Fever | 97.800394% | 3.356831% |
| Loss of Appetite | 38.378201% | 0.085543% |
| Blood in Sputum | 1.181878% | 0% |
| Night Sweats | 53.906763% | 0.121803% |
| Weight Loss | 11.260670% | 1.166181% |
| Shortness of Breath | 78.233749% | 0% |
| Tiredness | 73.965857% | 1.065246% |
- Removing rows missing symptom duration (when symptom is present) reduces number of patients down to 2890 (-155 patients).
- Some duration strings are malformed (e.g. ‘days’ doesn’t specify a number of days). We removed these rows bringing it down to 2843 patients.
- Duration of symptoms:
| Symptom | Presence % | Count | Mean | Std | Min | Max |
|---|
| Cough | 98.628210% | 2804 | 13.119472 | 4.860853 | 2 | 28 |
| Expectoration | 82.553641% | 2347 | 12.862804 | 3.720741 | 2 | 28 |
| Chest Pain | 84.417868% | 2400 | 11.727500 | 4.070440 | 1 | 21 |
| Fever | 97.748857% | 2779 | 10.744512 | 4.219696 | 2 | 21 |
| Loss of Appetite | 40.801970% | 1160 | 12.850862 | 7.93335 | 1 | 30 |
| Blood in Sputum | 1.195920% | 34 | 7.911765 | 5.900014 | 2 | 14 |
| Night Sweats | 54.449525% | 1548 | 7.529716 | 5.974284 | 2 | 30 |
| Weight Loss | 11.044671% | 314 | 22.754777 | 15.274916 | 1 | 90 |
| Shortness of Breath | 81.322547% | 2312 | 9.556228 | 4.895476 | 1 | 21 |
| Tiredness | 74.885684% | 2129 | 9.459371 | 5.584566 | 1 | 30 |
- Significantly more patients consumed raw fresh water crabs:
| Count | Percent |
|---|
| Fresh Water Crab | | |
| Raw_fresh_water_crabs | 402 | 14.14 |
| Roasted_fresh_water_crabs | 25 | 0.879353 |
| Smoked_fresh_water_crabs | 0 | 0 |
| Soup_fresh_water_crabs | 2803 | 98.593 |
| Pickled_fresh_water_crabs | 0 | 0 |
| Cooked_fresh_water_crabs | 146 | 5.13542 |
| Others_fresh_water_crabs | 2843 | 100 |
| Cray Fish | | |
| Raw_cray_fish | 0 | 0 |
| Roasted_cray_fish | 0 | 0 |
| Smoked_cray_fish | 0 | 0 |
| Soup_cray_fish | 20 | 0.703482 |
| Pickled_cray_fish | 0 | 0 |
| Cooked_cray_fish | 0 | 0 |
| Others_cray_fish | 0 | 0 |
| Wild Boar | | |
| Raw_wild_boar_meat | 0 | 0 |
| Roasted_wild_boar_meat | 0 | 0 |
| Smoked_wild_boar_meat | 0 | 0 |
| Soup_wild_boar_meat | 0 | 0 |
| Pickled_wild_boar_meat | 0 | 0 |
| Cooked_wild_boar_meat | 0 | 0 |
| Others_wild_boar_meat | 0 | 0 |
| Rodents | | |
| Raw_rodents | 0 | 0 |
| Roasted_rodents | 0 | 0 |
| Smoked_rodents | 0 | 0 |
| Soup_rodents | 0 | 0 |
| Pickled_rodents | 0 | 0 |
| Cooked_rodents | 0 | 0 |
| Others_rodents | 0 | 0 |