#Datacleaning can be especially problematic in the case of surveys.
Fieldwork companies will typically do some cleaning but, for IP reasons, are often reluctant to share the details of what they have done and on what grounds.
Cleaning is not as simple as flagging "bad" respondents. Many respondents answer some questions quite diligently but others less so. Their answers may reflect their true feelings, though, and are not necessarily "bad."
So, when conducting modeling, statisticians need to think carefully about the questions they may use for a particular model and which respondents (if any) to exclude from the modeling.
Even when the analytics will consist of simple cross tabs, it may not be safe to assume the data are "clean enough." It might be smart to have your statistician check the data before you begin your analysis.
We can also be proactive and alert our fieldwork company in advance regarding response patterns which suggest satisficing, or even include our own traps in the #questionnaire.
✴️ @AI_Python_EN
Fieldwork companies will typically do some cleaning but, for IP reasons, are often reluctant to share the details of what they have done and on what grounds.
Cleaning is not as simple as flagging "bad" respondents. Many respondents answer some questions quite diligently but others less so. Their answers may reflect their true feelings, though, and are not necessarily "bad."
So, when conducting modeling, statisticians need to think carefully about the questions they may use for a particular model and which respondents (if any) to exclude from the modeling.
Even when the analytics will consist of simple cross tabs, it may not be safe to assume the data are "clean enough." It might be smart to have your statistician check the data before you begin your analysis.
We can also be proactive and alert our fieldwork company in advance regarding response patterns which suggest satisficing, or even include our own traps in the #questionnaire.
✴️ @AI_Python_EN
data cleaning.pdf
356.1 KB
Step by Step Guide to Data Cleaning With Python(NumPy and Pandas)
#machinelearning #artificialintelligence #datascience #ml #ai #deeplearning #datacleaning #python
✴️ @AI_Python_EN
#machinelearning #artificialintelligence #datascience #ml #ai #deeplearning #datacleaning #python
✴️ @AI_Python_EN