检查点
数据清洗导航
学习如何使用实用的、分步的数据整理工作流程，将杂乱无章、分散的数据转化为清晰、结构化的洞察，这是每个成功数据分析项目的基础。
...
...
Data wrangling’s key processes
| Source of Dirty Data | What It Means | Common Examples of Issues |
|---|---|---|
| Human entry errors | Mistakes during manual data input. | Typos (“Californa”), transposed numbers (12,000 vs 21,000), inconsistent labels (“NY” vs. “New York”). |
| System and integration Issues | Problems when data flows between different systems or applications. | Date format mismatches, broken characters (café), repurposed fields storing wrong data. |
| Automated collection noise | Errors from non-human data gathering (sensors, web tools). | Faulty sensor spikes, optical recognition misreads (“8” for “B”), incomplete web responses. |
| Metadata and context gaps | Missing information about what data values represent. | Undefined units (“100” without knowing if it’s temperature or revenue), meaningless code labels (“1” vs “0” status). |
Table
| Dimension | Key Question | Examples |
|---|---|---|
| Completeness | Is all required data present? | Empty fields, missing rows after merges |
| Accuracy | Does the data reflect real-world values? | Impossible ages (e.g., -4), logical conflicts, misspellings |
| Noise and outliers | Are there random errors or extreme values? | Sensor spikes, garbled text, extreme highs/lows |
| Consistency | Is data uniformly formatted and labeled? | Date format variations, mixed units, inconsistent spellings |
| Uniqueness | Does each record represent a unique entity? | Duplicate rows, slightly varied records for the same person |
| Timeliness | Is the data current and relevant? | Outdated customer info, delayed reports |
| Validity and integrity | Does the data meet expected formats and rules? | Text in number fields, negative quantities, invalid characters |
Data-quality issues and techniques
| Data-Quality Dimension | Core Cleaning Techniques |
|---|---|
| Completeness | Imputation |
| Accuracy | Validation rules, outlier detection and treatment |
| Consistency | Parsing and standardization |
| Uniqueness | Deduplication |
| Timeliness | Automation and monitoring |
| Validity and integrity | Validation rules and constraints |
| Noise and outliers | Outlier detection and treatment, text-specific cleanup |
Quiz
单选题
数据清洗工作流的哪个阶段涉及将来自不同分散位置的数据引入统一的工作环境？
1 / 5