Data Science Methodology
Definition
A structured, highly iterative process (IBM’s CRISP-DM-style methodology) for turning a business question into a deployed analytical solution. It can run top-down (define the business problem, then analyze data for a solution) or bottom-up (start with data, then frame a problem around it), and it never truly ends — deployment feeds back into refinement.
Core Ideas
Stages
- Business understanding — establish the goal.
- Analytic approach — pick the type: predictive, descriptive, or classification (e.g. a recommender system is chosen at this stage). Decision trees are a common model.
- Data requirements — define what data is needed.
- Data collection — data scientists work with DBAs; it’s fine to defer decisions about unavailable data and acquire it later; eliminate redundant data.
- Data understanding — descriptive statistics (univariate stats, pairwise correlations, histograms) and quality checks (missing, invalid, misleading).
- Data preparation — the most time-consuming stage (70–90%): cleaning, feature engineering, text analysis.
- Modeling — build the model (e.g. split 70/30 train/test).
- Evaluation — ensure data is handled and interpreted correctly and the model works as intended, via diagnostic measures and statistical significance testing.
- Deployment → feedback → refinement → redeployment — the iterative loop.
Evaluation details
- ROC curve (Receiver Operating Characteristic) — plots true-positive vs false-positive rate across misclassification-cost values to select the optimal model (AUC summarizes it).
- Type I error — rejecting a true null hypothesis (false positive).
- Type II error — failing to reject a false null hypothesis (false negative).
- A training set is used for predictive modeling, not descriptive modeling.
Relationships
- Machine Learning — the modeling techniques applied within the methodology
- Data Quality — understanding/preparation depends on clean, valid data
- Data Leakage — an evaluation pitfall to guard against when splitting data
- Gradient Boosting — a common model choice for tabular prediction
- AI & Machine Learning — parent topic
See Also
- Regression and Statistical Inference — the statistical techniques underneath the modeling stage