The ML Project Checklist for Engineers

The hardest part of ML isn't the algorithms

You've read 18 articles. You understand features and labels, train/test splits, overfitting, decision trees, Random Forests, XGBoost, cross-validation, evaluation metrics, feature importance, neural networks, surrogate modelling, and SHAP. You can train a model, evaluate it honestly, and explain its predictions.

And yet, when you sit down at your desk on Monday morning, you'll face a question that none of those articles directly answers:

"Where would ML actually help in my work?"

This is the question that separates engineers who successfully apply ML from engineers who learn the theory and never use it. The algorithms are the easy part. Spotting the right problem, scoping a feasible project, and delivering value — that's the hard part.

Most engineers walk past ML opportunities every day. The 200 FEA runs that could be a surrogate. The manual inspection that could be a classifier. The process parameters that have hidden interactions nobody has systematically analysed. The opportunities are there — you just need the mental framework to recognise them.

This article gives you that framework, followed by a step-by-step checklist for taking an opportunity from "I think ML could help here" to a completed project. Print it. Reference it regularly. It'll save you more time than any hyperparameter tuning trick ever will.

Phase 0: Spotting ML opportunities in your own workflow

Before you can evaluate whether ML is the right tool, you need to recognise when you're looking at an ML-shaped problem. Here are five patterns that signal an opportunity. As you read each one, think about your own daily work.

Pattern 1: "We do this manually, and it's slow"

Any prediction, classification, or judgement call that a human makes repeatedly based on data is a potential ML application.

A quality inspector examines each part and decides pass or fail. A process engineer glances at a trend chart and decides whether the process is drifting. A maintenance technician listens to a machine and judges whether the bearing sounds healthy. A design engineer runs a quick hand calculation to estimate whether a concept will meet the stress requirement.

Each of these is a human doing pattern recognition — looking at inputs, applying experience, and producing an output. If a human can do it from data, a model might learn to do it too. Faster, more consistently, and at scale.

Ask yourself: What decisions do I or my colleagues make repeatedly, based on data or experience, that are time-consuming or inconsistent?

Pattern 2: "We run this simulation hundreds of times"

This is the surrogate modelling opportunity, and it's one of the highest-value ML applications in engineering.

If your workflow involves running expensive simulations (FEA, CFD, multi-body dynamics, electromagnetic analysis) with varying inputs — parameter sweeps, design space exploration, optimisation loops, sensitivity studies — you're spending computation time that ML could dramatically reduce.

The economics are compelling: 200 simulation runs to train a surrogate (a one-time cost) can replace 100,000 future evaluations. If each simulation takes 30 minutes, the surrogate pays for itself after the 201st evaluation.

Ask yourself: Is there a simulation I run repeatedly with different inputs? Does each run take more than a few minutes? Do I ever wish I could evaluate thousands of design points instead of dozens?

Pattern 3: "We have data, but nobody has looked at it systematically"

This is the hidden goldmine in most engineering organisations. Sensor logs, production databases, test archives, maintenance records — data that's collected because regulations or procedures require it, but never analysed beyond basic trending or monthly reports.

Unsupervised learning — clustering, anomaly detection, dimensionality reduction — can reveal structure that nobody knew was there. Machine operating modes that were never documented. Material batches that subtly differ from the rest. Process periods that produce consistently different quality without an obvious cause.

Ask yourself: What data does my company collect that nobody has systematically analysed? Are there spreadsheets, databases, or log files that might contain patterns nobody has looked for?

Pattern 4: "We know the outcome, but not what drives it"

You have historical data with both inputs and outcomes, but the relationship between them is too complex for simple analysis. Defect rates that vary unpredictably across production runs. Material properties that depend on dozens of processing variables in ways nobody can track mentally. Energy consumption that fluctuates for reasons that aren't obvious.

This is the classic supervised learning setup. The feature importance and SHAP analysis from such a model often delivers more value than the predictions themselves. "We've known that defect rates vary, but now we know that melt temperature and injection pressure account for 55% of the variation" is a finding that directly drives process improvement.

Ask yourself: What outcomes in my work vary in ways I can't fully explain? Do I have historical data that includes both the inputs and the outcome?

Pattern 5: "The expert is retiring"

Every engineering organisation has people whose expertise lives in their head — not in documentation, not in procedures, not in equations. The process engineer who can tell from the sound of the machine that something is about to go wrong. The experienced inspector who spots borderline defects that junior staff miss. The toolmaker who "just knows" which parameter settings will work for a new part geometry.

This institutional knowledge is extremely valuable and extremely fragile. When that person retires, changes roles, or is simply unavailable, the capability disappears.

If that expertise can be captured as data — the expert's decisions (labels) paired with the information they based those decisions on (features) — ML can learn to reproduce their judgement. Not perfectly, but well enough to preserve and scale knowledge that would otherwise be lost.

Ask yourself: Whose expertise would be hardest to replace if they left tomorrow? Could their decisions be recorded as training data?

Turning observations into candidates

If any of these patterns triggered a thought — "wait, we have exactly that situation" — write it down. Right now. Don't evaluate it yet, don't judge whether it's feasible, just capture the idea.

You should aim to identify 3–5 candidates before evaluating any of them. The best ML project is rarely the first one you think of — it's the one that combines a real business need, available data, and a technically feasible problem.

Phase 1: Should I use ML for this specific problem?

You've identified a candidate. Now put it through five filter questions. If you can't answer "yes" to all five, stop and reconsider — or choose a different candidate from your list.

1. Is there a pattern to learn?

ML finds patterns in historical data. If the output you're trying to predict is genuinely random — or driven by factors you can't measure — no algorithm will help.

Ask yourself: would an experienced engineer, given unlimited time to study the historical data, be able to make better-than-random predictions? If yes, ML can probably automate and scale that intuition. If no — if the outcome is truly unpredictable even in hindsight — ML won't create patterns where none exist.

2. Do I have enough data?

Rules of thumb for minimum dataset sizes:

Algorithm	Rough minimum	Why
Linear regression	30–50 samples	Few parameters to learn
Decision tree / Random Forest	100–300 samples	Needs enough data for meaningful splits
XGBoost	200–500 samples	Sequential correction needs sufficient diversity
Neural network	1,000–10,000+ samples	Many parameters to learn

The practical test: if you have fewer than 100 labelled data points, start with linear regression or k-NN. If those show a signal, you have a genuine pattern. If they don't, consider whether more data is obtainable before investing further.

3. Is the data representative?

Your training data must represent the conditions under which the model will be used. A model trained on summer production data may fail on winter production data. A model trained on one material grade won't necessarily work on another.

Ask: does my historical data cover the full range of conditions the model will encounter in practice?

4. Would a simpler method work?

Before reaching for XGBoost, ask:

• Is there a known physics equation? Use it.
• Would a simple threshold work? You need an if-statement, not a model.
• Would a lookup table work? Faster and more transparent for small design spaces.
• Would a linear regression work? A trendline might be sufficient.

ML earns its keep when the relationship is non-linear, involves many interacting variables, or when the volume of decisions exceeds what a human can handle manually.

5. Can I define what "good enough" means?

Before training a single model, define your success criteria:

• What metric will I use? (R², RMSE, F1, recall — decided by the engineering context)
• What threshold makes the model useful? (R² > 0.85? Recall > 90%? RMSE < 5 MPa?)
• What's the baseline I need to beat? (Current manual method? Expert judgement? A simple rule?)

Define success before you start — not after you see the results.

Phase 2: Frame the problem

Once you've decided ML is worth trying, frame the problem precisely.

Define the inputs and output

Be specific. "Predict quality" is too vague. "Predict whether injection-moulded parts will pass dimensional inspection (within ±0.05mm tolerance) from 8 process parameters recorded per cycle" is a well-defined ML problem.

Identify where labels come from

Common label sources in engineering:

• Inspection records: pass/fail, defect type, dimensional measurements
• Simulation outputs: stress, temperature, flow rate from FEA/CFD
• Test results: tensile strength, fatigue life, surface roughness
• Maintenance logs: failure dates, fault codes, repair actions
• Expert annotations: an experienced operator classifying patterns

If you don't have labels, you can't do supervised learning. You might still use unsupervised methods — but you can't build a predictive model without something to predict.

Define the deployment scenario

• Offline analysis: accuracy matters most, latency doesn't
• Near-real-time: moderate latency, needs to be robust
• Real-time: tight latency, model must be small and fast
• One-time study: explanations matter more than deployment

Phase 3: Prepare the data

Audit your data

• How many rows? Enough for your chosen algorithm?
• How many columns? More features than samples is a warning sign.
• Are there missing values? How many? Random or systematic?
• Are there obvious errors? Plot every column and look for absurdity.
• Is the data balanced? For classification, what's the ratio of classes?
• Are there data leakage risks? Features that carry information about the label that wouldn't be available at prediction time.

Engineer your features

• Create physics-informed features (ratios, products, derived quantities)
• Extract statistical features from time-series data (RMS, peaks, trends)
• Remove features that can't possibly be relevant
• Consider whether interactions between features are physically meaningful

Split the data

• Set aside 20% as a test set — touch it only at the very end
• Use cross-validation on the remaining 80% for model development
• For time-dependent data, split chronologically
• For imbalanced classification, use stratified splits

Phase 4: Build and evaluate the model

Start simple, then add complexity

1. Linear regression / logistic regression — the baseline

2. Random Forest — the reliable workhorse

3. XGBoost with early stopping — the performance maximiser

4. More complex approaches — only if 1–3 clearly aren't sufficient

At each step, compare cross-validated performance. Stop when the improvement isn't worth the added complexity.

Evaluate honestly

• Cross-validation for reliable performance estimates
• The right metric for your problem
• Feature importance to verify the model makes physical sense
• SHAP for individual prediction explanations when needed

Check for red flags

• Training vs test gap: large gap = overfitting
• Physically sensible features: if batch number is the top feature, investigate
• Stable performance across folds: wildly varying fold scores = data problem
• Performance on edge cases: does the model behave reasonably at extremes?

Phase 5: Communicate the results

For your engineering team

Focus on feature importance and SHAP. The feature importance bar chart and a few SHAP waterfall plots are usually more valuable than the accuracy score.

For your manager

A one-page summary:

• Problem: what are you predicting, and why does it matter?
• Result: key metric, with uncertainty (e.g. "R² = 0.88 ± 0.03")
• Insight: top 3 features driving the prediction
• Recommendation: deploy, collect more data, or investigate a process parameter
• Limitations: where the model shouldn't be trusted

For a regulator or quality team

Focus on traceability and validation. SHAP explanations, cross-validation results, and a clear description of the training data and its limitations.

The one-page ML opportunity brief

Before committing to a full project, write a one-page brief. If you can't fill it in convincingly, the project isn't ready.

An engineer who can write this brief is 10× more valuable than one who can only call model.fit(). This is the skill that turns ML knowledge into ML impact.

The complete knowledge base in one paragraph

You started with a question: "What is machine learning?" The answer: it's pattern recognition, automated. You learned that ML models need features and labels, that you must split your data to evaluate honestly, and that supervised learning predicts from labelled examples while unsupervised learning discovers hidden structure. You learned five algorithms — from linear regression to XGBoost — and when to use each. You learned to evaluate models with cross-validation, choose the right metric, and identify which features matter. You learned when neural networks help and when they don't, how to build surrogate models that replace expensive simulations, and how to explain any prediction with SHAP.

But none of that matters if you can't spot the opportunities. The five patterns — manual tasks ripe for automation, simulations begging for surrogates, unexplored data hiding patterns, unknown drivers of known outcomes, and expert knowledge at risk of walking out the door — are your radar. Train yourself to see them, and you'll find ML applications everywhere you look.

The algorithms are tools. The evaluation methods are quality assurance. The explanations are how you build trust. And this checklist is how you make sure you're solving the right problem in the first place.

The next step is yours.

Key takeaways

Finding the right problem is harder than solving it. The five opportunity patterns — manual tasks, repeated simulations, unexplored data, unknown drivers, retiring experts — are your radar for spotting ML applications in your own workflow. Train yourself to recognise them.

Ask "should I use ML?" before "which algorithm should I use?" The five filter questions will save you more time than any technical skill. If the answer to any of them is no, fix that before writing code.

Frame the problem precisely. "Predict quality" is vague. "Predict dimensional pass/fail from 8 process parameters using 2,000 historical cycles" is actionable.

Start simple, add complexity only when justified. Linear regression → Random Forest → XGBoost. Stop when the improvement isn't worth the added complexity.

Communicate results in terms your audience cares about. Feature importance for engineers. Business impact for managers. Traceability for regulators.

Write the one-page brief before committing. If you can't articulate the problem, the data, the success criterion, and the risk in one page, the project isn't ready.