01
Deploying a model isn't the finish line
An ML model isn't software you build once and walk away from. You collect data, train, evaluate and deploy, and then you keep watching how it performs and retraining it on new data, over and over. The scene below shows that loop. Try picking a stage.
- Deploy
- Put the model into production
The ML lifecycle: retraining at the end loops back to preparing data
Deploy: Put the model into production. The light-blue stages (deploy, monitor, retrain) are where MLOps does most of its work.
With ordinary software, if the code doesn't change, the behavior doesn't either. An ML model is different: even with the exact same code, its performance changes when the incoming data changes. That's why you have to version and watch not just the code, but the data and the model too. MLOps takes DevOps, the practice that ties software development and operations together, and stretches it to fit machine learning.
Research on real ML systems found that the code that actually trains the model is only a small part of the whole. The work around it, like collecting and validating data, building features, deploying and monitoring, is much bigger. MLOps automates and records that surrounding work, so there are fewer mistakes from people doing it by hand.
| Maturity level | What it looks like |
|---|---|
| Manual | Someone trains by hand in a notebook and hands over a model file. Hard to rebuild |
| Pipeline automation | A pipeline runs from data to registration, and retrains automatically when new data arrives |
| CI/CD automation | Testing and deploying the pipeline’s own code is automated too |
If a question says… → pick this
- “A repeating loop of data → training → evaluation → deployment → monitoring → retraining”ML lifecycle
- “Automating and standardizing how ML models are built and run”MLOps
- “If the code doesn’t change, model performance won’t either”Wrong — performance changes when the data changes
- “Versioning code, data and models together”Reproducibility in MLOps
The first step is turning the work people did by hand into a pipeline that runs the same steps in the same order.
02
Pipelines and reproducibility
A training pipeline runs data validation, feature engineering, training, evaluation and model registration automatically, in a fixed order. It kicks off when new data piles up or on a schedule. The evaluation step has a quality bar, and any model that falls short doesn't get through to the next step. In the scene below, try making this model miss the bar.
- Validate data
- Passed
- Build features
- Passed
- Train
- Passed
- Evaluate
- Passed
- Register model
- Passed
A training pipeline that runs automatically whenever new data arrives
Every step passed automatically and the new model was registered. Because no one does it by hand, the same steps run in the same order every time, and you can rebuild exactly this model later.
Experiment tracking
Every training run records the data, code version, hyperparameters and resulting metrics so you can compare runs.
Metadata and lineage
Traces which data and which steps a model came from, end to end.
Model registry
Keeps model versions, evaluation results and approval status (staging, production) in one place, and lets you roll back.
Data validation
Automatically catches missing values, out-of-range values and columns whose format changed, before training starts.
If a question says… → pick this
- “Runs training, evaluation and deployment steps automatically, one after another”ML pipeline (orchestration)
- “Logs and compares hyperparameters and metrics for each run”Experiment tracking
- “Traces which data a model came from”Metadata and lineage
- “Manages model versions and approval status, and rolls back”Model registry
- “Stops a model that misses the bar from deploying automatically”Quality gate at the evaluation step
Now it's time to release the registered model into the real service. There's more than one way to do that.
Advertisement
03
Deployment: when to predict, and how to swap models
First, there are two ways to serve predictions, depending on when you make them. Batch inference predicts on a large amount of data at once on a schedule and stores the results. Real-time (online) inference predicts the moment a request comes in and sends the answer straight back. There's also edge inference, where the prediction runs right on the device.
| Type | Good fit | What it’s like |
|---|---|---|
| Batch inference | Scoring churn risk for every customer each night | Cheap and simple. Results stay the same until the next run |
| Real-time inference | Spotting fraud at checkout, search recommendations | Has to respond fast. The endpoint stays up and scales up and down |
| Edge inference | A phone camera, factory machines | Fast with no network. The model has to be small |
When you switch to a new model, it's common to spread the risk out instead of switching everything at once. In the scene below, try each rollout method.
- Requests v2 answered
- 2 / 20
- Rollback
- Only 10% affected; set the share back to 0
Ways to roll out a new model v2 (green = requests v2 answered)
Only 10% of requests go to v2. You watch how it does, and if it looks fine you raise the share step by step. If something goes wrong, only a few users are affected.
Whichever method you use, you check with real metrics that the new model is actually better than the old one. A/B testing in the evaluation metrics guide is how you do that. You also keep earlier versions in the registry so you can roll back at any time.
If a question says… → pick this
- “Predicts on large volumes on a schedule”Batch inference
- “Predicts instantly with low latency for each request”Real-time (online) inference, endpoint
- “Sends part of the traffic to the new model and gradually increases it”Canary deployment
- “The new model gets copies of requests and only predicts; users never see its output”Shadow deployment
- “Two environments, traffic switched all at once, instant rollback”Blue/green deployment
Once a model is deployed, you can't just relax. The world the model sees keeps changing.
Pro only
That’s the free part of this guide
The other 5 chapters are open to Pro subscribers. Each one continues with 3D visuals and “if a question says…” tips, and the last chapter lets you check yourself with a short quiz.
- 04Monitoring and drift
- 05Retraining
- 06Training-serving skew and feature stores
- 07LLMOps
- 08Choosing an approach
PasslyExam
You have read the concept. Now test it on exam questions
PasslyExam offers original practice questions, explanations and full mock exams built around each certification’s exam guide. The explanations cover why the right option is right and why the look-alike options are wrong. Sign up to practice for free every day.
Exams that cover this concept