ഇന്ത്യൻ ML ടീമുകളിൽ 85% പ്രോജക്ടുകളും Jupyter notebook-ൽ തന്നെ അവസാനിക്കുന്നു — production-ൽ deploy ചെയ്യപ്പെടാതെ. FastAPI + Docker + AWS Mumbai (ap-south-1) എന്ന combination ഉപയോഗിച്ച് ₹500-1,000/month ചെലവിൽ ഒരു ML model serve ചെയ്യാം. Model monitoring-നായി Evidently AI (open source) ഉപയോഗിക്കുക — data drift കണ്ടെത്തി silent model degradation തടയാം.
Deploying a machine learning model to production in India requires four components: a FastAPI inference endpoint, Docker container for packaging, a cloud host in the AWS Mumbai or GCP Mumbai region, and monitoring for data drift. The gap between Jupyter notebook accuracy and production reliability closes when teams standardise on model serialisation, input validation, health checks, and automated retraining triggers.
Why 85% of Indian ML Projects Never Reach Production
Walk into any mid-sized tech office in Bengaluru or Hyderabad, and you will find data scientists who can build a gradient boosting model that outperforms industry benchmarks — but whose models never actually make it into the hands of users. The notebook-to-production gap is one of the most expensive silent failures in Indian ML teams, and it has three distinct root causes that rarely appear in post-mortems.
The first is infrastructure ownership confusion. Data scientists own the model; product engineers own the backend; DevOps owns the servers. Nobody owns the piece in between — the model serving layer. When a trained model sits as a .pkl file in a data scientist's Google Drive, it is not a deployed system. It is an experiment that stopped at the interesting part.
The second root cause is the absence of a standardised serving pattern. Teams that have deployed three models without a pattern will deploy the fourth one differently again. One model becomes a Flask script that runs directly on a VM. The next one gets embedded into a Django view. A third ends up in a Jupyter notebook that someone opens manually when a business stakeholder needs a prediction. Each approach creates its own maintenance burden, and none of them scales.
The third root cause is no monitoring plan. Indian ML teams that do manage to get a model into production often declare victory and move on. Six months later, the model is still running — but the predictions are quietly degrading because production data has shifted away from the distribution the model was trained on. No alert fires. Business metrics slowly worsen. The model gets blamed last, after every other hypothesis has been exhausted.
Consider a healthcare startup in Kochi that built a patient churn prediction model with 87% validation accuracy. The data science team spent four months tuning it. When the model was ready, there was no agreed-upon API format, no serving infrastructure, and no engineering sprint allocated to deployment. The model sat untouched for eight months while the product team manually reviewed churn risks in spreadsheets. When the model was finally productionised, the training data was eighteen months old — and the first drift report showed it was performing at 61% on live data. The opportunity cost of not deploying earlier was compounded by the cost of deploying a model that had already aged out.
The Minimal Production Stack
The de facto standard across Indian ML teams in 2026 is FastAPI for model serving combined with Docker for packaging. This combination emerged because it resolves the two most common failure modes: inconsistent environments and undocumented input contracts.
FastAPI earns its position for three specific reasons. Its async support handles concurrent prediction requests without blocking — important when your inference endpoint is receiving multiple calls from different parts of your application simultaneously. Its automatic OpenAPI documentation generation means that when your API is running, any engineer on the team can open /docs and see exactly what inputs the model expects and what outputs it returns, without reading your code. And Pydantic-based input validation rejects malformed requests before they reach your model — catching type mismatches and missing fields at the boundary layer rather than inside your prediction logic.
Docker earns its position by eliminating the single most demoralising phrase in any ML deployment: "it works on my machine." A containerised model carries its Python version, library versions, and runtime dependencies inside the image. The same container that runs on a data scientist's MacBook will run identically on an Ubuntu EC2 instance in Mumbai.
Every ML service needs three files as its minimum viable structure. main.py contains the FastAPI application, the model loading logic, the prediction endpoint, and the health check endpoint. Dockerfile defines how to build the container image — base image, dependency installation, app code copy, and startup command. requirements.txt pins every library version, including transitive dependencies, so that a rebuild six months later produces an identical runtime environment.
Model Serialisation — Picking the Right Format
How you save a trained model determines how reliably you can load it in production, how portable it is across different serving environments, and whether you can roll back to a previous version when something goes wrong.
For sklearn models, joblib outperforms pickle for numpy-heavy objects — the serialisation speed difference becomes meaningful when your model contains large arrays (random forests with hundreds of trees, for instance). Save with joblib.dump(model, 'model_v1.2.joblib') and use the .joblib extension to make the format explicit at a glance.
For PyTorch, you have two paths. Saving the state dict (torch.save(model.state_dict(), 'model.pt')) is the standard approach for continued training or fine-tuning. For deployment, TorchScript produces a serialised model that does not require the original class definition to load — this matters when your serving container does not have the training codebase available. Use torch.jit.script(model) or torch.jit.trace(model, example_input) depending on whether your model uses dynamic control flow.
For TensorFlow models, the SavedModel format (model.save('saved_model/')) is the correct choice for deployment. It bundles the computation graph, weights, and serving signatures into a directory that TensorFlow Serving or any inference framework can load directly — without needing your training code or even TensorFlow itself in some cases.
ONNX deserves mention as the cross-framework format. If your team trains in PyTorch but needs to serve with ONNX Runtime (which is faster than PyTorch inference on CPU for many model types), or if you need to move a model between frameworks, ONNX is the conversion layer. The torch.onnx.export function handles most standard architectures.
Version management should follow semantic versioning — churn_model_v2.1.0.joblib with a corresponding metadata JSON file that records training date, training data date range, validation metrics, and the feature list. Store both in an S3 bucket in ap-south-1. Your FastAPI startup logic can pull the latest approved version from S3 on container startup, which means deploying a model update requires only updating a pointer in your config — not rebuilding and redeploying the container image.
Building the FastAPI Inference Endpoint
The single most common mistake made by Indian ML beginners writing their first serving script is loading the model inside the prediction function — which means deserialising a potentially multi-hundred-megabyte file on every single API request. A model that takes 800ms to load gets called 200 times per minute and suddenly your inference endpoint has an 800ms floor on every response.
The correct pattern loads the model once at application startup using FastAPI's lifespan context or the deprecated but still widely used @app.on_event("startup") decorator. The model object is stored in application state and reused across all requests.
A well-structured FastAPI ML service has four components. First, a Pydantic input schema that defines the exact shape of a prediction request — field names, types, and constraints. If your churn model expects days_since_last_login: int and receives a string, Pydantic rejects the request with a 422 error and a descriptive message before your model ever sees the input. Second, a /predict endpoint that wraps the prediction call in a try/except block — catching model-side errors and returning structured error responses rather than 500 stack traces. Third, a /health endpoint that returns 200 OK with the model version and load timestamp, used by your load balancer to confirm the service is ready before routing traffic. Fourth, request batching support — accepting a list of inputs and returning a list of predictions in a single call, which dramatically improves throughput when your client can batch requests.
Logging every request's input features and output prediction to a structured log store (CloudWatch Logs in AWS, or a simple file that gets shipped to S3 hourly) is not optional — it is the prerequisite for any debugging, drift detection, or model auditing you will need to do later.
Containerising with Docker
A Dockerfile for a Python ML service has a layer ordering that maximises Docker's build cache — critical when you are iterating on application code and want rebuilds to complete in seconds rather than minutes.
Start with python:3.11-slim as your base image. The full Python image includes compilers, headers, and build tools that your running container does not need — slim saves over 500MB from the final image size. Copy requirements.txt first and run pip install before copying your application code. Docker caches each layer; if your requirements have not changed, the pip install layer is reused on the next build even after you modify main.py.
Large model files present a specific challenge. Baking a 2GB model file into a Docker image means every deployment pulls a 2GB image, and your container registry costs scale with image size. The better pattern: store the model in S3 and download it at container startup using boto3. Your FastAPI lifespan function downloads the model file to /tmp/model.joblib on first start. This keeps your image under 200MB and means model updates require no image rebuild at all — just update the S3 pointer.
For local development, a docker-compose.yml file lets you run the service with environment variables injected from a .env file — AWS credentials, model S3 path, feature thresholds. Never bake credentials into the Dockerfile or image. On AWS, the container picks up credentials from the EC2 instance's IAM role automatically; locally, you use the .env file via docker-compose.
Indian Cloud Context — Choosing a Region
AWS ap-south-1 (Mumbai) is the default for most Indian ML teams, and the reasoning is practical: it offers the lowest latency for users in India, has the broadest selection of ML-relevant instance types including GPU instances, and the largest ecosystem of Indian developers who have already solved the region-specific quirks. Most Indian fintech, healthtech, and edtech companies already have their primary AWS presence in Mumbai, so co-locating your ML inference endpoint with your application reduces cross-region data transfer costs.
GCP asia-south1 (Mumbai) has two compelling advantages: competitive pricing on standard compute compared to AWS, and TPU access for TensorFlow workloads. If your team trains large TensorFlow models and has considered switching away from GPU clusters, GCP's TPU v4 availability in Mumbai is worth evaluating. GCP's Vertex AI also has stronger managed ML pipeline support than AWS SageMaker Pipelines for teams already in the Google ecosystem.
Azure Central India is the right choice primarily when the business is already a Microsoft enterprise customer — Azure DevOps integration, Active Directory for access control, and existing Azure credit agreements make it practical. Azure's ML-specific instance variety in the India region is narrower than AWS Mumbai, which can constrain GPU options for inference-heavy workloads.
On GPU costs: AWS p3.2xlarge instances (1x NVIDIA V100) run approximately ₹220/hour on spot pricing and ₹600/hour on-demand in ap-south-1. For batch inference jobs that can tolerate interruption, spot instances cut cost by 60-70%. For latency-sensitive real-time inference where you cannot afford cold starts, on-demand is necessary. For many classification and regression models, quantised CPU inference on a ₹500/month server is genuinely viable — reducing weights from float32 to int8 cuts memory usage by approximately 75% and inference time by 2-4x on CPU, often bringing a model that required a GPU into CPU-only territory at acceptable latency.
MLOps Maturity Levels
Google's MLOps maturity framework describes three levels, and understanding where your team sits determines what you should build next.
Level 0 is manual everything. A data scientist trains a model on their local machine or a cloud notebook, validates it by hand, copies the serialised file to a server, and restarts the Flask script. Retraining happens when someone notices performance has degraded — usually from a business complaint. This describes most Indian startups with ML today. It is not shameful; it is where every team starts. The problem is that it does not scale: the data scientist becomes the single point of failure for every model update.
Level 1 introduces an automated training pipeline. Fresh data flows into a scheduled retraining job. MLflow tracks every experiment — hyperparameters, metrics, model artifacts, and the data snapshot used for training. When the retrained model passes validation thresholds, it gets pushed to a staging endpoint. A/B testing or shadow mode deployment verifies that the new model performs at least as well as the current production model before traffic shifts. This is where most Indian ML teams should aim to be in 2026. Kedro is the preferred pipeline orchestration tool in Bengaluru-based teams; MLflow handles experiment tracking across both Bengaluru and Kerala teams.
Level 2 is full CI/CD for ML: data drift automatically triggers a retraining job, the pipeline runs unattended, and human review only happens when the automated validation tests fail. This requires mature data infrastructure, reliable automated testing, and confidence built over months of Level 1 operation. It is appropriate for ML products where retraining frequency is high and the model directly drives revenue. It remains rare in Indian startups. Jumping from Level 0 directly to Level 2 almost always fails — the complexity overwhelms teams that have not yet built the observability and testing foundations that Level 2 depends on. Level 1 delivers roughly 80% of the operational value at 20% of the complexity.
Model Monitoring — What Indian Teams Skip
Silent model degradation is a specific failure pattern where a deployed model's predictions become progressively less accurate over time, no automated alert fires, and the degradation only surfaces when a business analyst notices that a KPI has been trending wrong for months. It happens because production data drifts away from the distribution the model was trained on — user behaviour changes, upstream data pipelines shift column encodings, a new product feature changes the meaning of an existing feature. The model sees inputs that increasingly differ from its training distribution and responds with confident but wrong predictions.
Evidently AI is the open-source tool that Indian ML teams use to build monitoring without standing up a complex paid platform. It generates data drift reports that compare a window of production inputs against your training data baseline, flagging which features have drifted beyond a statistical threshold. Self-hosted, it runs in a Docker container for free. The output is an HTML report or a JSON summary that your monitoring pipeline can parse.
Three metrics that every deployed Indian ML model should track: input feature distribution shifts (are the values coming into the model still in the range the model was trained on?), prediction distribution changes (is the model returning a different proportion of class labels than it did at launch?), and business KPI correlation (does the predicted outcome still correlate with the actual outcome in your labelled feedback data, if you collect it?).
A practical minimum viable monitoring setup: run an Evidently drift report weekly as a cron job on your EC2 or Lambda, compare the past 7 days of production inputs against the training data snapshot stored in S3, and email the ML team a JSON summary. If drift exceeds your threshold on more than two features, the email subject line changes to include an alert flag. This takes one afternoon to set up and prevents the class of problems where a model has been quietly failing for months before anyone noticed.
Common Deployment Mistakes by Indian ML Teams
Not validating inputs at the boundary. When a model receives an unexpected data type — a null where it expects a float, a string category it has never seen — it often does not raise an exception. It silently produces a wrong prediction that downstream code treats as valid. Pydantic at the API layer catches these before they reach the model.
No model versioning. Deploying a new model by overwriting the old file means you cannot roll back when the new model underperforms in production. Every model version should have a unique identifier, and your serving infrastructure should be able to swap between versions by changing a config value — not by redeploying code.
Hardcoded classification thresholds. The default 0.5 threshold for binary classification is almost never the optimal operating point for production. A fraud detection model needs a different precision-recall trade-off than a content recommendation model. The threshold should be configurable via environment variable, not buried in code — so that calibrating it in production requires a config change, not a deployment.
Missing request and response logging. Without a log of what inputs the model received and what predictions it returned, you cannot debug production anomalies, audit model decisions, or build the labelled dataset you will need for the next retraining cycle. Log every prediction with a timestamp and a unique request ID from day one.
No graceful degradation. What does your application do when the ML inference endpoint is unreachable? If the answer is "it crashes" or "it returns a 500 error to the user," you have a reliability problem that has nothing to do with ML and everything to do with system design. Define a fallback behaviour — a rule-based fallback prediction, a cached result, or a "service temporarily unavailable" response — and implement it at the client before you go live.
Frequently Asked Questions
What is the cheapest way to serve a machine learning model in India?
For low-traffic internal tools, the cheapest production-viable option is a ₹500/month DigitalOcean droplet (1 vCPU, 1GB RAM) running a FastAPI app inside Docker with a pre-loaded sklearn or small PyTorch model. For higher traffic, AWS Lambda (ap-south-1 Mumbai) with a containerised model costs near-zero for moderate request volumes — under 1 million invocations/month fits within the free tier. GPU inference in India currently runs ₹250-600/hour on AWS p3 spot instances; for production inference at scale, model quantisation (reducing from float32 to int8) typically cuts memory by 75% and allows CPU inference at acceptable latency for many use cases, eliminating the GPU cost entirely.
How do Indian startups handle model retraining in production?
Most Indian ML teams at the startup stage use a manual trigger approach: data scientists monitor model metrics weekly, and when accuracy degrades past a threshold, they manually kick off a retraining job on a cloud notebook or script. Automated retraining pipelines (Level 1 MLOps) involve scheduled jobs that pull fresh labelled data, retrain the model, run validation tests, and push the new model to a staging endpoint for A/B testing. Kedro and MLflow are the most common open-source tools used by Bengaluru and Kerala ML teams for pipeline orchestration and experiment tracking respectively. Fully automated CI/CD for ML (Level 2 MLOps) — where a data drift trigger automatically initiates retraining — is still rare in Indian startups; most are at Level 0 or transitioning to Level 1.
Should I use AWS SageMaker or build my own ML serving infrastructure in India?
For teams with fewer than 3 ML engineers, AWS SageMaker (Mumbai region) reduces operational overhead substantially — managed endpoints handle scaling, monitoring, and deployment without infrastructure expertise. The trade-off is cost: a SageMaker ml.t2.medium real-time endpoint runs ₹3,500-4,500/month continuously, versus ₹500-1,000/month for a self-managed FastAPI server on a small EC2 instance. For batch inference (not real-time), SageMaker Batch Transform is cost-effective since you only pay per inference job. The practical guidance: use SageMaker if your team lacks DevOps expertise or if model serving reliability is critical; build your own FastAPI+Docker stack if you have the engineering bandwidth and want cost control at scale.