𝐁𝐞𝐟𝐨𝐫𝐞 𝐜𝐡𝐚𝐧𝐠𝐢𝐧𝐠 𝐲𝐨𝐮𝐫 𝐌𝐋 𝐦𝐨𝐝𝐞𝐥, 𝐜𝐡𝐞𝐜𝐤 𝐲𝐨𝐮𝐫 𝐝𝐚𝐭𝐚.
When a model performs badly, the first thing we often do is try a different algorithm.

Sometimes that works.

But before doing that, I usually look at the dataset. 🔍

I check things like:

🔹 Missing values
🔹 Duplicate records
🔹 Outliers
🔹 Wrong data types
🔹 Class imbalance
🔹 Data leakage
🔹 High-cardinality columns
🔹 Features with little useful information

There is no point spending hours tuning a model if the dataset itself has problems. ⚠️

A simple workflow I prefer is:

📥 𝑹𝒂𝒘 𝑫𝒂𝒕𝒂
↓
🔎 𝑪𝒉𝒆𝒄𝒌 𝑸𝒖𝒂𝒍𝒊𝒕𝒚
↓
🧹 𝑪𝒍𝒆𝒂𝒏
↓
📊 𝑨𝒏𝒂𝒍𝒚𝒛𝒆
↓
🤖 𝑻𝒓𝒂𝒊𝒏
↓
📈 𝑴𝒐𝒏𝒊𝒕𝒐𝒓

Data quality is not just something to deal with before machine learning. It affects every step that comes after it.

So when a model is not performing as expected, don't immediately change the model.

🔍 Take another look at the data first. https://youtube.com/playlist?list=PL0nX4ZoMtjYHTtowSzzB2gVH2AuuoF9WW&si=EhLKvJCVlYQXknOs
Use Data quality checker tool: https://datasetdoctor.fastapicloud.dev

What is FastAPI? https://www.youtube.com/watch?v=yUsDgLZPDyI

Your model can look excellent and still be wrong.

One of the first things I check when evaluating an ML dataset is data leakage. 🔍

Data leakage happens when information that would not actually be available at prediction time gets into the training data.

For example:

🏥 Healthcare
You are predicting whether a patient will be admitted, but your dataset includes a field recorded after admission.

💳 Fraud detection
You are predicting fraud, but one of the features is created after the transaction has already been investigated.

📦 Customer churn
You are predicting who will leave, but the training data contains information that only becomes available after the customer leaves.

The result?

Your model may show:

📈 98% accuracy
📈 Excellent validation results
📈 Great performance during testing

Then you put it into production...

And the performance drops.

The problem was not necessarily the model.

The model had access to information it would never have in the real world.

That is why I don't look at model performance alone.

I also ask:

🔎 Where did each feature come from?
⏱️ When was it created?
🎯 Would this information actually be available when making the prediction?

A high score is not always a good score.

Sometimes, it is a warning sign.

Check out data quality issues
https://youtube.com/playlist?list=PL0nX4ZoMtjYHTtowSzzB2gVH2AuuoF9WW&si=EhLKvJCVlYQXknOs

Also checkout data quality checker tool https://datasetdoctor.fastapicloud.dev

#MachineLearning #DataScience #AI #DataLeakage #MLOps #Python

Set Up FastAPI Development Environment with uv & VS Code | FastAPI Full Course(Episode 4)
https://www.youtube.com/watch?v=G60LwkySnwQ

FastAPI Full Course Episode 5: FastAPI Parameters & Request Bodies(Path, Query & Pydantic) https://www.youtube.com/watch?v=-tkww4I4Vfg&t=638s

FastAPI Episode 6: Pydantic Validation Requests and Response Models
https://youtu.be/G5cKA88-6vc

The hardest part of building AI applications isn't writing the prompt or calling the model. In the last two weeks, I learned that keeping the backend from turning into spaghetti code once you move past the tutorial phase.

When you're wiring up an AI document pipeline in FastAPI, a few things quickly become non-negotiable:



• Payload Guardrails: If your Pydantic schemas aren't catching malformed JSON, missing nested fields, or bad Enums at the door, your AI service will fail unpredictably downstream.



• Route Isolation: Mixing your raw API endpoints with validation logic and business rules makes refactoring a nightmare by week three.



• The Persistence Gap: Transitioning from mock in-memory data structures to a real relational database and a vector store for RAG is where most clean prototypes start to break down.



If you're building production backends for AI and ML features, where do you usually draw the line between keeping things simple and over-engineering your architecture?



You can learn about FastAPI: https://www.youtube.com/playlist?list=PLQNCas8_eikM



#FastAPI #Python #BackendEngineering #SoftwareArchitecture #APIs #Pydantic #ArtificialIntelligence #MachineLearning #RAG

FastAPI Episode 7: Advanced Pydentic Model Design
https://www.youtube.com/watch?v=j4lLM6tWKKk

When building a FastAPI application, I pay close attention to how data is validated before it reaches the business logic.

That is where Pydantic becomes particularly useful.

Pydantic lets us define the structure and rules for the data our application accepts, instead of scattering validation checks throughout the codebase.

I use it to define and enforce the data contract at the application boundary.

For example:

𝑐𝑙𝑎𝑠𝑠 𝑃𝑟𝑜𝑐𝑒𝑠𝑠𝑖𝑛𝑔𝐶𝑜𝑛𝑓𝑖𝑔(𝐵𝑎𝑠𝑒𝑀𝑜𝑑𝑒𝑙):

𝑐ℎ𝑢𝑛𝑘_𝑠𝑖𝑧𝑒: 𝑖𝑛𝑡 = 𝐹𝑖𝑒𝑙𝑑(𝑔𝑡=0)

𝑜𝑣𝑒𝑟𝑙𝑎𝑝: 𝑖𝑛𝑡 = 𝐹𝑖𝑒𝑙𝑑(𝑔𝑒=0)

@𝑚𝑜𝑑𝑒𝑙_𝑣𝑎𝑙𝑖𝑑𝑎𝑡𝑜𝑟(𝑚𝑜𝑑𝑒="𝑎𝑓𝑡𝑒𝑟")

𝑑𝑒𝑓 𝑣𝑎𝑙𝑖𝑑𝑎𝑡𝑒_𝑐𝑜𝑛𝑓𝑖𝑔(𝑠𝑒𝑙𝑓):

𝑖𝑓 𝑠𝑒𝑙𝑓.𝑜𝑣𝑒𝑟𝑙𝑎𝑝 >= 𝑠𝑒𝑙𝑓.𝑐ℎ𝑢𝑛𝑘_𝑠𝑖𝑧𝑒:

𝑟𝑎𝑖𝑠𝑒 𝑉𝑎𝑙𝑢𝑒𝐸𝑟𝑟𝑜𝑟("𝑜𝑣𝑒𝑟𝑙𝑎𝑝 𝑚𝑢𝑠𝑡 𝑏𝑒 𝑠𝑚𝑎𝑙𝑙𝑒𝑟 𝑡ℎ𝑎𝑛 𝑐ℎ𝑢𝑛𝑘_𝑠𝑖𝑧𝑒")

𝑟𝑒𝑡𝑢𝑟𝑛 𝑠𝑒𝑙𝑓

Both fields can be individually valid, while their combination is not.

That is the difference between:

Field validation → Is this value valid?

Model validation → Is this combination valid?

With Pydantic, 𝙁𝙞𝙚𝙡𝙙(), 𝙛𝙞𝙚𝙡𝙙_𝙫𝙖𝙡𝙞𝙙𝙖𝙩𝙤𝙧(), 𝙖𝙣𝙙 𝙢𝙤𝙙𝙚𝙡_𝙫𝙖𝙡𝙞𝙙𝙖𝙩𝙤𝙧() let us keep these rules close to the schema.

The result is a cleaner boundary:

𝙍𝙚𝙦𝙪𝙚𝙨𝙩 → 𝙑𝙖𝙡𝙞𝙙𝙖𝙩𝙞𝙤𝙣 → 𝘽𝙪𝙨𝙞𝙣𝙚𝙨𝙨 𝙇𝙤𝙜𝙞𝙘 → 𝘿𝙖𝙩𝙖𝙗𝙖𝙨𝙚 / 𝘼𝙄

For production AI applications, this matters. Document metadata, processing parameters, search filters, and structured AI outputs all need predictable contracts.

Good schemas do more than describe data. They protect the rest of the system.


You can explore more: https://www.youtube.com/playlist?list=PLQNCas8_eikM


#Python #Pydantic #PydanticV2 #FastAPI #BackendEngineering #AIEngineering

✅ FastAPI Episode 8: Organizing Routes with APIRouter
https://youtu.be/SGPXI7bzEDM

This is the github starter for the project based FastAPI course tutorial
https://github.com/epythonlab2/ai-document-api