๐ก ๐๐ผ๐ ๐ฑ๐ผ๐ฒ๐ ๐ฎ๐ป ๐๐๐ (๐น๐ฎ๐ฟ๐ด๐ฒ ๐น๐ฎ๐ป๐ด๐๐ฎ๐ด๐ฒ ๐บ๐ผ๐ฑ๐ฒ๐น) ๐ฎ๐ฐ๐๐๐ฎ๐น๐น๐ ๐น๐ฒ๐ฎ๐ฟ๐ป?
Itโs a journey through 3 key phases:
1๏ธโฃ ๐ฆ๐ฒ๐น๐ณ-๐ฆ๐๐ฝ๐ฒ๐ฟ๐๐ถ๐๐ฒ๐ฑ ๐๐ฒ๐ฎ๐ฟ๐ป๐ถ๐ป๐ด (๐จ๐ป๐ฑ๐ฒ๐ฟ๐๐๐ฎ๐ป๐ฑ๐ถ๐ป๐ด ๐๐ฎ๐ป๐ด๐๐ฎ๐ด๐ฒ)
The model is trained on massive text datasets (Wikipedia, blogs, websites). This is where the transformer architecture comes into picture which you can simply think of it as neural networks that sees words and predicts what comes next.
For example:
โA flash flood watch will be in effect all _____.โ
The model ranks possible answers like โnight,โ โday,โ or even โgiraffe.โ Over time, it gets really good at picking the right one.
2๏ธโฃ ๐ฆ๐๐ฝ๐ฒ๐ฟ๐๐ถ๐๐ฒ๐ฑ ๐๐ฒ๐ฎ๐ฟ๐ป๐ถ๐ป๐ด (๐จ๐ป๐ฑ๐ฒ๐ฟ๐๐๐ฎ๐ป๐ฑ๐ถ๐ป๐ด ๐๐ป๐๐๐ฟ๐๐ฐ๐๐ถ๐ผ๐ป๐)
Next, we teach it how humans like their answers. Thousands of examples of questions and well-crafted responses are fed to the model. This step is smaller but crucial, itโs where the model learns to align with human intent.
3๏ธโฃ ๐ฅ๐ฒ๐ถ๐ป๐ณ๐ผ๐ฟ๐ฐ๐ฒ๐บ๐ฒ๐ป๐ ๐๐ฒ๐ฎ๐ฟ๐ป๐ถ๐ป๐ด (๐๐บ๐ฝ๐ฟ๐ผ๐๐ถ๐ป๐ด ๐๐ฒ๐ต๐ฎ๐๐ถ๐ผ๐ฟ)
Finally, the model learns to improve its behavior based on feedback. Humans rate its answers (thumbs up or thumbs down), and the model adjusts.
This helps it avoid harmful or wrong answers and focus on being helpful, honest, and safe.
Through this process, the model learns patterns and relationships in language, which are stored as numerical weights. These weights are then compressed into the parameter file, the core of what makes the model function.
โ๏ธ So what happens when you ask a question?
The model breaks your question into tokens (small pieces of text, turned into numbers). It processes these numbers through its neural networks and predicts the most likely response.
For example:
โWhat should I eat today?โ might turn into numbers like [123, 11, 45, 78], which the model uses to calculate the next best words to give you the answer.
โ๏ธBut hereโs something important: every model has a token limit -> a maximum number of tokens it can handle at once. This can vary between small and larger models. Once it reaches that limit, it forgets the earlier context and focuses only on the most recent tokens.
Finally, you can imagine an LLM as just two files:
โก๏ธ ๐ฃ๐ฎ๐ฟ๐ฎ๐บ๐ฒ๐๐ฒ๐ฟ ๐ณ๐ถ๐น๐ฒ โ This is the big file, where all the knowledge lives. Think of it like a giant zip file containing everything the model has learned about language.
โก๏ธ ๐ฅ๐๐ป ๐ณ๐ถ๐น๐ฒ โ This is the set of instructions needed to use the parameter file. It defines the modelโs architecture, handles text tokenization, and manages how the model generates outputs.
Thatโs a very simple way to break down how LLMs work!
These models are the backbone of AI agents, so lets not forget about them ๐
Itโs a journey through 3 key phases:
1๏ธโฃ ๐ฆ๐ฒ๐น๐ณ-๐ฆ๐๐ฝ๐ฒ๐ฟ๐๐ถ๐๐ฒ๐ฑ ๐๐ฒ๐ฎ๐ฟ๐ป๐ถ๐ป๐ด (๐จ๐ป๐ฑ๐ฒ๐ฟ๐๐๐ฎ๐ป๐ฑ๐ถ๐ป๐ด ๐๐ฎ๐ป๐ด๐๐ฎ๐ด๐ฒ)
The model is trained on massive text datasets (Wikipedia, blogs, websites). This is where the transformer architecture comes into picture which you can simply think of it as neural networks that sees words and predicts what comes next.
For example:
โA flash flood watch will be in effect all _____.โ
The model ranks possible answers like โnight,โ โday,โ or even โgiraffe.โ Over time, it gets really good at picking the right one.
2๏ธโฃ ๐ฆ๐๐ฝ๐ฒ๐ฟ๐๐ถ๐๐ฒ๐ฑ ๐๐ฒ๐ฎ๐ฟ๐ป๐ถ๐ป๐ด (๐จ๐ป๐ฑ๐ฒ๐ฟ๐๐๐ฎ๐ป๐ฑ๐ถ๐ป๐ด ๐๐ป๐๐๐ฟ๐๐ฐ๐๐ถ๐ผ๐ป๐)
Next, we teach it how humans like their answers. Thousands of examples of questions and well-crafted responses are fed to the model. This step is smaller but crucial, itโs where the model learns to align with human intent.
3๏ธโฃ ๐ฅ๐ฒ๐ถ๐ป๐ณ๐ผ๐ฟ๐ฐ๐ฒ๐บ๐ฒ๐ป๐ ๐๐ฒ๐ฎ๐ฟ๐ป๐ถ๐ป๐ด (๐๐บ๐ฝ๐ฟ๐ผ๐๐ถ๐ป๐ด ๐๐ฒ๐ต๐ฎ๐๐ถ๐ผ๐ฟ)
Finally, the model learns to improve its behavior based on feedback. Humans rate its answers (thumbs up or thumbs down), and the model adjusts.
This helps it avoid harmful or wrong answers and focus on being helpful, honest, and safe.
Through this process, the model learns patterns and relationships in language, which are stored as numerical weights. These weights are then compressed into the parameter file, the core of what makes the model function.
โ๏ธ So what happens when you ask a question?
The model breaks your question into tokens (small pieces of text, turned into numbers). It processes these numbers through its neural networks and predicts the most likely response.
For example:
โWhat should I eat today?โ might turn into numbers like [123, 11, 45, 78], which the model uses to calculate the next best words to give you the answer.
โ๏ธBut hereโs something important: every model has a token limit -> a maximum number of tokens it can handle at once. This can vary between small and larger models. Once it reaches that limit, it forgets the earlier context and focuses only on the most recent tokens.
Finally, you can imagine an LLM as just two files:
โก๏ธ ๐ฃ๐ฎ๐ฟ๐ฎ๐บ๐ฒ๐๐ฒ๐ฟ ๐ณ๐ถ๐น๐ฒ โ This is the big file, where all the knowledge lives. Think of it like a giant zip file containing everything the model has learned about language.
โก๏ธ ๐ฅ๐๐ป ๐ณ๐ถ๐น๐ฒ โ This is the set of instructions needed to use the parameter file. It defines the modelโs architecture, handles text tokenization, and manages how the model generates outputs.
Thatโs a very simple way to break down how LLMs work!
These models are the backbone of AI agents, so lets not forget about them ๐
Q: How would you scale a dense retrieval system for billions of documents while ensuring query efficiency?
Scaling a dense retrieval system to handle billions of documents can be tricky, but here are some strategies to ensure both efficiency and accuracy:
1๏ธโฃ Leverage ANN (Approximate Nearest Neighbor) Search:
Use algorithms like HNSW and IVF-PQ. These algorithms provide fast, scalable searches by narrowing down the search space, allowing for quicker retrieval without compromising on quality.
2๏ธโฃ Compress Vectors for Efficiency:
Reduce the size of vectors using techniques like PCA or Product Quantization. This helps save memory and speeds up similarity searches, making it easier to scale.
3๏ธโฃ Multi-Stage Retrieval for Better Accuracy:
First, apply ANN to retrieve a large candidate pool. Then, refine the results with a more precise model (like a cross-encoder) to improve the relevance of the results.
4๏ธโฃ Implement Caching to Speed Up Responses:
Cache frequently queried results and precomputed embeddings. This reduces redundant processing and ensures faster query response times.
5๏ธโฃ Distribute the Load:
Use sharding to divide the document index across multiple nodes, enabling parallel processing. Also, replicate the index to handle heavy traffic and ensure reliability.
Scaling a dense retrieval system to handle billions of documents can be tricky, but here are some strategies to ensure both efficiency and accuracy:
1๏ธโฃ Leverage ANN (Approximate Nearest Neighbor) Search:
Use algorithms like HNSW and IVF-PQ. These algorithms provide fast, scalable searches by narrowing down the search space, allowing for quicker retrieval without compromising on quality.
2๏ธโฃ Compress Vectors for Efficiency:
Reduce the size of vectors using techniques like PCA or Product Quantization. This helps save memory and speeds up similarity searches, making it easier to scale.
3๏ธโฃ Multi-Stage Retrieval for Better Accuracy:
First, apply ANN to retrieve a large candidate pool. Then, refine the results with a more precise model (like a cross-encoder) to improve the relevance of the results.
4๏ธโฃ Implement Caching to Speed Up Responses:
Cache frequently queried results and precomputed embeddings. This reduces redundant processing and ensures faster query response times.
5๏ธโฃ Distribute the Load:
Use sharding to divide the document index across multiple nodes, enabling parallel processing. Also, replicate the index to handle heavy traffic and ensure reliability.
Missing data occurs when no value is stored for a variable in an observation, which is common in real-world datasets. Incomplete data can skew analyses, reduce the validity of conclusions, and hinder machine learning model performance. The goal of this blog is to cover how to identify, understand, and handle the missing values effectively to maintain the data integrity.
Impact: Often times, missing data can lead to:
Biased results, especially if missingness is related to the data itself.
Reduced sample size, leading to less robust analysis.
Poor model generalization, if handled improperly.
This article is divided into following 4 sections:
Identifying missing data
Types of missing data
Methods to handle missing data
Best Practices and Considerations
1. Identifying Missing Data:
Data Profiling:
We can use profiling tools like pandas-profiling or Sweetviz that generate automated reports with insights on missing values.
Example: Use pandas-profiling to generate a profile report in Python.
import pandas as pd
from pandas_profiling import ProfileReport
df = pd.read_csv("data.csv")
profile = ProfileReport(df, minimal=True)
profile.to_file("data_report.html")
Visualization Techniques:
Use library like missingno to create heatmaps and barplots showing missing data patterns.
import missingno as msno
msno.matrix(df) # Matrix plot to view missing values pattern
msno.bar(df) # Bar plot of total missing values per column
Custom Exploratory Functions:
This is my favorite, where we write custom function to display missing data counts and percentages.
def missing_data_summary(df):
missing_data = df.isnull().sum()
missing_percentage = (missing_data / len(df)) * 100
return pd.DataFrame({'Missing Count': missing_data, 'Missing Percentage': missing_percentage})
print(missing_data_summary(df))
2. Types of Missing Data:
Missing Completely at Random (MCAR):
Definition: The missing values are independent of both observed and unobserved data.
Example: Survey data where respondents skipped random questions unrelated to their characteristics.
Can be detected with Statistical tests (e.g., Little's MCAR test)
Missing at Random (MAR):
Definition: Missing values depend on observed data but is not related to the missing values themselves.
Example: Income data may be missing but is related to age or education level.
It can be addressed effectively with conditional imputation, as we can predict missingness based on related variables.
Missing Not at Random (MNAR):
Definition: Missingness depends on unobserved data or the missing values themselves.
Example: People may be unwilling to disclose income if it is very high or low.
Addressing MNAR is challenging, and solutions may require domain knowledge, assumptions, or advanced modeling techniques.
3. Methods To Handle Missing Data:
Listwise Deletion:
Discuss cases where removing rows with missing data may be appropriate (e.g., when missing values are very low or MCAR).
Example: If a dataset has <5% missing values, deletion may suffice, though itโs risky for larger proportions.
Mean/Median/Mode Imputation:
For numerical data, impute using mean or median values; for categorical data, use mode.
Pros & Cons: Simple but can introduce bias and reduce variance in the data. If not handled properly, the model will be impacted by biased data.
df['column'] = df['column'].fillna(df['column'].mean())
Advanced Imputation Techniques:
K-Nearest Neighbors (KNN):
Use neighbors to fill missing values based on similarity.
Pros & Cons: Maintains relationship between variables, but computationally it can be expensive.
from sklearn.impute import KNNImputer
imputer = KNNImputer(n_neighbors=5)
df_imputed = imputer.fit_transform(df)
Multivariate Imputation by Chained Equations (MICE):
Iteratively imputes missing values by treating each column as a function of others.
Best suited for MAR, though it can also be computationally expensive.
Sklearn has features like fancyimpute or IterativeImputer to apply MICE.
Impact: Often times, missing data can lead to:
Biased results, especially if missingness is related to the data itself.
Reduced sample size, leading to less robust analysis.
Poor model generalization, if handled improperly.
This article is divided into following 4 sections:
Identifying missing data
Types of missing data
Methods to handle missing data
Best Practices and Considerations
1. Identifying Missing Data:
Data Profiling:
We can use profiling tools like pandas-profiling or Sweetviz that generate automated reports with insights on missing values.
Example: Use pandas-profiling to generate a profile report in Python.
import pandas as pd
from pandas_profiling import ProfileReport
df = pd.read_csv("data.csv")
profile = ProfileReport(df, minimal=True)
profile.to_file("data_report.html")
Visualization Techniques:
Use library like missingno to create heatmaps and barplots showing missing data patterns.
import missingno as msno
msno.matrix(df) # Matrix plot to view missing values pattern
msno.bar(df) # Bar plot of total missing values per column
Custom Exploratory Functions:
This is my favorite, where we write custom function to display missing data counts and percentages.
def missing_data_summary(df):
missing_data = df.isnull().sum()
missing_percentage = (missing_data / len(df)) * 100
return pd.DataFrame({'Missing Count': missing_data, 'Missing Percentage': missing_percentage})
print(missing_data_summary(df))
2. Types of Missing Data:
Missing Completely at Random (MCAR):
Definition: The missing values are independent of both observed and unobserved data.
Example: Survey data where respondents skipped random questions unrelated to their characteristics.
Can be detected with Statistical tests (e.g., Little's MCAR test)
Missing at Random (MAR):
Definition: Missing values depend on observed data but is not related to the missing values themselves.
Example: Income data may be missing but is related to age or education level.
It can be addressed effectively with conditional imputation, as we can predict missingness based on related variables.
Missing Not at Random (MNAR):
Definition: Missingness depends on unobserved data or the missing values themselves.
Example: People may be unwilling to disclose income if it is very high or low.
Addressing MNAR is challenging, and solutions may require domain knowledge, assumptions, or advanced modeling techniques.
3. Methods To Handle Missing Data:
Listwise Deletion:
Discuss cases where removing rows with missing data may be appropriate (e.g., when missing values are very low or MCAR).
Example: If a dataset has <5% missing values, deletion may suffice, though itโs risky for larger proportions.
Mean/Median/Mode Imputation:
For numerical data, impute using mean or median values; for categorical data, use mode.
Pros & Cons: Simple but can introduce bias and reduce variance in the data. If not handled properly, the model will be impacted by biased data.
df['column'] = df['column'].fillna(df['column'].mean())
Advanced Imputation Techniques:
K-Nearest Neighbors (KNN):
Use neighbors to fill missing values based on similarity.
Pros & Cons: Maintains relationship between variables, but computationally it can be expensive.
from sklearn.impute import KNNImputer
imputer = KNNImputer(n_neighbors=5)
df_imputed = imputer.fit_transform(df)
Multivariate Imputation by Chained Equations (MICE):
Iteratively imputes missing values by treating each column as a function of others.
Best suited for MAR, though it can also be computationally expensive.
Sklearn has features like fancyimpute or IterativeImputer to apply MICE.
from sklearn.experimental import enable_iterative_imputer
from sklearn.impute import IterativeImputer
imputer = IterativeImputer()
df_imputed = imputer.fit_transform(df)
selective closeup photo of brown guinea pig
Photo by Bonnie Kittle on Unsplash
Use Machine Learning Models:
Impute missing values by training a model on non-missing values.
Example: Predict missing income based on age, education, and occupation.
Build categories of similar data points. Find a group of similar people and use non-missing salary to get the median/ mean for missing ones.
Pros: Potentially accurate; Cons: Requires careful validation to prevent overfitting.
Data Augmentation:
For small sample sizes, consider generating synthetic data points.
Use models like GANs for more advanced augmentation.
4. Best Practices and Considerations:
Assess Method Effectiveness: Compare imputed values with actual values when possible or test multiple imputation methods to evaluate which yields better model performance.
Use Domain Knowledge: Understanding the domain can be crucial for addressing missing values, for example in MNAR data and/or identifying appropriate imputation techniques, or deciding to drop.
Monitor Impact: It is important to track model accuracy before and after handling missing data. Also we need to measure the gain based on implementation cost/ challenges.
Data Imbalance: We have to very careful while using simple methods (upsampling) for imbalanced datasets as they may not accurately reflect minority class values.
To summarize, missing values are an inevitable challenge, but addressing them effectively is key to successful data science projects. Understanding the type of missing data and choosing the right handling methodโwhether simple imputation or advanced techniques like MICE or KNNโis crucial. Thereโs no one-size-fits-all solution, so leveraging domain knowledge and validating your approach can ensure data integrity and reliable outcomes.
from sklearn.impute import IterativeImputer
imputer = IterativeImputer()
df_imputed = imputer.fit_transform(df)
selective closeup photo of brown guinea pig
Photo by Bonnie Kittle on Unsplash
Use Machine Learning Models:
Impute missing values by training a model on non-missing values.
Example: Predict missing income based on age, education, and occupation.
Build categories of similar data points. Find a group of similar people and use non-missing salary to get the median/ mean for missing ones.
Pros: Potentially accurate; Cons: Requires careful validation to prevent overfitting.
Data Augmentation:
For small sample sizes, consider generating synthetic data points.
Use models like GANs for more advanced augmentation.
4. Best Practices and Considerations:
Assess Method Effectiveness: Compare imputed values with actual values when possible or test multiple imputation methods to evaluate which yields better model performance.
Use Domain Knowledge: Understanding the domain can be crucial for addressing missing values, for example in MNAR data and/or identifying appropriate imputation techniques, or deciding to drop.
Monitor Impact: It is important to track model accuracy before and after handling missing data. Also we need to measure the gain based on implementation cost/ challenges.
Data Imbalance: We have to very careful while using simple methods (upsampling) for imbalanced datasets as they may not accurately reflect minority class values.
To summarize, missing values are an inevitable challenge, but addressing them effectively is key to successful data science projects. Understanding the type of missing data and choosing the right handling methodโwhether simple imputation or advanced techniques like MICE or KNNโis crucial. Thereโs no one-size-fits-all solution, so leveraging domain knowledge and validating your approach can ensure data integrity and reliable outcomes.
You will be 10x better at Machine Learning
๐๐ ๐ฒ๐จ๐ฎ ๐๐จ๐ฏ๐๐ซ ๐ญ๐ก๐๐ฌ๐ 5 ๐ค๐๐ฒ ๐ญ๐จ๐ฉ๐ข๐๐ฌ:
1. ๐ ๐ฎ๐ง๐๐๐ฆ๐๐ง๐ญ๐๐ฅ๐ฌ ๐จ๐ ๐๐๐๐ก๐ข๐ง๐ ๐๐๐๐ซ๐ง๐ข๐ง๐
โณ Supervised vs. Unsupervised Learning
โณ Regression, Classification, Clustering
โณ Overfitting, Underfitting, and Regularization
โณ Bias-Variance Tradeoff
2. ๐๐๐ญ๐ ๐๐ซ๐๐ฉ๐๐ซ๐๐ญ๐ข๐จ๐ง ๐๐ง๐ ๐ ๐๐๐ญ๐ฎ๐ซ๐ ๐๐ง๐ ๐ข๐ง๐๐๐ซ๐ข๐ง๐
โณ Data Cleaning (Handling Missing Values and outliers)
โณ Feature Scaling (Normalization, Standardization)
โณ Feature Selection and Dimensionality Reduction
โณ Encoding Categorical Variables
3. ๐๐ฅ๐ ๐จ๐ซ๐ข๐ญ๐ก๐ฆ๐ฌ ๐๐ง๐ ๐๐จ๐๐๐ฅ ๐๐ซ๐๐ข๐ง๐ข๐ง๐
โณ Linear Models (Linear Regression, Logistic Regression)
โณ Tree-Based Models (Random Forest, XGBoost)
โณ Neural Networks (ANN, CNN, RNN)
โณ Optimization Techniques
4. ๐๐จ๐๐๐ฅ ๐๐ฏ๐๐ฅ๐ฎ๐๐ญ๐ข๐จ๐ง ๐๐ง๐ ๐๐ฎ๐ง๐ข๐ง๐
โณ Cross-Validation (K-Fold, Leave-One-Out)
โณ Metrics (Accuracy, Precision, Recall, F1-Score, AUC-ROC)
โณ Hyperparameter Tuning (GridSearch, RandomSearch, etc.)
โณ Handling Imbalanced Datasets
5. ๐๐๐๐ก๐ข๐ง๐ ๐๐๐๐ซ๐ง๐ข๐ง๐ ๐ข๐ง ๐๐ซ๐จ๐๐ฎ๐๐ญ๐ข๐จ๐ง - ๐๐ฉ๐ฆ ๐๐ฐ๐ด๐ต ๐ช๐ฎ๐ฑ๐ฐ๐ณ๐ต๐ข๐ฏ๐ต ๐ฐ๐ฏ๐ฆ!
โณ Model Deployment (StreamLit, Azure ML, AWS SageMaker)
โณ Monitoring and Maintaining Models
โณ Pipelines for MLOps
โณ Interpretability and Explainability (SHAP, LIME, Feature Importance)
Why is this important?
Models in notebooks are cool... until theyโre not!
๐ฅ Deploying and scaling them into real, production-ready applications is where most ML practitioners face their biggest roadblocks. If your models are still sitting in Jupyter notebooks, youโre missing out on a HUGE opportunity!
๐๐ ๐ฒ๐จ๐ฎ ๐๐จ๐ฏ๐๐ซ ๐ญ๐ก๐๐ฌ๐ 5 ๐ค๐๐ฒ ๐ญ๐จ๐ฉ๐ข๐๐ฌ:
1. ๐ ๐ฎ๐ง๐๐๐ฆ๐๐ง๐ญ๐๐ฅ๐ฌ ๐จ๐ ๐๐๐๐ก๐ข๐ง๐ ๐๐๐๐ซ๐ง๐ข๐ง๐
โณ Supervised vs. Unsupervised Learning
โณ Regression, Classification, Clustering
โณ Overfitting, Underfitting, and Regularization
โณ Bias-Variance Tradeoff
2. ๐๐๐ญ๐ ๐๐ซ๐๐ฉ๐๐ซ๐๐ญ๐ข๐จ๐ง ๐๐ง๐ ๐ ๐๐๐ญ๐ฎ๐ซ๐ ๐๐ง๐ ๐ข๐ง๐๐๐ซ๐ข๐ง๐
โณ Data Cleaning (Handling Missing Values and outliers)
โณ Feature Scaling (Normalization, Standardization)
โณ Feature Selection and Dimensionality Reduction
โณ Encoding Categorical Variables
3. ๐๐ฅ๐ ๐จ๐ซ๐ข๐ญ๐ก๐ฆ๐ฌ ๐๐ง๐ ๐๐จ๐๐๐ฅ ๐๐ซ๐๐ข๐ง๐ข๐ง๐
โณ Linear Models (Linear Regression, Logistic Regression)
โณ Tree-Based Models (Random Forest, XGBoost)
โณ Neural Networks (ANN, CNN, RNN)
โณ Optimization Techniques
4. ๐๐จ๐๐๐ฅ ๐๐ฏ๐๐ฅ๐ฎ๐๐ญ๐ข๐จ๐ง ๐๐ง๐ ๐๐ฎ๐ง๐ข๐ง๐
โณ Cross-Validation (K-Fold, Leave-One-Out)
โณ Metrics (Accuracy, Precision, Recall, F1-Score, AUC-ROC)
โณ Hyperparameter Tuning (GridSearch, RandomSearch, etc.)
โณ Handling Imbalanced Datasets
5. ๐๐๐๐ก๐ข๐ง๐ ๐๐๐๐ซ๐ง๐ข๐ง๐ ๐ข๐ง ๐๐ซ๐จ๐๐ฎ๐๐ญ๐ข๐จ๐ง - ๐๐ฉ๐ฆ ๐๐ฐ๐ด๐ต ๐ช๐ฎ๐ฑ๐ฐ๐ณ๐ต๐ข๐ฏ๐ต ๐ฐ๐ฏ๐ฆ!
โณ Model Deployment (StreamLit, Azure ML, AWS SageMaker)
โณ Monitoring and Maintaining Models
โณ Pipelines for MLOps
โณ Interpretability and Explainability (SHAP, LIME, Feature Importance)
Why is this important?
Models in notebooks are cool... until theyโre not!
๐ฅ Deploying and scaling them into real, production-ready applications is where most ML practitioners face their biggest roadblocks. If your models are still sitting in Jupyter notebooks, youโre missing out on a HUGE opportunity!
As machine learning continues to transform industries and drive innovation, mastering both fundamental concepts and advanced techniques has become essential for aspiring data scientists and machine learning engineers. Whether youโre preparing for a job interview or simply looking to deepen your understanding, itโs important to be ready for a range of questions that test your knowledge across various levels of complexity.
In this article, weโll explore ten common interview questions that cover the breadth and depth of machine learning, providing not only answers but also real-world examples to help contextualize these concepts. This guide aims to equip you with the insights needed to confidently tackle your next machine learning interview.
Disclaimer: This guide is not a comprehensive guide, but it can serve as a quick refresher to spend 15 mins and get a wider coverage in machine learning.
General Breadth Questions
Explain the curse of dimensionality. How does it affect machine learning models?
Answer: The curse of dimensionality refers to the phenomenon where the feature space becomes increasingly sparse as the number of dimensions (features) increases. This sparsity makes it difficult for models to generalize well because the volume of the space grows exponentially, requiring more data to maintain the same level of accuracy. It can lead to overfitting and increased computational complexity.
Example: In a k-Nearest Neighbors (k-NN) classifier, as the number of features increases, the distance between points becomes less meaningful, leading to poor classification performance. Techniques like dimensionality reduction (PCA, t-SNE) are often used to mitigate this issue.
What is the difference between a generative and a discriminative model?
Answer: A generative model learns the joint probability distribution and can be used generate new instances of data, while a discriminative model learns the conditional probability and is focused on predicting the output labels given the inputs. Generative models are generally used for unsupervised tasks like data generation, while discriminative models are commonly used for supervised learning tasks like classification.
Example: Naive Bayes is a generative model that can generate new data points based on the learned distribution, while Logistic Regression is a discriminative model used to predict binary outcomes.
Large Language Models (LLMs) are also examples of generative models, as they generative tokens and predict next word in the sentence. While generative models are not the best for classification approach, it can be used.
One of the best paper to learn generative vs. discriminative models.
Describe how you would handle an imbalanced dataset.
Answer: Handling an imbalanced dataset can be done through several methods:
Resampling techniques: Oversampling the minority class or under-sampling the majority class.
Synthetic data generation: Techniques like SMOTE (Synthetic Minority Over-sampling Technique) create synthetic examples of the minority class.
Algorithmic adjustments: Using algorithms that account for class imbalance, such as adjusting class weights in models like SVM or Random Forest.
Evaluation metrics: Using metrics like Precision-Recall AUC instead of accuracy to better reflect model performance on imbalanced data.
Example: In fraud detection, where fraudulent transactions are rare, oversampling or SMOTE can be used to create a balanced dataset, and the model's performance is evaluated using the Precision-Recall curve rather than accuracy.
What is the role of a loss function in machine learning, and can you provide an example of how different loss functions are used?
Answer: A loss function measures how well the model's predictions match the true labels. It guides the optimization process during training by quantifying the error, which the algorithm then minimizes. Different tasks require different loss functions:
Mean Squared Error (MSE): Used for regression tasks, penalizing larger errors more heavily.
In this article, weโll explore ten common interview questions that cover the breadth and depth of machine learning, providing not only answers but also real-world examples to help contextualize these concepts. This guide aims to equip you with the insights needed to confidently tackle your next machine learning interview.
Disclaimer: This guide is not a comprehensive guide, but it can serve as a quick refresher to spend 15 mins and get a wider coverage in machine learning.
General Breadth Questions
Explain the curse of dimensionality. How does it affect machine learning models?
Answer: The curse of dimensionality refers to the phenomenon where the feature space becomes increasingly sparse as the number of dimensions (features) increases. This sparsity makes it difficult for models to generalize well because the volume of the space grows exponentially, requiring more data to maintain the same level of accuracy. It can lead to overfitting and increased computational complexity.
Example: In a k-Nearest Neighbors (k-NN) classifier, as the number of features increases, the distance between points becomes less meaningful, leading to poor classification performance. Techniques like dimensionality reduction (PCA, t-SNE) are often used to mitigate this issue.
What is the difference between a generative and a discriminative model?
Answer: A generative model learns the joint probability distribution and can be used generate new instances of data, while a discriminative model learns the conditional probability and is focused on predicting the output labels given the inputs. Generative models are generally used for unsupervised tasks like data generation, while discriminative models are commonly used for supervised learning tasks like classification.
Example: Naive Bayes is a generative model that can generate new data points based on the learned distribution, while Logistic Regression is a discriminative model used to predict binary outcomes.
Large Language Models (LLMs) are also examples of generative models, as they generative tokens and predict next word in the sentence. While generative models are not the best for classification approach, it can be used.
One of the best paper to learn generative vs. discriminative models.
Describe how you would handle an imbalanced dataset.
Answer: Handling an imbalanced dataset can be done through several methods:
Resampling techniques: Oversampling the minority class or under-sampling the majority class.
Synthetic data generation: Techniques like SMOTE (Synthetic Minority Over-sampling Technique) create synthetic examples of the minority class.
Algorithmic adjustments: Using algorithms that account for class imbalance, such as adjusting class weights in models like SVM or Random Forest.
Evaluation metrics: Using metrics like Precision-Recall AUC instead of accuracy to better reflect model performance on imbalanced data.
Example: In fraud detection, where fraudulent transactions are rare, oversampling or SMOTE can be used to create a balanced dataset, and the model's performance is evaluated using the Precision-Recall curve rather than accuracy.
What is the role of a loss function in machine learning, and can you provide an example of how different loss functions are used?
Answer: A loss function measures how well the model's predictions match the true labels. It guides the optimization process during training by quantifying the error, which the algorithm then minimizes. Different tasks require different loss functions:
Mean Squared Error (MSE): Used for regression tasks, penalizing larger errors more heavily.
Cross-Entropy Loss: Used for classification tasks, particularly in deep learning, measuring the difference between predicted probabilities and actual labels.
Hinge Loss: Used in Support Vector Machines, encouraging a margin between classes.
Example: In a binary classification problem using logistic regression, cross-entropy loss helps adjust the model weights to minimize the difference between predicted probabilities and the actual binary labels.
Explain what a convolutional neural network (CNN) is and how it differs from a traditional fully connected network.
Answer: A Convolutional Neural Network (CNN) is a type of deep learning model specifically designed to process structured grid data, like images. Unlike fully connected networks where each neuron is connected to every neuron in the previous layer, CNNs use convolutional layers that apply filters to capture spatial hierarchies in data. This makes them efficient for tasks like image recognition, as they reduce the number of parameters and take advantage of the local structure of the data.
Example: In image classification, a CNN can automatically learn to detect edges, textures, and shapes through its convolutional layers, leading to highly accurate models for tasks like object detection.
Some Depth Questions
Describe how you would implement and evaluate a machine learning model for time series forecasting.
Answer: Implementing a time series forecasting model involves several steps:
Data preparation: Handle missing data, decompose the series into trend, seasonality, and residual components, and create lag features.
Model selection: Choose models that are well-suited for temporal data, such as ARIMA, SARIMA, or LSTM (Long Short-Term Memory) networks.
Evaluation: Use metrics like Mean Absolute Error (MAE) or Root Mean Squared Error (RMSE) and evaluate the model using cross-validation methods like Time Series Cross-Validation to ensure that the temporal order is preserved.
Example: In predicting electricity demand, using LSTM can capture long-term dependencies in the data, and the model can be evaluated on unseen data to ensure that it generalizes well over time.
What is the significance of eigenvectors and eigenvalues in machine learning? How are they used in Principal Component Analysis (PCA)?
Answer: Eigenvectors and eigenvalues are fundamental in linear algebra and are used in PCA to reduce the dimensionality of data. Eigenvectors represent directions in the data space, and eigenvalues indicate the magnitude of variance in those directions. In PCA, the eigenvectors corresponding to the largest eigenvalues are selected as principal components, which capture the most significant variance in the data, reducing dimensionality while preserving as much information as possible.
Example: In facial recognition, PCA can be used to reduce the dimensionality of the image data by projecting the faces onto the principal components, making the model more efficient without losing essential features.
For simplicity of understanding, the diagram below shows 3-D to 2-D conversion. In reality, the input features are in high dimension (most of the times in hundreds). When we do clustering, and want to visualize the data, we can apply PCA to bring the features into 2-D to visualize.
Explain the concept of reinforcement learning and the difference between Q-learning and deep Q-learning.
Answer: Reinforcement learning is a type of machine learning where an agent learns to make decisions by interacting with an environment to maximize cumulative rewards. Q-learning is a value-based method where the agent learns a Q-function, which estimates the value of taking a particular action in a given state. Deep Q-learning extends this by using a deep neural network to approximate the Q-function, allowing it to handle more complex and high-dimensional state spaces.
Hinge Loss: Used in Support Vector Machines, encouraging a margin between classes.
Example: In a binary classification problem using logistic regression, cross-entropy loss helps adjust the model weights to minimize the difference between predicted probabilities and the actual binary labels.
Explain what a convolutional neural network (CNN) is and how it differs from a traditional fully connected network.
Answer: A Convolutional Neural Network (CNN) is a type of deep learning model specifically designed to process structured grid data, like images. Unlike fully connected networks where each neuron is connected to every neuron in the previous layer, CNNs use convolutional layers that apply filters to capture spatial hierarchies in data. This makes them efficient for tasks like image recognition, as they reduce the number of parameters and take advantage of the local structure of the data.
Example: In image classification, a CNN can automatically learn to detect edges, textures, and shapes through its convolutional layers, leading to highly accurate models for tasks like object detection.
Some Depth Questions
Describe how you would implement and evaluate a machine learning model for time series forecasting.
Answer: Implementing a time series forecasting model involves several steps:
Data preparation: Handle missing data, decompose the series into trend, seasonality, and residual components, and create lag features.
Model selection: Choose models that are well-suited for temporal data, such as ARIMA, SARIMA, or LSTM (Long Short-Term Memory) networks.
Evaluation: Use metrics like Mean Absolute Error (MAE) or Root Mean Squared Error (RMSE) and evaluate the model using cross-validation methods like Time Series Cross-Validation to ensure that the temporal order is preserved.
Example: In predicting electricity demand, using LSTM can capture long-term dependencies in the data, and the model can be evaluated on unseen data to ensure that it generalizes well over time.
What is the significance of eigenvectors and eigenvalues in machine learning? How are they used in Principal Component Analysis (PCA)?
Answer: Eigenvectors and eigenvalues are fundamental in linear algebra and are used in PCA to reduce the dimensionality of data. Eigenvectors represent directions in the data space, and eigenvalues indicate the magnitude of variance in those directions. In PCA, the eigenvectors corresponding to the largest eigenvalues are selected as principal components, which capture the most significant variance in the data, reducing dimensionality while preserving as much information as possible.
Example: In facial recognition, PCA can be used to reduce the dimensionality of the image data by projecting the faces onto the principal components, making the model more efficient without losing essential features.
For simplicity of understanding, the diagram below shows 3-D to 2-D conversion. In reality, the input features are in high dimension (most of the times in hundreds). When we do clustering, and want to visualize the data, we can apply PCA to bring the features into 2-D to visualize.
Explain the concept of reinforcement learning and the difference between Q-learning and deep Q-learning.
Answer: Reinforcement learning is a type of machine learning where an agent learns to make decisions by interacting with an environment to maximize cumulative rewards. Q-learning is a value-based method where the agent learns a Q-function, which estimates the value of taking a particular action in a given state. Deep Q-learning extends this by using a deep neural network to approximate the Q-function, allowing it to handle more complex and high-dimensional state spaces.
Example: In a self-driving car simulation, reinforcement learning can be used to train the car to navigate roads by maximizing the reward (e.g., staying on the road, avoiding collisions), with deep Q-learning enabling it to learn from high-dimensional inputs like images from cameras.
How would you approach a situation where your model performs well on the training data but poorly on the test data?
Answer: This situation indicates overfitting. The approach to address it includes:
Cross-validation: To ensure that the model generalizes well to unseen data.
Regularization: Techniques like L1 or L2 regularization to penalize complex models and reduce overfitting.
Simplifying the model: Reducing the complexity by decreasing the number of features or layers in the model.
Gathering more data: To help the model learn a broader range of patterns.
Ensembling: Using methods like bagging or boosting to improve model robustness.
Example: If a neural network overfits on a small dataset, adding dropout layers during training or using a simpler model might help it generalize better to the test data.
What is transfer learning, and when would you use it?
Answer: Transfer learning is a technique where a model developed for a particular task is reused as the starting point for a model on a second task. It is particularly useful when the second task has limited data. Instead of training a model from scratch, the pre-trained modelโs knowledge is transferred, and only the final layers are fine-tuned.
Example: In image classification, using a pre-trained model like VGG16 on ImageNet and fine-tuning it for a specific task like identifying specific animals can lead to good performance even with a small dataset.
To Summarize
Preparing for a machine learning interview requires a balance of theoretical knowledge and practical understanding. The questions covered in this article highlight key areas of machine learning, from foundational principles like the curse of dimensionality and loss functions to more complex topics like reinforcement learning and transfer learning. By familiarizing yourself with these questions and their corresponding answers, youโll be better positioned to demonstrate your expertise and critical thinking skills during an interview.
How would you approach a situation where your model performs well on the training data but poorly on the test data?
Answer: This situation indicates overfitting. The approach to address it includes:
Cross-validation: To ensure that the model generalizes well to unseen data.
Regularization: Techniques like L1 or L2 regularization to penalize complex models and reduce overfitting.
Simplifying the model: Reducing the complexity by decreasing the number of features or layers in the model.
Gathering more data: To help the model learn a broader range of patterns.
Ensembling: Using methods like bagging or boosting to improve model robustness.
Example: If a neural network overfits on a small dataset, adding dropout layers during training or using a simpler model might help it generalize better to the test data.
What is transfer learning, and when would you use it?
Answer: Transfer learning is a technique where a model developed for a particular task is reused as the starting point for a model on a second task. It is particularly useful when the second task has limited data. Instead of training a model from scratch, the pre-trained modelโs knowledge is transferred, and only the final layers are fine-tuned.
Example: In image classification, using a pre-trained model like VGG16 on ImageNet and fine-tuning it for a specific task like identifying specific animals can lead to good performance even with a small dataset.
To Summarize
Preparing for a machine learning interview requires a balance of theoretical knowledge and practical understanding. The questions covered in this article highlight key areas of machine learning, from foundational principles like the curse of dimensionality and loss functions to more complex topics like reinforcement learning and transfer learning. By familiarizing yourself with these questions and their corresponding answers, youโll be better positioned to demonstrate your expertise and critical thinking skills during an interview.
DoorDash now uses RAG to solve complex issues for their Dashers accurately.
Hereโs how --
The first step is to identify the core issue the Dasher is facing based on their messages, so that the system can retrieve articles on previous similar cases.
They break down complex tasks in the Dasherโs request, and use chain-of-thought prompting to generate an answer for it.
The most interesting part is their LLM Guardrail system:
1. First it applies a semantic similarity check between responses and knowledge base articles.
2. If the response fails, an LLM-powered evaluation checks for grounding, coherence, and compliance, so that it can prevent hallucinations and escalate problematic cases.
The knowledge base is continuously updated for completeness and accuracy, guided by the LLM quality assessments. For regression prevention they benchmark prompt changes before deployment as well.
Hereโs how --
The first step is to identify the core issue the Dasher is facing based on their messages, so that the system can retrieve articles on previous similar cases.
They break down complex tasks in the Dasherโs request, and use chain-of-thought prompting to generate an answer for it.
The most interesting part is their LLM Guardrail system:
1. First it applies a semantic similarity check between responses and knowledge base articles.
2. If the response fails, an LLM-powered evaluation checks for grounding, coherence, and compliance, so that it can prevent hallucinations and escalate problematic cases.
The knowledge base is continuously updated for completeness and accuracy, guided by the LLM quality assessments. For regression prevention they benchmark prompt changes before deployment as well.
Suppose you're tasked with building a recommendation system for Instagram's Stories. How would you design the architecture?
This is a system-design question you get for Applied Data Science or Machine Learning Engineer rounds.
These rounds are typically interactive where you want to understand what specific problem the interviewer wants you to solve.
A system like stories needs to be highly scalable and handle-real time updates.
Keeping this in mind a few architecture aspect you could dive into are -
๐๐๐ญ๐ ๐๐ง๐ ๐๐ฌ๐ญ๐ข๐จ๐ง ๐๐๐ฒ๐๐ซ
Needs to handle streaming data from multiple sources:
Event Streams: User interactions (views, likes, skips, etc.) and creator activity.
Content Metadata: Story metadata like geotags, captions, etc.
Real-Time Features: Temporal features like time since Story creation.
User Features: Long-term (preferences, demographics) and short-term (session-based) behavioral data.
Kafka for streaming, Spark Streaming for processing, and a NoSQL database like DynamoDB for fast feature lookups - could be a good tech stack
๐๐๐ง๐๐ข๐๐๐ญ๐ ๐๐๐ง๐๐ซ๐๐ญ๐ข๐จ๐ง
Narrow down the pool of millions of Stories to hundreds of candidates.
๐ Compute embeddings for users and Stories using models like two-tower architectures - Deep Average Network (DAN) for user embeddings based on interaction history, Story embeddings derived from metadata and visual features (using models like CLIP or Vision Transformers).
๐ Use ANN (Approximate Nearest Neighbors) methods (e.g., ScaNN) to retrieve top-N candidates efficiently.
๐๐๐ง๐ค๐ข๐ง๐ ๐๐๐ฒ๐๐ซ
Score and rank the candidates for personalization.
๐ Gradient Boosted Decision Trees (e.g., LightGBM) or BERT4Rec to capture sequence-level dependencies in user interactions.
๐ Use bandit algorithms (e.g., Thompson Sampling or CMABs) to explore less popular Stories while exploiting known preferences.
๐๐ฏ๐๐ฅ๐ฎ๐๐ญ๐ข๐จ๐ง
๐ Offline Metrics - NDCG (Normalized Discounted Cumulative Gain), and diversity scores.
๐ Online Metrics - View-through rate, engagement time, and session duration.
This is a system-design question you get for Applied Data Science or Machine Learning Engineer rounds.
These rounds are typically interactive where you want to understand what specific problem the interviewer wants you to solve.
A system like stories needs to be highly scalable and handle-real time updates.
Keeping this in mind a few architecture aspect you could dive into are -
๐๐๐ญ๐ ๐๐ง๐ ๐๐ฌ๐ญ๐ข๐จ๐ง ๐๐๐ฒ๐๐ซ
Needs to handle streaming data from multiple sources:
Event Streams: User interactions (views, likes, skips, etc.) and creator activity.
Content Metadata: Story metadata like geotags, captions, etc.
Real-Time Features: Temporal features like time since Story creation.
User Features: Long-term (preferences, demographics) and short-term (session-based) behavioral data.
Kafka for streaming, Spark Streaming for processing, and a NoSQL database like DynamoDB for fast feature lookups - could be a good tech stack
๐๐๐ง๐๐ข๐๐๐ญ๐ ๐๐๐ง๐๐ซ๐๐ญ๐ข๐จ๐ง
Narrow down the pool of millions of Stories to hundreds of candidates.
๐ Compute embeddings for users and Stories using models like two-tower architectures - Deep Average Network (DAN) for user embeddings based on interaction history, Story embeddings derived from metadata and visual features (using models like CLIP or Vision Transformers).
๐ Use ANN (Approximate Nearest Neighbors) methods (e.g., ScaNN) to retrieve top-N candidates efficiently.
๐๐๐ง๐ค๐ข๐ง๐ ๐๐๐ฒ๐๐ซ
Score and rank the candidates for personalization.
๐ Gradient Boosted Decision Trees (e.g., LightGBM) or BERT4Rec to capture sequence-level dependencies in user interactions.
๐ Use bandit algorithms (e.g., Thompson Sampling or CMABs) to explore less popular Stories while exploiting known preferences.
๐๐ฏ๐๐ฅ๐ฎ๐๐ญ๐ข๐จ๐ง
๐ Offline Metrics - NDCG (Normalized Discounted Cumulative Gain), and diversity scores.
๐ Online Metrics - View-through rate, engagement time, and session duration.
Fundamentals of a ๐ฉ๐ฒ๐ฐ๐๐ผ๐ฟ ๐๐ฎ๐๐ฎ๐ฏ๐ฎ๐๐ฒ.
With the rise of GenAI, Vector Databases skyrocketed in popularity. The truth is that a Vector Database is also useful for different kinds of AI Systems outside of a Large Language Model context.
When it comes to Machine Learning, we often deal with Vector Embeddings. Vector Databases were created to perform specifically well when working with them:
โก๏ธ Storing.
โก๏ธ Updating.
โก๏ธ Retrieving.
When we talk about retrieval, we refer to retrieving set of vectors that are most similar to a query in a form of a vector that is embedded in the same Latent space. This retrieval procedure is called Approximate Nearest Neighbour (ANN) search.
A query here could be in a form of an object like an image for which we would like to find similar images. Or it could be a question for which we want to retrieve relevant context that could later be transformed into an answer via a LLM.
Letโs look into how one would interact with a Vector Database:
๐ช๐ฟ๐ถ๐๐ถ๐ป๐ด/๐จ๐ฝ๐ฑ๐ฎ๐๐ถ๐ป๐ด ๐๐ฎ๐๐ฎ.
1. Choose a ML model to be used to generate Vector Embeddings.
2. Embed any type of information: text, images, audio, tabular. Choice of ML model used for embedding will depend on the type of data.
3. Get a Vector representation of your data by running it through the Embedding Model.
4. Store additional metadata together with the Vector Embedding. This data would later be used to pre-filter or post-filter ANN search results.
5. Vector DB indexes Vector Embedding and metadata separately. There are multiple methods that can be used for creating vector indexes, some of them: Random Projection, Product Quantization, Locality-sensitive Hashing.
6. Vector data is stored together with indexes for Vector Embeddings and metadata connected to the Embedded objects.
๐ฅ๐ฒ๐ฎ๐ฑ๐ถ๐ป๐ด ๐๐ฎ๐๐ฎ.
7. A query to be executed against a Vector Database will usually consist of two parts:
โก๏ธ Data that will be used for ANN search. e.g. an image for which you want to find similar ones.
โก๏ธ Metadata query to exclude Vectors that hold specific qualities known beforehand. E.g. given that you are looking for similar images of apartments - exclude apartments in a specific location.
8. You execute Metadata Query against the metadata index. It could be done before or after the ANN search procedure.
9. You embed the data into the Latent space with the same model that was used for writing the data to the Vector DB.
10. ANN search procedure is applied and a set of Vector embeddings are retrieved. Popular similarity measures for ANN search include: Cosine Similarity, Euclidean Distance, Dot Product.
With the rise of GenAI, Vector Databases skyrocketed in popularity. The truth is that a Vector Database is also useful for different kinds of AI Systems outside of a Large Language Model context.
When it comes to Machine Learning, we often deal with Vector Embeddings. Vector Databases were created to perform specifically well when working with them:
โก๏ธ Storing.
โก๏ธ Updating.
โก๏ธ Retrieving.
When we talk about retrieval, we refer to retrieving set of vectors that are most similar to a query in a form of a vector that is embedded in the same Latent space. This retrieval procedure is called Approximate Nearest Neighbour (ANN) search.
A query here could be in a form of an object like an image for which we would like to find similar images. Or it could be a question for which we want to retrieve relevant context that could later be transformed into an answer via a LLM.
Letโs look into how one would interact with a Vector Database:
๐ช๐ฟ๐ถ๐๐ถ๐ป๐ด/๐จ๐ฝ๐ฑ๐ฎ๐๐ถ๐ป๐ด ๐๐ฎ๐๐ฎ.
1. Choose a ML model to be used to generate Vector Embeddings.
2. Embed any type of information: text, images, audio, tabular. Choice of ML model used for embedding will depend on the type of data.
3. Get a Vector representation of your data by running it through the Embedding Model.
4. Store additional metadata together with the Vector Embedding. This data would later be used to pre-filter or post-filter ANN search results.
5. Vector DB indexes Vector Embedding and metadata separately. There are multiple methods that can be used for creating vector indexes, some of them: Random Projection, Product Quantization, Locality-sensitive Hashing.
6. Vector data is stored together with indexes for Vector Embeddings and metadata connected to the Embedded objects.
๐ฅ๐ฒ๐ฎ๐ฑ๐ถ๐ป๐ด ๐๐ฎ๐๐ฎ.
7. A query to be executed against a Vector Database will usually consist of two parts:
โก๏ธ Data that will be used for ANN search. e.g. an image for which you want to find similar ones.
โก๏ธ Metadata query to exclude Vectors that hold specific qualities known beforehand. E.g. given that you are looking for similar images of apartments - exclude apartments in a specific location.
8. You execute Metadata Query against the metadata index. It could be done before or after the ANN search procedure.
9. You embed the data into the Latent space with the same model that was used for writing the data to the Vector DB.
10. ANN search procedure is applied and a set of Vector embeddings are retrieved. Popular similarity measures for ANN search include: Cosine Similarity, Euclidean Distance, Dot Product.
๐1
Three different learning styles in machine learning algorithms:
1. Supervised Learning
Input data is called training data and has a known label or result such as spam/not-spam or a stock price at a time.
A model is prepared through a training process in which it is required to make predictions and is corrected when those predictions are wrong. The training process continues until the model achieves a desired level of accuracy on the training data.
Example problems are classification and regression.
Example algorithms include: Logistic Regression and the Back Propagation Neural Network.
2. Unsupervised Learning
Input data is not labeled and does not have a known result.
A model is prepared by deducing structures present in the input data. This may be to extract general rules. It may be through a mathematical process to systematically reduce redundancy, or it may be to organize data by similarity.
Example problems are clustering, dimensionality reduction and association rule learning.
Example algorithms include: the Apriori algorithm and K-Means.
3. Semi-Supervised Learning
Input data is a mixture of labeled and unlabelled examples.
There is a desired prediction problem but the model must learn the structures to organize the data as well as make predictions.
Example problems are classification and regression.
Example algorithms are extensions to other flexible methods that make assumptions about how to model the unlabeled data.
1. Supervised Learning
Input data is called training data and has a known label or result such as spam/not-spam or a stock price at a time.
A model is prepared through a training process in which it is required to make predictions and is corrected when those predictions are wrong. The training process continues until the model achieves a desired level of accuracy on the training data.
Example problems are classification and regression.
Example algorithms include: Logistic Regression and the Back Propagation Neural Network.
2. Unsupervised Learning
Input data is not labeled and does not have a known result.
A model is prepared by deducing structures present in the input data. This may be to extract general rules. It may be through a mathematical process to systematically reduce redundancy, or it may be to organize data by similarity.
Example problems are clustering, dimensionality reduction and association rule learning.
Example algorithms include: the Apriori algorithm and K-Means.
3. Semi-Supervised Learning
Input data is a mixture of labeled and unlabelled examples.
There is a desired prediction problem but the model must learn the structures to organize the data as well as make predictions.
Example problems are classification and regression.
Example algorithms are extensions to other flexible methods that make assumptions about how to model the unlabeled data.
๐2
๐ฆTop 10 Data Science Tools๐ฆ
Here we will examine the top best Data Science tools that are utilized generally by data researchers and analysts. But prior to beginning let us discuss about what is Data Science.
๐ฐWhat is Data Science ?
Data science is a quickly developing field that includes the utilization of logical strategies, calculations, and frameworks to extract experiences and information from organized and unstructured data .
๐ฝTop Data Science Tools that are normally utilized :
1.) Jupyter Notebook : Jupyter Notebook is an open-source web application that permits clients to make and share archives that contain live code, conditions, representations, and narrative text .
2.) Keras : Keras is a famous open-source brain network library utilized in data science. It is known for its usability and adaptability.
Keras provides a range of tools and techniques for dealing with common data science problems, such as overfitting, underfitting, and regularization.
3.) PyTorch : PyTorch is one more famous open-source AI library utilized in information science. PyTorch also offers easy-to-use interfaces for various tasks such as data loading, model building, training, and deployment, making it accessible to beginners as well as experts in the field of machine learning.
4.) TensorFlow : TensorFlow allows data researchers to play out an extensive variety of AI errands, for example, image recognition , natural language processing , and deep learning.
5.) Spark : Spark allows data researchers to perform data processing tasks like data control, investigation, and machine learning , rapidly and effectively.
6.) Hadoop : Hadoop provides a distributed file system (HDFS) and a distributed processing framework (MapReduce) that permits data researchers to handle enormous datasets rapidly.
7.) Tableau : Tableau is a strong data representation tool that permits data researchers to make intuitive dashboards and perceptions. Tableau allows users to combine multiple charts.
8.) SQL : SQL (Structured Query Language) SQL permits data researchers to perform complex queries , join tables, and aggregate data, making it simple to extricate bits of knowledge from enormous datasets. It is a powerful tool for data management, especially for large datasets.
9.) Power BI : Power BI is a business examination tool that conveys experiences and permits clients to make intuitive representations and reports without any problem.
10.) Excel : Excel is a spreadsheet program that broadly utilized in data science. It is an amazing asset for information the board, examination, and visualization .Excel can be used to explore the data by creating pivot tables, histograms, scatterplots, and other types of visualizations.
Here we will examine the top best Data Science tools that are utilized generally by data researchers and analysts. But prior to beginning let us discuss about what is Data Science.
๐ฐWhat is Data Science ?
Data science is a quickly developing field that includes the utilization of logical strategies, calculations, and frameworks to extract experiences and information from organized and unstructured data .
๐ฝTop Data Science Tools that are normally utilized :
1.) Jupyter Notebook : Jupyter Notebook is an open-source web application that permits clients to make and share archives that contain live code, conditions, representations, and narrative text .
2.) Keras : Keras is a famous open-source brain network library utilized in data science. It is known for its usability and adaptability.
Keras provides a range of tools and techniques for dealing with common data science problems, such as overfitting, underfitting, and regularization.
3.) PyTorch : PyTorch is one more famous open-source AI library utilized in information science. PyTorch also offers easy-to-use interfaces for various tasks such as data loading, model building, training, and deployment, making it accessible to beginners as well as experts in the field of machine learning.
4.) TensorFlow : TensorFlow allows data researchers to play out an extensive variety of AI errands, for example, image recognition , natural language processing , and deep learning.
5.) Spark : Spark allows data researchers to perform data processing tasks like data control, investigation, and machine learning , rapidly and effectively.
6.) Hadoop : Hadoop provides a distributed file system (HDFS) and a distributed processing framework (MapReduce) that permits data researchers to handle enormous datasets rapidly.
7.) Tableau : Tableau is a strong data representation tool that permits data researchers to make intuitive dashboards and perceptions. Tableau allows users to combine multiple charts.
8.) SQL : SQL (Structured Query Language) SQL permits data researchers to perform complex queries , join tables, and aggregate data, making it simple to extricate bits of knowledge from enormous datasets. It is a powerful tool for data management, especially for large datasets.
9.) Power BI : Power BI is a business examination tool that conveys experiences and permits clients to make intuitive representations and reports without any problem.
10.) Excel : Excel is a spreadsheet program that broadly utilized in data science. It is an amazing asset for information the board, examination, and visualization .Excel can be used to explore the data by creating pivot tables, histograms, scatterplots, and other types of visualizations.
Walmart built an AI semantic search system processing millions of queries with 99% recall.
- When a user searches for a product, the query goes through a Siamese network with pre-trained tokenisers. This architecture allows the model to use the context of the input queries effectively.
- Different attributes are concatenated to the query title using a special token. This ensures that the model can distinguish among different product characteristics, like brand or colour, when processing a query.
- During training, the model employs a sampled softmax loss function, where both relevant and irrelevant products are considered for each query -- this helps improve the accuracy in distinguishing between different product matches.
- The architecture combines multiple embeddings for both queries and products. This lets the system capture the varying meanings of common queries, improving the model's flexibility and interpretation.
- When a user searches for a product, the query goes through a Siamese network with pre-trained tokenisers. This architecture allows the model to use the context of the input queries effectively.
- Different attributes are concatenated to the query title using a special token. This ensures that the model can distinguish among different product characteristics, like brand or colour, when processing a query.
- During training, the model employs a sampled softmax loss function, where both relevant and irrelevant products are considered for each query -- this helps improve the accuracy in distinguishing between different product matches.
- The architecture combines multiple embeddings for both queries and products. This lets the system capture the varying meanings of common queries, improving the model's flexibility and interpretation.
Data Science Interview Question
Q. What is the difference between bagging and boosting in decision trees?
Answer:
Bagging or bootstrap aggregation, is the process of randomly sampling with replacement multiple times from your original dataset and fitting a decision tree to each of the datasets. Then, you average the predictions made across multiple datasets for regression, or take the majority vote for classification. This averaging approach reduces the variance of the ensemble, and performs better than a single decision tree which fits the data hard and is likely to overfit.
Boosting, on the other hand, is a sequential learning approach. Given the original dataset, the boosting approach does not attempt to fit the data hard, but learns slowly. In boosting, given the current model, the algorithm fits a small, shrunken tree to the residuals of the model. Then, it adds this shrunken tree to the original tree to update the residuals. It continues this process as more small, shrunken trees are fit to the residuals of the model. By focusing on improving residual error, and using this stepwise, sequential approach, the function improves in areas that it usually does not perform well in.
Q. What is the difference between bagging and boosting in decision trees?
Answer:
Bagging or bootstrap aggregation, is the process of randomly sampling with replacement multiple times from your original dataset and fitting a decision tree to each of the datasets. Then, you average the predictions made across multiple datasets for regression, or take the majority vote for classification. This averaging approach reduces the variance of the ensemble, and performs better than a single decision tree which fits the data hard and is likely to overfit.
Boosting, on the other hand, is a sequential learning approach. Given the original dataset, the boosting approach does not attempt to fit the data hard, but learns slowly. In boosting, given the current model, the algorithm fits a small, shrunken tree to the residuals of the model. Then, it adds this shrunken tree to the original tree to update the residuals. It continues this process as more small, shrunken trees are fit to the residuals of the model. By focusing on improving residual error, and using this stepwise, sequential approach, the function improves in areas that it usually does not perform well in.
Walk away from people and situations where you do not get respect. When someone does not value you, just ignore them and move on. The more we walk away from such situations, the more we open ourselves up to future possibilities in life.