Confusion matrix (TP, FP, TN, FN), clearly explained

🔑 Tags: #PYTHON #AI #ML

https://t.me/CodeProgrammer ✅

🏳️‍🌈 Python became GitHub's first language!

👨🏻‍💻 In a recent GitHub report, with the expansion of artificial intelligence, Python could finally overtake JavaScript and become the most popular language on GitHub in 2024. This happened after 10 years of JavaScript dominance and it is not very strange.

✔️ Because with the growth of artificial intelligence, developers are turning to Python more than ever, and Python's applications in data science and analytics are increasing every day. You can read the full GitHub report here:👇

┌ 🐱 Top programming along GitHub
├ 💰 Report


⏪ I also introduced the most important Python libraries for working with data and AI here: 👇


🖥 Data Manipulation & Analysis
▶️ pandas
▶️ Apache Spark
▶️ Polars
▶️ DuckDB


📊 Data Visualization
➡️ matplotlib
➡️ plotly
➡️ seaborn


🖥 Machine & Deep Learning
➡️ TensorFlow
➡️ PyTorch
➡️ Keras
➡️ scikit-learn
➡️ XGBoost
➡️ LightGBM
➡️ Prophet


🌫 NLP & Large Language Models
➡️ Hugging Face Transformers
➡️ LangChain
➡️ LlamaIndex

🔑 Tags: #PYTHON #AI #ML #NLP

https://t.me/CodeProgrammer ✅

Pandas 🐼 to Polars Guide

🔑 Tags: #PYTHON #AI #ML #pandas #Polars

https://t.me/CodeProgrammer ✅

What is a 𝗩𝗲𝗰𝘁𝗼𝗿 𝗗𝗮𝘁𝗮𝗯𝗮𝘀𝗲?

With the rise of Foundational Models, Vector Databases skyrocketed in popularity. The truth is that a Vector Database is also useful outside of a Large Language Model context.

When it comes to Machine Learning, we often deal with Vector Embeddings. Vector Databases were created to perform specifically well when working with them:

➡️ Storing.
➡️ Updating.
➡️ Retrieving.

When we talk about retrieval, we refer to retrieving set of vectors that are most similar to a query in a form of a vector that is embedded in the same Latent space. This retrieval procedure is called Approximate Nearest Neighbour (ANN) search.

A query here could be in a form of an object like an image for which we would like to find similar images. Or it could be a question for which we want to retrieve relevant context that could later be transformed into an answer via a LLM.

Let’s look into how one would interact with a Vector Database:

𝗪𝗿𝗶𝘁𝗶𝗻𝗴/𝗨𝗽𝗱𝗮𝘁𝗶𝗻𝗴 𝗗𝗮𝘁𝗮.

1. Choose a ML model to be used to generate Vector Embeddings.
2. Embed any type of information: text, images, audio, tabular. Choice of ML model used for embedding will depend on the type of data.
3. Get a Vector representation of your data by running it through the Embedding Model.
4. Store additional metadata together with the Vector Embedding. This data would later be used to pre-filter or post-filter ANN search results.
5. Vector DB indexes Vector Embedding and metadata separately. There are multiple methods that can be used for creating vector indexes, some of them: Random Projection, Product Quantization, Locality-sensitive Hashing.
6. Vector data is stored together with indexes for Vector Embeddings and metadata connected to the Embedded objects.

𝗥𝗲𝗮𝗱𝗶𝗻𝗴 𝗗𝗮𝘁𝗮.

7. A query to be executed against a Vector Database will usually consist of two parts:

➡️ Data that will be used for ANN search. e.g. an image for which you want to find similar ones.
➡️ Metadata query to exclude Vectors that hold specific qualities known beforehand. E.g. given that you are looking for similar images of apartments - exclude apartments in a specific location.

8. You execute Metadata Query against the metadata index. It could be done before or after the ANN search procedure.
9. You embed the data into the Latent space with the same model that was used for writing the data to the Vector DB.
10. ANN search procedure is applied and a set of Vector embeddings are retrieved. Popular similarity measures for ANN search include: Cosine Similarity, Euclidean Distance, Dot Product.

How are you using Vector DBs? Let me know in the comment section!

#RAG #LLM #DataEngineering

https://t.me/CodeProgrammer ✅

📹 3blue1brown presented the shortest and most understandable lecture on neural networks!

In the new episode, he talks about the mechanism of attention and transformers. The lecture has become even more concise and exciting!

Ideal for absolute beginners and even those who are far from technical.

The author managed to explain the key aspects of the neural network in just 9 minutes using bright graphics and simple examples.

📌 Original

📂 Tags: #DataScience #Python #ML #AI #LLM #BIGDATA #Courses #Transformer

http://t.me/codeprogrammer ⭐️

📕 Python Basics Made Simple!

📷 Course: AI Python for Beginners

👨‍💻 Instructor: Andrew Ng

In the #AIPythonforBeginners course series you'll learn how to identify strings, integers, and floats with the type() function, and build a solid Python foundation for your AI journey.

Enroll Free: https://learn.deeplearning.ai/courses/ai-python-for-beginners

📂 Tags: #DataScience #Python #ML #AI #LLM #BIGDATA #Courses #Transformer

http://t.me/codeprogrammer ⭐️

💠 All free Kaggle courses for data science
📁 Along with the course completion certificate

✅ Python ⬅️ link

✅ An introduction to machine learning ⬅️ link

✅ Pandas ⬅️ link

✅ Medium machine learning ⬅️ link

✅ Data visualization ⬅️ link

✅ Feature engineering ⬅️ link

✅ An introduction to the SQL language ⬅️ link

✅ Advanced SQL language ⬅️ link

✅ An introduction to deep learning ⬅️ link

✅ Computer vision ⬅️ link

✅ Time series ⬅️ link

✅ Data cleanup ⬅️ link

✅ Geographical analysis ⬅️ link

✅ Explainability of machine learning ⬅️ link

📂 Tags: #DataScience #Python #ML #AI #LLM #BIGDATA #Courses #Transformer

http://t.me/codeprogrammer ⭐️

Python List Methods clearly Explained

📂 Tags: #DataScience #Python #ML #AI #LLM #BIGDATA #Courses #Transformer

http://t.me/codeprogrammer ⭐️

🐱 7 of the best GitHub repos
✅ To enter the world of data analysis and data science


1⃣ 100 Days of ML Code repo

✍️ A hundred-day program for learning and practicing machine learning coding.


🔢 Awesome Data Science repo

✍️ A curated list of great data science resources such as books, software, and tools.


🔢 Data Science for Beginners repo

✍️ A repository from Microsoft that has a 10-week course with 20 lessons for beginners. Each lesson includes videos, quizzes, challenges and more.


🔢 Data Science Interviews Repo

✍️ A repository of questions and answers for science job interviews.


🔢 ML Technical Interviews repo

✍️ A good guide for machine learning and artificial intelligence technical interviews.


🔢 ML Interviews repo

✍️ A repository containing machine learning interview questions from basic topics to complex topics such as neural networks and reinforcement learning.


🔢 Data Science Python Notebooks repo

✍️ A collection of notebooks in various fields of data science such as deep learning, machine learning, data analysis and Python topics.

📂 Tags: #DataScience #Python #ML #AI #LLM #BIGDATA #Courses #Transformer

http://t.me/codeprogrammer ⭐️

List of Running Processes using Python

📂 Tags: #DataScience #Python #ML #AI #LLM #BIGDATA #Courses #Transformer

http://t.me/codeprogrammer ⭐️

Pandas is getting outdated.

5 reasons you should move to FireDucks 👇

1. Requires changing ONLY ONE line of code:
↳ Replace "𝗶𝗺𝗽𝗼𝗿𝘁 𝗽𝗮𝗻𝗱𝗮𝘀 𝗮𝘀 𝗽𝗱" with "𝗶𝗺𝗽𝗼𝗿𝗲 𝗳𝗶𝗿𝗲𝗱𝘂𝗰𝗸𝘀.𝗽𝗮𝗻𝗱𝗮𝘀 𝗮𝘀 𝗽𝗱"
↳ The rest of the entire code remains the same.
↳ So, if you know Pandas, you already know how to use FireDucks.
↳ Done!

2. Ridiculously faster as per official benchmarks:
↳ Modin had an average speed-up of 0.9x over Pandas.
↳ Polars had an average speed-up of 39x over Pandas.
↳ But FireDucks had an average speed-up of 50x over Pandas.

3. Pandas is single-core; FireDucks is multi-core.

4. Pandas follows eager execution; FireDucks is based on lazy execution. This way, FireDucks can build a logical execution plan and apply possible optimizations.

5. That said, even under eager execution, FireDucks is way faster than Pandas, as depicted in the image below.

📂 Tags: #DataScience #Python #ML #AI #LLM #BIGDATA #Courses #Pandas #FireDucks

http://t.me/codeprogrammer ⭐️

5 Free Courses for Mastering LLMs

Link: https://machinelearningmastery.com/5-free-courses-mastering-llms

https://t.me/CodeProgrammer ✅

Master #DataAnalytics for real-world problems. I have curated a list of 40 Data Analytics Projects (solved & explained) that will help you build analytics skills using #Python.

Includes projects like:

1. Rainfall Trends in India Analysis
2. Netflix Content Strategy Analysis
3. Creating a Mutual Fund Plan
4. Stock Market Portfolio Optimization
5. Metro Operations Optimization
6. Analyzing the Impact of Carbon Emissions

Find this list of projects here:
https://thecleverprogrammer.com/2024/11/01/data-analytics-projects-with-python/

📂 Tags: #DataScience #Python #ML #AI #LLM #BIGDATA #Courses #Pandas #deeplearning #Projects

http://t.me/codeprogrammer ⭐️

Best LLMs Courses

Link: https://www.mltut.com/best-large-language-models-courses/

#MachineLearning #DeepLearning #BigData #Datascience #ML #HealthTech #DataVisualization #ArtificialInteligence #SoftwareEngineering #GenAI #deeplearning #ChatGPT #OpenAI #python #AI #keras #SQL #Statistics #LLMs #AIagents

http://t.me/codeprogrammer ⭐️

🐼 20 of the most used Pandas + PDF functions

👨🏻‍💻 The first time I used Pandas, I was supposed to quickly clean and organize a raw and complex dataset with the help of Pandas functions. Using the groupby function, I was able to categorize the data and get in-depth analysis of customer behavior. Best of all, it was when I used loc and iloc that I could easily filter the data.

✔️ Since then I decided to prepare a list of the most used Pandas functions that I use on a daily basis. Now this list is ready! In the following, I will introduce 20 of the best and most used Pandas functions:



🏳️‍🌈 read_csv(): Fast data upload from CSV files

🏳️‍🌈 head(): look at the first five rows of the database to start..

🏳️‍🌈 info(): Checking data structure such as data type and empty values.

🏳️‍🌈 describe(): Generate descriptive statistics for numeric columns.

🏳️‍🌈 loc[ ]: accesses rows and columns by label or condition.

🏳️‍🌈 iloc[ ]: Access data by row number.

🏳️‍🌈 merge(): Merge dataframes with common columns.

🏳️‍🌈 groupby(): Grouping for easier analysis.

🏳️‍🌈 pivot_table(): Summarize data in pivot table format.

🏳️‍🌈 to_csv(): Save data as a CSV file.

🏳️‍🌈 pd.concat(): Concatenate multiple dataframes in rows or columns.

🏳️‍🌈 pd.melt(): Convert wide format data to long format.

🏳️‍🌈 pd.pivot_table(): Create a pivot table with multiple levels.

🏳️‍🌈 pd.cut(): Split the data into specific intervals.

🏳️‍🌈 pd.qcut(): Sort data by percentage.

🏳️‍🌈 pd.merge(): Merge data in database style for advanced linking.

🏳️‍🌈 DataFrame.apply(): Apply a custom function to the data.

🏳️‍🌈 DataFrame.groupby(): Analyze grouped data.

🏳️‍🌈 DataFrame.drop_duplicates(): Drop duplicate rows.

🏳️‍🌈 DataFrame.to_excel(): Save data directly to Excel file.


┌ 🐼 Pandas Functions
└ 📄 PDF

#MachineLearning #DeepLearning #BigData #Datascience #ML #Pandas #DataVisualization #ArtificialInteligence #SoftwareEngineering #GenAI #deeplearning #ChatGPT #OpenAI #python #AI #keras #SQL #Statistics #LLMs #AIagents

http://t.me/codeprogrammer ⭐️

Regression & Classification Loss Functions

#MachineLearning #DeepLearning #BigData #Datascience #ML #Pandas #DataVisualization #ArtificialInteligence #SoftwareEngineering #GenAI #deeplearning #ChatGPT #OpenAI #python #AI #keras #SQL #Statistics #LLMs #AIagents

http://t.me/codeprogrammer ⭐️