M𝗼𝘀𝘁 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝘀 𝘂𝘀𝗲 #𝗣𝘆𝗦𝗽𝗮𝗿𝗸 𝗲𝘃𝗲𝗿𝘆 𝗱𝗮𝘆… 𝗯𝘂𝘁 𝗳𝗲𝘄 𝗸𝗻𝗼𝘄 𝘄𝗵𝗶𝗰𝗵 𝗳𝘂𝗻𝗰𝘁𝗶𝗼𝗻𝘀 𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝗺𝗮𝘅𝗶𝗺𝗶𝘇𝗲 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲.

Ever written long UDFs, confusing joins, or bulky transformations?
Most of that effort is unnecessary — #Spark already gives you built-ins for almost everything.

𝐊𝐞𝐲 𝐈𝐧𝐬𝐢𝐠𝐡𝐭𝐬 (𝐟𝐫𝐨𝐦 𝐭𝐡𝐞 𝐏𝐃𝐅)
• Core Ops: select(), withColumn(), filter(), dropDuplicates()
• Aggregations: groupBy(), countDistinct(), collect_list()
• Strings: concat(), split(), regexp_extract(), trim()
• Window: row_number(), rank(), lead(), lag()
• Date/Time: current_date(), date_add(), last_day(), months_between()
• Arrays/Maps: array(), array_union(), MapType

Just mastering these ~20 functions can simplify 70% of your transformations.

https://t.me/DataAnalyticsX

I'm pleased to invite you to join my private Signal group.

All my resources will be free and unrestricted there. My goal is to build a clean community exclusively for smart programmers, and I believe Signal is the most suitable platform for this (Signal is the second most popular app after WhatsApp in the US), making it particularly suitable for us as programmers.

https://signal.group/#CjQKIPcpEqLQow53AG7RHjeVk-4sc1TFxyym3r0gQQzV-OPpEhCPw_-kRmJ8LlC13l0WiEfp

❗️LISA HELPS EVERYONE EARN MONEY!$29,000 HE'S GIVING AWAY TODAY!

Everyone can join his channel and make money! He gives away from $200 to $5.000 every day in his channel

https://t.me/+YDWOxSLvMfQ2MGNi

⚡️FREE ONLY FOR THE FIRST 500 SUBSCRIBERS! FURTHER ENTRY IS PAID! 👆👇

https://t.me/+YDWOxSLvMfQ2MGNi

Python Libraries You Should Know ✅

⦁ NumPy: Numerical Computing ⚙️
NumPy is the foundation for numerical operations in Python. It provides fast arrays and math functions.

Example:

import numpy as np

arr = np.array([1, 2, 3])
print(arr * 2)  # [2 4 6]

Challenge: Create a 3x3 matrix of random integers from 1–10.

matrix = np.random.randint(1, 11, size=(3, 3))
print(matrix)

⦁ Pandas: Data Analysis 🐼
Pandas makes it easy to work with tabular data using DataFrames.

Example:

import pandas as pd

data = {"Name": ["Alice", "Bob"], "Age": [25, 30]}
df = pd.DataFrame(data)
print(df)

Challenge: Load a CSV file and show the top 5 rows.

df = pd.read_csv("data.csv")
print(df.head())

⦁ Matplotlib: Data Visualization 📊
Matplotlib helps you create charts and plots.

Example:

import matplotlib.pyplot as plt

x = [1, 2, 3]
y = [2, 4, 1]

plt.plot(x, y)
plt.title("Simple Line Plot")
plt.show()

Challenge: Plot a bar chart of fruit sales.

fruits = ["Apples", "Bananas", "Cherries"]
sales = [30, 45, 25]

plt.bar(fruits, sales)
plt.title("Fruit Sales")
plt.show()

⦁ Seaborn: Statistical Plots 🎨
Seaborn builds on Matplotlib with beautiful, high-level charts.

Example:

import seaborn as sns
import matplotlib.pyplot as plt

tips = sns.load_dataset("tips")
sns.boxplot(x="day", y="total_bill", data=tips)
plt.show()

Challenge: Create a heatmap of correlation.

corr = tips.corr()
sns.heatmap(corr, annot=True, cmap="coolwarm")
plt.show()

⦁ Requests: HTTP for Humans 🌐
Requests makes it easy to send HTTP requests.

Example:

import requests

response = requests.get("https://api.github.com")
print(response.status_code)
print(response.json())

Challenge: Fetch and print your IP address.

res = requests.get("https://api.ipify.org?format=json")
print(res.json()["ip"])

⦁ Beautiful Soup: Web Scraping 🍜
Beautiful Soup helps you extract data from HTML pages.

Example:

from bs4 import BeautifulSoup
import requests

url = "https://example.com"
html = requests.get(url).text
soup = BeautifulSoup(html, "html.parser")

print(soup.title.text)

Challenge: Extract all links from a webpage.

links = soup.find_all("a")
for link in links:
    print(link.get("href"))

Next Steps:
⦁ Combine these libraries for real-world projects
⦁ Try scraping data and analyzing it with Pandas
⦁ Visualize insights with Seaborn and Matplotlib

Double Tap ♥️ For More

These Python commands cover 90% of data cleaning tasks you'll ever need 👇

Want to get into Data Analysis?
Here are paid courses with certificates to build real skills:

1️⃣ Google Data Analytics Certificate
https://lnkd.in/dqEU-yht

2️⃣ IBM Data Science Certificate
https://lnkd.in/dQz58dY6

3️⃣ SQL Basics for Data Science
https://lnkd.in/dcFHHm28

4️⃣ Google Business Intelligence Certificate
https://lnkd.in/d4gbdF24

5️⃣ Microsoft Python Development Certificate
https://lnkd.in/dDXX_AHM

Which data skill are you focusing on now?

Selecting with Transformations and Conditional Logic

#### Data Setup

#### pandas

import pandas as pd

data = {
    'product_id': [101, 102, 103, 104, 105, 106, 107, 108],
    'product_name': ['Laptop', 'Mouse', 'Keyboard', 'Monitor', 'Webcam', 'Microphone', 'Speakers', 'Charger'],
    'category': ['Electronics', 'Electronics', 'Electronics', 'Electronics', 'Peripherals', 'Peripherals', 'Audio', 'Accessories'],
    'price': [1200.00, 25.00, 75.00, 300.00, 50.00, 80.00, 150.00, 15.00],
    'stock_quantity': [50, 200, 150, 70, 100, 60, 40, 0]
}
df_pd = pd.DataFrame(data)

#### polars

import polars as pl

data = {
    'product_id': [101, 102, 103, 104, 105, 106, 107, 108],
    'product_name': ['Laptop', 'Mouse', 'Keyboard', 'Monitor', 'Webcam', 'Microphone', 'Speakers', 'Charger'],
    'category': ['Electronics', 'Electronics', 'Electronics', 'Electronics', 'Peripherals', 'Peripherals', 'Audio', 'Accessories'],
    'price': [1200.00, 25.00, 75.00, 300.00, 50.00, 80.00, 150.00, 15.00],
    'stock_quantity': [50, 200, 150, 70, 100, 60, 40, 0]
}
df_pl = pl.DataFrame(data)

#### SQL (Conceptual Table Structure and Data)

-- CREATE TABLE products (
--     product_id INT PRIMARY KEY,
--     product_name VARCHAR(255),
--     category VARCHAR(255),
--     price DECIMAL(10, 2),
--     stock_quantity INT
-- );

-- INSERT INTO products VALUES
-- (101, 'Laptop', 'Electronics', 1200.00, 50),
-- (102, 'Mouse', 'Electronics', 25.00, 200),
-- (103, 'Keyboard', 'Electronics', 75.00, 150),
-- (104, 'Monitor', 'Electronics', 300.00, 70),
-- (105, 'Webcam', 'Peripherals', 50.00, 100),
-- (106, 'Microphone', 'Peripherals', 80.00, 60),
-- (107, 'Speakers', 'Audio', 150.00, 40),
-- (108, 'Charger', 'Accessories', 15.00, 0);

---

Creating New Columns with Expressions (SELECT col1, col2 + col3 AS new_col)

#### pandas

# Select 'product_name', 'price', and calculate 'total_inventory_value'
result_pd = df_pd.assign(
    total_inventory_value=df_pd['price'] * df_pd['stock_quantity'],
    discounted_price=df_pd['price'] * 0.9
)[['product_name', 'price', 'total_inventory_value', 'discounted_price']]
print(result_pd)

#### polars

# Select 'product_name', 'price', and calculate 'total_inventory_value'
result_pl = df_pl.select(
    'product_name',
    'price',
    (pl.col('price') * pl.col('stock_quantity')).alias('total_inventory_value'),
    (pl.col('price') * 0.9).alias('discounted_price')
)
print(result_pl)

#### SQL

-- Select product_name, price, and calculate total_inventory_value and discounted_price
SELECT
    product_name,
    price,
    price * stock_quantity AS total_inventory_value,
    price * 0.9 AS discounted_price
FROM products;

---

Conditional Column Creation (CASE WHEN equivalent)

#### pandas

# Create 'price_level' based on price and 'stock_status'
def get_price_level(price):
    if price > 200:
        return 'High'
    elif price > 50:
        return 'Medium'
    else:
        return 'Low'

def get_stock_status(stock):
    if stock == 0:
        return 'Out of Stock'
    elif stock < 50:
        return 'Low Stock'
    else:
        return 'In Stock'

result_pd = df_pd.assign(
    price_level=df_pd['price'].apply(get_price_level),
    stock_status=df_pd['stock_quantity'].apply(get_stock_status)
)[['product_name', 'price', 'price_level', 'stock_quantity', 'stock_status']]
print(result_pd)

#### polars

# Create 'price_level' based on price and 'stock_status'
result_pl = df_pl.select(
    'product_name',
    'price',
    pl.when(pl.col('price') > 200).then(pl.lit('High'))
    .when(pl.col('price') > 50).then(pl.lit('Medium'))
    .otherwise(pl.lit('Low'))
    .alias('price_level'),
    'stock_quantity',
    pl.when(pl.col('stock_quantity') == 0).then(pl.lit('Out of Stock'))
    .when(pl.col('stock_quantity') < 50).then(pl.lit('Low Stock'))
    .otherwise(pl.lit('In Stock'))
    .alias('stock_status')
)
print(result_pl)

#### SQL

-- Create price_level and stock_status based on conditions
SELECT
    product_name,
    price,
    CASE
        WHEN price > 200 THEN 'High'
        WHEN price > 50 THEN 'Medium'
        ELSE 'Low'
    END AS price_level,
    stock_quantity,
    CASE
        WHEN stock_quantity = 0 THEN 'Out of Stock'
        WHEN stock_quantity < 50 THEN 'Low Stock'
        ELSE 'In Stock'
    END AS stock_status
FROM products;

---

String Transformations in Select

#### pandas

# Select product_name in uppercase and first 3 characters of category
result_pd = df_pd.assign(
    product_name_upper=df_pd['product_name'].str.upper(),
    category_prefix=df_pd['category'].str.slice(0, 3)
)[['product_name', 'product_name_upper', 'category', 'category_prefix']]
print(result_pd)

#### polars

# Select product_name in uppercase and first 3 characters of category
result_pl = df_pl.select(
    'product_name',
    pl.col('product_name').str.to_uppercase().alias('product_name_upper'),
    'category',
    pl.col('category').str.slice(0, 3).alias('category_prefix')
)
print(result_pl)

#### SQL

-- Select product_name in uppercase and first 3 characters of category
SELECT
    product_name,
    UPPER(product_name) AS product_name_upper,
    category,
    SUBSTRING(category, 1, 3) AS category_prefix -- Or LEFT(category, 3) in some SQL dialects
FROM products;

---

Selecting with Advanced Filtering (IN, BETWEEN equivalents)

#### pandas

# Select products in 'Electronics' or 'Audio' categories
print("Products in Electronics or Audio:")
print(df_pd[df_pd['category'].isin(['Electronics', 'Audio'])])

# Select products with price between 50 and 200 (inclusive)
print("\nProducts with price between 50 and 200:")
print(df_pd[df_pd['price'].between(50, 200)])

#### polars

1. What is the primary purpose of the pandas library?
A. Working with unstructured multimedia data
B. Creating and manipulating structured tabular data
C. Building machine learning models
D. Visualizing neural networks

Correct answer: B.

2. Which pandas object is one-dimensional and enforces a homogeneous data type?
A. DataFrame
B. Index
C. Series
D. Panel

Correct answer: C.

3. How can a pd.Series be best compared to an Excel structure?
A. Entire worksheet
B. Row
C. Column
D. Pivot table

Correct answer: C.

4. Which object in pandas represents labels for rows or columns?
A. Series
B. DataFrame
C. Index
D. ndarray

Correct answer: C.

5. What happens if no index is provided when creating a pd.Series?
A. An error is raised
B. A random index is created
C. A RangeIndex starting at 0 is created
D. Index values must be inferred manually

Correct answer: C.

6. Which argument is used to explicitly set the data type of a pd.Series?
A. type=
B. data_type=
C. dtype=
D. astype=

Correct answer: C.

7. What is the default value of the name attribute of a pd.Series if not provided?
A. Empty string
B. Undefined
C. None
D. "Series"

Correct answer: C.

8. Which structure allows heterogeneous column data types?
A. Series
B. Index
C. ndarray
D. DataFrame

Correct answer: D.

9. When constructing a DataFrame from a dictionary, what do the dictionary keys represent?
A. Row labels
B. Index levels
C. Column labels
D. Data types

Correct answer: C.

10. Which attribute returns the number of rows in a pd.Series?
A. size
B. shape
C. len()
D. index

Correct answer: B.

11. What does the pd.Series.shape attribute return?
A. An integer
B. A list
C. A one-element tuple
D. A two-element tuple

Correct answer: C.

12. Which attribute of a DataFrame returns a Series of column data types?
A. dtype
B. dtypes
C. types
D. schema

Correct answer: B.

13. What does len(df) return for a DataFrame?
A. Number of columns
B. Total number of elements
C. Number of rows
D. Size of memory used

Correct answer: C.

14. In basic DataFrame selection using df["a"], what is returned?
A. A DataFrame
B. A scalar
C. A NumPy array
D. A Series

Correct answer: D.

15. What does df[["a"]] return?
A. A Series
B. A DataFrame
C. A scalar
D. A NumPy array

Correct answer: B.

16. When using [] with a Series that has a non-default integer index, selection is done by:
A. Position
B. Order of insertion
C. Label
D. Data type

Correct answer: C.

17. Which method should be used for explicit position-based selection in a Series?
A. loc
B. at
C. iloc
D. ix

Correct answer: C.

18. What does ser.iloc[1] return?
A. All rows with label 1
B. The value at position 1
C. A slice of the Series
D. A DataFrame

Correct answer: B.

19. How many indexers are required when using DataFrame.iloc?
A. One
B. Two
C. Three
D. Unlimited

Correct answer: B.

20. What does df.iloc[:, 0] return?
A. First row
B. First column as a Series
C. First column as a DataFrame
D. Entire DataFrame

Correct answer: B.

21. Which method performs label-based selection in a Series?
A. iloc
B. at
C. loc
D. take

Correct answer: C.

22. What is a key difference between slicing with loc and iloc?
A. loc excludes the stop value
B. iloc includes labels
C. loc includes the stop label
D. iloc works only with strings

Correct answer: C.

23. Which operation may raise a KeyError when using loc?
A. Slicing with ordered unique labels
B. Selecting existing labels
C. Slicing with non-unique unordered labels
D. Selecting with lists

Correct answer: C.

24. In a DataFrame, df.loc["Jack", :] selects:
A. All rows named Jack
B. All columns named Jack
C. All columns for the row labeled Jack
D. Only numeric columns

Correct answer: C.