Python Learning Series Part-9


Web Scraping with BeautifulSoup and Requests:

Web scraping involves extracting data from websites. BeautifulSoup is a Python library for pulling data out of HTML and XML files, and the Requests library is used to send HTTP requests.

1. Extracting Data from Websites:
   - Installation:
     - Install BeautifulSoup and Requests using:
     

       pip install beautifulsoup4
       pip install requests
       

   - Making HTTP Requests:
     - Use the Requests library to send GET requests to a website.
     

       import requests

       response = requests.get('https://example.com')
       

2. Parsing HTML with BeautifulSoup:
   - Creating a BeautifulSoup Object:
     - Parse the HTML content of a webpage.
     

       from bs4 import BeautifulSoup

       soup = BeautifulSoup(response.text, 'html.parser')
       

   - Navigating the HTML Tree:
     - Use BeautifulSoup methods to navigate and extract data from HTML elements.
     

       title = soup.title
       paragraphs = soup.find_all('p')
       

3. Scraping Data from a Website:
   - Extracting Text:
     - Get the text content of HTML elements.
     

       title_text = soup.title.text
       paragraph_text = soup.find('p').text
       

   - Extracting Attributes:
     - Retrieve specific attributes of HTML elements.
     

       image_url = soup.find('img')['src']
       

4. Handling Multiple Pages and Dynamic Content:
   - Pagination:
     - Iterate through multiple pages by modifying the URL.
     

       for page in range(1, 6):
           url = f'https://example.com/page/{page}'
           response = requests.get(url)
           # Process the page content
       

   - Dynamic Content:
     - Use tools like Selenium for websites with dynamic content loaded by JavaScript.

Web scraping is a powerful technique for collecting data from the web, but it's important to be aware of legal and ethical considerations.


Hope it helps :)

Python Learning Series Part-10


SQL for Data Analysis:

Structured Query Language (SQL) is a powerful language for managing and manipulating relational databases. Understanding SQL is crucial for working with databases and extracting relevant information for data analysis.

1. Basic SQL Commands:
   - SELECT Statement:
     - Retrieve data from one or more tables.
     

       SELECT column1, column2 FROM table_name WHERE condition;
       

   - INSERT Statement:
     - Insert new records into a table.
     

       INSERT INTO table_name (column1, column2) VALUES (value1, value2);
       

   - UPDATE Statement:
     - Modify existing records in a table.
     

       UPDATE table_name SET column1 = value1 WHERE condition;
       

   - DELETE Statement:
     - Remove records from a table.
     

       DELETE FROM table_name WHERE condition;
       

2. Data Filtering and Sorting:
   - WHERE Clause:
     - Filter data based on specified conditions.
     

       SELECT * FROM employees WHERE department = 'Sales';
       

   - ORDER BY Clause:
     - Sort the result set in ascending or descending order.
     

       SELECT * FROM products ORDER BY price DESC;
       

3. Aggregate Functions:
   - SUM, AVG, MIN, MAX, COUNT:
     - Perform calculations on groups of rows.
     

       SELECT AVG(salary) FROM employees WHERE department = 'Marketing';
       

4. Joins and Relationships:
   - INNER JOIN, LEFT JOIN, RIGHT JOIN:
     - Combine rows from two or more tables based on a related column.
     

       SELECT employees.name, departments.department_name
       FROM employees
       INNER JOIN departments ON employees.department_id = departments.department_id;
       

   - Primary and Foreign Keys:
     - Establish relationships between tables for efficient data retrieval.
     

       CREATE TABLE employees (
           employee_id INT PRIMARY KEY,
           name VARCHAR(50),
           department_id INT FOREIGN KEY REFERENCES departments(department_id)
       );
       

Understanding SQL is essential for working with databases, especially in scenarios where data is stored in relational databases like MySQL, PostgreSQL, or SQLite.


Hope it helps :)

Python Learning Series Part-11

Advanced Data Visualization:

Advanced data visualization goes beyond basic charts and explores more sophisticated techniques to represent data effectively.

1. Interactive Visualizations with Plotly:
   - Creating Interactive Plots:
     - Plotly provides a higher level of interactivity for charts.
     

       import plotly.express as px

       fig = px.scatter(df, x='X-axis', y='Y-axis', color='Category', size='Size', hover_data=['Details'])
       fig.show()
       

   - Dash for Web Applications:
     - Dash, built on top of Plotly, allows you to create interactive web applications with Python.
     

       import dash
       import dash_core_components as dcc
       import dash_html_components as html

       app = dash.Dash(__name__)

       app.layout = html.Div(children=[
           dcc.Graph(
               id='example-graph',
               figure=fig
           )
       ])

       if __name__ == '__main__':
           app.run_server(debug=True)
       

2. Geospatial Data Visualization:
   - Folium for Interactive Maps:
     - Folium is a Python wrapper for Leaflet.js, enabling the creation of interactive maps.
     

       import folium

       m = folium.Map(location=[latitude, longitude], zoom_start=10)
       folium.Marker(location=[point_latitude, point_longitude], popup='Marker').add_to(m)
       m.save('map.html')
       

   - Geopandas for Spatial Data:
     - Geopandas extends Pandas to handle spatial data and integrates with Matplotlib for visualization.
     

       import geopandas as gpd
       import matplotlib.pyplot as plt

       gdf = gpd.read_file('shapefile.shp')
       gdf.plot()
       plt.show()
       

3. Customizing Visualizations:
   - Matplotlib Customization:
     - Customize various aspects of Matplotlib plots for a polished look.
     

       plt.title('Customized Title', fontsize=16)
       plt.xlabel('X-axis Label', fontsize=12)
       plt.ylabel('Y-axis Label', fontsize=12)
       

   - Seaborn Themes:
     - Seaborn provides different themes to quickly change the overall appearance of plots.
     

       import seaborn as sns

       sns.set_theme(style='whitegrid')
       

Advanced visualization techniques help convey complex insights effectively.


Hope it helps :)

Interview QnA | Date: 01-04-2024
Company Name: Accenture
Role: Data Scientist
Topic: Silhouette, trend seasonality, bag of words, bagging boosting

1. What do you understand by the term silhouette coefficient?

The silhouette coefficient is a measure of how well clustered together a data point is with respect to the other points in its cluster. It is a measure of how similar a point is to the points in its own cluster, and how dissimilar it is to the points in other clusters. The silhouette coefficient ranges from -1 to 1, with 1 being the best possible score and -1 being the worst possible score.


2. What is the difference between trend and seasonality in time series?

Trends and seasonality are two characteristics of time series metrics that break many models. Trends are continuous increases or decreases in a metric’s value. Seasonality, on the other hand, reflects periodic (cyclical) patterns that occur in a system, usually rising above a baseline and then decreasing again.


3. What is Bag of Words in NLP?

Bag of Words is a commonly used model that depends on word frequencies or occurrences to train a classifier. This model creates an occurrence matrix for documents or sentences irrespective of its grammatical structure or word order.


4. What is the difference between bagging and boosting?

Bagging is a homogeneous weak learners’ model that learns from each other independently in parallel and combines them for determining the model average. Boosting is also a homogeneous weak learners’ model but works differently from Bagging. In this model, learners learn sequentially and adaptively to improve model predictions of a learning algorithm

+50 Git Use Cases.pdf

Thanks for the amazing response. Here are the answers to each question 👇👇

1. How do you reverse a string?
Example:

   def reverse_string(s):
       return s[::-1]
   
   print(reverse_string("hello"))  # Output: "olleh"
   

2. How do you determine if a string is a palindrome?
Example:

   def is_palindrome(s):
       return s == s[::-1]
   
   print(is_palindrome("radar"))  # Output: True
   

3. How do you calculate the number of numerical digits in a string?
Example:

   def count_digits(s):
       return sum(1 for char in s if char.isdigit())
   
   print(count_digits("abc123def456"))  # Output: 6

Interview QnA | 07-04-2024
Company - The Math Company
Role- Data Analyst

1.How to create filters in Power BI?

Filters are an integral part of Power BI reports. They are used to slice and dice the data as per the dimensions we want. Filters are created in a couple of ways.

Using Slicers: A slicer is a visual under Visualization Pane. This can be added to the design view to filter our reports. When a slicer is added to the design view, it requires a field to be added to it. For example- Slicer can be added for Country fields. Then the data can be filtered based on countries.

Using Filter Pane: The Power BI team has added a filter pane to the reports, which is a single space where we can add different fields as filters. And these fields can be added depending on whether you want to filter only one visual(Visual level filter), or all the visuals in the report page(Page level filters), or applicable to all the pages of the report(report level filters)


2.How to sort data in Power BI?

Sorting is available in multiple formats. In the data view, a common sorting option of alphabetical order is there. Apart from that, we have the option of Sort by column, where one can sort a column based on another column. The sorting option is available in visuals as well. Sort by ascending and descending option by the fields and measure present in the visual is also available.

3.How to convert pdf to excel?

Open the PDF document you want to convert in XLSX format in Acrobat DC.

Go to the right pane and click on the “Export PDF” option.

Choose spreadsheet as the Export format.

Select “Microsoft Excel Workbook.”

Now click “Export.”

Download the converted file or share it.



4. How to enable macros in excel?

Click the file tab and then click “Options.”

A dialog box will appear. In the “Excel Options” dialog box, click on the “Trust Center” and then “Trust Center Settings.”

Go to the “Macro Settings” and select “enable all macros.”

Click OK to apply the macro settings.

Python Learning Series Part-12

Complete Python Topics for Data Analysis:

Natural Language Processing (NLP)

Natural Language Processing involves working with human language data, enabling computers to understand, interpret, and generate human-like text.

1. Text Preprocessing:
   - Tokenization:
     - Break text into words or phrases (tokens).
     

       from nltk.tokenize import word_tokenize

       text = "Natural Language Processing is fascinating!"
       tokens = word_tokenize(text)
       

   - Stopword Removal:
     - Eliminate common words (stopwords) that often don't contribute much meaning.
     

       from nltk.corpus import stopwords

       stop_words = set(stopwords.words('english'))
       filtered_tokens = [word for word in tokens if word.lower() not in stop_words]
       

2. Text Analysis:
   - Frequency Analysis:
     - Analyze the frequency of words in a text.
     

       from nltk.probability import FreqDist

       freq_dist = FreqDist(filtered_tokens)
       

   - Word Clouds:
     - Visualize word frequency using a word cloud.
     

       from wordcloud import WordCloud
       import matplotlib.pyplot as plt

       wordcloud = WordCloud().generate_from_frequencies(freq_dist)
       plt.imshow(wordcloud, interpolation='bilinear')
       plt.axis("off")
       plt.show()
       

3. Sentiment Analysis:
   - VADER Sentiment Analysis:
     - Assess the sentiment (positive, negative, neutral) of a piece of text.
     

       from nltk.sentiment import SentimentIntensityAnalyzer

       analyzer = SentimentIntensityAnalyzer()
       sentiment_score = analyzer.polarity_scores("I love NLP!")
       

4. Named Entity Recognition (NER):
   - Spacy for NER:
     - Identify entities (names, locations, organizations) in text.
     

       import spacy

       nlp = spacy.load('en_core_web_sm')
       doc = nlp("Apple Inc. is headquartered in Cupertino.")
       for ent in doc.ents:
           print(ent.text, ent.label_)
       

5. Topic Modeling:
   - Latent Dirichlet Allocation (LDA):
     - Identify topics within a collection of text documents.
     

       from gensim import corpora, models

       dictionary = corpora.Dictionary(documents)
       corpus = [dictionary.doc2bow(text) for text in documents]
       lda_model = models.LdaModel(corpus, num_topics=3, id2word=dictionary)
     

Hope it helps :)

Top 40 commonly asked DSA questions :

𝗔𝗿𝗿𝗮𝘆𝘀 𝗮𝗻𝗱 𝗦𝘁𝗿𝗶𝗻𝗴𝘀:
1. Find the missing number in an array of integers.
2. Implement an algorithm to rotate an array.
3. Check if a string is a palindrome.
4. Find the first non-repeating character in a string.
5. Implement an algorithm to reverse a linked list.
6. Merge two sorted arrays.
7. Implement a stack using arrays/linked list.
8. Write a program to remove duplicates from a sorted array.

𝗟𝗶𝗻𝗸𝗲𝗱 𝗟𝗶𝘀𝘁𝘀:
1. Detect a cycle in a linked list.
2. Find the intersection point of two linked lists.
3. Reverse a linked list in groups of k.
4. Implement a function to add two numbers represented by linked lists.
5. Clone a linked list with next and random pointer.

𝗧𝗿𝗲𝗲𝘀 𝗮𝗻𝗱 𝗕𝗶𝗻𝗮𝗿𝘆 𝗦𝗲𝗮𝗿𝗰𝗵 𝗧𝗿𝗲𝗲𝘀 (𝗕𝗦𝗧):
1. Find the height of a binary tree.
2. Check if a binary tree is balanced.
3. Find the lowest common ancestor in a binary tree.
4. Serialize and deserialize a binary tree.
5. Implement an algorithm for in-order traversal without recursion.
6. Convert a BST to a sorted doubly linked list.

You can check these amazing resources for DSA Preparation

All the best 👍👍

Python Learning Series Part-13

Deep Learning Basics with TensorFlow:

Deep Learning is a subset of machine learning that involves neural networks with multiple layers (deep neural networks). TensorFlow is an open-source deep learning library developed by Google.

1. Introduction to Neural Networks:
   - Perceptrons and Activation Functions:
     - Basic building blocks of neural networks.
     

       import tensorflow as tf

       # Create a simple perceptron
       perceptron = tf.keras.layers.Dense(units=1, activation='sigmoid', input_shape=(input_size,))
       

   - Activation Functions:
     - Functions like ReLU or sigmoid introduce non-linearity.
     

       activation_relu = tf.keras.layers.Activation('relu')
       activation_sigmoid = tf.keras.layers.Activation('sigmoid')
       

2. Building Neural Networks:
   - Sequential Model:
     - A linear stack of layers.
     

       model = tf.keras.Sequential([
           tf.keras.layers.Dense(64, activation='relu', input_shape=(input_size,)),
           tf.keras.layers.Dense(1, activation='sigmoid')
       ])
       

   - Compiling the Model:
     - Specify optimizer, loss function, and metrics.
     

       model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
       

3. Training Neural Networks:
   - Fit Method:
     - Train the model on training data.
     

       model.fit(X_train, y_train, epochs=10, batch_size=32, validation_data=(X_val, y_val))
       

   - Model Evaluation:
     - Assess the model's performance on test data.
     

       test_loss, test_accuracy = model.evaluate(X_test, y_test)
       

4. Convolutional Neural Networks (CNNs):
   - Convolutional Layers:
     - Specialized layers for image data.
     

       model.add(tf.keras.layers.Conv2D(filters=64, kernel_size=(3, 3), activation='relu', input_shape=(height, width, channels)))
       

   - Pooling Layers:
     - Reduce dimensionality.
     

       model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2)))
       

5. Recurrent Neural Networks (RNNs):
   - LSTM Layers:
     - Handle sequences of data.
     

       model.add(tf.keras.layers.LSTM(units=50, return_sequences=True, input_shape=(timesteps, features)))
       

   - Embedding Layers:
     - Convert words to vectors in natural language processing.
     

       model.add(tf.keras.layers.Embedding(input_dim=vocab_size, output_dim=embedding_dim, input_length=max_length))
       

Deep learning with TensorFlow is powerful for handling complex tasks like image recognition and sequence processing.

Hope it helps :)

Ace your big data interview with the latest Apache Flink Course
👇👇
https://www.udemy.com/course/apache-flink-with-scala-3/?couponCode=LETSLEARNNOWPP

Promo Code: NEWRELEASE

Python Learning Series Part-14

14. Transfer Learning with Pre-trained Models:

Transfer learning involves using pre-trained models as a starting point for a new task. It's a powerful technique that leverages the knowledge gained from training on large datasets.

1. Introduction to Transfer Learning:
   - Why Transfer Learning?
     - Utilize knowledge learned from one task to improve performance on a different, but related, task.

   - Pre-trained Models:
     - Models trained on massive datasets, such as ImageNet, that capture general features of images, text, or other data.

2. Transfer Learning in Computer Vision:
   - Fine-tuning Pre-trained Models:
     - Adjust the weights of a pre-trained model on a smaller dataset for a specific task.
     

       base_model = tf.keras.applications.MobileNetV2(weights='imagenet', include_top=False, input_shape=(224, 224, 3))
       base_model.trainable = False  # Freeze the pre-trained layers

       model = tf.keras.Sequential([
           base_model,
           tf.keras.layers.GlobalAveragePooling2D(),
           tf.keras.layers.Dense(10, activation='softmax')
       ])
       

   - Feature Extraction:
     - Use pre-trained models as feature extractors.
     

       base_model = tf.keras.applications.VGG16(weights='imagenet', include_top=False, input_shape=(224, 224, 3))

       for layer in base_model.layers:
           layer.trainable = False  # Freeze pre-trained layers

       model = tf.keras.Sequential([
           base_model,
           tf.keras.layers.Flatten(),
           tf.keras.layers.Dense(10, activation='softmax')
       ])
       

3. Transfer Learning in Natural Language Processing:
   - Using Pre-trained Embeddings:
     - Utilize word embeddings trained on large text corpora.
     

       embeddings_index = load_pretrained_word_embeddings()
       embedding_matrix = create_embedding_matrix(word_index, embeddings_index)
       embedding_layer = tf.keras.layers.Embedding(input_dim=vocab_size, output_dim=embedding_dim, weights=[embedding_matrix], input_length=max_length)
       

   - Fine-tuning Language Models:
     - Fine-tune models like BERT for specific tasks.
     

       bert_model = TFBertModel.from_pretrained('bert-base-uncased')
       

Transfer learning accelerates model development by leveraging pre-existing knowledge.

Hope it helps :)

How to send follow up email to a recruiter 👇👇

Dear [Recruiter’s Name],

I hope this email finds you doing well. I wanted to take a moment to express my sincere gratitude for the time and consideration you have given me throughout the recruitment process for the [position] role at [company].

I understand that you must be extremely busy and receive countless applications, so I wanted to reach out and follow up on the status of my application. If it’s not too much trouble, could you kindly provide me with any updates or feedback you may have?

I want to assure you that I remain genuinely interested in the opportunity to join the team at [company] and I would be honored to discuss my qualifications further. If there are any additional materials or information you require from me, please don’t hesitate to let me know.

Thank you for your time and consideration. I appreciate the effort you put into recruiting and look forward to hearing from you soon.

Warmest regards,

Like if helps

👉Telegram Link: https://t.me/addlist/wcoDjKedDTBhNzFl

All the best 👍👍

Leetcode Questions you can check to Learn DSA from scratch 👇👇

1️⃣ Arrays: Data structures, such as arrays, store elements in contiguous memory locations. They are versatile and useful for a wide variety of purposes.
LeetCode Problems:
• Search in Rotated Sorted Array (Problem #33)
• Product of Array Except Self (Problem #238)
• Find the Missing Number (Problem #268)

2️⃣Two Pointers: In Two Pointers, two pointers are maintained in the collection and can be manipulated to solve a problem efficiently.
LeetCode problems:
• Trapping Rain Water (Problem #42)
• Longest Substring Without Repeating Characters (Problem #3)
• Squares of a Sorted Array (Problem #977)

3️⃣In-place Linked List Traversal: As an explanation, in-place traversal is a technique for modifying linked list nodes without using extra space.
LeetCode Problems:
• Remove Nth Node From End of List (Problem #19)
• Reorder List (Problem #143)

4️⃣Fast & Slow Pointers: This pattern uses two pointers to traverse a sequence at different speeds (fast and slow), often used to detect cycles or find a specific position in the sequence.
LeetCode Problems:
• Happy Number (Problem #202)
• Subarray Sum Equals K (Problem #560)
• Intersection of Two Linked Lists (Problem #160)

5️⃣Merge Intervals: This pattern involves merging overlapping intervals in a collection, often used in problems dealing with intervals or ranges.
LeetCode problems:
• Non-overlapping Intervals (Problem #435)
• Minimum Number of Arrows to Burst Balloons (Problem #452)

Join for more: https://t.me/crackingthecodinginterviews

ENJOY LEARNING 👍👍

RtBrick questions

1) explain types of ipc( inter-process techniques )

2) how would you implement a custom memory allocator , given that you should also be able to detect memory leaks , and detection to follow sequential order of allocations.

3) if I connect two pc's directly using a cable , will ip be needed to send packets

4) difference between private and public IP

5) why shared_memory is faster than other techniques

6) how to optimise a code from data perspective, how to optimise from loop( instructions ) perspective

7) why pointer sizes remains constant

8) why struct padding is needed

9) meta and data blocks

10) how to debug a core file , what steps , how to backtrace

11) location of shared_memory , static vs shared libs

12) connect 2 routers, now in order to ping these two, will arp be used or not , and how does ping work

13) when to use TCP and UDP

14) different regions in a process memory and where do mapping happen

15) why virtual memory and how does process know by demand paging which page to fetch?? How do you even divide a process into pages as a process / program is a file running

16) when to use an event I/O mechanism and pub - sub model

17) if 2 process are exceeding cpu cycles , and you know the root cause is some anomaly in the event i/o mechanism, how would you go about debugging it

18) explain symmetric vs asymmetric cryptography, what is a private key and public key, explain any common cryptography asymmetric algo

19) when you use a vpn, from which side is it feasible to inspect the public IP of the source ? What are SSL certificates, how to authenticate using SSL certificates

20) explain master to peer slave architecture, how to debug issues in a typical consensus self election algothrm

21) what is a little endian and big endian system and how to detect one . When sending data into a network , can you directly send little endian  data to big endian or not ??? If yes , how, if no , why and how ?
------------DSA-------------
BST, DLL, insertion deletion searching in them, any balanced BST, longest subarray without repeat chars, common but manipulation techniques