Today's Quiz - How would you evaluate a logistic regression model?
Company Name - Infosys
Role - Data Scientist
Model Evaluation is a very important part in any analysis to answer the following questions,
How well does the model fit the data? Which predictors are most important? Are the predictions accurate?
So, the following are the criterion to access the model performance,
1. Akaike Information Criteria (AIC): In simple terms, AIC estimates the relative amount of information lost by a given model. So, the less information lost the higher the quality of the model. Therefore, we always prefer models with minimum AIC.
2. Receiver operating characteristics (ROC curve): ROC curve illustrates the diagnostic ability of a binary classifier. It is calculated/ created by plotting True Positive against False Positive at various threshold settings. The performance metric of ROC curve is AUC (area under curve). Higher the area under the curve, better the prediction power of the model.
3. Confusion Matrix: In order to find out how well the model does in predicting the target variable, we use a confusion matrix/ classification rate. It is nothing but a tabular representation of actual Vs predicted values which helps us to find the accuracy of the model.
Company Name - Infosys
Role - Data Scientist
Model Evaluation is a very important part in any analysis to answer the following questions,
How well does the model fit the data? Which predictors are most important? Are the predictions accurate?
So, the following are the criterion to access the model performance,
1. Akaike Information Criteria (AIC): In simple terms, AIC estimates the relative amount of information lost by a given model. So, the less information lost the higher the quality of the model. Therefore, we always prefer models with minimum AIC.
2. Receiver operating characteristics (ROC curve): ROC curve illustrates the diagnostic ability of a binary classifier. It is calculated/ created by plotting True Positive against False Positive at various threshold settings. The performance metric of ROC curve is AUC (area under curve). Higher the area under the curve, better the prediction power of the model.
3. Confusion Matrix: In order to find out how well the model does in predicting the target variable, we use a confusion matrix/ classification rate. It is nothing but a tabular representation of actual Vs predicted values which helps us to find the accuracy of the model.
DS Job Opportunities for Freshers
1. Role: Junior Data Scientist
Company: A Client of Analytics Vidhya
Experince : 0-2 Years of Experience
Job link: https://lnkd.in/eVzfJJ37
2. Role: Integration Analyst
Company: Zycus
Experience: 0-1 years
Job link: https://lnkd.in/eBxzw8ff
3. Role: Data Scientist
Company: A Client of Analytics Vidhya
Experince: 0-2 Years
Job link: https://lnkd.in/e5z6_XFc
4. Role: Data Analyst Intern
Company: Trell
Experience: 0-1 years
Job link: https://lnkd.in/gJXXjEen
5. Role:Data Scientist
Company: Myna Solutions
Experience: 0-1 years
Job link: https://lnkd.in/e5NC_wjh
1. Role: Junior Data Scientist
Company: A Client of Analytics Vidhya
Experince : 0-2 Years of Experience
Job link: https://lnkd.in/eVzfJJ37
2. Role: Integration Analyst
Company: Zycus
Experience: 0-1 years
Job link: https://lnkd.in/eBxzw8ff
3. Role: Data Scientist
Company: A Client of Analytics Vidhya
Experince: 0-2 Years
Job link: https://lnkd.in/e5z6_XFc
4. Role: Data Analyst Intern
Company: Trell
Experience: 0-1 years
Job link: https://lnkd.in/gJXXjEen
5. Role:Data Scientist
Company: Myna Solutions
Experience: 0-1 years
Job link: https://lnkd.in/e5NC_wjh
Today's Quiz - What is Pruning in Decision Trees, and How Is It Done?
Company Name - Landmark Group
Role - Data Scientist
Pruning is a data compression technique in machine learning and search algorithms that reduces the size of decision trees by removing sections of the tree that are non-critical and redundant to classify instances. Pruning reduces the complexity of the final classifier, and hence improves predictive accuracy by the reduction of overfitting.
Pruning processes can be divided into two types (pre- and post-pruning).
Pre-pruning procedures prevent a complete induction of the training set by replacing a stop () criterion in the induction algorithm. Pre-pruning methods are considered to be more efficient because they do not induce an entire set, but rather trees remain small from the start
Post-pruning is the most common way of simplifying trees. Here, nodes and subtrees are replaced with leaves to reduce complexity.
The procedures are differentiated on the basis of their approach in the tree (top-down or bottom-up).
Top-down fashion: It will traverse nodes and trim subtrees starting at the root.
Bottom-up fashion: It will begin at the leaf nodes.
There is a popular pruning algorithm called reduced error pruning, in which starting at the leaves, each node is replaced with its most popular class. If the prediction accuracy is not affected, the change is kept.
Company Name - Landmark Group
Role - Data Scientist
Pruning is a data compression technique in machine learning and search algorithms that reduces the size of decision trees by removing sections of the tree that are non-critical and redundant to classify instances. Pruning reduces the complexity of the final classifier, and hence improves predictive accuracy by the reduction of overfitting.
Pruning processes can be divided into two types (pre- and post-pruning).
Pre-pruning procedures prevent a complete induction of the training set by replacing a stop () criterion in the induction algorithm. Pre-pruning methods are considered to be more efficient because they do not induce an entire set, but rather trees remain small from the start
Post-pruning is the most common way of simplifying trees. Here, nodes and subtrees are replaced with leaves to reduce complexity.
The procedures are differentiated on the basis of their approach in the tree (top-down or bottom-up).
Top-down fashion: It will traverse nodes and trim subtrees starting at the root.
Bottom-up fashion: It will begin at the leaf nodes.
There is a popular pruning algorithm called reduced error pruning, in which starting at the leaves, each node is replaced with its most popular class. If the prediction accuracy is not affected, the change is kept.
Today's Quiz - Difference between forecasting and prediction?
Company Name - L&T Infotech
Role - Data Scientist
A forecast refers to a calculation or an estimation which uses data from previous events, combined with recent trends to come up a future event outcome.
On the other hand, a prediction is an actual act of indicating that something will happen in the future with or without prior information.
Accuracy: A Forecast is more accurate compared to a prediction. This is because forecasts are derived by analyzing a set of past data from the past and presents trends.
On the other hand, a prediction can be right or wrong. For example, if you predict the outcome of a football match, the result depends on how well the teams played no matter their recent performance or players.
Bias: Forecasting uses mathematical formulas and as a result, they are free from personal as well as intuition bias.
On the other hand, predictions are in most cases subjective and fatalistic in nature.
Quantification: When using a model to do a forecast, it’s possible to come up with the exact quantity. For example, the World Bank uses economic trends, and the previous GDP values and other inputs to come up with a percentage value for a country economic growth.
However, when doing prediction, since there is no data for processing, one can only say the economy of a given country will grow or not.
Application: Forecasts are only applicable in the economic and meteorology field where there is a lot of information about the subject matter.
On the contrary, prediction can be applied anywhere as long as there is an expected future outcome.
Company Name - L&T Infotech
Role - Data Scientist
A forecast refers to a calculation or an estimation which uses data from previous events, combined with recent trends to come up a future event outcome.
On the other hand, a prediction is an actual act of indicating that something will happen in the future with or without prior information.
Accuracy: A Forecast is more accurate compared to a prediction. This is because forecasts are derived by analyzing a set of past data from the past and presents trends.
On the other hand, a prediction can be right or wrong. For example, if you predict the outcome of a football match, the result depends on how well the teams played no matter their recent performance or players.
Bias: Forecasting uses mathematical formulas and as a result, they are free from personal as well as intuition bias.
On the other hand, predictions are in most cases subjective and fatalistic in nature.
Quantification: When using a model to do a forecast, it’s possible to come up with the exact quantity. For example, the World Bank uses economic trends, and the previous GDP values and other inputs to come up with a percentage value for a country economic growth.
However, when doing prediction, since there is no data for processing, one can only say the economy of a given country will grow or not.
Application: Forecasts are only applicable in the economic and meteorology field where there is a lot of information about the subject matter.
On the contrary, prediction can be applied anywhere as long as there is an expected future outcome.
Today's Quiz - Why do we need a validation set and a test set?
Company Name - Infosys
Role - Data Scientist
The data is split into three different categories while creating a model:
Training set: We use the training set for building the model and adjusting the model’s variables. But we cannot rely on the correctness of the model build on top of the training set. The model might give incorrect outputs on feeding new inputs.
Validation set: We use a validation set to look into the model’s response on top of the samples that don’t exist in the training dataset. Then, we will tune hyperparameters on the basis of the estimated benchmark of the validation data.
When we are evaluating the model’s response using the validation set, we are indirectly training the model with the validation set. This may lead to the overfitting of the model to specific data. So, this model won’t be strong enough to give the desired response to the real-world data.
Test set: The test dataset is the subset of the actual dataset, which is not yet used to train the model. The model is unaware of this dataset. So, by using the test dataset, we can compute the response of the created model on hidden data. We evaluate the model’s performance on the basis of the test dataset.
The model is always exposed to the test dataset after tuning the hyperparameters on top of the validation set.
As we know, the evaluation of the model on the basis of the validation set would not be enough. Thus, we use a test set for computing the efficiency of the model.
Company Name - Infosys
Role - Data Scientist
The data is split into three different categories while creating a model:
Training set: We use the training set for building the model and adjusting the model’s variables. But we cannot rely on the correctness of the model build on top of the training set. The model might give incorrect outputs on feeding new inputs.
Validation set: We use a validation set to look into the model’s response on top of the samples that don’t exist in the training dataset. Then, we will tune hyperparameters on the basis of the estimated benchmark of the validation data.
When we are evaluating the model’s response using the validation set, we are indirectly training the model with the validation set. This may lead to the overfitting of the model to specific data. So, this model won’t be strong enough to give the desired response to the real-world data.
Test set: The test dataset is the subset of the actual dataset, which is not yet used to train the model. The model is unaware of this dataset. So, by using the test dataset, we can compute the response of the created model on hidden data. We evaluate the model’s performance on the basis of the test dataset.
The model is always exposed to the test dataset after tuning the hyperparameters on top of the validation set.
As we know, the evaluation of the model on the basis of the validation set would not be enough. Thus, we use a test set for computing the efficiency of the model.
Today's Quiz - What are some ways I can make my model more robust to outliers?
Company Name - Bridgei2i Analytics.
Role - Data Scientist
There are several ways to make a model more robust to outliers, from different points of view (data preparation or model building). An outlier in the question and answer is assumed being unwanted, unexpected, or a must-be-wrong value to the human’s knowledge so far (e.g. no one is 200 years old) rather than a rare event which is possible but rare.
Outliers are usually defined in relation to the distribution. Thus outliers could be removed in the pre-processing step (before any learning step), by using standard deviations (Mean +/- 2*SD), it can be used for normality. Or interquartile ranges Q1 - Q3, Q1 - is the "middle" value in the first half of the rank-ordered data set, Q3 - is the "middle" value in the second half of the rank-ordered data set. It can be used for not normal/unknown as threshold levels.
Moreover, data transformation (e.g. log transformation) may help if data have a noticeable tail. When outliers related to the sensitivity of the collecting instrument which may not precisely record small values, Winsorization may be useful. This type of transformation has the same effect as clipping signals (i.e. replaces extreme data values with less extreme values). Another option to reduce the influence of outliers is using mean absolute difference rather mean squared error.
For model building, some models are resistant to outliers (e.g. tree-based approaches) or non-parametric tests. Similar to the median effect, tree models divide each node into two in each split. Thus, at each split, all data points in a bucket could be equally treated regardless of extreme values they may have.
Company Name - Bridgei2i Analytics.
Role - Data Scientist
There are several ways to make a model more robust to outliers, from different points of view (data preparation or model building). An outlier in the question and answer is assumed being unwanted, unexpected, or a must-be-wrong value to the human’s knowledge so far (e.g. no one is 200 years old) rather than a rare event which is possible but rare.
Outliers are usually defined in relation to the distribution. Thus outliers could be removed in the pre-processing step (before any learning step), by using standard deviations (Mean +/- 2*SD), it can be used for normality. Or interquartile ranges Q1 - Q3, Q1 - is the "middle" value in the first half of the rank-ordered data set, Q3 - is the "middle" value in the second half of the rank-ordered data set. It can be used for not normal/unknown as threshold levels.
Moreover, data transformation (e.g. log transformation) may help if data have a noticeable tail. When outliers related to the sensitivity of the collecting instrument which may not precisely record small values, Winsorization may be useful. This type of transformation has the same effect as clipping signals (i.e. replaces extreme data values with less extreme values). Another option to reduce the influence of outliers is using mean absolute difference rather mean squared error.
For model building, some models are resistant to outliers (e.g. tree-based approaches) or non-parametric tests. Similar to the median effect, tree models divide each node into two in each split. Thus, at each split, all data points in a bucket could be equally treated regardless of extreme values they may have.
Hey Folks
Our Popular Data Science in on SUPER DISCOUNT today and this offer will end by 11:59PM (IST)
You still have chance to grap this opportunity.
Regular price: ₹3499/-
Today's Price: ₹999/-
You still have 2 Hours.
Don't Miss this Offer
Get hands on practical learning experience with one one course support mentorship.
Discount Link: https://www.cloudyml.com/ai-for-all-discount/
#LearnByDoing
Our Popular Data Science in on SUPER DISCOUNT today and this offer will end by 11:59PM (IST)
You still have chance to grap this opportunity.
Regular price: ₹3499/-
Today's Price: ₹999/-
You still have 2 Hours.
Don't Miss this Offer
Get hands on practical learning experience with one one course support mentorship.
Discount Link: https://www.cloudyml.com/ai-for-all-discount/
#LearnByDoing
Today's Quiz - How would you predict who will renew their subscription next month? What data would you need to solve this? What analysis would you do? Would you build predictive models? If so, which algorithms?
Company Name - Cognizant
Role - Data Scientist
Let’s assume that we’re trying to predict renewal rate for Netflix subscription. So our problem statement is to predict which users will renew their subscription plan for the next month.
Next, we must understand the data that is needed to solve this problem. In this case, we need to check the number of hours the channel is active for each household, the number of adults in the household, number of kids, which channels are streamed the most, how much time is spent on each channel, how much has the watch rate varied from last month, etc. Such data is needed to predict whether or not a person will continue the subscription for the upcoming month.
After collecting this data, it is important that you find patterns and correlations. For example, we know that if a household has kids, then they are more likely to subscribe. Similarly, by studying the watch rate of the previous month, you can predict whether a person is still interested in a subscription. Such trends must be studied.
The next step is analysis. For this kind of problem statement, you must use a classification algorithm that classifies customers into 2 groups:
Customers who are likely to subscribe next month
Customers who are not likely to subscribe next month
Would you build predictive models? Yes, in order to achieve this you must build a predictive model that classifies the customers into 2 classes like mentioned above.
Which algorithms to choose? You can choose classification algorithms such as Logistic Regression, Random Forest, Support Vector Machine, etc.
Once you’ve opted the right algorithm, you must perform model evaluation to calculate the efficiency of the algorithm. This is followed by deployment.
Company Name - Cognizant
Role - Data Scientist
Let’s assume that we’re trying to predict renewal rate for Netflix subscription. So our problem statement is to predict which users will renew their subscription plan for the next month.
Next, we must understand the data that is needed to solve this problem. In this case, we need to check the number of hours the channel is active for each household, the number of adults in the household, number of kids, which channels are streamed the most, how much time is spent on each channel, how much has the watch rate varied from last month, etc. Such data is needed to predict whether or not a person will continue the subscription for the upcoming month.
After collecting this data, it is important that you find patterns and correlations. For example, we know that if a household has kids, then they are more likely to subscribe. Similarly, by studying the watch rate of the previous month, you can predict whether a person is still interested in a subscription. Such trends must be studied.
The next step is analysis. For this kind of problem statement, you must use a classification algorithm that classifies customers into 2 groups:
Customers who are likely to subscribe next month
Customers who are not likely to subscribe next month
Would you build predictive models? Yes, in order to achieve this you must build a predictive model that classifies the customers into 2 classes like mentioned above.
Which algorithms to choose? You can choose classification algorithms such as Logistic Regression, Random Forest, Support Vector Machine, etc.
Once you’ve opted the right algorithm, you must perform model evaluation to calculate the efficiency of the algorithm. This is followed by deployment.
Some Data Science Internship opportunities for freshers:
1. Company: Dataweave Pvt Ltd
Role: Data Engineering Internship
Experience: 0 year
Job Link:https://lnkd.in/eAZTF5EH
2. Company: Cisco
Role: Data Scientist - (Intern)
Experience: 0 years
Job Link: https://lnkd.in/erisCMfV
3. Company: Amazon
Role: Data Scientist Intern
Experience: 0 year
Job Link: https://lnkd.in/ebPXt2UC
4. Company: Al Statistics
Role: Data Science Intern
Experience: 0 year
Job Link: https://lnkd.in/e8ggDsam
5. Company: Microsoft
Role: Data Scientist Interns
Experience: 0-1 years
Job Link: https://lnkd.in/e4aVz64p
1. Company: Dataweave Pvt Ltd
Role: Data Engineering Internship
Experience: 0 year
Job Link:https://lnkd.in/eAZTF5EH
2. Company: Cisco
Role: Data Scientist - (Intern)
Experience: 0 years
Job Link: https://lnkd.in/erisCMfV
3. Company: Amazon
Role: Data Scientist Intern
Experience: 0 year
Job Link: https://lnkd.in/ebPXt2UC
4. Company: Al Statistics
Role: Data Science Intern
Experience: 0 year
Job Link: https://lnkd.in/e8ggDsam
5. Company: Microsoft
Role: Data Scientist Interns
Experience: 0-1 years
Job Link: https://lnkd.in/e4aVz64p
https://cutshort.io/
Dataweave Pvt Ltd is hiring Data Engineering Internship job in Bengaluru (Bangalore) | Cutshort
Apply to Data Engineering Internship job at Dataweave Pvt Ltd in Bengaluru (Bangalore) from 0 - 1 years of experience.
Today's Quiz - What is backpropagation? How does it work? Why do we need it?
Company Name - EY
Role - Data Scientist
The Backpropagation algorithm looks for the minimum value of the error function in weight space using a technique called the delta rule or gradient descent. The weights that minimize the error function is then considered to be a solution to the learning problem.
We need backpropagation because,
Calculate the error – How far is your model output from the actual output.
Minimum Error – Check whether the error is minimized or not.
Update the parameters – If the error is huge then, update the parameters (weights and biases). After that again check the error.
Repeat the process until the error becomes minimum.
Model is ready to make a prediction – Once the error becomes minimum, you can feed some inputs to your model and it will produce the output.
Company Name - EY
Role - Data Scientist
The Backpropagation algorithm looks for the minimum value of the error function in weight space using a technique called the delta rule or gradient descent. The weights that minimize the error function is then considered to be a solution to the learning problem.
We need backpropagation because,
Calculate the error – How far is your model output from the actual output.
Minimum Error – Check whether the error is minimized or not.
Update the parameters – If the error is huge then, update the parameters (weights and biases). After that again check the error.
Repeat the process until the error becomes minimum.
Model is ready to make a prediction – Once the error becomes minimum, you can feed some inputs to your model and it will produce the output.
**Today's Quiz -**
Three zebras are sitting on each corner of an equilateral triangle. Each zebra randomly picks a direction and only runs along the outline of the triangle to either opposite edge of the triangle. What is the probability that none of the zebras collide?
Company Name - Wipro
Role - Data Scientist**
• Let's imagine all of the zebras on an equilateral triangle. They each have two options of directions to go in if they are running along the outline to either edge. Given the case is random, let's compute the possibilities in which they fail to collide.
• There are only really two possibilities. The zebras will either all choose to run in a clockwise direction or a counter-clockwise direction.
• Let's calculate the probabilities of each. The probability that every zebra will choose to go clockwise will be the product of each zebra choosing the clockwise direction. Given there are two choices (counterclockwise or clockwise), that would be 1/2 * 1/2 * 1/2 = 1/8
• The probability of every zebra going counter-clockwise is the same at 1/8. Therefore, if we sum up the probabilities, we get the correct probability of 1/4 or 25%.
Three zebras are sitting on each corner of an equilateral triangle. Each zebra randomly picks a direction and only runs along the outline of the triangle to either opposite edge of the triangle. What is the probability that none of the zebras collide?
Company Name - Wipro
Role - Data Scientist**
• Let's imagine all of the zebras on an equilateral triangle. They each have two options of directions to go in if they are running along the outline to either edge. Given the case is random, let's compute the possibilities in which they fail to collide.
• There are only really two possibilities. The zebras will either all choose to run in a clockwise direction or a counter-clockwise direction.
• Let's calculate the probabilities of each. The probability that every zebra will choose to go clockwise will be the product of each zebra choosing the clockwise direction. Given there are two choices (counterclockwise or clockwise), that would be 1/2 * 1/2 * 1/2 = 1/8
• The probability of every zebra going counter-clockwise is the same at 1/8. Therefore, if we sum up the probabilities, we get the correct probability of 1/4 or 25%.
**Today's Quiz - If your model suffers from low bias and high variance, which algorithm would you use to tackle it? Why?
Company Name - KPMG
Role - Data Scientist**
The error of a model can either be of bias and/or variance. Very low bias but high variance indicates overfitting, as well as complexity of the model. By averaging these out, we can reduce the variance and increase the bias.
a) A bagging algorithm can handle the high variance. The dataset is randomly subsampled mm times and the model trained using each subsample. Then the models are averaged by averaging out the predictions of each mode.
b) By using the k-nearest neighbour algorithm, the trade-off between bias and variance can be achieved. The value of k is increased to increase the number of neighbours that contribute to the prediction, and this in turn increases the bias of the model.
c) By using the support vector machine algorithm, the trade-off can be achieved by increasing the C parameter that influences the number of violations of the margin allowed in the training data, and this in turn increases the bias but decreases the variance.
Company Name - KPMG
Role - Data Scientist**
The error of a model can either be of bias and/or variance. Very low bias but high variance indicates overfitting, as well as complexity of the model. By averaging these out, we can reduce the variance and increase the bias.
a) A bagging algorithm can handle the high variance. The dataset is randomly subsampled mm times and the model trained using each subsample. Then the models are averaged by averaging out the predictions of each mode.
b) By using the k-nearest neighbour algorithm, the trade-off between bias and variance can be achieved. The value of k is increased to increase the number of neighbours that contribute to the prediction, and this in turn increases the bias of the model.
c) By using the support vector machine algorithm, the trade-off can be achieved by increasing the C parameter that influences the number of violations of the margin allowed in the training data, and this in turn increases the bias but decreases the variance.
**Today's Quiz - We know that one hot encoding increases the dimensionality of a dataset, but label encoding doesn’t. How?
Company Name - JP Morgan
Role - Data Scientist**
• When we use one-hot encoding, there is an increase in the dimensionality of a dataset.
• The reason for the increase in dimensionality is that, for every class in the categorical variables, it forms a different variable.
• Example: Suppose, there is a variable ‘Color.’ It has three sub-levels as Yellow, Purple, and Orange. So, one hot encoding ‘Color’ will create three different variables as Color.Yellow, Color.Purple, and Color.Orange.
• In label encoding, the sub-classes of a certain variable get the value as 0 and 1. So, we use label encoding only for binary variables.
• This is the reason that one hot encoding increases the dimensionality of data and label encoding does not.
Company Name - JP Morgan
Role - Data Scientist**
• When we use one-hot encoding, there is an increase in the dimensionality of a dataset.
• The reason for the increase in dimensionality is that, for every class in the categorical variables, it forms a different variable.
• Example: Suppose, there is a variable ‘Color.’ It has three sub-levels as Yellow, Purple, and Orange. So, one hot encoding ‘Color’ will create three different variables as Color.Yellow, Color.Purple, and Color.Orange.
• In label encoding, the sub-classes of a certain variable get the value as 0 and 1. So, we use label encoding only for binary variables.
• This is the reason that one hot encoding increases the dimensionality of data and label encoding does not.
Some Data Science Internship opportunities for freshers:
1. Company: Hotstar
Role: Data Analyst
Experience: 1-3 years
Job link: https://lnkd.in/eAJd-gnd
2. Company: iAnalyst Infotech Pvt. Ltd.
Role: Data Scientist Intern
Experience: 0 years
Job link: https://lnkd.in/dDAT3wyb
3. Company: Emplay
Role: Data Science Intern
Experience: 0 years
Job link: https://lnkd.in/dwasDHqN
4. Company: Merkle
Role: Data Science- Analyst (Intern)
Experience: 0 years
Job link: https://lnkd.in/d_qtaqje
5. Company: Cisco
Role: Data Scientist - (Intern)
Experience: 0 years
Job link: https://lnkd.in/djJaVujF
1. Company: Hotstar
Role: Data Analyst
Experience: 1-3 years
Job link: https://lnkd.in/eAJd-gnd
2. Company: iAnalyst Infotech Pvt. Ltd.
Role: Data Scientist Intern
Experience: 0 years
Job link: https://lnkd.in/dDAT3wyb
3. Company: Emplay
Role: Data Science Intern
Experience: 0 years
Job link: https://lnkd.in/dwasDHqN
4. Company: Merkle
Role: Data Science- Analyst (Intern)
Experience: 0 years
Job link: https://lnkd.in/d_qtaqje
5. Company: Cisco
Role: Data Scientist - (Intern)
Experience: 0 years
Job link: https://lnkd.in/djJaVujF
Linkedin
Hotstar hiring Data Analyst in Mumbai, Maharashtra, India | LinkedIn
Posted 1:54:46 PM. Disney+ Hotstar is the fastest growing entertainment brand in India with one of the largest…See this and similar jobs on LinkedIn.
**Today's Quiz - Two candidates Aman and Mohan appear for a Data Science Job interview. The probability of Aman cracking the interview is 1/8 and that of Mohan is 5/12. What is the probability that at least one of them will crack the interview?
Company Name - TheMathCompany
Role - Data Scientist Trainee**
The probability of Aman getting selected for the interview is 1/8 P(A) = 1/8 The probability of Mohan getting selected for the interview is 5/12 P(B)=5/12
Now, the probability of at least one of them getting selected can be denoted at the Union of A and B, which means
P(A U B) =P(A)+ P(B) – (P(A ∩ B)) ………………………(1)
Where P(A ∩ B) stands for the probability of both Aman and Mohan getting selected for the job. To calculate the final answer, we first have to find out the value of P(A ∩ B) So, P(A ∩ B) = P(A) * P(B)
1/8 * 5/12
5/96
Now, put the value of P(A ∩ B) into equation 1
P(A U B) =P(A)+ P(B) – (P(A ∩ B))
1/8 + 5/12 -5/96
So, the answer will be 47/96.
Company Name - TheMathCompany
Role - Data Scientist Trainee**
The probability of Aman getting selected for the interview is 1/8 P(A) = 1/8 The probability of Mohan getting selected for the interview is 5/12 P(B)=5/12
Now, the probability of at least one of them getting selected can be denoted at the Union of A and B, which means
P(A U B) =P(A)+ P(B) – (P(A ∩ B)) ………………………(1)
Where P(A ∩ B) stands for the probability of both Aman and Mohan getting selected for the job. To calculate the final answer, we first have to find out the value of P(A ∩ B) So, P(A ∩ B) = P(A) * P(B)
1/8 * 5/12
5/96
Now, put the value of P(A ∩ B) into equation 1
P(A U B) =P(A)+ P(B) – (P(A ∩ B))
1/8 + 5/12 -5/96
So, the answer will be 47/96.
**Today's Quiz - I know that a linear regression model is generally evaluated using Adjusted R² or F value. How would you evaluate a logistic regression model?
Company Name - EY
Role - Data Scientist**
We can use the following methods:
• Since logistic regression is used to predict probabilities, we can use AUC-ROC curve along with confusion matrix to determine its performance.
• Also, the analogous metric of adjusted R² in logistic regression is AIC. AIC is the measure of fit which penalizes model for the number of model coefficients. Therefore, we always prefer model with minimum AIC value.
• Null Deviance indicates the response predicted by a model with nothing but an intercept. Lower the value, better the model. Residual deviance indicates the response predicted by a model on adding independent variables. Lower the value, better the model.
Company Name - EY
Role - Data Scientist**
We can use the following methods:
• Since logistic regression is used to predict probabilities, we can use AUC-ROC curve along with confusion matrix to determine its performance.
• Also, the analogous metric of adjusted R² in logistic regression is AIC. AIC is the measure of fit which penalizes model for the number of model coefficients. Therefore, we always prefer model with minimum AIC value.
• Null Deviance indicates the response predicted by a model with nothing but an intercept. Lower the value, better the model. Residual deviance indicates the response predicted by a model on adding independent variables. Lower the value, better the model.
**Today's Quiz - How many iPhones are currently being used in India?
Company Name - Ericsson
Role - Data Analyst**
Clarify with the interviewers whether the question is about only a single version of the iPhone or all versions put together. Here, we shall assume that all iPhones put together are being talked about.
The first step toward solving this query will be segmentation. There are many ways in which India’s population can be segmented. Here, we shall first assume that only people who have attained a working age and are under the age of retirement own an iPhone. Children and old citizens do not own an iPhone. This removes 20% of the population as children and 20% as senior citizens.
The next assumption will be that only the upper stratum of India’s income range can afford an iPhone. This metric assumes that only 5% of the eligible citizens from the previous filter can own an iPhone.
Now, it is not necessary that every member of this upper stratum will own an iPhone. Other options, such as OnePlus, Samsung, etc., are also available. However, a fair assumption would be that 50% of the eligible population from the previous filter owns an iPhone.
Calculating the proportion of the population that owns an iPhone –
0.6 x 0.05 x 0.5 = 0.015
Total iPhones in India = 0.015 x 130 crore = 1.95 crore
Company Name - Ericsson
Role - Data Analyst**
Clarify with the interviewers whether the question is about only a single version of the iPhone or all versions put together. Here, we shall assume that all iPhones put together are being talked about.
The first step toward solving this query will be segmentation. There are many ways in which India’s population can be segmented. Here, we shall first assume that only people who have attained a working age and are under the age of retirement own an iPhone. Children and old citizens do not own an iPhone. This removes 20% of the population as children and 20% as senior citizens.
The next assumption will be that only the upper stratum of India’s income range can afford an iPhone. This metric assumes that only 5% of the eligible citizens from the previous filter can own an iPhone.
Now, it is not necessary that every member of this upper stratum will own an iPhone. Other options, such as OnePlus, Samsung, etc., are also available. However, a fair assumption would be that 50% of the eligible population from the previous filter owns an iPhone.
Calculating the proportion of the population that owns an iPhone –
0.6 x 0.05 x 0.5 = 0.015
Total iPhones in India = 0.015 x 130 crore = 1.95 crore
**Today's Quiz - You have been asked to evaluate a regression model based on R², adjusted R² and tolerance. What will be your criteria?
Company Name - Deloitte
Role - Data Science Trainee**
Tolerance (1 / VIF) is used as an indicator of multicollinearity. It is an indicator of percent of variance in a predictor which cannot be accounted by other predictors. Large values of tolerance are desirable.
We will consider adjusted R² as opposed to R² to evaluate model fit because R² increases irrespective of improvement in prediction accuracy as we add more variables.
But, adjusted R² would only increase if an additional variable improves the accuracy of model, otherwise stays same. It is difficult to commit a general threshold value for adjusted R² because it varies between data sets.
For example: a gene mutation data set might result in lower adjusted R² and still provide fairly good predictions, as compared to a stock market data where lower adjusted R² implies that model is not good.
Company Name - Deloitte
Role - Data Science Trainee**
Tolerance (1 / VIF) is used as an indicator of multicollinearity. It is an indicator of percent of variance in a predictor which cannot be accounted by other predictors. Large values of tolerance are desirable.
We will consider adjusted R² as opposed to R² to evaluate model fit because R² increases irrespective of improvement in prediction accuracy as we add more variables.
But, adjusted R² would only increase if an additional variable improves the accuracy of model, otherwise stays same. It is difficult to commit a general threshold value for adjusted R² because it varies between data sets.
For example: a gene mutation data set might result in lower adjusted R² and still provide fairly good predictions, as compared to a stock market data where lower adjusted R² implies that model is not good.
👇Key Highlights of this Course:
☑️ Assignments-based learning for Hands-on experience and Confidence build-up.
☑️25+ Assignments have nicely curated questions to cover all the important topics of Python, Machine Learning, SQL, Statistics, Model Deployment and AWS (SageMaker, Lambda Functions, API Gateway).
☑️10+ Data Science end to end Capstone Projects that you can put in your Resume.
☑️1-1 Skype course support mentorship available from 10 am to 10 pm every day (including weekends) to resolve Course,Assignment and Project related queries.
☑️250+ Frequently asked Interview Questions and answers PDF Collection
☑️Self Paced Course with Lifetime access
☑️Reference videos and blog collections in a structured way to learn all the Concepts.
☑️Get Data Science Interview Questions and Answers in the form of Newsletters daily.
☑️Suitable even for beginners with no prior coding experience
☑️Final Course Completion Certificate.
Course link: https://rzp.io/l/mfGimAk
☑️ Assignments-based learning for Hands-on experience and Confidence build-up.
☑️25+ Assignments have nicely curated questions to cover all the important topics of Python, Machine Learning, SQL, Statistics, Model Deployment and AWS (SageMaker, Lambda Functions, API Gateway).
☑️10+ Data Science end to end Capstone Projects that you can put in your Resume.
☑️1-1 Skype course support mentorship available from 10 am to 10 pm every day (including weekends) to resolve Course,Assignment and Project related queries.
☑️250+ Frequently asked Interview Questions and answers PDF Collection
☑️Self Paced Course with Lifetime access
☑️Reference videos and blog collections in a structured way to learn all the Concepts.
☑️Get Data Science Interview Questions and Answers in the form of Newsletters daily.
☑️Suitable even for beginners with no prior coding experience
☑️Final Course Completion Certificate.
Course link: https://rzp.io/l/mfGimAk
Razorpay
Pay for Machine Learning for Data Science Course by CloudyML
Discounted Course Fee: Rs. 1999/-. (GST Included in the price). . . **After making payment, you will get login credentials for our learning portal instantly via Email..
Today's Interview Quest
Company Name- Capital One
Role- Data Scientist
Q. How would you build a model to predict credit card fraud?
A. Use Kaggle's Credit card fraud dataset, start with EDA (Exploratory Data Analysis). Applying train, test split over the data and then finally choosing any model like logistic regression, XGBoost or Random Forest. After Hyperparameter tuning and fitting the model, the final step would be evaluating its performance.
Q. How would you derive new features from features that already exist?
A. Feature engineering is applied first to generate additional features, and then feature selection is done to eliminate irrelevant, redundant, or highly correlated features. This includes techniques like Binning, Data manipulation etc.
Q. If you’re attempting to predict a customer’s gender, and you only have 100 data points, what problems could arise?
A. Overfitting because we might learn too much into some particular patterns within this small sample set so we lose generalization abilities on other datasets.
Q. Suppose you were given two years of transaction history. What features would you use to predict credit risk?
A. Following are the features that can be used in such case.
Transaction amount,
Transaction count,
Transaction frequency,
transaction category: bar, grocery, jwery etc.
transaction channels: credit card, debit card, international wire transfer etc.
distance between transaction address and mailing address,
fraud/ risk score
Q. Explain overfitting and what steps you can take to prevent it.
A. Overfitting happens when a model learns the detail and noise in the training data to the extent that it negatively impacts the performance of the model on new data. Some steps that we can take to avoid it:
1. Data augmentation
2. L1/L2 Regularization
3. Remove layers / number of units per layer
4. Cross-validation
Q. Why does SVM need to maximize the margin between support vectors?
A. Our goal is to maximize the margin because the hyperplane for which the margin is maximum is the optimal hyperplane. Thus SVM tries to make a decision boundary in such a way that the separation between the two classes is as wide as possible in the plane.
Company Name- Capital One
Role- Data Scientist
Q. How would you build a model to predict credit card fraud?
A. Use Kaggle's Credit card fraud dataset, start with EDA (Exploratory Data Analysis). Applying train, test split over the data and then finally choosing any model like logistic regression, XGBoost or Random Forest. After Hyperparameter tuning and fitting the model, the final step would be evaluating its performance.
Q. How would you derive new features from features that already exist?
A. Feature engineering is applied first to generate additional features, and then feature selection is done to eliminate irrelevant, redundant, or highly correlated features. This includes techniques like Binning, Data manipulation etc.
Q. If you’re attempting to predict a customer’s gender, and you only have 100 data points, what problems could arise?
A. Overfitting because we might learn too much into some particular patterns within this small sample set so we lose generalization abilities on other datasets.
Q. Suppose you were given two years of transaction history. What features would you use to predict credit risk?
A. Following are the features that can be used in such case.
Transaction amount,
Transaction count,
Transaction frequency,
transaction category: bar, grocery, jwery etc.
transaction channels: credit card, debit card, international wire transfer etc.
distance between transaction address and mailing address,
fraud/ risk score
Q. Explain overfitting and what steps you can take to prevent it.
A. Overfitting happens when a model learns the detail and noise in the training data to the extent that it negatively impacts the performance of the model on new data. Some steps that we can take to avoid it:
1. Data augmentation
2. L1/L2 Regularization
3. Remove layers / number of units per layer
4. Cross-validation
Q. Why does SVM need to maximize the margin between support vectors?
A. Our goal is to maximize the margin because the hyperplane for which the margin is maximum is the optimal hyperplane. Thus SVM tries to make a decision boundary in such a way that the separation between the two classes is as wide as possible in the plane.