🔍 Understanding the Impact of Feature Selection vs. Feature Extraction in Dimensionality Reduction for Big Data 📊
In the era of big data, working with high-dimensional datasets presents major challenges in processing, visualization, and model performance. A recent study titled "Comparison of Feature Selection and Feature Extraction Role in Dimensionality Reduction of Big Data" (Journal of Techniques, 2023) offers a comprehensive evaluation of Feature Selection (FS) and Feature Extraction (FE) using the ANSUR II dataset — a U.S. Army anthropometric dataset with 109 features and 6068 observations.
📌 Study Goals
To compare FS and FE techniques in terms of:
➡️ Dimensionality reduction
➡️ Predictive performance
➡️ Information retention
⚙️ Techniques Explored
🧹 Feature Selection:
🔸 Highly Correlated Filter – removes features with correlation > 0.88
🔸 Recursive Feature Elimination (RFE) – eliminates the least important features iteratively
🔄 Feature Extraction:
🔹 Principal Component Analysis (PCA) – transforms original features into orthogonal components
🧪 Methodology
🧼 Data preprocessing using Missing Value Ratio
🧠 Classification using ML models:
✅ K-Nearest Neighbors (KNN)
✅ Decision Tree
✅ Support Vector Machine (SVM)
✅ Neural Network
✅ Random Forest
🔍 Post-reduction classification using the same models
📈 Key Results
🏆 KNN consistently performed best, maintaining 83% accuracy pre- and post-reduction
🧠 RFE showed the highest accuracy among reduction techniques with 66% post-reduction accuracy
🧩 PCA effectively reduced features but slightly decreased accuracy and interpretability
💡 Takeaways
✅ Use Feature Selection when interpretability and maintaining original structure are important
✅ Use Feature Extraction for noisy or highly redundant datasets
🎯 The choice depends on your data and modeling objectives
📖 Read the full paper here: DOI: 10.51173/jt.v5i1.1027
This is an excellent reference for anyone navigating the complexities of dimensionality reduction in ML pipelines. Whether you're optimizing models or just curious about FS vs. FE, this study is gold! 🧠✨
#MachineLearning #DataScience #FeatureEngineering #DimensionalityReduction #BigData #AI #KNN #PCA #RFE #MLResearch #DataAnalytics
In the era of big data, working with high-dimensional datasets presents major challenges in processing, visualization, and model performance. A recent study titled "Comparison of Feature Selection and Feature Extraction Role in Dimensionality Reduction of Big Data" (Journal of Techniques, 2023) offers a comprehensive evaluation of Feature Selection (FS) and Feature Extraction (FE) using the ANSUR II dataset — a U.S. Army anthropometric dataset with 109 features and 6068 observations.
📌 Study Goals
To compare FS and FE techniques in terms of:
➡️ Dimensionality reduction
➡️ Predictive performance
➡️ Information retention
⚙️ Techniques Explored
🧹 Feature Selection:
🔸 Highly Correlated Filter – removes features with correlation > 0.88
🔸 Recursive Feature Elimination (RFE) – eliminates the least important features iteratively
🔄 Feature Extraction:
🔹 Principal Component Analysis (PCA) – transforms original features into orthogonal components
🧪 Methodology
🧼 Data preprocessing using Missing Value Ratio
🧠 Classification using ML models:
✅ K-Nearest Neighbors (KNN)
✅ Decision Tree
✅ Support Vector Machine (SVM)
✅ Neural Network
✅ Random Forest
🔍 Post-reduction classification using the same models
📈 Key Results
🏆 KNN consistently performed best, maintaining 83% accuracy pre- and post-reduction
🧠 RFE showed the highest accuracy among reduction techniques with 66% post-reduction accuracy
🧩 PCA effectively reduced features but slightly decreased accuracy and interpretability
💡 Takeaways
✅ Use Feature Selection when interpretability and maintaining original structure are important
✅ Use Feature Extraction for noisy or highly redundant datasets
🎯 The choice depends on your data and modeling objectives
📖 Read the full paper here: DOI: 10.51173/jt.v5i1.1027
This is an excellent reference for anyone navigating the complexities of dimensionality reduction in ML pipelines. Whether you're optimizing models or just curious about FS vs. FE, this study is gold! 🧠✨
#MachineLearning #DataScience #FeatureEngineering #DimensionalityReduction #BigData #AI #KNN #PCA #RFE #MLResearch #DataAnalytics