dev.to
1. Web Scraping With Puppeteer for Total Noobs
Web scraping is something I never thought I'd do. I'm primarily a UI developer, although my career started as a backend developer. I've never been asked at a job to perform any scraping tasks, and I'd never had a personal project that required me to scrape some data until recently. I'll share what I've learned, which is honestly probably just scratching the surface of what you can do with a technology like Puppeteer. In this post I'll walk you through installing and writing a Node.js script that…
#javascript #beginners #tutorial #webdev
2. Data Pipelines with Great Expectations | Introduction
My background is mostly in web development, but I am learning data engineering because I am interested in business intelligence and want to level-up my knowledge. One of the things that I have recently discovered is data validation within a data pipeline. Let me share a quick example from a previous job I had that illustrates the need for data validation in a data pipeline.
Example of a faulty data pipeline
We had a web app that allowed our clients to log into an account and view performance d…
#datascience #dataengineering #datavalidation #greatexpectations
3. A Comprehensive Guide to Cloud Migration from On-Prem Datacenter to AWS
Cloud migration is the process of moving data, applications, or other business elements from an on-premise environment to a cloud-based environment. Migrating to Amazon Web Services (AWS) is a popular choice for organizations looking to take advantage of the benefits of cloud computing.
Phases of Cloud Migration
Assessment: In this phase, organizations conduct a thorough assessment of their existing on-premise infrastructure and identify the elements that need to be migrated to AWS. The asses…
#aws #cloudmigration #cloud #migration
1. Web Scraping With Puppeteer for Total Noobs
Web scraping is something I never thought I'd do. I'm primarily a UI developer, although my career started as a backend developer. I've never been asked at a job to perform any scraping tasks, and I'd never had a personal project that required me to scrape some data until recently. I'll share what I've learned, which is honestly probably just scratching the surface of what you can do with a technology like Puppeteer. In this post I'll walk you through installing and writing a Node.js script that…
#javascript #beginners #tutorial #webdev
2. Data Pipelines with Great Expectations | Introduction
My background is mostly in web development, but I am learning data engineering because I am interested in business intelligence and want to level-up my knowledge. One of the things that I have recently discovered is data validation within a data pipeline. Let me share a quick example from a previous job I had that illustrates the need for data validation in a data pipeline.
Example of a faulty data pipeline
We had a web app that allowed our clients to log into an account and view performance d…
#datascience #dataengineering #datavalidation #greatexpectations
3. A Comprehensive Guide to Cloud Migration from On-Prem Datacenter to AWS
Cloud migration is the process of moving data, applications, or other business elements from an on-premise environment to a cloud-based environment. Migrating to Amazon Web Services (AWS) is a popular choice for organizations looking to take advantage of the benefits of cloud computing.
Phases of Cloud Migration
Assessment: In this phase, organizations conduct a thorough assessment of their existing on-premise infrastructure and identify the elements that need to be migrated to AWS. The asses…
#aws #cloudmigration #cloud #migration