🐙 GitHub repo: TikTokLive

Python library to receive live stream events (comments, gifts, etc.) in realtime from TikTok LIVE.

https://github.com/isaackogan/TikTokLive

🗓 Update on the Scraping Universe

Did you know that TikTok's parent company scrapes all around the web massively? One of the most aggressive scraping on the internet! 🤯

ByteDance, the company behind TikTok, has been scraping data from websites at an insane rate.

According to Sam Crowther, the CEO of Kasada, the bot called Bytespider is blowing away the competition, hoovering up data 25 times faster than GPTbot, which scrapes data for ChatGPT. And 3,000 times quicker than ClaudeBot, the scraper bot used by Anthropic. 🤖

What do you think guys? 😀

In case you miss Extract Summit 2024 event by Zyte, you can access full days talks on Youtube

💼 Job market requirement insight for Scrapers

Zyte is opening a position as Principal Reverse Engineer and these are skills they required for the candidate:

• Hacker mindset
• Understand techniques and tools for crawling, extracting, and processing data
• Proficiency in programming languages: JavaScript/Node.js, Python, Java
• Reverse engineering skills: static, dynamic, and concolic analysis
• Understand operating systems and computer networking concepts
• Can use tools like Wireshark, Burp Suite, etc to intercept and debug network traffic
• Understand browser engines, browser fingerprinting, and ad-blocker mechanisms

And will be liked if:
• Experience with Decompilers, IDA Pro, Ghidra or Frida, Jadx, and Babel
• Experience with C/C++
• Core contributions to Mozilla or Chromium projects

🐙 GitHub repo: google-maps-scraper

scrape data data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place

https://github.com/gosom/google-maps-scraper

⚙ Tech Stack at Apify

Frontend: React.js, styled-components, Storybook, Cypress

Backend: TypeScript/Node.js, Next.js, Nest.js, Docusaurus, Jest

Infra: AWS, Kubernetes, Helm, MongoDB, Redis, DynamoDB, S3, GitHub Actions

Monitoring: New Relic, LogDNA, Sentry, PagerDuty

Tools: GitHub, ZenHub, Notion, GSuite

AI Tools: Langchain, LlamaIndex, Pinecone, OpenAI API, Web agents

🐙 GitHub repo: google-play-scraper

Google play scraper for Python

https://github.com/JoMingyu/google-play-scraper

🐙 GitHub repo: Scrapegraph-ai

Python scraper based on AI

https://github.com/ScrapeGraphAI/Scrapegraph-ai

🐙 GitHub repo: Google-Maps-Scraper

Google maps scraper with gui

https://github.com/Zubdata/Google-Maps-Scraper

🐙 GitHub repo: scrape

CLI utility to scrape emails from websites

https://github.com/lawzava/scrape

Stream the data (JSON) and handle auto-generated authentication using browser automation 🔥

https://youtu.be/g0EgwfQJew8

Business leads collecting from Google Maps scraping with Python + Selenium and view 🙂

🐙 GitHub repo: jaypyles/Scraperr

Self-hosted webscraper

https://github.com/jaypyles/Scraperr

JA3 transport is a way to identify a client’s TLS configuration. It includes the list of cipher suites supported, the extensions sent, and other details.

This fingerprint can be used to recognize the browser or device making the connection, even if it's using encryption.

Fingerprinting is a common method to detect automation bot and crawler.

How JA3 fingerprints can be impersonated? 🤔

Camouflage 🦎 with Firefox-based 🦊 automation. What a fresh idea 💡

https://camoufox.com