55 Repositories
Python scraping Libraries
A python package to get amazon product and search data in json form. The package does not require any API keys as it works by scraping the amazon page.
amazondata A python package to get amazon product and search data in json form. The package does not require any API keys as it works by scraping the
Grab the content you want and push it to you, Support rss, html, json scraping
Notify_Rss 项目结构 使用 python3 本项目采用Python 3.10.5开发并测试如果你还未安装python3 请点击python3 pip3 install -r requirements.txt python3 setup.py -c [config_path] docker(
Using tweepy and snscrape fetch tweets from a list of twitter usernames and automatically like those tweets.
Twitter Auto Liker Using tweepy and snscrape fetch tweets from a list of twitter usernames and automatically like those tweets. Usage python main.py -
This program basically shows the festival that occur in a particular date including the tithi. It scraps the data from hamropatro. It is just for educational purpose , if the scraping causes any problem to the website then i will take down the code.
Nepali_Event_Viewer This program basically shows the festival that occur in a particular date including the tithi. It scraps the data from hamropatro.
Simple CLI-based tool for extracting and storing relevant job info. from bulk job searches on indeed.com
indeed-jobs-searcher Simple CLI-based tool for extracting and storing relevant job info. from bulk job searches on indeed.com. It would be remiss of m
🐦 Apuntes, ejercicios y antecedentes del curso impartido en el Congreso de Ciencias Básicas de la Universidad Autónoma de Aguascalientes, 2022 de creación de Bots de Twitter con Python y Web Scraping y APIS.
🤖 Creando BOTS de Twitter con Python y Web Scraping 📅 Impartido 3, 4 y 5 de Octubre, 2022 ⏰ Seis horas teórico-prácticas 💻 Congreso de Ciencias Bás
This repo contains a full-fledged Python-based script that scrapes a JavaScript-rendered website, cleans the data, and pushes the results to a cloud-based database. The workflow is orchestrated on Airflow to run automatically
python_scrapy_airflow_pipeline This repo contains a full-fledged Python-based script that scrapes a JavaScript-rendered website homzmart.com/en, clean
An open-source proxy scraper able to scrape from almost every site
KC-Scraper An open-source proxy scraper able to scrape from almost every website this tool is made for educational purposes only Features • Threads •
Scraping sephora.com all skincare products (Category Wise)
sephora-products-scraper Scraping sephora.com all skincare products (Category Wise) This django app will scrape all skin care products data from sepho
Yet another hands-on over Web Scraping, this time using Startup Decks to extract some files and store them in the Local System
Startup Scrapping A Web Scrapping project, which extracts multimedia from Startup Decks (https://startupdecks.co/decks/) and stores it in your local d
Scraping popular Job Portal using Python Programming Language with packages Beautiful Soup and Selenium.
Web-Scraping There are many jobs portals on the internet. One of them is id.indeed.com. A lot of information on job vacancies that are needed. So, thi
Enhanced web scraping tool for handling embedded links, tables and lists
Spiderman Web Scraper After reading through the BeautifulSoup documentation, I realised that many common operations are not in the module. As such, I
Project for MLPipeline with Crypto data.
Crypto Project Project for Web Scraping, MLOps and Cloud Computing with Crypto data. Step 1 - Web Scraping + Database architectures Core points: Web S
I'm scraping movies metadata from imdb website, and each movies songs from tunefind website. I'm planning to store the scraped in a postgresql database and use it in a web app.
movies - musics database: I'm scraping movies metadata from imdb website, and each movies songs from tunefind website. I'm planning to store the scrap
Web Scraping que permite filtrar Fundos Imobiliários listados na Bolsa de Valores Brasileira e armazená-los em uma planilha.
FIIs Filter Script que permite filtrar Fundos Imobiliários listados na B3 de acordo com parâmetros informados pelo usuário. Tabela de conteúdos Sobre
Python library for scraping inside Airflow.
as-scraper Python library for scraping inside Airflow. Installation The as-scraper library uses Geckodriver (Firefox) for scraping with the Selenium l
Scraping de sites de imobiliárias para achar casas que caibam no meu orçamento
Mudança Raspagem de dados de imobiliárias para achar uma casa que caiba no meu orçamento TODO Raspagem: Paginação Simplificação das URLs no yaml Limpe
Web Scraping que faz a busca de produtos/marcas em diversos sites e retorna-os juntamente com um json
Web Scraping Sites com BeautifulSoup Este projeto é somente para fins educacionais. O código mostra como fazer web scraping de páginas de conteúdo din
An open-source proxy scraper able to scrape from almost every site
KC-Scraper An open-source proxy scraper able to scrape from almost every website Features • Threads • Auto remove not reachable websites How to use
Advanced news reader website using django and celery.
Django News Reader Advanced news reader website using django and celery. How to Run Project Download Codes git clone https://github.com/dori-dev/news-
Simple requests based manhwa downloader which uses the Lezhin API.
Lezhin Downloader 📚 Simple requests based manhwa scraper which uses the Lezhin API. Installation ✨ Install the requirements Edit config in scrape.py
A proxy server to bypass cloudflare I am under attack mode using playwright, written in python
Cloudfire Cloudfire is a reverse proxy to bypass cloudflare's javascript challenge (I'm under attack mode) using playwright. How it works Cloudfire us
A collection of rock, paper and scissor, hangman, web scraping, number guessing etc
This collection, will comprise of rock, paper and scissor, hangman, web scraping and number guessing. This collection is to help me start my programming process on GitHub, learn how to utilize git language properly and learn how to formally present any project on GitHub.
This project extracts (web scraping) movie data from IMDB and send a customized email letter with random popular movies.
Email-Movie-Recommender-Web-Scrapping Web Scraping Project Description This project extracts movie data from IMDB and send a customized email letter w
Web scraping house discord
web-scraping-house-discord Features Discord Oklouer Contributing To contribute follow these steps: Fork this repository. Create a branch with clear na
Telegram tool for scraping group members and sending DMs to the collected targets.
Telegram Scraper and Message Sender Telegram Scraper and Mass DM sender • API Setup Go to http://my.telegram.org and log in. Click on API development
Tool for enumerate subdomains by Brute-force, or by using different options while scraping results.
Subxenum Subdomain enumerate tool (Subxenum) using different techniques and scraping. With Subxenum, you can brute-force subdomains using a wordlist
Teledown - Telegram image downloader
Teledown Teledown is a python script that automate downloading pictures from multiple telegram channels at once. Prerequisites Before your can use Tel
Code examples of Web Scraping with Python. Includes usage of BeautifulSoup4
What Is Web Scraping? The automated gathering of data from the internet is nearly as old as the internet itself. Although web scraping is not a new te
Three different ways to get the weather with python
Weather-Python How to get weather data through Python in different ways. In this repository there are 3 folders that contain files about 3 different w
Telegram code for scraping members and adding them
This is a code for scraping members of a telegram group and adding them into another group (NOTE: please do not use this code to spam people or mass-add people, this code does not supercede Telegram's laws so please use it carefully. We are also not responsible for any illegal activity using this code.)
Python library for scraping google search.
rapisearch rapisearch is a python library for scraping google search Installation Download as a zip file or clone this repository git clone https://gi
API for scraping football players information from "Football Transfers"
api-fps API for scraping football players information from "Football Transfers" Features easy to use useful fast filterable API Reference Default Rout
Scraping F1 data from official website. Seasonal data of drivers and teams or specific racing data of drivers and teams. Now you can search drivers by name.
f1DataScraper Scraping F1 data from official website. Seasonal data of drivers and teams. Racing data of drivers and teams. Check out sample images fo
Jupyter Notebooks demonstrating scraping data from social media sites. Written in Python.
Data Scraper Experiments This repo demonstrates the use of SNScrape to scrape data from the Social Media sites like Twitter, Facebook and Instagram. O
Amazon Web Scraper to buy products from Amazon at the best prices. 💸
AmazonWebScraper 💸 Python program that makes it possible to predefine a price from which the user should receive an email as soon as the specified pr
Scrape various open data directories to create an index of what's available out there
scrape-open-data Scrapes every available dataset from Socrata and stores them as newline-delimited JSON in this repository, to track changes over time
Freely available & most common web scraping techniques intended to bypass simple anti-scraping measures
HTTP requests spoofer Freely available & most common web scraping techniques intended to bypass simple anti-scraping measures. What is this? It's a li
This tools is used to create automatically discord threads by scraping ctfd plateform and collecting name , category , description and points of challenges.
Discord bot CTFD-Thread-Parser Description: This tools is used to create automatically discord threads by scraping ctfd plateform and collecting name
Amazon Web Scraper to buy products from Amazon at the best prices. 💸
AmazonWebScraper 💸 Python program that makes it possible to predefine a price from which the user should receive an email as soon as the specified pr
This script will download the Tesla Service Manual onto a local doc folder for offline access.
Tesla Service Manual Scraper This script will download the Tesla Service Manual onto a local doc folder for offline access. This is geared for Windows
Code relating to scraping U.S. Supreme court arguments audio and transcripts
Supreme Court Scrapers This repository contains the code responsible for scraping from the Supreme Court website Schedule https://www.supremecourt.gov
Create project web scraping to capture comments in website tripadvisor.com.br
Web Scraping Tripadvisor - Extract Comment's Users Table of Contents About • Features • Revised Concepts • Installation • Get Started 📌 About Constru
Automação Web Scraping com Selenium
Projeto de Automação Web Scraping com Selenium e Excel Temos como base uma Importadora que trabalha com Dólar, Euro e Ouro. Este código usaremos a int
Python SDK for browserlify.com, PDF generation and Web Scraping
Browserlify Python SDK The Browserlify API Python Library Usage Requirements You should be signed up as a developer on the browserlify.com so that you
A universal scraping tool to acquire CS:GO demofiles from professional esports events provided by hltv.org
GoScrape 🐙 : Universal hltv.org demofile scraper Go scrape is a little open source project I created to make it easy to bulk download demofiles for t
A REST API for film2subtitle.com website written in Python and FastAPI ⚡
Film2Subtitle API Introduction 🗣️ It is a REST API for film2subtitle.com website built with Python and FastAPI web framework. You can search and down
Raspagem de dados no site do BGG - Os top#1000 jogos e requisições nos detalhes sobre eles. Os dados obtidos foram adicionados no MySQL e exportados em csv.
BoardGameGeek (BGG) Web Scraping. Sumário Habilidades Desenvolvido Estrutura arquivos pastas Ambiente Virtual Raspagem Contatos Habilidades Utilizar o
Bot desenvolvido para participar do processo seletivo da QIPU.
Bot desenvolvido para Participar do processo seletivo da QIPU. Para utilizar o bot siga os seguintes passos: 01 - Primeiro crie um ambiente virtual, e
Fast TikTok NO Watermark Video Downloader (username or url)
💎 TD [ TikDown v3 ] Star ⭐ if you want more New * Download by profile (@xxx) * Faster To come * Download by # Features * Download TikTok vide
Monitorando o IX através do Zabbix usando Web Scraping em Python
Monitorando o IX através do Zabbix usando Web Scraping em Python
Benson turns a list of URLs into mp3s of the contents of each web page - take control over your reading backlog!
Benson.py turns URLs into mp3s of the web pages' content — take charge of your reading backlog! Report Bug · Request Feature Table of Contents About T
Command-line Utility for Web Scraping
SCR Command-line Utility for Web Scraping Core Features Matches web content using XPath-, Regex- and Python Format Expressions Crawls through complex
This is a Web Scraping course group project, we scraped useful events for chain hotels and stored them in MongoDB database
Brief Intro This is a coursework group project, our team leveraged ticketmaster API, positionstack API and eventbrite website to scrape useful events
DergiPark - Web Scraping Project
DergiPark Project Description DergiPark is one of the biggest websites that provides electronic hosting for academic peer-reviewed articles in Turkey.