Python package, scraping recipes from all over the internet
Project description
A simple web scraping tool for recipe sites.
pip install recipe-scrapers
then:
from recipe_scrapers import scrape_me # give the url as a string, it can be url from any site listed below scraper = scrape_me('https://www.allrecipes.com/recipe/158968/spinach-and-feta-turkey-burgers/') # Q: What if the recipe site I want to extract information from is not listed below? # A: You can give it a try with the wild_mode option! If there is Schema/Recipe available it will work just fine. scraper = scrape_me('https://www.feastingathome.com/tomato-risotto/', wild_mode=True) scraper.title() scraper.total_time() scraper.yields() scraper.ingredients() scraper.instructions() scraper.image() scraper.host() scraper.links() scraper.nutrients() # if available
Note: scraper.links() returns a list of dictionaries containing all of the <a> tag attributes. The attribute names are the dictionary keys.
Scrapers available for:
- https://www.750g.com
- https://claudia.abril.com.br/
- https://www.acouplecooks.com
- https://allrecipes.com/
- https://amazingribs.com/
- https://ambitiouskitchen.com/
- https://archanaskitchen.com/
- https://www.atelierdeschefs.fr/
- https://averiecooks.com/
- https://bakingmischief.com/
- https://bbc.com/
- https://bbc.co.uk/
- https://bbcgoodfood.com/
- https://bettycrocker.com/
- https://blueapron.com/
- https://bonappetit.com/
- https://bowlofdelicious.com/
- https://budgetbytes.com/
- https://cdkitchen.com/
- https://chefkoch.de/
- https://closetcooking.com/
- https://cookeatshare.com/
- https://cookieandkate.com/
- https://cookpad.com/
- https://cookstr.com/
- https://copykat.com/
- https://countryliving.com/
- https://cuisineaz.com/
- https://cybercook.com.br/
- https://delish.com/
- https://domesticate-me.com/
- https://downshiftology.com/
- https://www.dr.dk/
- https://www.eatingbirdfood.com/
- https://eatsmarter.com/
- https://eatsmarter.de/
- https://eatwhattonight.com/
- https://epicurious.com/
- https://recipes.farmhousedelivery.com/
- https://fifteenspatulas.com/
- https://finedininglovers.com/
- https://fitmencook.com/
- https://food.com/
- https://foodandwine.com/
- https://foodnetwork.com/
- https://foodrepublic.com/
- https://geniuskitchen.com/
- https://giallozafferano.it/
- https://gimmesomeoven.com/
- https://recietas.globo.com/
- https://gonnawantseconds.com/
- https://gousto.co.uk/
- https://greatbritishchefs.com/
- https://halfbakedharvest.com/
- https://www.hassanchef.com/
- https://www.heb.com/
- https://heinzbrasil.com.br/
- https://hellofresh.com/
- https://hellofresh.co.uk/
- https://www.hellofresh.de/
- https://hostthetoast.com/
- https://101cookbooks.com/
- https://receitas.ig.com.br/
- https://inspiralized.com/
- https://jamieoliver.com/
- https://justbento.com/
- https://kennymcgovern.com/
- https://www.kingarthurbaking.com
- https://kochbar.de/
- https://kuchnia-domowa.pl/
- https://littlespicejar.com/
- http://livelytable.com/
- https://lovingitvegan.com/
- https://lecremedelacrumb.com/
- https://marmiton.org/
- https://matprat.no/
- https://www.melskitchencafe.com/
- http://mindmegette.hu/
- https://minimalistbaker.com/
- https://misya.info/
- https://momswithcrockpots.com/
- http://motherthyme.com/
- https://mybakingaddiction.com/
- https://myrecipes.com/
- https://healthyeating.nhlbi.nih.gov/
- https://nourishedbynutrition.com/
- https://nutritionbynathalie.com/blog
- https://cooking.nytimes.com/
- https://ohsheglows.com/
- https://www.paleorunningmomma.com/
- https://www.panelinha.com.br/
- https://paninihappy.com/
- https://popsugar.com/
- https://przepisy.pl/
- https://purelypope.com/
- https://purplecarrot.com/
- https://rachlmansfield.com/
- https://realsimple.com/
- https://recipietineats.com/
- https://seriouseats.com/
- https://simplyquinoa.com/
- https://simplyrecipes.com/
- https://simplywhisked.com/
- https://skinnytaste.com/
- https://southernliving.com/
- https://spendwithpennies.com/
- https://www.thespruceeats.com/
- https://steamykitchen.com/
- https://streetkitchen.hu/
- https://sunbasket.com/
- https://sweetpeasandsaffron.com/
- https://tastesoflizzyt.com
- https://tasteofhome.com
- https://tasty.co
- https://tastykitchen.com/
- https://thehappyfoodie.co.uk/
- https://thekitchn.com/
- https://thenutritiouskitchen.co/
- https://thepioneerwoman.com/
- https://thespruceeats.com/
- https://thevintagemixer.com/
- https://thewoksoflife.com/
- https://tine.no/
- https://tudogostoso.com.br/
- https://twopeasandtheirpod.com/
- https://vanillaandbean.com/
- https://vegolosi.it/
- https://watchwhatueat.com/
- https://whatsgabycooking.com/
- https://en.wikibooks.org/
- https://yummly.com/
Contribute
Part of the reason I want this open sourced is because if a site makes a design change, the scraper for it should be modified.
If you spot a design change (or something else) that makes the scraper unable to work for a given site - please fire an issue asap.
If you are programmer PRs with fixes are warmly welcomed and acknowledged with a virtual beer.
If you want a scraper for a new site added
Open an Issue providing us the site name, as well as a recipe link from it.
You are a developer and want to code the scraper on your own:
If Schema is available on the site - you can do this
Otherwise, scrape the HTML - like this
Generating a new scraper class:
python generate.py <ClassName> <URL>
- ClassName: The name of the new scraper class.
- URL: The URL of an example recipe from the target site. The content will be stored in test_data to be used with the test class.
For Devs / Contribute
Assuming you have python3 installed, navigate to the directory where you want this project to live in and drop these lines
git clone git@github.com:hhursev/recipe-scrapers.git && cd recipe-scrapers && python3 -m venv .venv && source .venv/bin/activate && pip install -r requirements-dev.txt && pre-commit install && python -m coverage run -m unittest && python -m coverage report
In case you want to run a single unittest for a newly developed scraper
python -m coverage run -m unittest tests.test_myscraper
FAQ
- How do I know if a website has a Recipe Schema? Run in python shell:
from recipe_scrapers import scrape_me scraper = scrape_me('<url of a recipe from the site>', wild_mode=True) # if no error is raised - there's schema available: scraper.title() scraper.instructions() # etc.
Special thanks to:
All the contributors that helped improving the package. You are awesome!
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
| Filename, size | File type | Python version | Upload date | Hashes |
|---|---|---|---|---|
| Filename, size recipe_scrapers-12.0.4-py3-none-any.whl (255.0 kB) | File type Wheel | Python version py3 | Upload date | Hashes View |
| Filename, size recipe_scrapers-12.0.4.tar.gz (41.9 kB) | File type Source | Python version None | Upload date | Hashes View |
Close
Hashes for recipe_scrapers-12.0.4-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 9da54a34704ff3f7a2c2571d19c07f6a45f45d4cf9d3aa1f1577924161e4fd15 |
|
| MD5 | 07f380114fd9a9c092fbe5a754fe0f1c |
|
| BLAKE2-256 | 449273e5743755f1f71a2988f3cdfb2240974a6ae989d36714128726cec6b532 |
