recipe-scrapers 13.3.0

Python package, scraping recipes from all over the internet

---

A simple web scraping tool for recipe sites.

pip install recipe-scrapers

then:

from recipe_scrapers import scrape_me

# give the url as a string, it can be url from any site listed below
scraper = scrape_me('https://www.allrecipes.com/recipe/158968/spinach-and-feta-turkey-burgers/')

# Q: What if the recipe site I want to extract information from is not listed below?
# A: You can give it a try with the wild_mode option! If there is Schema/Recipe available it will work just fine.
scraper = scrape_me('https://www.feastingathome.com/tomato-risotto/', wild_mode=True)

scraper.title()
scraper.total_time()
scraper.yields()
scraper.ingredients()
scraper.instructions()
scraper.image()
scraper.host()
scraper.links()
scraper.nutrients()  # if available

Notes:

• Starting from v13.0.0 the packaged stopped suppressing scraper exceptions by default. If you want the previous behaviour

import os
from recipe_scrapers import scrape_me

os.environ["RECIPE_SCRAPERS_SETTINGS"] = "recipe_scrapers.settings.v12_settings"

scraper = scrape_me(...)  # etc.

• scraper.links() returns a list of dictionaries containing all of the <a> tag attributes. The attribute names are the dictionary keys.

Scrapers available for:

• https://www.acouplecooks.com

• https://claudia.abril.com.br/

• http://www.afghankitchenrecipes.com/

• https://allrecipes.com/

• https://alltommat.se/

• https://amazingribs.com/

• https://ambitiouskitchen.com/

• https://archanaskitchen.com/

• https://www.atelierdeschefs.fr/

• https://averiecooks.com/

• https://bbc.com/

• https://bbc.co.uk/

• https://bbcgoodfood.com/

• https://bakingmischief.com/

• https://bettycrocker.com/

• https://bigoven.com/

• https://blueapron.com/

• https://bonappetit.com/

• https://bowlofdelicious.com/

• https://budgetbytes.com/

• https://castironketo.net/

• https://cdkitchen.com/

• https://chefkoch.de/

• https://closetcooking.com/

• https://cookeatshare.com/

• https://cookpad.com/

• https://cookieandkate.com/

• https://cookinglight.com/

• https://cookstr.com/

• https://copykat.com/

• https://countryliving.com/

• https://cucchiaio.it/

• https://cuisineaz.com/

• https://cybercook.com.br/

• https://delish.com/

• https://domesticate-me.com/

• https://downshiftology.com/

• https://www.dr.dk/

• https://eatwhattonight.com/

• https://www.eatingbirdfood.com/

• https://eatsmarter.com/

• https://eatsmarter.de/

• https://epicurious.com/

• https://recipes.farmhousedelivery.com/

• https://fifteenspatulas.com/

• https://finedininglovers.com/

• https://fitmencook.com/

• https://food.com/

• https://food52.com/

• https://foodandwine.com/

• https://foodnetwork.com/

• https://foodrepublic.com/

• https://www.forksoverknives.com/

• https://www.750g.com

• https://geniuskitchen.com/

• https://giallozafferano.it/

• https://gimmesomeoven.com/

• https://recietas.globo.com/

• https://gonnawantseconds.com/

• https://gousto.co.uk/

• https://greatbritishchefs.com/

• https://www.heb.com/

• https://halfbakedharvest.com/

• https://www.hassanchef.com/

• https://heinzbrasil.com.br/

• https://hellofresh.com/

• https://hellofresh.co.uk/

• https://www.hellofresh.de/

• https://www.hellofresh.fr/

• https://hostthetoast.com/

• https://101cookbooks.com/

• https://receitas.ig.com.br/

• https://indianhealthyrecipes.com

• https://www.innit.com/

• https://inspiralized.com/

• https://jamieoliver.com/

• https://justbento.com/

• https://kennymcgovern.com/

• https://www.kingarthurbaking.com

• https://kochbar.de/

• https://kuchnia-domowa.pl/

• https://lecremedelacrumb.com/

• https://littlespicejar.com/

• http://livelytable.com/

• https://lovingitvegan.com/

• https://marmiton.org/

• https://www.marthastewart.com/

• https://matprat.no/

• https://www.melskitchencafe.com/

• http://mindmegette.hu/

• https://minimalistbaker.com/

• https://misya.info/

• https://momswithcrockpots.com/

• http://motherthyme.com/

• https://mybakingaddiction.com/

• https://www.myplate.gov/

• https://myrecipes.com/

• https://healthyeating.nhlbi.nih.gov/

• https://cooking.nytimes.com/

• https://nourishedbynutrition.com/

• https://nutritionbynathalie.com/blog

• https://ohsheglows.com/

• https://101cookbooks.com/

• https://www.paleorunningmomma.com/

• https://www.panelinha.com.br/

• https://paninihappy.com/

• https://popsugar.com/

• https://przepisy.pl/

• https://purelypope.com/

• https://purplecarrot.com/

• https://rachlmansfield.com/

• https://realsimple.com/

• https://recipietineats.com/

• https://redhousespice.com/

• https://reishunger.de/

• https://sallysbakingaddiction.com

• https://www.saveur.com/

• https://seriouseats.com/

• https://simplyquinoa.com/

• https://simplyrecipes.com/

• https://simplywhisked.com/

• https://skinnytaste.com/

• https://southernliving.com/

• https://spendwithpennies.com/

• https://steamykitchen.com/

• https://streetkitchen.hu/

• https://sunbasket.com/

• https://sweetcsdesigns.com/

• https://sweetpeasandsaffron.com/

• https://tasteofhome.com

• https://tastesoflizzyt.com

• https://tasty.co

• https://tastykitchen.com/

• https://theclevercarrot.com/

• https://thehappyfoodie.co.uk/

• https://thekitchn.com/

• https://thenutritiouskitchen.co/

• https://thepioneerwoman.com/

• https://thespruceeats.com/

• https://thevintagemixer.com/

• https://thewoksoflife.com/

• https://timesofindia.com/

• https://tine.no/

• https://tudogostoso.com.br/

• https://twopeasandtheirpod.com/

• https://vanillaandbean.com/

• https://vegrecipesofindia.com/

• https://vegolosi.it/

• https://watchwhatueat.com/

• https://whatsgabycooking.com/

• https://www.wholefoodsmarket.com/

• https://www.wholefoodsmarket.co.uk/

• https://en.wikibooks.org/

• https://yemek.com/

• https://yummly.com/

Contribute

Part of the reason I want this open sourced is because if a site makes a design change, the scraper for it should be modified.

If you spot a design change (or something else) that makes the scraper unable to work for a given site - please fire an issue asap.

If you are programmer PRs with fixes are warmly welcomed and acknowledged with a virtual beer.

If you want a scraper for a new site added

• Open an Issue providing us the site name, as well as a recipe link from it.

• You are a developer and want to code the scraper on your own:

  • If Schema is available on the site - you can do this

    • How do I know if a schema is available on my site?

  • Otherwise, scrape the HTML - like this

  • Generating a new scraper class:

    python generate.py <ClassName> <URL>

    • ClassName: The name of the new scraper class.

    • URL: The URL of an example recipe from the target site. The content will be stored in test_data to be used with the test class.

For Devs / Contribute

Assuming you have python3 installed, navigate to the directory where you want this project to live in and drop these lines

git clone [e-mail]:hhursev/recipe-scrapers.git &&
cd recipe-scrapers &&
python3 -m venv .venv &&
source .venv/bin/activate &&
pip install -r requirements-dev.txt &&
pre-commit install &&
python -m coverage run -m unittest &&
python -m coverage report

In case you want to run a single unittest for a newly developed scraper

python -m coverage run -m unittest tests.test_myscraper

FAQ

• How do I know if a website has a Recipe Schema? Run in python shell:

from recipe_scrapers import scrape_me
scraper = scrape_me('<url of a recipe from the site>', wild_mode=True)
# if no error is raised - there's schema available:
scraper.title()
scraper.instructions()  # etc.

Special thanks to:

All the contributors that helped improving the package. You are awesome!