Introducing LeetScrape - a powerful and efficient Python package designed to scrape problem statements and their topic and company tags, difficulty, test cases, hints, and code stubs from LeetCode.com. Easily download and save LeetCode problems to your local machine, making it convenient for offline practice and studying. It is perfect for anyone preparing for coding interviews. With the LeetScrape, you can boost your coding skills and improve your chances of landing your dream job.
Project description
LeetScrape
Introducing the LeetScrape - a powerful and efficient Python package designed to scrape problem statements and basic test cases from LeetCode.com. With this package, you can easily download and save LeetCode problems to your local machine, making it convenient for offline practice and studying. It is perfect for software engineers and students preparing for coding interviews. The package is lightweight, easy to use and can be integrated with other tools and IDEs. With the LeetScrape, you can boost your coding skills and improve your chances of landing your dream job.
Use this package to get the list of Leetcode questions, their topic and company tags, difficulty, question body (including test cases, constraints, hints), and code stubs in any of the available programming languages.
Detailed documentation available here.
Installation
Start by installing the package from pip or conda:
pip install leetscrape
# or using conda:
conda install leetscrape
# or using poetry:
poetry add leetscrape
Usage
Command Line
After installing the package, run the following command to get a code stub and a pytest test file for a given Leetcode question:
$ leetscrape --titleSlug two-sum --qid 1
At least one of the two arguments is required.
titleSlug
is the slug of the leetcode question that is in the url of the question, andqid
is the number associated with the question.
Other classes
Import the relevant classes from the package:
from leetscrape.GetQuestionsList import GetQuestionsList
from leetscrape.GetQuestionInfo import GetQuestionInfo
from leetscrape.utils import combine_list_and_info, get_all_questions_body
Scrape the list of problems
Get the list of questions, companies, topic tags, categories using the GetQuestionsList
class:
ls = GetQuestionsList()
ls.scrape() # Scrape the list of questions
ls.to_csv(directory_path="../data/") # Save the scraped tables to a directory
Get Question statement and other information
Query individual question's information such as the body, test cases, constraints, hints, code stubs, and company tags using the GetQuestionInfo
class:
# This table can be generated using the previous commnd
questions_info = pd.read_csv("../data/questions.csv")
# Scrape question body
questions_body_list = get_all_questions_body(
questions_info["titleSlug"].tolist(),
questions_info["paidOnly"].tolist(),
save_to="../data/questionBody.pickle",
)
# Save to a pandas dataframe
questions_body = pd.DataFrame(
questions_body_list
).drop(columns=["titleSlug"])
questions_body["QID"] = questions_body["QID"].astype(int)
Note The above code stub is time consuming (10+ minutes) since there are 2500+ questions.
Create a new dataframe with all the questions and their metadata and body information.
questions = combine_list_and_info(
info_df = questions_body, list_df=ls.questions, save_to="../data/all.json"
)
Upload scraped data to a Database
Create a PostgreSQL database using the SQL dump and insert data using sqlalchemy
.
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
engine = create_engine("<database_connection_string>", echo=True)
questions.to_sql(con=engine, name="questions", if_exists="append", index=False)
# Repeat the same for tables ls.topicTags, ls.categories,
# ls.companies, # ls.questionTopics, and ls.questionCategory
Use the queried_questions_list
PostgreSQL function (defined in the SQL dump) to query for questions containy query terms:
select * from queried_questions_list('<query term>');
Use the all_questions_list
PostgreSQL function (defined in the SQL dump) to query for all the questions in the database:
select * from all_questions_list();
Use the get_similar_questions
PostgreSQL function (defined in the SQL dump) to query for all questions similar to a given question:
select * from get_similar_questions(<QuestionID>);
Use the extract_solutions
method to extract solution code stubs from your python script. Note that the solution method should be a part of a class named Solution
(see here for an example):
# Returns a dict of the form {QuestionID: solutions}
solutions = extract_solutions(filename=<path_to_python_script>)
Use the upload_solutions
method to upload the extracted solution code stubs from your python script to the PosgreSQL database.
upload_solutions(engine=<sqlalchemy_engine>, row_id = <row_id_in_table>, solutions: <solutions_dict>)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for leetscrape-0.1.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cad480265c2b94061783a1e1117e2af890d70f66d0a58ac7afa1b8ed8ff52e7c |
|
MD5 | f134da1cfdc72bc29284a16b07cdd000 |
|
BLAKE2b-256 | ca7145aacc0dcdd70897638b0634cc6e9d28bb5be7801046d9d8428af035dbe8 |