Use Groq.com as llm
Project description
GROQ without API
This projects uses playwright to Access GROQ using python
Inspiration(Story)
On a long summer evening of april 2023 I was sitting in front of cranky old fan working on my News-app. I needed a llm api to summarize the news articles. As a poor man I do not have money to buy llm subscription, I chose to use Gemini (limit 1 req/sec), it took avg 8-9 sec per request to process, felt dissapointed.
At this point I have heard about Groq and its new LPU hardware that outputs insanely fast. There wasn't any free api for poor people like me. So I decided to emulate the user using playwright and can query and get the llm response. Finally as the thousands of seconds passed I could manage to make it work. Now you can too.
I did it for poor people.
Working
- It emulates user using playwright and queries the selected model and ouputs the response as a json object.
- It uses cookie to login.(For the first time you have to login in under 120 seconds(browser will be open for 2 minutes), your cookies will be saved and will be used again next time you query)
- This code doesn't share your cookie or any kind of data with anyone.
- One way query.
Installation
pip install groq-on
make sure you have playwright installed, if not do this
pip install playwright
#install firefox
playwright install firefox
Usage
CLI
groq "how old is sun" "how to fuck around and find out"\
--model llama3-70b\
--cookie_file ./groq_cookie.json\
--headless\
--save_output\
--output_dir ./output/\
Code
help
groq-on --help
positional arguments:
queries one or alot of quoted string like \
'what is the purpose of life?'\
'why everyone hates meg griffin?'
options:
-h, --help show this help message and exit
--model MODEL Available models are gemma-7b llama2-70b llama3-70b llama3-8b mixtral-8x7b
--cookie_file COOKIE_FILE
--headless set true to not see the browser
--save_output set true to save the groq output with its query name.json
--output_dir OUTPUT_DIR
Path to save the output file. Defaults to current working directory.
TODO
- Keep updated
- Add something
- Use Better parser
- Use color for better Visual
- Multiround chat
Contribution
Feel free to add features and keep it maintained and do pull requests.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.