10 projects
S3IncrementalProcessor
Incremental S3 File Processor is a Python package that enables efficient, incremental file processing from an Amazon S3 bucket. It keeps track of processed files using a checkpointing mechanism, ensuring that only new or modified files are processed in subsequent runs. The package supports batch processing, allows resetting checkpoints for reprocessing all data, and correctly handles edge cases like files with identical timestamps.
clickup-to-s3-migration
My coworkers and I created this free and open-source library. Anyone can now transfer entire clicks folders, tickets, and comments to AWS S3. When we switched from Clickup to JIRA software, this task was essentially done to backup the entire Clickup system. This threaded script employs the maximum number of threads to use ClickUP APIS and migrate all data into Data Lake S3 so that users or teams can use Athena to query using regular SQL if necessary.
athena-usage-metrics-extractor
Athena usage is simple python library that allows you to extract all usage information for given date range and for given workgroup
elasticsearchquerygenerator
Create Complex Elastic Search Query in Seconds Please see documentation for more details
elasticsearchmappinggenerator
Generate Elastic Search Index Mapping Quickly with this helper class see examples on how to use it
gcpcsvhunt
This package allows to download all CSV Files from GCP bucket
llogpy
just add @logMethod and @logClass to add log files for your methods
pyproxywebscrapper
This Package will be useful for people who are doing webscrapping usually when doing web scrapping if you make to many request the server will block your IP Address There is a way to trick the server using Random Ip from different country.Every Request we make we will use unique Ip and Port number check out doccumentation for more details
youtubetaggenerator
This Library allows Developers to get Youtube Tag nased on youtube title adding this tags in video will boost the views of videos
ytdvideo
A Python package to get Youtube Videos from any channel can save the data as JSON and EXCEL
