Python TrendHunter API

Overview

The TrendHunter API is a command line interface for scraping information from the TrendHunter Ideas database.

Installation

Optional Create a dedicated directory, create an isolated virtual environment, and activate the environment.

mkdir trendhunter && cd "$_"
python3 -m venv venv
source venv/bin/activate

Using pip, install the trendhunter library from the Python Package Index,

pip install trendhunter

Contributing

Before contributing, please install the test and dev versions of the library,

pip install trendhunter[dev,test]

If you contribute and add tests to your contributions, you can run ./bin/test from the root project directory. The script runs pytest using the /tests directory as the root. It is not currently configured to accept additional pytest options.

Once the package is ready to be uploaded, please complete the following in order,

• Increment the version template in pyproject.toml
• Set the PYPI_TOKEN environment variable
• Run /bin/twine

Usage

Before using the API, ensure you are in the correct directory and have activated your virtual environment. You can find your created trendhunter directory with,

find ~ -type d -name trendhunter

Copy the correct location and use it to navigate and start your virtual environment. Using /jbal/trendhunter as my directory,

cd /jbal/trendhunter && source venv/bin/activate

Upgrade your environment to hold the latest version of trendhunter, as it may have been updated recently,

pip install trendhunter --upgrade

The API is implemented through Python Click, a command line interface. The API implements 5 subcommands,

• assortment
• trends
• lists
• categories
• search

The trends, lists, categories, and searchcommands are very similar, because they accept a variety of optional arguments, and one required argument. The required argument is the name of the TrendHunter site from which you want to base your query. This argument is called theuid`, and must come after the subcommand.

trendhunter trends [OPTIONS] UID

The assortment subcommand is unique, in that it does not accept a uid argument. Instead, it requires at least one -i, --item optional argument. The assortment continues to accept a variety of other optional arguments, with a couple exceptions detailed below.

trendhunter assortment [OPTIONS]

The following excerpt describes every optional argument that can be passed to the API via the console,

  -b, --best                      Specify that the API should use the 'best'
                                  searching algorithm. By default the API is
                                  configured to query the default results
                                  page.
  -c, --concurrency INTEGER       "Number of concurrent requests. The API
                                  default is to send 5 "concurrent requests,
                                  but can be increased to 100. You may want to
                                  "limit concurrency to avoid 429 errors on
                                  the TrendHunter API.  [default: 5]
  -f, --format [0|1]              The output format. The API default (0) is to
                                  format the output to the console. If the
                                  user would prefer to output the details to a
                                  PowerPoint file, the 1 value can be used.
                                  [default: 0]
  -i, --item <TEXT CHOICE INTEGER INTEGER>...
                                  The assortment feature. Provides the ability
                                  to query trends, lists, categories, and
                                  search in one command. The entire assortment
                                  is specialized to ignore duplicates. Each
                                  item must be specified as a tuple of uid,
                                  type of article, n (number of articles), and
                                  m (chunk size for processing). The order is
                                  important, and must be (uid, type, n, m).
                                  The type of article must be an integer from
                                  1 through 4 inclusive,
                                  
                                  [1] trends
                                  [2] lists
                                  [3] categories
                                  [4] search
                                  [required]
  -l, --loglevel [10|20|30|40|50]
                                  The log level of the root Python logger. The
                                  API default is to log anything at or above
                                  the INFO level. Decrease the value to view
                                  more verbose logs.  [default: 20]
  -m INTEGER                      Size of a simultaneously-processed article
                                  chunk. The API default is to process 100
                                  articles at one time. Decrease this value to
                                  reduce memory usage.  [default: 100]
  -n INTEGER                      Number of articles. The API default is to
                                  return 50 articles matching the provided
                                  uid, but the `n` option is used to customize
                                  this value.  [default: 50]
  -p, --path PATH                 The path to write any output files. If one
                                  is not passed, the output path will be the
                                  current path.
  -s, --size <INTEGER INTEGER>...
                                  The maximum resolution of any created image
                                  files. The API default is to limit a
                                  thumbnail to a dimension of (300, 300). If
                                  an image is not equal in width and height
                                  dimension, the increase in resolution will
                                  be halted when the aspect ratio forces the
                                  larger dimension to hit the boundary
                                  specified here.  [default: 800, 800]
  -t, --timeout INTEGER           Number of seconds until a request times out.
                                  The API default is to allow 10 seconds for a
                                  request to complete. If you are receiving
                                  several timeout exceptions, try to increaase
                                  this value.  [default: 10]
  -y, --proxy TEXT                The HTTP url of a proxy server. The API
                                  default is to not use a proxy server, but if
                                  the TrendHunter API bans your IP address,
                                  you can provide one here. Please try to use
                                  a VPN before resorting to using a proxy
                                  server. If you do need to use a proxy,
                                  please be aware of the considerable risk if
                                  the provider is not secure.
  -r, --rate FLOAT                Rate is used to limit the API request
                                  frequency. Plug in a positive integer or
                                  float to represent rate / second. This will apply
                                  to every outbound request.
  --help                          Show this message and exit.

Note -b, --best is only recognized when using the assortment, categories, or search subcommand.

Note -i, --item is only recognized when using the assortment subcommand. Further, the -m and -n options are not recognized for the assortment subcommand, because they are enveloped in the -i, --item option.

At any point, if you get stuck and would like to reference the required and optional arguments, you can view the Click help page for either the base trendhunter command or one of the subcommands,

trendhunter --help

trendhunter trends --help

Examples

trendhunter trends -n 5 -f 1 -p ../test/ holiday-giveaways

trendhunter lists 2023-tech-trends

trendhunter categories -l 10 -f 1 -n 200 -m 3 food

trendhunter search candy