webrecon 0.0.2

Get information from various web resources

Web Recon

Source Code This package is designed to help users search the interwebz for pretty much anything (legal). You can use this package, which wraps other cloud packages and SDKs for ease of use and integration.

Goals of this project

• Create a single source code repository that can handle searching
• Allow other projects to use the search with minimal integration and code changes
  • Serverless frameworks like AWS Lambda, Google Cloud Functions, and Azure Automations. If the platform allows python with dependencies, webrecon should be able to run there.
• Container clusters like ECS, EKS, Kubernetes in general, etc. All the Dockers, basically.

Use

Searching with Google CSE

To use Google Custom Search Engine you will want to use the gcse.search() function. You can include the library in your source code by importing webrecon.gcse. See the docstring for up to date notes on params etc. You can send in any of the parameters that the GCSE API from python offers up and a couple more too. The updated list of params from the Alphabet-Googlez folks can be found here and the current supported list of params for the search from webrecon.gsce.search()

• q : This parameter is the actual search term. It can be helpful to use Google search operators.
• filters <[String]>: Send a list of keys that might be found in the results from the search so that you can filter the results you deal with in your actual final result set.
• key : The developer key for Google. You can find this in your GCSE console (manually log into the browser, amigoritos).
• cx : The GCSE ID. Same story as the key parameter.
• kwargs : You can pass any number of arguments in as key value pairs, at the end of your parameters list. These arguments will be dutifully passed along to the actual Google custom search method.

Example

from webrecon import gcse
...
res = gcse.search('intext:HUGO BOSS Genesis 2 Virgin Wool Dress Pants',
				  'title',
				  'link',
				  num=3,
				  exactTerm='Genesis 2')

print(res)
[
	{
		'title': 'Hugo Boss Pants',
		'link': 'https://www.macys.com/shop/b/hugo-boss-pants?id=78110'
	},
	{
		'title': 'HUGO BOSS 100% Wool Pants for Men for sale | eBay',
		'link': 'https://www.ebay.com/b/HUGO-BOSS-100-Wool-Pants-for-Men/57989/bn_4243898'
	},
	{
		'title': "HUGO BOSS | Trousers for Men | Elegant and Casual Men's Trousers",
		'link': 'https://www.hugoboss.com/us/men-pants/'
	}
]