Python idiomatic SDK for Cortex™ Data Lake.
Project description
Palo Alto Networks Cortex™ Data Lake SDK
Python idiomatic SDK for the Cortex™ Data Lake.
The Palo Alto Networks Cortex Data Lake Python SDK was created to assist developers with programmatically interacting with the Palo Alto Networks Cortex™ Data Lake API.
The primary goal is to provide full, low-level API coverage for the following Cortex™ Data Lake services:
- Query Service
The secondary goal is to provide coverage, in the form of helpers, for common tasks/operations.
-
Log/event pagination
-
OAuth 2.0 and token refreshing
-
More, coming soon!
-
Documentation: https://cortex.pan.dev
-
Free software: ISC license
Features
- HTTP client wrapper for the popular Requests library with full access to its features.
- Language bindings for Query Service.
- Helper methods for performing common tasks, such as log/event pagination.
- Support for OAuth 2.0 grant code authorization flow.
- Library of example scripts illustrating how to leverage the SDK.
- Support for API Explorer Developer Tokens for easier access to API!
Status
The Palo Alto Networks Cortex™ Data Lake Python SDK is considered alpha at this time.
Installation
From PyPI:
$ pip install pan-cortex-data-lake
Obtaining and Using OAuth 2.0 Tokens
If you're an app developer, work with your Developer Relations representative to obtain your OAuth2 credentials. API Explorer may optionally be used to generate a Developer Token, which can also be used to authenticate with the API. For details on API Explorer developer tokens, please visit https://cortex.pan.dev/docs/learn/developer_tokens.
Example
>>> from pan_cortex_data_lake import Credentials, QueryService
>>> c = Credentials()
>>> qs = QueryService(credentials=c)
>>> query_params = {
... "query": "SELECT * FROM `1234567890.firewall.traffic` LIMIT 1",
... }
>>> q = qs.create_query(query_params=query_params)
>>> q.status_code
201
>>> q.json()
{'jobId': '40fedce6-ddf5-44cf-9af2-7c3d5303f388', 'uri': '/query/v2/jobs/40fedce6-ddf5-44cf-9af2-7c3d5303f388'}
>>> results = qs.get_job_results(job_id='40fedce6-ddf5-44cf-9af2-7c3d5303f388')
>>> results.json()
Contributors
History
2.0.0-alpha9 (2020-04-04)
- Now checking if static
access_token
exists before attempting a developer token refresh (#147) - Reduced the number of times
json()
property is called inQueryService
andHTTPClient
(#148) - Updated
clientType
andclientVersion
increate_query()
method (#149) - Updated
iter_job_results()
to usepageCursor
instead of deprecatedscrollToken
(#145) - Switched to using
rowsInPage
instead ofrowsInJob
when updatingstats.records
inget_job_results()
(#146)
2.0.0-alpha8 (2020-03-25)
- Updated docstrings
- Bug fix for issue #143
2.0.0-alpha7 (2020-03-23)
- Updated SDK name in HTTPClient default headers
- Added support for specifying Developer Token provider using envar or kwarg
- Updated
credentials.json
parent dir name
2.0.0-alpha6 (2020-03-04)
- Changed
iter_job_results()
method to allow overridingenforce_json
- Updated default API gateway URL
- Minor corrections to docstrings
- Removeed suppression of urllib3 warnings
- Updated example scripts
- Updated README
- Reformatted/refactoring with black
- Added
transactions
attribute toApiStats
with default value - Reimplemented module-level logging
- Applied change to
HTTPClient
to avoid applying header credentials twice when no method credentials are present
2.0.0-alpha5 (2020-03-03)
- Fixed
Credentials
storage adapter import
2.0.0-alpha4 (2020-03-03)
- Updated MANIFEST
2.0.0-alpha3 (2020-02-20)
- Renamed
pancloud
package directory topan_cortex_data_lake
- Published to PyPI under new name,
pan-cortex-data-lake
- Moved from RST to MD format for docs files
- Droped support for
pipenv
- Updated setup.cfg and setup.py
- Updated tests and test config files
2.0.0-alpha2 (2020-01-09)
- Fixed invalid use of
HTTPClient
classpath
keyword arg inCredentials
2.0.0-alpha1 (2019-11-21)
- Removed all legacy modules, e.g. logging, event and directorysync
- Removed all references to legacy modules and classes
- Removed all legacy tests
- Added util module
- Added query module
- Added example query script
- Updated httpclient module
1.5.1 (2019-04-26)
- Updated Pipfile.lock.
- Replaced recursion with loop pattern in
LoggingService
xpoll()
method. - Added
decode('utf-8')
to base64 decoded JWT to ensure compatibility withjson.loads()
. - Addressed minor typos in docs.
1.5.0 (2019-02-27)
- Added
flush()
method toEventService
class. - Added
auto_refresh
support toHTTPClient
_apply_credentials()
method. - Removed
auto_retry
feature fromHTTPClient
class. - Refactored
HTTPClient
classrequest()
method keyword argument overrides. - Removed unused
token_revoke_url
keyword argument. - Added support for API Explorer Developer Tokens to
Credentials
class. - Refactored
Credentials
refresh()
method.
1.4.0 (2018-10-04)
- Added default URL to
HTTPClient
class. - Updated docstrings for
StorageAdapter
,TinyDBStore
andCredentials
classes. - Now returning
state
asstr
instead ofUUID
inget_authorization_url()
method. - Now unifying display results for -m/-s/--write. For -m you now need an output specifier (-j/-p) to print the response.
- Added
decode_jwt_payload()
method to allow for extracting/using all JWT fields. - Added -s option to allow for invocation of setter methods. This allows modifying of credential store fields.
- Added credential setters to allow for modifying credentials.
- Updated examples.
- Switched from using
requests
toHTTPClient
inCredentials
class. - Now checking JWT access_token
exp
to determine if refresh if needed. - Now generating a new
state
each timeget_authorization_url()
is called. - Added
__repr__
toCredentials
class with support for masking secrets. - Updated -E --ack,nack,poll options usage to be accurate.
JOB_FAILED
response inxpoll()
queryStatus now includes errorCode.
1.3.0 (2018-08-04)
- Added support for custom read/write credentials path.
TinyDBStore
fetch_credential()
now returnsNone
instead of emptystr
.- Fixed bug that caused
_resolve_credential()
to be executed an inefficient number of times. - Now updating
HTTPClient
headers instead of overriding them which previously broke HTTP persistence. - Added JMESPath
isotime()
function tosummit.py
which prints epoch. - Added argument to
EventService
xpoll()
method to support sleeping between polls. - Added
PAN_
prefix to envars to avoid name collisions. - Fixed bug when
R['R1_obj']['LoggingService.query']
is None and allow json=None case with special case of --end -1 which will not set a default end of now. - Fixed bug that nulled out credentials if an error occurred during a
fetch_tokens()
orrefresh()
operation. - Now enforcing strict credential resolution. Previous behavior allowed for merging credentials from different providers.
- Added support for caching
refresh_token
to support rolling. - Added support for writing logs to
summit.py
. - Switch from using
data
param tojson
param in client/service methods. - Now defaulting
R1
toNone
so don't send body unless specified. - Now printing request headers and body at debug level 3 in
summit.py
. - Added enhancements to
summit.py
for specifyingstartTime
andendTime
.
1.2.3 (2018-06-21)
- Reversed the
access_token
lookup order in get_credentials() method. - Added
_resolve_credential()
toaccess_token()
property method to support token caching. - Now comparing passed
access_token
inrefresh()
method to value returned by property method.
1.2.2 (2018-06-20)
- Added pancloud.adapters to find_packages include list to resolve issue with PyPI package.
1.2.1 (2018-06-20)
- Added requirements.txt to MANIFEST.in to address build issues with PyPI package.
- Fixed issue in README.rst that prevented PyPI from properly rendering.
1.2.0 (2018-06-20)
- Updated README.rst
- Updated RTD API Reference.
- Added
pancloud
to requirements_dev.txt - Added docstrings to
Credentials
property methods. - Changed logging xpoll() to return entire log entry instead of just
_source
dictionary. - Added
Credentials
Storage Adapter feature and movedTinyDB
code totinydb_adapter.py
, the default storage adapter. - Automatically carry
queryId
fromlogging --query
response to--id
in subsequent--poll
,--xpoll
and--delete
insummit.py
. - Various bug fixes and improvements to
summit.py
. - Added support for caching
access_token
in credentials store. - Added
write()
method toLoggingService
class to support writing logs. - Fixed issues with
Credentials
get_authorization_url
andfetch_tokens
methods. - Added
logging_write.py
to examples.
1.1.0 (2018-05-08)
- Updated logging
xpoll()
to reflect behavior of current API. - Added
credentials.py
module to support OAuth2 operations. - Added
auto\_refresh
andauto_retry
support toHTTPClient()
. - Added
PartialCredentialsError
exception to handle cases where incomplete credentials are passed toCredentials
. - Removed extraneous dependencies, e.g. pyopenssl.
- Cleaned up requirements.txt and PipLock files.
- Added
TinyDB
package for reading/writingcredentials.json
file. - Now raising
PanCloudError
forenforce_json
errors. - Removed logging
poll_all()
method and example script. - Now checking if
sequence_no
isNone
before incrementing in loggingiter_poll()
method. - Added event
xpoll()
method to support iterating through events. - Introducing
summit.py
, a command-line interface forpancloud
. - Added
credentials
support tosummit.py
to enable writingcredentials.json
file and passingCredentials
with requests. - Updated example scripts library to incorporate new credentials feature.
1.0.3 (2018-04-26)
- Overhauled and updated RTDs
- Updated
__init__.py
to allow package-level imports - Fixed issue where starrifying authorization token in
__repr__
overrode value in request header.
1.0.2 (2018-03-20)
- Updated .gitignore to exclude .pytest_cache
- Excluding certain functions/classes from tests until API Gateway is ready
- Renamed tests modules to all lowercase
- Bumped version to 1.0.2 from 1.0.1
- Updated 'query' and 'poll' endpoints to reflect current working state of APIs
- Fixed .travis.yml branch regex
- Updated example scripts
- Removed pyopenssl property method from HTTPClient
- Removed test_repr() as not all tested python versions support ordered kwargs
1.0.1 (2018-03-19)
- Default to empty
path
1.0.0 (2018-03-16)
- First release.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pan-cortex-data-lake-2.0.0a9.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3a3ebaa87dab274fda06bb7b548adc0cdf7efbc7bebc0b545393727a0ab2c890 |
|
MD5 | 31f1f04513348f2a25f5efc6311f718a |
|
BLAKE2b-256 | f9c302dd316b5e2061e1fba724ce0c9db5f16ce25d9519d71f8fc4accb3db720 |
Hashes for pan_cortex_data_lake-2.0.0a9-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bd5aa73356465c5f5fa440239e9edbd072c5c3cbd3863feb3576409b80e9e16b |
|
MD5 | 3df6477f050537daf332a2a89f87ffe7 |
|
BLAKE2b-256 | c010efcab7ded6bff47a3b87695218097b4a21575cdfd6a0e7a5a88633147973 |