fp-NGFW-SMC-python-monitoring 1.5.3

Forcepoint NGFW Management Center Monitoring

Monitoring API for Forcepoint NGFW Security Management Center

smc-python-monitoring is made of up two core capabilities:

• Monitor, Query or view live events from the SMC

• Pub/Sub (Subscribe) to SMC element events by supported API type

The smc-python-monitoring API makes it possible to obtain real time monitoring information from the SMC with respects to:

• Logs

• Connections

• Blacklist

• Routing

• Users

• SSL VPN Users

• VPNs

• Alerts

This correlates to the area of the SMC under “Monitoring”.

Every query can be refined by using filters that allow a variety of boolean operations to control the results.

The smc-python-monitoring API also provides the ability to subscribe to element event changes in the SMC. Any element type that is exposed as an SMC API entry point can be used as a subscriber target. Event type examples that would fire events would be created, updated, or deleted (for example - although not an exhaustive list).

Compatibility

Requires Security Management Center version 6.2 or newer.

This package has been tested with Python 2.7, 3.5 and 3.6.

Requirements

fp_NGFW_SMC_python >= 1.0.0

websocket-client

Install

pip install fp-NGFW-SMC-python-monitoring

Usage

In order to leverage monitoring or subscriber features, you must first obtain a valid smc session as normal:

from smc import session
session.login()
...

Once a valid session is obtained, the session information is re-used for the web socket connection, including SSL related data such as validating SSL through the client side SSL certificate before establishing the connection.

Monitoring and Queries

Making queries is uniform across all query types. There are small exceptions to this rule with respects to LogQuery which provides more options on how to control the batched query size, custom time ranges for the query, etc. See documentation on LogQuery for more details.

A return data format for queries can be configured when calling fetch on the query. Results can be obtained in a variety of formatted outputs such as CSV, Table format or as a raw dict. You can also provide your own formatter to the query. See smc_monitoring.models.formatter for more info.

By default each query type has a pre-defined set of field_ids that define a basic set of fields for the query type. This specifies which fields are returned in the query. You can customize these fields on the query by using the query.format.field_names or query.format.field_ids methods. See the documentation for more info.

All queries can be made as each ‘batch’ queries or ‘live’ style queries. The type used depends on the method called on the query instance. All queries done via smc-python-monitoring follow the same rules as implemented in the SMC UI.

For example, ‘Monitoring’ queries (i.e. Connections, Blacklist, Users, etc) do not allow time/date ranges on the query, however filtering by fields work. The same field filtering works in the same way with Log style queries.

Example of making a basic log query in real time. Note the default return format is Table for a cleaner output, assuming this is being run from a command window:

query = LogQuery()
for log in query.fetch_live():
    print(log)

Making a simple LogQuery with a fetch size of 50 and returning in a raw dict format:

query = LogQuery(fetch_size=50)
for log in query.fetch_raw():
    print(log)

Making a more sophisticated query that uses a timezone, then adds an “AND” filter that will match if the entry has an alert severity of “HIGH” and only if the ACTION and APPLICATION fields have values:

query = LogQuery(fetch_size=50)
query.format.timezone('CST')

query.add_and_filter(
    [InFilter(FieldValue(LogField.ALERTSEVERITY), [ConstantValue(Alerts.HIGH)]),
     DefinedFilter(FieldValue(LogField.ACTION)),
     DefinedFilter(FieldValue(LogField.IPSAPPID))])

for log in query.fetch_batch(TableFormat):
    print(log)

Query for the last 10 records if the source IP is 192.168.4.84 and return only fields timestamp, source, destination and service:

query = LogQuery(fetch_size=10)
query.format.timezone('CST')

query.format.field_ids([LogField.TIMESTAMP, LogField.SRC, LogField.DST, LogField.IPSAPPID])

query.add_and_filter(
    [InFilter(FieldValue(LogField.SRC), [IPValue('192.168.4.84')]),
     DefinedFilter(FieldValue(LogField.IPSAPPID))])

for log in query.fetch_batch(TableFormat):
    ...

Examples of other monitoring type queries:

Obtain all current connections on a given engine. Output in CSV:

query = ConnectionQuery('sg_vm')
for record in query.fetch_batch(CSVFormat):
    print(record)

Obtain all authenticated users on a given engine, output as ‘User’ object instances:

query = UserQuery('sg_vm')
for record in query.fetch_as_element():
    print(record)

Obtain all VPN SA’s on given engine, output at ‘VPNSecurityAssoc’ object instances:

query = VPNSAQuery('sg_vm')
for record in query.fetch_as_element():
    print(record)

Obtain all current routes for a given engine, output as a list of raw dict items:

query = RoutingQuery('sg_vm')
for record in query.fetch_batch(RawDictFormat):
    print(record)

Obtain all SSL VPN connections for a given engine, output in table format:

query = SSLVPNQuery('sg_vm')
for record in query.fetch_batch(TableFormat):
    print(record)

Obtain all active alerts from the Shared Domain:

query = ActiveAlertQuery('Shared Domain', timezone='America/Chicago')
for record in query.fetch_batch():
print(record)

Subscribing to Events

Using smc-python-monitoring you can also subscribe to events published by the SMC API when changes are made. As long as the entry point exists for the element type, you can set up a “channel” to receive real-time updates when the element type is modified.

To listen for events you must first obtain an SMC session as usual.

Then obtain an instance of Notification, specifying the events of interest.

Subscribe to a single element event (SMC api entry point):

notification = Notification('network')

Subscribe to multiple element events on a single channel (subscription_id):

notification = Notification('network,host,iprange')

Subscribe to multiple element events, each with it’s own channel (subscription_id):

notification = Notification('network')
notification.subscribe('host')
notification.subscribe('layer2_policy')

Return the events as instance of “Event” (optional). Otherwise raw json returned.

for event in notification.notify(as_type=Event):
        print(event)

Event(subscription_id=151,action=delete,element=https://xxxx/elements/host/1087)
Event(subscription_id=152,action=delete,element=https://xxxx/elements/layer2_policy/27)

Release History

1.0.0 (2017-09-03)

• Initial release of SMC Monitoring package

• Requires SMC 6.3.0, 6.3.1

1.1.0 (2018-02-02)

• Requires SMC >= 6.3.2

• ActiveAlertQuery implemented for interfacing with Alerts (requires SMC >= 6.3.3). Currently read-only

• BlacklistQuery modified to use generic log fields for all queries except fetch_as_element

1.2

• VPN SA’s can be deleted (requires SMC >= 6.4.2)

• Updated requirement to websocket-client 0.48.0. Fixes issue with IndexError when Tomcat 8 does not include the Status-Message in the initial handshake: https://github.com/websocket-client/websocket-client/issues/406

1.3.0

• Set max_recv on queries to terminate the websocket after a specified number of receives

1.3.1

• Remove kwargs from websocket call to super to support older versions of websocket-client

• Fixed fetch_batch to only return a single payload, previous versions were unlimited

1.4.0

• Update requirements to fp_NGFW_SMC_python >= 1.0.0

1.5.0

-fixed blacklist entry issue -added first_fetch support -added support to NumberValue and TranslatedValue in QueryFilter -blacklist -> block_list renaming -support “use ssl for session id” (Fix session lost issue) -Adding asynchronious callback support (@see SMCSocketAsyncProtocol) -support certificate options in wsocket

1.5.1

TODO