Skip to main content

Distributed Network Packet Analysis Pipeline for Layer 2, 3 and 4 Frames

Project description

Python 3 AI-ready framework for recording network traffic in a data pipeline. Once recorded, you can train a deep neural network (DNN) to identify attack and non-attack traffic on your network. Included demo DNN has over 83% accuracy predicting attack vs non-attack records. Currently supports recording ethernet and arp (layer 2), ipv4, ipv6 and icmp (layer 3) and also tcp, udp frames (layer 4) frames and datagrams. Messages are auto-forwarded over to redis or rabbitmq for distributed processing in realtime. Why should I use this? This framework can help build, train and tune your own defensive machine learning models to help defend your own infrastructure at the network layer. Once the data is auto-saved as a csv file, then you can build models within Jupyter notebooks: https://github.com/jay-johnson/celery-connectors#running-jupyterhub-with-postgres-and-ssl or your ML/AI framework of choice. This pip also has an example for training a Keras Deep Neural Network model to predict attack and non-attack records using a captured and prepared dataset. There are test tools installed with this pip to quickly send mock: TCP, UDP, ARP and ICMP packets. This build currently utilizes scapy-python3 for packet recording: https://github.com/phaethon/scapy Future builds will utilize the multiprocessing engine included but does not filter src/dst ports correctly yet.The license will be full Apache 2 once that migration is done.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
network_pipeline-1.2.1-py2.py3-none-any.whl (83.3 kB) Copy SHA256 hash SHA256 Wheel 3.6 Jul 8, 2018
network-pipeline-1.2.1.tar.gz (49.4 kB) Copy SHA256 hash SHA256 Source None Jul 8, 2018

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page