NGSI Python framework intended to build a Fiware NGSI Agent
Project description
pyngsi
What is it ?
pyngsi
is a Python framework that helps you write a pipeline for your Fiware dataflow.
Writing a NGSI agent that relies on pyngsi avoids all the plumbing so you can focus on writing your own logic to build NGSI entities.
Key Features
- NGSI v2 support
- Map Python-native data to NGSI entities
- Write NGSI entities to Fiware Orion
- Handle incoming data through a common interface
- Compute statistics
- Allow visualization/debugging facilities
Where to get it
The source code is currently hosted on GitHub at : https://github.com/pixel-ports/pyngsi
Binary installer for the latest released version is available at the Python package index.
pip install pyngsi
Getting started
Build your first NGSI entity
from pyngsi.ngsi import DataModel
m = DataModel(id="Room1", type="Room")
m.add_url("dataProvider", "https://fiware-orion.readthedocs.io/en/master/user/walkthrough_apiv2/index.html#entity-creation")
m.add("pressure", 720)
m.add("temperature", 23.0)
m.pprint()
The resulting JSON looks like this :
{
"id": "Room1",
"type": "Room",
"dataProvider": {
"value": "https://fiware-orion.readthedocs.io/en/master/user/walkthrough_apiv2/index.html#entity-creation",
"type": "URL"
},
"pressure": {
"value": 720,
"type": "Integer"
},
"temperature": {
"value": 23.0,
"type": "Float"
}
}
Send the NGSI entity to the Orion broker
from pyngsi.sink import SinkOrion
sink = SinkOrion()
sink.write(m.json())
Develop your own NGSI Agent
Let's quickly create a CSV file to store values from our room sensors
echo -e "Room1;23;720\nRoom2;21;711" > room.csv
Let's code a function that converts incoming rows to NGSI entities
def build_entity(row: Row) -> DataModel:
id, temperature, pressure = row.record.split(';')
m = DataModel(id=id, type="Room")
m.add_url("dataProvider", row.provider)
m.add("temperature", float(temperature))
m.add("pressure", int(pressure))
return m
Let's use it in in our new NGSI Agent
from pyngsi.sources.source import Source, Row
from pyngsi.sink import SinkOrion
from pyngsi.agent import NgsiAgent
src = Source.from_file("room.csv")
sink = SinkOrion()
agent = NgsiAgent.create_agent(src, sink, process=build_entity)
agent.run()
This basic example shows how the pyngsi framework is used to build a NGSI Agent.
Here data are stored on the local filesystem.
By changing just one line you could retrieve incoming data from a FTP server or HTTP server.
from pyngsi.sources.source import Source, Row
from pyngsi.sources.server import ServerHttpUpload
from pyngsi.sink import SinkOrion
from pyngsi.agent import NgsiAgent
src = ServerHttpUpload() # line updated !
sink = SinkOrion()
agent = NgsiAgent.create_agent(src, sink, process=build_entity)
agent.run()
The HTTP server is running. Now you can send the file to the endpoint.
curl -F file=@room.csv http://127.0.0.1:8880/upload
JSON and text formats are natively supported.
Many sources and sinks are provided, i.e. SinkStdout to just displays entities, eliminating the need of having an Orion server running.
One could create a custom Source to handle custom data. The MicrosoftExcelSource is given as exemple.
One could extend the framework according to his needs.
Dependencies
License
Documentation
The official documentation is hosted at https://pixel-ports.github.io/pyngsi-tutorial.html
Known Issues
SourceMicrosoftExcel may fail to open some odd Excel files due to an openpyxl bug (i.e. sometimes cannot read graphs).
In this case try to remove the offending sheet or prefer working with CSV files.
Background
Work on pyngsi
started at Orange in 2019 for the needs of the PIXEL european project.
Funding
pyngsi
has been developed as part of the PIXEL project, H2020, funded by the EC under Grant Agreement number 769355.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.