A set of tools for working with IPFS in Python: a programmer-friendly API wrapper, P2P data-transmission machinery, and accelerated connections to known peers
Project description
A library for working with IPFS in Python.
It includes a programmer-friendly wrapper called IPFS-API for the official IPFS Python API (ipfshttpclient), a module called IPFS-DataTransmission for direct P2P data transmission between two IPFS-Nodes and a module called IPFS-LNS for remembering the multiaddrs of known IPFS nodes ("contacts") to speed up connection times.
Version 0.2.X is out!
- In IPFS_DataTransmission the transmission protocol which has so far been my schoolboy-style home-made buffer-management system has been replaced with the much more efficient ZeroMQ protocol. That means greater speed and greater reliability.
- Failure management in IPFS_DataTransmission: all transmission functions, such as TransmitData(), Conversation.Start(), Conversation.Say() and TransmitFile() now return a boolean to indicate whether or not they were successful. You can also specify timeouts.
- IPFS_DataTransmission.Conversation has a Listen() method: it allows you to block the calling thread as you wait for an incoming transmission.
Unfortunately, all these great changes and simplifications have deprecated some functions, so you may need to update some of your code which uses IPFS_API. See the ChangeLog for full details.
Package Contents:
Modules:
- IPFS_API: a wrapper for the module ipfshttpclient2 that makes it easier to work with IPFS in Python
- IPFS_DataTransmission: a Python module that enables P2P data transmission between IPFS nodes
- IPFS_LNS: a Python module that allows IPFS nodes and their multiaddresses to be stored in app data to make them easier to find in the IP layer of the internet (easier to connect to)
- ipfshttpclient2: a modified version of the official ipfshttpclient module that has been expanded to include the ipfs.p2p functions
Other:
- Examples: a folder of scripts demonstrating how to work with IPFS_DataTransmission
Getting started with IPFS-Toolkit:
- Install IPFS-Toolkit (see below)
- Read this Modules Contained in IPFS-Toolkit section below to learn what this package contains and how it works.
- For IPFS-DataTransmission: Read and try out the demo scripts in the Examples folder.
Setup
Installation
pip install IPFS-Toolkit
Prerequsites
IPFS-Toolkit is made for interacting with IPFS in the Python programming language. One configuration must be applied to IPFS in order to use IPFS-Toolkit's main features.
-
Install IPFS (Desktop version or CLI, doesn't matter). https://docs.ipfs.io/install/
-
Enable "Libp2pStreamMounting" in IPFS:
Desktop Version/WebUI: on the Settings tab, scroll down to "IPFS CONFIG" and change the line that reads:
change from:
"Libp2pStreamMounting": false,
to:
"Libp2pStreamMounting": true,
Click the "Save" button and restart IPFS.CLI Version: run:
ipfs config --json Experimental.Libp2pStreamMounting true
-
Python3, version 3.6 or above https://www.python.org/downloads/
-
Pip for Python3: https://pip.pypa.io/en/stable/installing/
-
If you want to use the source code, install the following prerequisite Python modules:
pip3 install setuptools
pip3 install wheel
pip3 install multiaddr
pip3 install appdirs
pip3 install multiaddr
pip3 install appdirs
pip3 install idna
pip3 install httpcore
pip3 install httpx
pip3 install zmq
Modules Contained in IPFS-Toolkit
IPFS-API
import IPFS_API
This has a simplified and more user-friendly yet limited API for interacting with IPFS compared to the ipfshttpclient.
Usage examples:
print(IPFS_API.MyID()) # print your IPFS peer ID
cid = IPFS_API.Publish('./SomeFileOrDir') # upload file or directory to IPFS and store it's CID in a variable
# Managing IPNS keys
IPFS_API.CreateIPNS_Record('MyWebsite') # generate a pair of IPNS name keys
IPFS_API.UpdateIPNS_Record('MyWebsite', './SomeFileOrDir') # upload file to IPFS & publish
IPFS_API.UpdateIPNS_RecordFromHash('MyWebsite', cid) # publish IPFS content to IPNS key
# IPFS PubSub
IPFS_API.SubscribeToTopic("test", print) # print is the eventhandler
IPFS_API.PublishToTopic("test", "Hello there!")
HTTP API access:
This allows you to access the official IPFS API (the same as the command line ipfs utility) through ipfshttpclients' client
object. At this stage of development it is incomplete.
Examples:
import IPFS_API
IPFS_API.http_client.add("helloworld.txt")
IPFS_API.dht.findpeer("QMHash")
print(IPFS_API.swarm.peers())
IPFS-Datatransmission
import IPFS_DataTransmission
A Python module for limitless, easy, private peer-to-peer data transmission other over the IPFS Network.
This module has 3 pairs functions for use by the user:
- Simple data transmission:
TransmitData(data, peerID, listener_name)
ListenForTransmissions(listener_name,
eventhandler)
- File Transmission:
TransmitFile(data, peerID, listener_name)
ListenForFileTransmissions(listener_name, eventhandler)
- Conversations (Ping-Pong data Transmission):
StartConversation(data, peerID, listener_name)
ListenForConversations(listener_name, eventhandler)
Listener Functions:
The listener functions (ListenForTransmissions, ListenForFileTransmissions and ListenForConversations) produce Listener objects () that run on their own threads waiting for incoming data transmission or conversation requests. When another computer wants to send the first computer somthing, the other computer sends a transmission request which the Listener object receives and reacts to by creating a Transmission reception object () which runs on its own separate thread to receive the transmission, or in the case of the conversation listener, calls its user-defined eventhandler so that the conversation can be joined or ignored. This system of receiving data transmissions/conversations allows a computer to receive multiple transmission addressed to the same listener simultaneously. Multiple Listeners of the same type (the types being DataTransmission, FileTransmission, or Conversation) can be run simultaneously and independently, for example by different programs. Those different listeners (instances of ListenForTransmissions) must me named to distinguish them from each other for addressing purposes. This name is the "listener_name" parameter in the listener functions. The name is chosen by the user when creating the listener (e.g. when calling ListenForTransmission()), and must be provided when starting a transmission (e.g. when calling TransmitData()).
IPFS-LNS:
import IPFS_LNS
IPFS-LNS (IPFS Local Name System) allows you to store IPFS nodes' peer IDs and names in a database stored on your local machine. IPFS-LNS also stores a list of multiaddresses (essentially IP addresses) that the peer has been known to use, as long as you use IPFS-LNS to connect or check the connection to that peer. Every time you try to connect to the peer, IPFS-LNS tries to specifically connect to the peer using the known multiaddresses from previous connections, which can dramatically increase the speed of doing so compared to searching the whole IPFS network for the peer.
Basic usage:
import IPFS_API
# check connection to peer of peerID "QmHash" and whom we call "Bob", add him to our list of known peers ("contacts") if he is not already added
IPFS_API.CheckPeerConnection("QmHash", "Bob")
import IPFS_LNS
peerID = IPFS_LNS.LookUpContact("Bob") # get the contact 'Bob's IPFS peer ID
IPFS Technicalities:
How does IPFS-DataTransmission use IPFS to to send data to another computer? After all, IPFS is a file system and made for sharing and storing files, creating a content addressed internet. Does it really provide the functionality for two computers to communicate directly which each other?
Yes, IPFS does provide that functionality, although as of November 2021 that is still an experimental feature. That's why you have to configure IPFS to enable that feature as described in Prerequisites.txt.
After all, IPFS relies on peers sending data to each other over its decentralised network using a communication technology called libp2p. IPFS-DataTransmission essentially uses the libp2p module inside of the IPFS process running on the computer to communicate to other computers via an access point to the module which IPFS provides, the experimental feature which IPFS calls Libp2pStreamMounting. Libp2pStreamMounting works by allowing the user to set up a port forwarding system on the computer between the user's program (in this case IPFS-DataTransmission) and the libp2p process inside of IPFS. After setting up this port forwarding the user (in this case the IPFS-DataTransmission script) can communicate to the chosen local port on their computer, and IPFS forwards that communication to the libp2p module runnning inside of IPFS on the other computer, which forwards it in turn to a preconfigured port on that computer, to be listened to by the user (in this case an IPFS-DataTransmission Listener object).
Because the Python API for interacting with IPFS (ipfshttpclient) doesn't yet support this experimental feature, I have updated the module myself, the result of which is the ipfshttpclient2 folder in this code project, which will remain part of this project until the changes are integrated into the official ipfshttpclient module.
Links
Naturally, I host this project on IPFS too
Source Code:
IPFS (faster):
https://ipfs.io/ipfs/QmU3wAuZLJsbaLy4rBRBNRi7aRivjRrc4FdoHkSzs7fasv
IPNS (more likely to be up-to-date):
https://ipfs.io/ipns/k2k4r8k7h909zdodsvbxe32sahpfdkqcqqn1npblummw4df6iv7dj5xh
Website:
IPFS (faster):
https://ipfs.io/ipfs/QmTxBa33rCcnr6VxyH3dDd2oNddjMM8WkF6aNpq5mew54F
IPNS (more likely to be up-to-date):
https://ipfs.io/ipns/k2k4r8m2dzqi5s8jm3shm77sr1728ex7bsds0fk6e9gkf2ld2f3mnhcy
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for IPFS_Toolkit-0.2.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 00341097b404d4f17f4c5bfd322c8386dff4c22c03918a0d8d8d2c0c4214ea8a |
|
MD5 | 58c44ce088b76ac6f16328426b66c4b3 |
|
BLAKE2b-256 | 5d339e0d7a2230766aebbe24e458108b9371d4984a8bde600b80399ad03e936b |