RCSB Python I/O Utility Classes
Introduction
This module contains a collection of utility classes for performing I/O operations on common file formats encountered in the PDB data repository.
Installation
Download the library source software from the project repository:
git clone --recurse-submodules https://github.com/rcsb/py-rcsb_utils_io.git
Optionally, run test suite (Python versions 3.9) using tox:
tox
Installation is via the program pip.
pip install rcsb.utils.io
or from the local repository:
pip install .
Usage
The MarshalUtil offers an easy way for reading in and writing out files in various formats, including CSV, JSON, pickle, mmCIF, bcif (BinaryCIF), fasta , and "list" files (plain text file in which each row is a list item).
Reading files
Let's say you have a JSON file, "data.json". You can read this in by:
from rcsb.utils.io.MarshalUtil import MarshalUtil
mU = MarshalUtil(workDir=".")
dataD = mU.doImport("data.json", fmt="json")
The same method works even if the file is compressed (e.g., "data.json.gz"):
dataD = mU.doImport("data.json.gz", fmt="json")
Note that this automatic handling of compressed gzip files applies to any type of input format.
You can also import remote files directly from the command line, e.g.:
dataD = mU.doImport("https://files.rcsb.org/pub/pdb/holdings/current_file_holdings.json.gz", fmt="json")
To read in a pickle file, "data.pic":
from rcsb.utils.io.MarshalUtil import MarshalUtil
mU = MarshalUtil()
dataD = mU.doImport("data.pic", fmt="pickle")
To read in and parse an mmCIF file, "4hhb.cif.gz":
from rcsb.utils.io.MarshalUtil import MarshalUtil
mU = MarshalUtil()
# Read all data containers from the mmCIF file into `dataContainerList`
dataContainerList = mU.doImport("https://files.rcsb.org/pub/pdb/data/structures/divided/mmCIF/hh/4hhb.cif.gz", fmt="mmcif")
# Get the first dataContainer (in most cases, there will only be one container in the file)
dataContainer = dataContainerList[0]
# Print the name of the container
eName = dataContainer.getName()
print(eName)
# Get the list of categories
catNameList = dataContainer.getObjNameList()
print(catNameList)
# Iterate over all the categories and attributes and store them in a new dictionary
cifDataD = {}
for catName in catNameList:
if not dataContainer.exists(catName):
continue
dObj = dataContainer.getObj(catName)
for ii in range(dObj.getRowCount()):
dD = dObj.getRowAttributeDict(ii)
cifDataD.setdefault(eName, {}).setdefault(catName, []).append(dD)
To read in and parse an BCIF file, "4hhb.bcif.gz":
from rcsb.utils.io.MarshalUtil import MarshalUtil
mU = MarshalUtil()
# Read all data containers from the mmCIF file into `dataContainerList`
dataContainerList = mU.doImport("https://files.rcsb.org/pub/pdb/data/structures/divided/mmCIF/hh/4hhb.bcif.gz", fmt="bcif")
**_For more examples, see [testMarshallUtil.py](https://github.com/rcsb/py-rcsb_utils_io/blob/master/rcsb/utils/tests-io/testMarshallUtil.py)._**
#### Writing files
You can use the `MarshalUtil` to write out the following data structures into the corresponding file formats:
Object | Output fmt
list | list dict | json or pickle DataContainerList | mmcif or bcif
For example, if you have a dictionary object, `dataD`, you can export it via:
```python
from rcsb.utils.io.MarshalUtil import MarshalUtil
mU = MarshalUtil()
dataD = {"name": "John Doe", "age": "33"}
mU.doExport("data.json", dataD, fmt="json", indent=2)
# Or, to export and compress as gzip:
mU.doExport("data.json.gz", dataD, fmt="json", indent=2)
Additionally, if you want to export a BCIF file, you can export in via:
from rcsb.utils.io.MarshalUtil import MarshalUtil
mU = MarshalUtil()
# Use dataContainerList (created for reading a file) as reference
mU.doExport("data.bcif", dataContainerList, fmt="bcif")
# Or, to export and compress as gzip:
mU.doExport("data.bcif.gz", dataContainerList, fmt="bcif")
Release files for rcsb.utils.io 1.55
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| rcsb_utils_io-1.55.tar.gz | 61.9 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| rcsb_utils_io-1.55-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 115.1 kB
Release files / rcsb_utils_io-1.55.tar.gz
| Download URL | rcsb_utils_io-1.55.tar.gz |
|---|---|
| Size | 61.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
774293c430a948e96ceb1b302f2ba63e05de987c0d8dae9e6398c50ca8f39ff7
|
|
BLAKE2b-256 checksum How to use checksums |
998ab5f643e4f046a87159e44f65ecb312e4e6958b2dd339cb41e4c3854b9e47
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.15
|
Release files / rcsb_utils_io-1.55-py3-none-any.whl
| Download URL | rcsb_utils_io-1.55-py3-none-any.whl |
|---|---|
| Size | 53.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
559011cf299216bc2c534560a9361126271c788c4eaac49b0357abbc536da665
|
|
BLAKE2b-256 checksum How to use checksums |
aec43f13d7a979b0e8bf870d78334389388e3de1965ff649188a25394b301582
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.15
|
