pyconquest 0.1.4

Python code which partly mimics the conquest pacs

PyConquest

Introduction

Python code to partly mimic the functionality of the Conquest Pacs system ( http://www.natura-ingenium.nl/dicom.html ). No (source) code of Conquest was used to write this python code. Program was optimised to be partly compatible (files and database), so queries etc. can be re-used.

Description:

Class is used to organise and index sets of dicom files. The dicom files are stored in a directory (default name : data), with the files of each patient stored in a subdirectory with the patientid as the name. The class will index all files in the directory and store information (tag information of the dicom files) in a sqlite database (default name : conquest.db). Which tags are stored in the database can be defined by the dicom.sql file, this file is optional, a standard set is used when not present.

Standard file/directory layout is :

conquest.db                       database file
data                              data_directory
    1                             subdirectory with name of PatientID
      file1_of_pat1.dcm           files of patient1
      file2_of_pat1.dcm
      ..
      ..
    2                             subdirectory with name of PatientID
      file1_of_pat2               files of patient2
      file2_of_pat2
      ..
      ..
[dicom.sql]                       optional definition of the database

Implemented features :

• indexer properties : read a dicom tree and build a sqlite database from this which conforms to the Conquest standard.
• use the conquest style dicom.sql file to define the columns
• add some special columns,with for instance the roinames in a RTSTRUCT file
• provide low level procedures to write and read to the database and helper files to create queries
• provide basic SCU and SCP functionality
• basic printing of data properties

What it is not and when you should use Conquest

• no attempt made to mimic full dicom functionality (this is no PACS)
• lua scripting
• image visualisation

Examples

Note : use the argument loglevel='INFO' when creating the instance to get logging output

(Re)create database and rebuild database

from pyconquest import pyconquest

c=pyconquest(loglevel='INFO')
c.create_standard_dicom_tables()
nr_files=c.rebuild_database_from_dicom()
# only rebuild a single MRN/patient
c.rebuild_database_from_dicom(mrn='1234567')
# build for every MRN, even if the MRN already exists in Database
c.rebuild_database_from_dicom(compute_only_missing=False)

Basic database summary

from pyconquest import pyconquest

c=pyconquest()
c.dicom_series_summary()

Extracting extra data from the dicomfiles

adds data concerning images to the dicomseries table

from pyconquest import pyconquest

c=pyconquest()
c.analyse_images()

Saving database information to a csv file

from pyconquest import pyconquest

c=pyconquest()

# by table(s) or views, specification of filenames is optional
c.dump_data_to_csv(table='dicomstudies')
c.dump_data_to_csv(table=['dicomseries','dicompatients'],
                   filename_dict={'dicomseries': 'dicomseries_filename', 'dicompatients': 'dicompatients_filename'} )
# by query
c.dump_data_to_csv(query='select * from dicomstudies',filename='query_filename.csv')

More advanced querying and extraction from the database:

from pyconquest import pyconquest

c=pyconquest()

# extract python list type of selected seriesuids 
query='select * from dicomseries where modality='CT'
list_of_seriesuid=c.execute_db_query(query,return_list_from_col='SeriesInst')
for seriesuid in list_of_seriesuid:
  print(seriesuid)
  
In dicomimages.ElementList information is saved in 'python like' nested lists as a string. 
You can use the following code to turn the string ( in Python ) into a real list:

beam_info =eval(' '.join([("'"+part+"'" if part not in ['[[', ']]', ',', ' ', '', ']', '[', '],'] else part)
                      for part in str(**ElementListString**).split(' ')]))

Using non standard names and directories

from pyconquest import pyconquest

c=pyconquest(
    data_directory='data2',
    sql_inifile_name='dicom2.sql',
    database_filename='test.db',
    connect_and_read_sql=True)
    
nr_files=c.rebuild_database_from_dicom()   
c.dicom_series_summary()

Adding and deleting files to the database

Adding a single file specified by name (optionally remove original)

from pyconquest import pyconquest

c=pyconquest()
c.store_dicom_file('1.2.840.113704.1.111....931a31.dcm')
c.store_dicom_file('1.2.840.113704.1.111....931a31.dcm', remove_after_store=True)

Enter all dicom files in a directory into the database

from pyconquest import pyconquest

c=pyconquest()
c.store_dicom_files_from_directory('input_directory_name')
c.store_dicom_files_from_directory('input_directory_name', remove_after_store=True)

# using below, the filename is made equal to the sopinstanceuid :
c.store_dicom_files_from_directory('input_directory_name', sopinstance_as_filename=True)

Deleting series or list of series from the database (optionally 'real' deletion of the file)

c.delete_series(seriesuid='2.16.840.1.11....152.20190524082318.537836')
c.delete_series(seriesuid=['2.16..7836','1.2.666...55'])
#or by query
c.delete_series(query='select * from dicomseries where modality="CT"')

# Really pyhsically remove the files (otherwise only the DB entries are removed)
c.delete_series('2.16.840.1.11....152.20190524082318.537836', delete_files=True)

Sending and receiving files via dicom connectivity

Send dicom files via dicom protocol, by patientid, seriesuid or query

from pyconquest import pyconquest

c=pyconquest()
c.connect_db()
c.send_dicom(addres='127.0.0.1', port=5678, patientid='1234567', ae_title=b'destinationAE',  sending_ae_title=b'MY_AE_TITLE')
c.send_dicom(seriesuid='2.16.840.1.113669.2.931128.880152.20190524082318.537836')

RTPLAN_query="select seriesinst from dicomseries where modality=\'RTPLAN\'"
c.send_dicom(query=RTPLAN_query)

Start receiver (received files are stored in data directory and database is updated )

from pyconquest import pyconquest

c=pyconquest(data_directory='data2')
c.start_dicom_listener(port=5678)
# Below does not update the database which saves time
c.start_dicom_listener(port=104, write_to_database=False)

Copying files of a series on the disk to another directory

by seriesuid or list of seriesuids

from pyconquest import pyconquest

c.copy_dicom_files_to_dest(seriesuid='1.2...8024597.12',destination='out', UseSubDirectories=True)
c.copy_dicom_files_to_dest(seriesuid=['1.2.84..97.12','2.16.840....33'],destination='data2')

Or by query (query should return seriesinst)

from pyconquest import pyconquest

c=pyconquest()
CTquery="select seriesinst from dicomseries where modality=\'CT\'"
c.copy_dicom_files_to_dest(query=CTquery, destination='CTdata',UseSubDirectories=True)

Changing tags of dicom files

Simple functionality to change a single tag of a dicom file. For each change the file is opened and saved, so not suitable for many changes per dicom file.

NOTE : The tags are changed in the dicom file only, not in the database. So if needed, a rebuild database needs to be done to synchronise the database.

from pyconquest import pyconquest

c=pyconquest()
# single tag of single file
c.change_tag('test.dcm', tag='PatientBirthDate', new_value='19000101')

# all tags from a list of filenames
c.change_all_tags(filename=['test.dcm', 'test.dcm'], new_value='19000102')

# using a helper function, change by series, study or patientid
lst = c.get_list_of_filenames(seriesuid='2.16.840.1.113669.2.931128.880152.20190524082317.937233'))
c.change_all_tags(filename=lst, tag='PatientBirthDate', new_value='19000101')

Deleting and changing ROI names in a RTSTRUCT file

It is possible to change or delete rois in a RTSTRUCT file using either of 3 modes:

• delete : delete the specific names
• leave : leave only the given names and delete all others
• change : change the given names to alternatives

There are flags to actualy write the file or not. Using delete_points you can indicate whether deletion of points is allowed

you can use get_list_of_filenames() to apply to multiple files.

NOTE : The tags are changed in the dicom file only, not in the database. So if needed, a rebuild database needs to be done to synchronise the database.

from pyconquest import pyconquest

c=pyconquest()
c.modify_rtstruct(filename='tst.dcm',mode='change',roiname=['Lung_R','Lung_L'],newname=['LR','LL'],write_file=True)
c.modify_rtstruct(filename='tst.dcm',mode='delete',roiname=['Lung_R','Lung_L'],write_file=True)
c.modify_rtstruct(filename='tst.dcm',mode='leave',roiname=['Lung_R','Lung_L'],write_file=True)
# standard mode is to NOT delete points, you can overrule
# and filename can be a list of filenames
c.modify_rtstruct(filename=['tst.dcm','tst2.dcm'],mode='leave',roiname=['Lung_R','Lung_L'],write_file=True,delete_points=True)

Definition of the database structure

The database can be defined using various ways

• If a file dicom.sql exist that file is used
• If an alternative file is specified on creation of the instance, that file is used
  • use sql_infilename='filename.sql' when creating
• If no file is found, a hardcoded definition is taken
  • You can add columns to this definition as illustrated below:

from pyconquest import pyconquest

c=pyconquest(sql_inifile_name='dicom.sql',loglevel='INFO')
c.add_column_to_database(tablename='DICOMpatients',column_definition=['0x0020', '0x000d', 'StudyInst'])
c.create_standard_dicom_tables()

the .sql files are compatible with the original Conquest file format

To install and upload to PyPi:

python setup.py sdist
twine upload dist/*

CHANGELOG

version 0.1.4

• For RTPLAN, beam information is saved in the column DicomImages.ElementList

version 0.1.3

• Added functionality change_tag() to change tags of DICOM files
• Added functionality modify_rtstruct() to change and delete roi's from files

version 0.1.2

• Added method analyse_images to analyse CT/MR/PET images and add data to DicomSeries table. Data added is number of slices, min/max slicepositions and a check for slice consistency is done.
• Added postprocessing routine database_postprocessing() of the database to enrich the database with entries that can only be added when the database is complete. For instance the Referenced seriesuid in DICOMseries table for RTDOSE. Routine is automatically executed at the end of rebuild_database_from_dicom()
• Added option called sopinstance_as_filename to store_dicom_files_from_directory() and store_dicom_file() to change the actual filename in the database from the original (default) to sopinstanceUID.dcm

version 0.1.1 (skipped to 0.1.1 because some systems take 0.0.51 as default)

• Added option dump_data_to_csv() to dump data from the database to csv files
• Logging of errors to a (rotating) log file : "pyconquest_error.log"
• Added option write_to_database to dicom_listener, if FALSE, sqlite db is not updated (for speed)
• For RTDOSE the referenced SOP UID of the RTPLAN is saved to DicomImages table
• For RTPLAN the referenced SOP UID of the RTSTUCT is saved to DicomImages table

version 0.0.7

• added option not to check for double entries when rebuilding the database (for speed)
• added fast option to delete a patient from the database
• database indices are now created when creating a new database
• made index tables non unique to improve robustness

version 0.0.6

• Rebuild_database_from_dicom() now has option to rebuild only missing directories
• When closing the database, the time elapsed is printed
• Referenced seriesuid is now extracted from RTSTRUCT and placed in dicomseries and dicomimages table
• Dicom_series_summary() now prints result directly to stdout, so no need for pandas to make a readable table
• The view v_series now only combines series and study table (previous version was too slow due to complexity)
• New view : v_seriesRT now combines series,study and image tables for only RT dicom objects (so not for images)

version 0.0.5

• a view (v_series) is added to the sqlite database that joins study,series and image table
• delete_series can now handle query to define the series to delete
• series argument of delete_series changed from positional to named
• added option to rebuild database for a single mrn (mrn='1234567')
• added option sending_ae_title to send_dicom() and send_dicom_file()
• bugfix in name of file in store_dicom (wrong path was used)
• execute_db_query now has an option to return (only) a list of a single column

version 0.0.4

• Added function insert_dict to class to easily insert your own data in the database. Also takes care of creating the table when the first dict is inserted
• If a database has no tables, tables are created automatically on db opening (so on instance creation)
• In create_buildquery, now columns can be defined with formats deviating from default and the default can be defined
• Number of fractions and number of beams saved from RTPLAN to database columns
• For RTDOSE, RTPLAN and RTSTRUCT file hashes are calculated and saved in dicomimage table
• Added delete_series function to delete the series from database and optionally delete the .dcm files
• Added filter_roinames() and set_roi_filter() methods to facilitate filtering of roi name lists for child classes

version 0.0.3

• Improved documentation and docstrings
• uniformity in calling send and copy routines