Skip to main content

Convert TART observation data to Measurement Sets

Project description

tart2ms

PyPI package version number License

Convert data from a TART radio telescope to measurement set format. This module relies on the excellent dask-ms module as a helper to create the measurement sets. This packate requires python-casacore to be installed on your system

Install

sudo aptitude install python3-casacore
sudo pip3 install tart2ms

Examples

Download data from the TART in real time via the RESTful API (defaults to using the API at https://api.elec.ac.nz/tart/mu-udm):

tart2ms --ms data.ms

To convert a previously downloads JSON file to a measurement set (MS):

tart2ms --json data.json --ms data.ms

JSON-based datasets currently may only contain a single timestamp. This limits their usefulness when it comes to more general imaging. It is possible to make a concatenated database from multiple single-dump json databases, e.g.

tart2ms --json ../tart_data/NZ_2022_10_19_json/*.json

HDF5 format archive files may contain multiple timestamps and may also be concatenated into longer observations as is the case for JSON archive files.

tart2ms --hdf ../tart_data/NZ_2022_10_19/*.hdf

JSON databases may be exported from HDF5 archives using

tart_vis2json --vis ../NZ_2022_10_19/*.hdf 

Currently each such exported JSON database will contain a single timestamp (thus multiple JSON databases may result from a single HDF5 archive).

Your telescope name may not be in the JPL list of recognized observatories which at present raises an error in casacore and hence some casa tasks like listobs or plotants, even though the antenna table contains valid ITRF coordinates for the antennae. We recommend that if problems are encountered the telescope name is changed to an existing observatory like kat-7 or MeerKAT.

tart2ms --json ../tart_data/NZ_2022_10_19_json/*.json -c --telescope_name 'kat-7'

Standard CASA tasks may be executed with the CASA memo 229-compliant (MSv2.0) databases written by tart2ms. These may include (tested):

  • listobs
  • plotms
  • fixvis
  • plotants
  • clean

To synthesize (using wsclean) the image from the measurement set:

wsclean -name test -size 1280 1280 -scale 0.0275 -niter 0 data.ms

This will create an image called test-image.fits. You will need to install wsclean on your system.

Usage

usage: tart2ms [-h] [--json JSON] [--ms MS] [--api API] [--catalog CATALOG]
            [--vis VIS] [--pol2]

Generate measurement set from a JSON file from the TART radio telescope.

optional arguments:
-h, --help         show this help message and exit
--json JSON        Snapshot observation JSON file (visiblities, positions
                    and more). (default: None)
--hdf HDF          Visibility hdf5 file (One minutes worth of visibility data). (default: None)
--ms MS            Output MS table name. (default: tart.ms)
--api API          Telescope API server URL. (default:
                    https://api.elec.ac.nz/tart/mu-udm)
--catalog CATALOG  Catalog API URL. (default:
                    https://tart.elec.ac.nz/catalog)
--vis VIS          Use a local JSON file containing the visibilities for
                    visibility data (default: None)
--pol2             Fake a second polarization. Some pipelines choke if there
                    is only one. (default: False)

Credits

Thanks to Simon Perkins and Oleg Smirnov for help in interpreting the measurement set documentation.

TODO

Changelog

  • 0.6.0b4 Correct debug calculation of rayleigh limit in read_ms.
  • 0.6.0b3 If WEIGHT_SPECTRUM is not present, then subsititude ones
  • 0.6.0b2 Clean up the logging code. Add a --debug flag. Get rid of the annoying printing of casa tables
  • 0.6.0b1 New casa_read_ms function similar to read_ms, but uses casacore - blindingly fast. made function signatures the same for both read_ms() and casa_read_ms().
  • 0.5.0b3 Remove noisy warnings. Add venv makefile. Output the snapshot direction in 12-ball format to make rephasing easier.
  • 0.5.0b2 Make prediction an optional dependency to remove codex-africanus from requirements.
  • 0.5.0b1 Add model prediction Add automatic catalog download
  • 0.4.0b3 Use the new catalog url functions from tart_tools
  • 0.4.0b2 Add a helper function get_array_location
  • 0.4.0b1 Add a tart2ms.read_ms function (from disko) Add utilities for resolution calculations.
  • 0.3.0b3 New features for keeping the phase center constant during a long measurement set (fringe phasing) Fix progress bar.
  • 0.3.0b1 Fix many CASA compatability issues. Antenna positions are done. Many thanks to Ben Hugo of SARAO.
  • 0.2.0b5 Fix bug in the timensions of the TIME_CENTROID column in the MAIN table. Issue 8.
  • 0.2.0b3 Fix bug in the timensions of the TIME column in the MAIN table. Issue 7.
  • 0.2.0b3 Move to the tart-telescope organization on github..
  • 0.2.0b2 Place all visibilities from HDF5 files into a single measurement set..
  • 0.2.0b1 Add importing of HDF5 files saved from the web app.
  • 0.1.4b4 clean up some bitrot in dask-ms (dealing with chunking objects)
  • 0.1.4b3 Add SIGMA, FLAG, FLAG_CATEGORY to main table (:/)
  • 0.1.4b1 Add RESOLUTION and EFFECTIVE_BW to the SPECTRAL_WINDOW
  • 0.1.3b1 Sort out the timestamps correctly, added a handy function for converting to epoch time.
  • 0.1.2 Correct pointing direction of the array (in J2000).
  • 0.1.1 Added -pol2 switch to generate a second polarization.
  • 0.1.0 first functioning release.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tart2ms-0.7.0rc1.tar.gz (6.3 MB view details)

Uploaded Source

Built Distribution

tart2ms-0.7.0rc1-py3-none-any.whl (6.5 MB view details)

Uploaded Python 3

File details

Details for the file tart2ms-0.7.0rc1.tar.gz.

File metadata

  • Download URL: tart2ms-0.7.0rc1.tar.gz
  • Upload date:
  • Size: 6.3 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for tart2ms-0.7.0rc1.tar.gz
Algorithm Hash digest
SHA256 a90a0e8ba7bcbc69866339ea6932af8b0626a4a3cbbf683a83495129d3d82510
MD5 7ba90f7620138316df9e3f3c0330aa83
BLAKE2b-256 cb2168e0ccf1e52e3b11f0b4b2df0de6dd50109a4281c0a40bcc5579644af217

See more details on using hashes here.

File details

Details for the file tart2ms-0.7.0rc1-py3-none-any.whl.

File metadata

  • Download URL: tart2ms-0.7.0rc1-py3-none-any.whl
  • Upload date:
  • Size: 6.5 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for tart2ms-0.7.0rc1-py3-none-any.whl
Algorithm Hash digest
SHA256 4fdae0b069295c11b1cb59bfea960624f12eb987fcbb77b790532a454653f3b6
MD5 5affad1c47aa42a38e20c369d7a34324
BLAKE2b-256 bbc8a634e0f43398e49e842ada53f57ee7ef60947b20a78a7211d1b8ef3a2d09

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page