Skip to main content

Virtual large arrays and lazy evaluation

Project description

Build Status

Virtual large arrays and lazy evaluation.

For example, we can combine multiple array data sources into a single virtual array:

>>> first_time_series = OrthoArrayAdapter(hdf_var_a)
>>> second_time_series = OrthoArrayAdapater(hdf_var_b)
>>> print first_time_series.shape, second_time_series.shape
(52000, 800, 600) (56000, 800, 600)
>>> time_series = biggus.LinearMosaic([first_time_series, second_time_series], axis=0)
>>> time_series
<LinearMosaic shape=(108000, 800, 600) dtype=dtype('float32')>

Any biggus Array can then be indexed, independent of underlying data sources:

>>> time_series[51999:52001, 10, 12]
<LinearMosaic shape=(2,) dtype=dtype('float32')>

And an Array can be converted to a numpy ndarray on demand:

>>> time_series[51999:52001, 10, 12].ndarray()
array([ 0.72151309,  0.54654914], dtype=float32)

Get in touch!

We’ve got lots of exciting plans underway for biggus, but we’re also very keen to hear from you.

  • How are you thinking of using biggus?
  • What capabilities does biggus need for it to be useful to you?
  • What capabilities does biggus already have that you find useful?

Further reading

To get more ideas of what Biggus can do, please browse the wiki, and its examples.

If you have any questions or feedback please feel free to post to the discussion group or raise an issue on the issue tracker.

Project details


Release history Release notifications

History Node

0.14.0

History Node

0.13.0

History Node

0.12.0

History Node

0.11.0

History Node

0.10.0

History Node

0.9.1

History Node

0.9.0

History Node

0.8.0

History Node

0.7.0

History Node

0.6.0

History Node

0.5.0

This version
History Node

0.4.0

History Node

0.3

History Node

0.2

History Node

0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
Biggus-0.4.0.tar.gz (30.9 kB) Copy SHA256 hash SHA256 Source None Feb 7, 2014

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page