Pass directories between metaflow steps.
Project description
Magic Directories
An experimental plugin for passing data in directories in between steps.
Warning: this package is highly experimental.
Installation
pip install metaflow
Usage
You can use @magicdir
to pass local directories between metaflow steps. This will also work remotely.
#flow.py
class MagicDirFlow(FlowSpec):
@magicdir(dir='mydir')
@step
def start(self):
with open('mydir/output1', 'w') as f:
f.write('hello world')
with open('mydir/output2', 'w') as f:
f.write('hello world again')
self.next(self.end)
@magicdir(dir='mydir')
@step
def end(self):
print('first', open('mydir/output1').read())
print('second', open('mydir/output1').read())
if __name__ == "__main__":
MagicDirFlow()
If you run the above flow, you will see:
python flow.py run --with batch
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for metaflow_plugin_magicdir-0.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0d24b134fc9453210a0125fbf04c8910dfdd00df908bcd9db6f600b405f6a055 |
|
MD5 | 280a70c944a6c7540b0e6933f6e9e916 |
|
BLAKE2b-256 | 5ce611b85cdcc82a979cab19c5eb07d9961b60a52f6611f418396906d4c16895 |
Close
Hashes for metaflow_plugin_magicdir-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 19681c1e365f72b965298d8d13b46b7b5ae8ee3dd8ad6e31c05dc9e817a88f7d |
|
MD5 | 28d4f90c3239241d535942bf944bdeb1 |
|
BLAKE2b-256 | 27543af9b5cadf336915c1a0019ac55cddba41eb73bc6afde187c347aaa53593 |