A collection of data processing Spark functions for the use in Statistics Norway
SSB Spark Tools
A collection of data processing Spark functions for the use in Statistics Norway (SSB)
The SSB Spark Tools Library is a colection of Data processing functions for the use in Data processing in Statistics Norway
pip install ssb_spark_tools
make help to see common development commands.
install-build-tools Install required tools for build/dev build Build dist test Run tests clean Clean all build artifacts release-validate Validate that a distribution will render properly on PyPI release-test Release a new version, uploading it to PyPI Test release Release a new version, uploading it to PyPI bump-version-patch Bump patch version, e.g. 0.0.1 -> 0.0.2 bump-version-minor Bump minor version, e.g. 0.0.1 -> 0.1.0
Refer to the
Makefile to see details about the different tasks.
Run tests for all python distributions using
This will require that your dev machine has the required python distributions installed locally. (You can install python distributions using pyenv.)
Before releasing, make sure you're working on a "new" version number. You can bump the version using the bumpversion tool.
Also, make sure to update release notes.
To release and publish a new version to PyPI:
This will run tests, build distribution packages and perform some rudimentary PyPI compliancy checking.
For a dress rehearsal, you can do a test release to the TestPyPI index. TestPyPI is very useful, as you can try all the steps of publishing a package without any consequences if you mess up. Read more about TestPyPI here.
To perform the actual release, run:
You should see the new release appearing here (it might take a couple of minutes for the index to update).
- Initial version with functions as in use on initiaition
Statistics Norway – https://github.com/statisticsnorway
Distributed under the MIT license. See
LICENSE for more information.
- Fork it (https://github.com/statisticsnorway/ssb-pseudonymization-py/fork)
- Create your feature branch (
git checkout -b feature/fooBar)
- Commit your changes (
git commit -am 'Add some fooBar')
- Push to the branch (
git push origin feature/fooBar)
- Create a new Pull Request
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size ssb_spark_tools-0.0.1-py3-none-any.whl (10.3 kB)||File type Wheel||Python version py3||Upload date||Hashes View|
|Filename, size ssb_spark_tools-0.0.1.tar.gz (5.1 kB)||File type Source||Python version None||Upload date||Hashes View|
Hashes for ssb_spark_tools-0.0.1-py3-none-any.whl