A new kind of Progress Bar, with real-time throughput, eta and very cool animations!
A new kind of Progress Bar, with real-time throughput, eta and very cool animations!
Ever found yourself in a remote ssh session, doing some lengthy operations, and every now and then you feel the need to hit [enter] just to ensure you didn't lose the connection? Ever wondered where your processing is in, and when will it finish? Ever needed to pause the progress bar for a while, return to the python REPL for a manual inspection or fixing an item, and then resume the process like it never happened? I did...
I've made this cool progress bar thinking about all that, the Alive-Progress bar! :)
I like to think of it as a new kind of progress bar for python, as it has among other things:
- a cool live spinner, which clearly shows your lengthy process did not hang and your ssh connection is healthy;
- a visual feedback of the current speed/throughput, as the spinner runs faster or slower according to the actual processing speed;
- an efficient multi-threaded bar, which updates itself at a fraction of the actual speed (1,000,000 iterations per second equates to roughly 60 frames per second refresh rate) to keep CPU usage low and avoid terminal spamming; (📌 new: you can now calibrate this!)
- an expected time of arrival (ETA), with a smart exponential smoothing algorithm that shows the remaining processing time in the most friendly way;
- a print() hook and (📌 new) logging support, which allows print statements and logging messages effortlessly in the midst of an animated bar, automatically cleaning the screen and even enriching it with the current position when that occurred;
- after your processing has finished, a nice receipt is printed with the statistics of that run, including the elapsed time and observed throughput;
- it tracks your desired count, not necessarily the actual iterations, to detect under and overflows, so it will look different if you send in less or more than expected;
- it automatically detects if there's an allocated tty, and if there isn't (like in a pipe redirection), only the final receipt is printed, so you can safely include it in any code and rest assure your log file won't get thousands of progress lines;
- you can pause it! I think that's an unprecedented feature for a progress bar! It's incredible to be able to manually operate on some items while inside a running progress bar context, and get the bar back like it had never stopped whenever you want;
- it is customizable, with a growing smorgasbord of different bar and spinner styles, as well as several factories to easily generate yours!
📌 New in 1.6 series!
- soft wrapping support - or lack thereof actually, it won't scroll your terminal desperately up if it doesn't fit the line anymore!
- hiding cursor support - more beautiful and professional appearance!
- python logging support - adequately clean and enriched messages from logging without any configuration or hack!
- exponential smoothing of ETA - way smoother when you need it the most!
- proper bar title - left aligned always visible title so you know what is expected from that processing!
- enhanced elapsed time and ETA representation - the smallest rendition possible, so you can maximize the animations!
bar.text()dedicated method - now you can change the situational message without making the bar going forward!
- performance optimizations - even less overhead, your processing won't even notice it!
Just install with pip:
$ pip install alive-progress
Open a context manager like this:
from alive_progress import alive_bar items = range(1000) # retrieve your set of items with alive_bar(len(items)) as bar: # declare your expected total for item in items: # iterate as usual # process each item bar() # call after consuming one item
And it's alive! 👏
In general lines, just retrieve the items, enter the
alive_bar() context manager, and iterate/process normally, calling
bar() once per item.
itemscan be any iterable, and usually will be some queryset;
- the first argument of the
alive_baris the expected total, so it can be anything that returns an integer, like
len(items)for iterables that support it, or even a static integer;
bar()call is what makes the bar go forward -- you usually call it in every iteration after consuming an item, but you can get creative! Remember the bar is counting for you independently of the iteration process, only when you call
bar()(something no other progress bar have), so you can use it to count anything you want! For example, you could call
bar()only when you find something expected and then know how many of those there were, including the percentage that it represents! Or call it more than once in the same iteration, no problem at all, you choose what you are monitoring! The ETA will not be that useful unfortunately;
bar()call returns the current count or percentage.
So, you could even use it without any loops, like for example:
with alive_bar(3) as bar: corpus = read_big_file() bar('file read, tokenizing') tokens = tokenize(corpus) bar('tokens ok, processing') process(tokens) bar()
Modes of operation
total argument is optional. Providing it makes the bar enter the definite mode, the one used for well-bounded tasks. This mode has all statistics widgets
alive-progress has to offer: counter, throughput and eta.
If you do not provide a
total, the bar enters the unknown mode. In this mode, the whole progress-bar is animated like the cool spinners, as it's not possible to determine the percentage of completion. Therefore, it's also not possible to compute an eta, but you still get the counter and throughput widgets.
The cool spinner are still present in this mode, so the animations from both bar and spinner runs concurrently and independently of each other, rendering a unique show in your terminal 😜.
Then you have the (📌 new) manual modes, where you get to actually control the bar! That way, you can put it in whatever position you want, including make it go backwards or act like a gauge of some sort!
Just pass a
manual=True argument to
config_handler.set_global()), and you get to send a percentage to the very same
bar() handler! For example to set it at 15%, you would call
bar(0.15), which is 15 / 100, as simple as that.
Call it as frequently as you need, the refresh rate will be asynchronously computed as usual, according to current progress and elapsed time.
And please provide the
total if you have it, to get all the same counter, throughput and eta widgets as the definite mode. The counter will be inferred from the supplied user percentage.
If you omit the
total, it's not possible to infer the counter widget, but you'll still kinda get the throughput and eta widgets, a simpler one with only "%/s" (percent per second) and a rough ETA to get to 100%, which are very inaccurate, but better than nothing.
Just remember: You do not have to think about which mode you should be using, just always pass a
totalif you know it, and use
manualif you need it! It will just work! 👏
To summarize it all:
|mode||completion||counter||throughput||eta||overflow and underflow|
|unknown||❌ (an animation runs)||✅||✅||❌||❌|
|manual (bounded)||✅ you choose||✅ inferred||✅||✅||✅|
|manual (unbounded)||✅ you choose||❌||⚠️ simpler||⚠️ rough||✅|
- in definite and unknown modes, it accepts an optional
intargument, which increments the counter by any positive number, like
bar(5)to increment the counter by 5 in one step ➔ relative positioning;
- in manual modes, it needs a mandatory
floatargument, which overwrites the progress percentage, like
bar(.35)to put the bar in the 35% position ➔ absolute positioning.
- and it always returns the updated counter/progress value.
bar()handlers used to also have a
textparameter which is being removed, more details here.
Wondering what styles does it have bundled? It's
Actually I've made these styles just to put to use all combinations of the factories I've created, but I think some of them ended up very very cool! Use them at will, or create your own!
There's also a bars
showtime, check it out! ;)
(📌 new) Now there are new commands in exhibition! Try the
from alive_progress import show_bars, show_spinners # call them and enjoy the show ;)
There's also a (📌 new) utility called
print_chars, to help finding that cool one to put in your customized spinner or bar, or to determine if your terminal do support unicode chars.
While in any alive progress context, you can display messages with:
- the usual Python
loggingframework, which properly clean the line, print or log an enriched message (including the current bar position) and continues the bar right below it;
- the (📌 new)
bar.text('message')call, which sets a situational message right within the bar, usually to display something about the items being processed or the phase the processing is in.
Deprecated: there's still a
bar(text='message')to update the situational message, but that did not allow you to update it without also changing the bar position, which was inconvenient. Now they are separate methods, and the message can be changed whenever you want.
DeprecationWarnings should be displayed to alert you if needed, please update your software to
bar.text('message'), since this will be removed in the next version.
Appearance and behavior
There are several options to customize appearance and behavior, most of them usable both locally and globally. But there's a few that only make sense locally, these are:
title: an optional yet always visible title if defined, that represents what is that processing;
calibrate: calibrates the fps engine (more details here)
Those used anywhere are [default values in brackets]:
40] number of characters to render the animated progress bar
spinner: the spinner to be used in all renditions it's a predefined name in
show_spinners(), or a custom spinner
bar: bar to be used in definite and both manual modes it's a predefined name in
show_bars(), or a custom bar
unknown: bar to be used in unknown mode (whole bar is a spinner) it's a predefined name in
show_spinners(), or a custom spinner
'smooth', which sets spinner, bar and unknown] theme name in alive_progress.THEMES
False] runs animations even without a tty (more details here)
False] set to manually control percentage
True] includes the bar position in print() and logging messages
0] fixed title length, or 0 for unlimited
To use them locally just send the option to
from alive_progress import alive_bar with alive_bar(total, title='Title here', length=20, ...): ...
To use them globally, set them before in
config_handler object, and any
alive_bar created after that will also use those options:
from alive_progress import alive_bar, config_handler config_handler.set_global(length=20, ...) with alive_bar(total, ...): # both sets of options will be active here! ...
And you can mix and match them, local options always have precedence over global ones!
You should now be completely able to use
alive-progress, have fun!
If you've appreciated my work and would like me to continue improving it, you could buy me a coffee! I would really appreciate that 😊! Thank you!
And if you want to do even more, exciting stuff lies ahead!
Calibration (📌 new)
alive-progressbars have a cool visual feedback of the current throughput, so you can instantly see how fast your processing is, as the spinner runs faster or slower with it. For this to happen, I've put together and implemented a few fps curves to empirically find which one gave the best feel of speed:
(interactive version here)
The graph shows the logarithmic (red), parabolic (blue) and linear (green) curves, as well as an adjusted logarithmic curve (dotted orange), with a few twists for small numbers. I've settled with the adjusted logarithmic curve, as it seemed to provide the best all around perceived speed changes. In the future and if someone would find it useful, it could be configurable.
alive-progresscalibration is 1,000,000 in auto (and manual bounded) modes, ie. it takes 1 million iterations per second for the bar to refresh itself at 60 frames per second. In the manual unbounded mode it is 1.0 (100%). Both enable a vast operating range and generally work well.
Let's say your processing hardly gets to 20 items per second, and you think
alive-progressis rendering sluggish, you could:with alive_bar(total, calibrate=20) as bar: ...
And it will be running waaaay faster... Perhaps too fast, consider calibrating to ~50% more, find the one you like the most! :)
Create your own animations
Make your own spinners and bars! All of the major components are individually customizable!
There's builtin support for a plethora of special effects, like frames, scrolling, bouncing, delayed and compound spinners! Get creative!
These animations are made by very advanced generators, defined by factories of factory methods: the first level receives and process the styling parameters to create the actual factory; this factory then receives operating parameters like screen length, to build the infinite animation generators.
These generators are capable of several different animation cycles, for example a bouncing ball has a cycle to the right and another to the left. They continually yield the next rendered animation frame in a cycle until it is exhausted. This just enables the next one, but does not start it! That has all kinds of cool implications: the cycles can have different animation sizes, different screen lengths, they do not need to be synchronized, they can create long different sequences by themselves, they can cooperate with each other to play cycles in sequence or simultaneously, and I can display several at once on the screen without any interferences! It's almost like they are alive! 😉
The types I've made are:
frames: draw any sequence of characters, that will be played frame by frame in sequence;
scrolling: pick a frame or a sequence of characters and make it flow smoothly from one side to the other, hiding behind or wrapping upon the invisible borders; if using a sequence, generates several cycles of distinct characters;
bouncing: aggregates two
scrollingin opposite directions, to make two frames or two sequences of characters flow interleaved from/to each side, hiding or immediately bouncing upon the invisible borders; supports several interleaved cycles too;
delayed: get any other animation generator, and copy it multiple times, skipping some frames at the start! very cool effects are made here;
compoundget a handful of generators and play them side by side simultaneously! why choose if you can have them all?
A small example (Click to see it in motion)
The Pause mechanism
Why would you want to pause it, I hear? To get to manually act on some items at will, I say! Suppose you need to reconcile payment transactions. You need to iterate over thousands of them, detect somehow the faulty ones, and fix them. This fix is not simple nor deterministic, you need to study each one to understand what to do. They could be missing a recipient, or have the wrong amount, or not be synced with the server, etc, it's hard to even imagine all possibilities. Typically you would have to let the detection process run until completion, appending to a list each inconsistency found, and waiting potentially a long time until you can actually start fixing them. You could of course mitigate this by processing in chunks or printing them and acting in another shell, but those have their own shortcomings. Now there's a better way, pause the actual detection for a moment! Then you have to wait only until the next one is found, and act in near real time!
To use the pause mechanism, you must be inside a function, which you should already be in your code (in the ipython shell just wrap it inside one). This requires a function to act as a generator and
yieldthe objects you want to interact with. The
barhandler includes a context manager for this, just do
with bar.pause(): yield transaction.def reconcile_transactions(): qs = Transaction.objects.filter() # django example, or in sqlalchemy: session.query(Transaction).filter() with alive_bar(qs.count()) as bar: for transaction in qs: if not validate(transaction): with bar.pause(): yield transaction bar()
That's it! Then you can use it in ipython (or your preferred REPL)! Just call the function to instantiate the generator and, whenever you want another transaction, call
next(gen, None)! The progress bar will run as usual while searching, but as soon as an inconsistency is found, the bar pauses itself and you get the prompt back with a transaction! How cool is that 😃?In : gen = reconcile_transactions() In : next(gen, None) |█████████████████████ | 105/200 [52%] in 5s (18.8/s, eta: 4s) Out: Transaction<#123>
When you're done, continue the process with the same
nextas before... The bar reappears and continues like nothing happened!! :)In : next(gen, None) |█████████████████████ | ▁▃▅ 106/200 [52%] in 5s (18.8/s, eta: 4s)
Forcing animations on non-interactive consoles (like Pycharm's)
Pycharm's python console for instance do not report itself as "interactive", so I've included a
force_ttyargument to be able to use the alive-progress bar in it.
So, just start it as:with alive_bar(1000, force_tty=True) as bar: for i in range(1000): time.sleep(.01) bar()
You can also set it system-wide in
Do note that this console is heavily instrumented and has more overhead, so the outcome may not be as fluid as you would expect.
- This whole project was implemented in functional style;
- It does not declare even a single class;
- It uses extensively (and very creatively) python Closures and Generators, they're in almost all modules (look for instance the spinners factories and spinner_player 😜);
- It does not have any dependencies.
- improve test coverage, hopefully achieving 100% branch coverage
- variable width bar rendition, listening to changes in terminal size
- enable multiple simultaneous bars, for nested or multiple statuses
- create a contrib system, to allow a simple way to share users' spinners and bars styles
- jupyter notebook support
- support colors in spinners and bars
- any other ideas welcome!
- create an unknown mode for bars (without a known total and eta)
- implement a pausing mechanism
- change spinner styles
- change bar styles
- include a global configuration system
- create generators for scrolling, bouncing, delayed and compound spinners
- create an exhibition of spinners and bars, to see them all in motion
- include theme support in configuration
- soft wrapping support
- hiding cursor support
- python logging support
- exponential smoothing of ETA time series
Python 2 EOL
alive_progress next major version 2.0 will support Python 3.5+ only. But if you still need support for Python 2, there is a full featured one you can use, just:
$ pip install -U "alive_progress<2"
- 1.6.0: soft wrapping support; hiding cursor support; python logging support; exponential smoothing of ETA time series; proper always visible bar title; enhanced elapsed time representation; new bar.text() dedicated method (deprecating 'text' parameter in bar()); performance optimizations
- 1.5.1: fix compatibility with python 2.7 (should be the last one, version 2 is in the works, with python 3 support only)
- 1.5.0: standard_bar accepts a background parameter instead of blank, which accepts arbitrarily sized strings and remains fixed in the background, simulating a bar going "over it"
- 1.4.4: restructure internal packages; 100% branch coverage of all animations systems, i.e., bars and spinners
- 1.4.3: protect configuration system against other errors (length='a' for example); first automated tests, 100% branch coverage of configuration system
- 1.4.2: sanitize text input, keeping \n from entering and replicating bar on screen
- 1.4.1: include license file in source distribution
- 1.4.0: print() enrichment can now be disabled (locally and globally), exhibits now have a real time fps indicator, new exhibit functions
show_bars, new utility
show_barsgains some advanced demonstrations (try it again!)
- 1.3.3: further improve stream compatibility with isatty
- 1.3.2: beautifully finalize bar in case of unexpected errors
- 1.3.1: fix a subtle race condition that could leave artifacts if ended very fast, flush print buffer when position changes or bar terminates, keep total argument from unexpected types
- 1.3.0: new fps calibration system, support force_tty and manual options in global configuration, multiple increment support in bar handler
- 1.2.0: filled blanks bar styles, clean underflow representation of filled blanks
- 1.1.1: optional percentage in manual mode
- 1.1.0: new manual mode
- 1.0.1: pycharm console support with force_tty, improve compatibility with python stdio streams
- 1.0.0: first public release, already very complete and mature
This software is licensed under the MIT License. See the LICENSE file in the top distribution directory for the full license text.
Did you like it?
Thank you for your interest!
I've put much ❤️ and effort into this. If you've appreciated my work and would like me to continue improving it, you could buy me a coffee! I would really appreciate that 😊! (the button is on the top-right corner) Thank you!
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size alive_progress-1.6.1-py3-none-any.whl (37.5 kB)||File type Wheel||Python version py3||Upload date||Hashes View|
|Filename, size alive-progress-1.6.1.tar.gz (41.4 kB)||File type Source||Python version None||Upload date||Hashes View|
Hashes for alive_progress-1.6.1-py3-none-any.whl