ProgIter documentation¶
ProgIter lets you measure and print the progress of an iterative process. This can be done either via an iterable interface or using the manual API. Using the iterable interface is most common.
ProgIter is unthreaded. This differentiates it from tqdm and rich.progress which use a threaded implementation. The choice of implementation has different tradeoffs and neither is strictly better than the other. An unthreaded progress bar provides synchronous uncluttered logging, increased stability, and — unintuitively —- speed (due to Python’s GIL). Meanwhile threaded progress bars are more responsive, able to update multiple stdout lines at a time, and can look prettier (unless you try to log stdout to disk).
ProgIter was originally developed independently of tqdm
, but the newer
versions of this library have been designed to be compatible with tqdm-API.
ProgIter
is now a (mostly) drop-in alternative to tqdm. The tqdm
library may be more appropriate in some cases. The main advantage of ProgIter
is that it does not use any python threading, and therefore can be safer with
code that makes heavy use of multiprocessing.
The reason
for this is that threading before forking may cause locks to be duplicated
across processes, which may lead to deadlocks.
ProgIter is simpler than tqdm, which may be desirable for some applications. However, this also means ProgIter is not as extensible as tqdm. If you want a pretty bar or need something fancy, use tqdm (or rich); if you want useful information about your iteration by default, use progiter.
Package level documentation can be found at: https://progiter.readthedocs.io/en/latest/
Example
The basic usage of ProgIter is simple and intuitive: wrap a python iterable.
The following example wraps a range
iterable and reports progress to stdout
as the iterable is consumed. The ProgIter
object accepts various keyword
arguments to modify the details of how progress is measured and reported. See
API documentation of the ProgIter
class here:
https://progiter.readthedocs.io/en/latest/progiter.progiter.html#progiter.progiter.ProgIter
>>> from progiter import ProgIter
>>> def is_prime(n):
... return n >= 2 and not any(n % i == 0 for i in range(2, n))
>>> for n in ProgIter(range(1000), verbose=2):
>>> # do some work
>>> is_prime(n)
0.00% 0/1000... rate=0 Hz, eta=?, total=0:00:00
0.60% 6/1000... rate=76995.12 Hz, eta=0:00:00, total=0:00:00
100.00% 1000/1000... rate=266488.22 Hz, eta=0:00:00, total=0:00:00
A Progress Iterator
ProgIter lets you measure and print the progress of an iterative process. This can be done either via an iterable interface or using the manual API. Using the iterable interface is most common.
The basic usage of ProgIter is simple and intuitive. Just wrap a python
iterable. The following example wraps a range
iterable and prints reported
progress to stdout as the iterable is consumed.
Example
>>> for n in ProgIter(range(1000)):
>>> # do some work
>>> pass
Note that by default ProgIter reports information about iteration-rate, fraction-complete, estimated time remaining, time taken so far, and the current wall time.
Example
>>> # xdoctest: +IGNORE_WANT
>>> def is_prime(n):
... return n >= 2 and not any(n % i == 0 for i in range(2, n))
>>> for n in ProgIter(range(1000), verbose=1):
>>> # do some work
>>> is_prime(n)
1000/1000... rate=114326.51 Hz, eta=0:00:00, total=0:00:00
For more complex applications is may sometimes be desirable to manually use the ProgIter API. This is done as follows:
Example
>>> # xdoctest: +IGNORE_WANT
>>> n = 3
>>> prog = ProgIter(desc='manual', total=n, verbose=3)
>>> prog.begin() # Manually begin progress iteration
>>> for _ in range(n):
... prog.step(inc=1) # specify the number of steps to increment
>>> prog.end() # Manually end progress iteration
manual 0/3... rate=0 Hz, eta=?, total=0:00:00
manual 1/3... rate=14454.63 Hz, eta=0:00:00, total=0:00:00
manual 2/3... rate=17485.42 Hz, eta=0:00:00, total=0:00:00
manual 3/3... rate=21689.78 Hz, eta=0:00:00, total=0:00:00
When working with ProgIter in either iterable or manual mode you can use the
prog.ensure_newline
method to guarantee that the next call you make to
stdout will start on a new line. You can also use the prog.set_extra
method
to update a dynamci “extra” message that is shown in the formatted output. The
following example demonstrates this.
Example
>>> # xdoctest: +IGNORE_WANT
>>> def is_prime(n):
... return n >= 2 and not any(n % i == 0 for i in range(2, n))
>>> _iter = range(1000)
>>> prog = ProgIter(_iter, desc='check primes', verbose=2, show_wall=True)
>>> for n in prog:
>>> if n == 97:
>>> print('!!! Special print at n=97 !!!')
>>> if is_prime(n):
>>> prog.set_extra('Biggest prime so far: {}'.format(n))
>>> prog.ensure_newline()
check primes 0/1000... rate=0 Hz, eta=?, total=0:00:00, wall=2020-10-23 17:27 EST
check primes 1/1000... rate=95547.49 Hz, eta=0:00:00, total=0:00:00, wall=2020-10-23 17:27 EST
check primes 4/1000...Biggest prime so far: 3 rate=41062.28 Hz, eta=0:00:00, total=0:00:00, wall=2020-10-23 17:27 EST
check primes 16/1000...Biggest prime so far: 13 rate=85340.61 Hz, eta=0:00:00, total=0:00:00, wall=2020-10-23 17:27 EST
check primes 64/1000...Biggest prime so far: 61 rate=164739.98 Hz, eta=0:00:00, total=0:00:00, wall=2020-10-23 17:27 EST
!!! Special print at n=97 !!!
check primes 256/1000...Biggest prime so far: 251 rate=206287.91 Hz, eta=0:00:00, total=0:00:00, wall=2020-10-23 17:27 EST
check primes 512/1000...Biggest prime so far: 509 rate=165271.92 Hz, eta=0:00:00, total=0:00:00, wall=2020-10-23 17:27 EST
check primes 768/1000...Biggest prime so far: 761 rate=136480.12 Hz, eta=0:00:00, total=0:00:00, wall=2020-10-23 17:27 EST
check primes 1000/1000...Biggest prime so far: 997 rate=115214.95 Hz, eta=0:00:00, total=0:00:00, wall=2020-10-23 17:27 EST