1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
|
---
title: '`tqdm`: A Fast, Extensible Progress Meter for Python and CLI'
tags:
- progressbar
- progressmeter
- progress-bar
- meter
- rate
- eta
- console
- terminal
- time
- progress
- bar
- gui
- python
- parallel
- cli
- utilities
- shell
- batch
authors:
- name: Casper O da Costa-Luis
orcid: 0000-0002-7211-1557
affiliation: 1
affiliations:
- name: "Independent (Non-affiliated)"
index: 1
date: 16 February 2019
bibliography: paper.bib
---
# Introduction
**`tqdm`** is a progress bar library designed to be fast and extensible. It is
written in Python, though ports in other languages are available. `tqdm` means
**progress** in Arabic (*taqadum* [@tqdm-ar]) and is an abbreviation for
**I love you so much** in Spanish (*te quiero demasiado* [@tqdm-es]).
It is a common programming problem to have iterative operations where progress
monitoring is desirable or advantageous. Including statements within a `for` loop to `print` out the current iteration number is a common strategy. However, there are many improvements which could be made in such a scenario:
- preventing excessive printing, such as only displaying every $n$^th^
iteration;
- displaying iteration rate;
- displaying elapsed and estimated completion times, and
- showing all of the above on one continuously updating line.
Addressing all these issues may well take up more developer time and effort than
the rest of the content of the loop. Any changes to iteration rates or attempts
to re-use the printing logic in a different loop may well result in suboptimal
display rates -- displaying every $n$^th^ iteration may be too (in)frequent --
requiring manual adjustment of $n$ to fix.
`tqdm` addresses all of these problems once and for all, taking advantage of
Pythonic patterns to make it a trivial task to add visually appealing,
customisable progress bars without any significant performance degradation even
in the most demanding of scenarios.
`tqdm` is intended to be used in frontends (giving end users a visual indication
of progress of computations or data transfer). It is also useful for developers
for debugging purposes, both as a profiling tool and also as a way of displaying
logging information of an iterative task (such as error during training of
machine learning algorithms). Due to its ease of use, the library is also an
ideal candidate for inclusion in Python educational courses. For general (not
necessarily Python) purposes, the command-line interface (CLI) mode further
presents a useful tool for CLI users and system administrators monitoring data
flow through pipes.
# Features
Exhaustive documentation may be found on the project's [home
page](https://github.com/tqdm/tqdm/#documentation).
The two basic use cases are within Python code and within a CLI:
## Python Iterable Wrapper
`tqdm`'s primary (and original) use is as a wrapper around Python iterables. A
simple case would be:
```python
from tqdm import tqdm
from time import sleep
for i in tqdm(range(100)):
sleep(0.1)
100%|#########################################| 100/100 [00:10<00:00, 9.95it/s]
```
Supported features include:
- Display customisation via arguments such as `desc`, `postfix` and `bar_format`
- Automatic limiting of display updates to avoid slowing down due to excessive
iteration rates [@stdout]
- Automatic detection of console width to fill the display
- Automatic use of Unicode to render smooth-filling progress bars on supported
terminals
- Support for custom rendering frontends, including:
* Command-line interface
* *Jupyter* HTML notebooks
* `matplotlib`
- Support for custom hooks/callbacks, including:
* `pandas`
* `keras` [@keras]
## Command-line Interface (CLI)
A CLI is also provided, where `tqdm` may be used a pipe:
```sh
# count lines of text in all *.txt files
$ cat *.txt | wc -l
1075075
# same but with continuously updating progress information
$ cat *.txt | python3 -m tqdm --unit loc --unit_scale | wc -l
1.08Mloc [00:07, 142kloc/s]
# same if `total` is known
$ cat *.txt | python3 -m tqdm --unit loc --unit_scale --total 1075075 | wc -l
100%|#####################################| 1.08/1.08M [00:07<00:00, 142kloc/s]
1075075
```
# Availability
The package supports both Python versions 2 and 3, and is available for download
via `conda` [@conda], `pip` [@pypi], `snap` [@snapcraft], `docker` [@docker],
and *Zenodo* [@zenodo].
Web-based Jupyter interactive demonstrations are also available
[@notebooks;@binder]
Unit tests are run at least weekly on cloud-based continuous integration
[@travis], with code style and security issues checked on
[Codacy](https://app.codacy.com/project/tqdm/tqdm/dashboard) [@code-review].
Coverage is reported on [Coveralls](https://coveralls.io/github/tqdm/tqdm) and
[Codecov](https://codecov.io/gh/tqdm/tqdm), and performance is monitored against
regression [@asv].
# Impact
As of January 2019, `tqdm` has accumulated over 20 million downloads
[@pypi-downloads], and 315 thousand code inclusions [@tqdm-results]. Dependants
of `tqdm` include 23 thousand repositories [@tqdm-dependents] and 7 thousand
libraries [@lib-io]. `tqdm` has a SourceRank of 22 [@sourcerank], placing it in
the world's top 20 Python packages as of early 2019 [@sourcerank-descending].
The source code of `tqdm` is hosted on GitHub, where it has received over 9
thousand stars [@stars;@stars-hist], and was top trending repository during a
period in December 2015 [@trend-hist]. The documentation has received over 500
thousand hits [@hits], with highest rates during weekdays. Historical reading
rates have also trended upwards at the end of holiday periods. This implies
widespread use in commercial and academic settings.
[OpenHub](https://www.openhub.net/p/tqdm) valuates the work according to the
constructive cost model (COCOMO) as being worth approximately $50,000.
The library has also been used in several textbooks [@miller;@boxel;@nandy] and
peer-reviewed scientific publications
[@stein;@cook;@madhikar;@palmer;@knight;@moriwaki;@jackson].
The [`tqdm` wiki](https://github.com/tqdm/tqdm/wiki) also lists other references
in public media.
# Licence
`tqdm`'s source code is OSS, and all versions are archived at the DOI
[10.5281/zenodo.595120](https://doi.org/10.5281/zenodo.595120). The primary
maintainer [Casper da Costa-Luis](https://github.com/casperdcl) releases
contributions under the terms of the MPLv2.0, while all other contributions are
released under the terms of the MIT licence [@licence].
# References
|
