diff options
author | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-03-02 08:20:07 +0000 |
---|---|---|
committer | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-03-02 08:20:07 +0000 |
commit | 3d2c9fd003c14a4969f383cd5eb0966b7b6a3d7b (patch) | |
tree | 96212b1fc6b9515e6bb63a5fc7869cb1da01d36d /DEMO.ipynb | |
download | tqdm-3d2c9fd003c14a4969f383cd5eb0966b7b6a3d7b.tar.xz tqdm-3d2c9fd003c14a4969f383cd5eb0966b7b6a3d7b.zip |
Adding upstream version 4.64.1.upstream/4.64.1
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'DEMO.ipynb')
-rw-r--r-- | DEMO.ipynb | 890 |
1 files changed, 890 insertions, 0 deletions
diff --git a/DEMO.ipynb b/DEMO.ipynb new file mode 100644 index 0000000..b892af4 --- /dev/null +++ b/DEMO.ipynb @@ -0,0 +1,890 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "<h1 align=\"center\">tqdm</h1>\n", + "<img src=\"https://img.tqdm.ml/logo.gif\" align=\"left\" />\n", + "\n", + "[![Py-Versions](https://img.shields.io/pypi/pyversions/tqdm.svg?logo=python&logoColor=white)](https://pypi.org/project/tqdm)|[![Versions](https://img.shields.io/pypi/v/tqdm.svg)](https://tqdm.github.io/releases)|[![Conda-Forge-Status](https://img.shields.io/conda/v/conda-forge/tqdm.svg?label=conda-forge&logo=conda-forge)](https://anaconda.org/conda-forge/tqdm)|[![Docker](https://img.shields.io/badge/docker-pull-blue.svg?logo=docker&logoColor=white)](https://hub.docker.com/r/tqdm/tqdm)|[![Snapcraft](https://img.shields.io/badge/snap-install-82BEA0.svg?logo=snapcraft)](https://snapcraft.io/tqdm)\n", + "-|-|-|-|-\n", + "\n", + "[![Build-Status](https://img.shields.io/github/workflow/status/tqdm/tqdm/Test/master?logo=GitHub)](https://github.com/tqdm/tqdm/actions?query=workflow%3ATest)|[![Coverage-Status](https://img.shields.io/coveralls/github/tqdm/tqdm/master?logo=coveralls)](https://coveralls.io/github/tqdm/tqdm)|[![Branch-Coverage-Status](https://codecov.io/gh/tqdm/tqdm/branch/master/graph/badge.svg)](https://codecov.io/gh/tqdm/tqdm)|[![Codacy-Grade](https://app.codacy.com/project/badge/Grade/3f965571598f44549c7818f29cdcf177)](https://www.codacy.com/gh/tqdm/tqdm/dashboard)|[![Libraries-Rank](https://img.shields.io/librariesio/sourcerank/pypi/tqdm.svg?logo=koding&logoColor=white)](https://libraries.io/pypi/tqdm)|[![PyPI-Downloads](https://img.shields.io/pypi/dm/tqdm.svg?label=pypi%20downloads&logo=PyPI&logoColor=white)](https://pepy.tech/project/tqdm)\n", + "-|-|-|-|-|-\n", + "\n", + "[![DOI](https://img.shields.io/badge/DOI-10.5281/zenodo.595120-blue.svg)](https://doi.org/10.5281/zenodo.595120)|[![LICENCE](https://img.shields.io/pypi/l/tqdm.svg)](https://raw.githubusercontent.com/tqdm/tqdm/master/LICENCE)|[![OpenHub-Status](https://www.openhub.net/p/tqdm/widgets/project_thin_badge?format=gif)](https://www.openhub.net/p/tqdm?ref=Thin+badge)|[![binder-demo](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tqdm/tqdm/master?filepath=DEMO.ipynb)|[![awesome-python](https://awesome.re/mentioned-badge.svg)](https://github.com/vinta/awesome-python)\n", + "-|-|-|-|-\n", + "\n", + "`tqdm` derives from the Arabic word *taqaddum* (تقدّم) which can mean\n", + "\"progress,\" and is an abbreviation for \"I love you so much\" in Spanish\n", + "(*te quiero demasiado*).\n", + "\n", + "Instantly make your loops show a smart progress meter - just wrap any\n", + "iterable with `tqdm(iterable)`, and you're done!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from tqdm import tqdm\n", + "for i in tqdm(range(10000)):\n", + " pass" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`trange(N)` can be also used as a convenient shortcut for\n", + "`tqdm(xrange(N))`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from tqdm import trange\n", + "for i in trange(10000, unit_scale=True, desc=\"hello\", unit=\"epoch\"):\n", + " pass" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![Screenshot](https://img.tqdm.ml/tqdm.gif)|[![Video](https://img.tqdm.ml/video.jpg)](https://tqdm.github.io/video) [![Slides](https://img.tqdm.ml/slides.jpg)](https://tqdm.github.io/PyData2019/slides.html) [![Merch](https://img.tqdm.ml/merch.jpg)](https://tqdm.github.io/merch)\n", + "-|-\n", + "\n", + "It can also be executed as a module with pipes:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "! seq 9999999 | tqdm --bytes | wc -l" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "```sh\n", + "tar -zcf - docs/ | tqdm --bytes --total `du -sb docs/ | cut -f1` > backup.tgz\n", + " 44%|██████████████▊ | 153M/352M [00:14<00:18, 11.0MB/s]\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Overhead is low -- about 60ns per iteration (80ns with `tqdm.gui`), and\n", + "is unit tested against performance regression. By comparison, the\n", + "well-established\n", + "[ProgressBar](https://github.com/niltonvolpato/python-progressbar) has\n", + "an 800ns/iter overhead.\n", + "\n", + "In addition to its low overhead, `tqdm` uses smart algorithms to predict\n", + "the remaining time and to skip unnecessary iteration displays, which\n", + "allows for a negligible overhead in most cases.\n", + "\n", + "`tqdm` works on any platform (Linux, Windows, Mac, FreeBSD, NetBSD,\n", + "Solaris/SunOS), in any console or in a GUI, and is also friendly with\n", + "IPython/Jupyter notebooks.\n", + "\n", + "`tqdm` does not require any dependencies (not even `curses`!), just\n", + "Python and an environment supporting `carriage return \\r` and\n", + "`line feed \\n` control characters.\n", + "\n", + "---\n", + "\n", + "## Usage\n", + "\n", + "`tqdm` is very versatile and can be used in a number of ways.\n", + "The three main ones are given below.\n", + "\n", + "### Iterable-based\n", + "\n", + "Wrap `tqdm()` around any iterable:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from tqdm import tqdm\n", + "from time import sleep" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "text = \"\"\n", + "for char in tqdm([\"a\", \"b\", \"c\", \"d\"]):\n", + " sleep(0.25)\n", + " text = text + char" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`trange(i)` is a special optimised instance of `tqdm(range(i))`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from tqdm import trange\n", + "\n", + "for i in trange(100):\n", + " sleep(0.01)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Instantiation outside of the loop allows for manual control over `tqdm()`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "pbar = tqdm([\"a\", \"b\", \"c\", \"d\"])\n", + "for char in pbar:\n", + " sleep(0.25)\n", + " pbar.set_description(\"Processing %s\" % char)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Manual\n", + "\n", + "Manual control of `tqdm()` updates using a `with` statement:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "with tqdm(total=100) as pbar:\n", + " for i in range(10):\n", + " sleep(0.1)\n", + " pbar.update(10)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If the optional variable `total` (or an iterable with `len()`) is\n", + "provided, predictive stats are displayed.\n", + "\n", + "`with` is also optional (you can just assign `tqdm()` to a variable,\n", + "but in this case don't forget to `del` or `close()` at the end:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "pbar = tqdm(total=100)\n", + "for i in range(10):\n", + " sleep(0.1)\n", + " pbar.update(10)\n", + "pbar.close()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Module" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Perhaps the most wonderful use of `tqdm` is in a script or on the\n", + "command line. Simply inserting `tqdm` (or `python -m tqdm`) between\n", + "pipes will pass through all `stdin` to `stdout` while printing progress\n", + "to `stderr`.\n", + "\n", + "The example below demonstrated counting the number of lines in all\n", + "Python files in the current directory, with timing information included." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "! time find . -name '*.py' -type f -exec cat \\{} \\; | wc -l" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "! time find . -name '*.py' -type f -exec cat \\{} \\; | tqdm | wc -l" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Note that the usual arguments for `tqdm` can also be specified." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "! find . -name '*.py' -type f -exec cat \\{} \\; | tqdm --unit loc --unit-scale --total 4104300 --null" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Backing up a large directory?\n", + "\n", + "```sh\n", + "tar -zcf - docs/ | tqdm --bytes --total `du -sb docs/ | cut -f1` > backup.tgz\n", + " 44%|██████████████▊ | 153M/352M [00:14<00:18, 11.0MB/s]\n", + "```\n", + "\n", + "This can be beautified further:\n", + "\n", + "```sh\n", + "BYTES=\"$(du -sb docs/ | cut -f1)\"\n", + "tar -cf - docs/ \\\n", + " | tqdm --bytes --total \"$BYTES\" --desc Processing | gzip \\\n", + " | tqdm --bytes --total \"$BYTES\" --desc Compressed --position 1 \\\n", + " > ~/backup.tgz\n", + "Processing: 100%|██████████████████████| 352M/352M [00:14<00:00, 30.2MB/s]\n", + "Compressed: 42%|█████████▎ | 148M/352M [00:14<00:19, 10.9MB/s]\n", + "```\n", + "\n", + "Or done on a file level using 7-zip:\n", + "\n", + "```sh\n", + "7z a -bd -r backup.7z docs/ | grep Compressing \\\n", + " | tqdm --total $(find docs/ -type f | wc -l) --unit files \\\n", + " | grep -v Compressing\n", + "100%|██████████████████████████▉| 15327/15327 [01:00<00:00, 712.96files/s]\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Documentation" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "tqdm?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "! tqdm --help" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Examples and Advance Usage\n", + "\n", + "- See the [examples](https://github.com/tqdm/tqdm/tree/master/examples)\n", + " folder;\n", + "- import the module and run `help()`;\n", + "- consult the [wiki](https://github.com/tqdm/tqdm/wiki)\n", + " - this has an\n", + " [excellent article](https://github.com/tqdm/tqdm/wiki/How-to-make-a-great-Progress-Bar)\n", + " on how to make a **great** progressbar;\n", + "- check out the [slides from PyData London](https://tqdm.github.io/PyData2019/slides.html), or\n", + "- run this file!\n", + "\n", + "### Description and additional stats\n", + "\n", + "Custom information can be displayed and updated dynamically on `tqdm` bars\n", + "with the `desc` and `postfix` arguments:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from tqdm import tqdm, trange\n", + "from random import random, randint\n", + "from time import sleep\n", + "\n", + "with trange(10) as t:\n", + " for i in t:\n", + " # Description will be displayed on the left\n", + " t.set_description('GEN %i' % i)\n", + " # Postfix will be displayed on the right,\n", + " # formatted automatically based on argument's datatype\n", + " t.set_postfix(loss=random(), gen=randint(1,999), str='h',\n", + " lst=[1, 2])\n", + " sleep(0.1)\n", + "\n", + "with tqdm(total=10, bar_format=\"{postfix[0]} {postfix[1][value]:>8.2g}\",\n", + " postfix=[\"Batch\", dict(value=0)]) as t:\n", + " for i in range(10):\n", + " sleep(0.1)\n", + " t.postfix[1][\"value\"] = i / 2\n", + " t.update()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Points to remember when using `{postfix[...]}` in the `bar_format` string:\n", + "\n", + "- `postfix` also needs to be passed as an initial argument in a\n", + " compatible format, and\n", + "- `postfix` will be auto-converted to a string if it is a `dict`-like\n", + " object. To prevent this behaviour, insert an extra item into the\n", + " dictionary where the key is not a string.\n", + "\n", + "Additional `bar_format` parameters may also be defined by overriding\n", + "`format_dict`, and the bar itself may be modified using `ascii`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from tqdm import tqdm\n", + "from time import sleep\n", + "\n", + "class TqdmExtraFormat(tqdm):\n", + " \"\"\"Provides a `total_time` format parameter\"\"\"\n", + " @property\n", + " def format_dict(self):\n", + " d = super(TqdmExtraFormat, self).format_dict\n", + " total_time = d[\"elapsed\"] * (d[\"total\"] or 0) / max(d[\"n\"], 1)\n", + " d.update(total_time=self.format_interval(total_time) + \" in total\")\n", + " return d\n", + "\n", + "for i in TqdmExtraFormat(\n", + " range(9), ascii=\" .oO0\",\n", + " bar_format=\"{total_time}: {percentage:.0f}%|{bar}{r_bar}\"):\n", + " if i == 4:\n", + " break" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Note that `{bar}` also supports a format specifier `[width][type]`.\n", + "\n", + "- `width`\n", + " + unspecified (default): automatic to fill `ncols`\n", + " + `int >= 0`: fixed width overriding `ncols` logic\n", + " + `int < 0`: subtract from the automatic default\n", + "- `type`\n", + " + `a`: ascii (`ascii=True` override)\n", + " + `u`: unicode (`ascii=False` override)\n", + " + `b`: blank (`ascii=\" \"` override)\n", + "\n", + "This means a fixed bar with right-justified text may be created by\n", + "using: `bar_format=\"{l_bar}{bar:10}|{bar:-10b}right-justified\"`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Nested progress bars\n", + "\n", + "`tqdm` supports nested progress bars. Here's an example:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from tqdm.auto import trange\n", + "from time import sleep\n", + "\n", + "for i in trange(4, desc='1st loop'):\n", + " for j in trange(5, desc='2nd loop'):\n", + " for k in trange(50, desc='3rd loop', leave=False):\n", + " sleep(0.01)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For manual control over positioning (e.g. for multi-processing use),\n", + "you may specify `position=n` where `n=0` for the outermost bar, `n=1`\n", + "for the next, and so on. However, it's best to check if tqdm can work\n", + "without manual position first.\n", + "\n", + "```python\n", + "from time import sleep\n", + "from tqdm import trange, tqdm\n", + "from multiprocessing import Pool, RLock, freeze_support\n", + "\n", + "L = list(range(9))\n", + "\n", + "def progresser(n):\n", + " interval = 0.001 / (n + 2)\n", + " total = 5000\n", + " text = \"#{}, est. {:<04.2}s\".format(n, interval * total)\n", + " for _ in trange(total, desc=text, position=n):\n", + " sleep(interval)\n", + "\n", + "if __name__ == '__main__':\n", + " freeze_support() # for Windows support\n", + " tqdm.set_lock(RLock()) # for managing output contention\n", + " p = Pool(initializer=tqdm.set_lock, initargs=(tqdm.get_lock(),))\n", + " p.map(progresser, L)\n", + "```\n", + "\n", + "Note that in Python 3, `tqdm.write` is thread-safe:\n", + "\n", + "```python\n", + "from time import sleep\n", + "from tqdm import tqdm, trange\n", + "from concurrent.futures import ThreadPoolExecutor\n", + "\n", + "L = list(range(9))\n", + "\n", + "def progresser(n):\n", + " interval = 0.001 / (n + 2)\n", + " total = 5000\n", + " text = \"#{}, est. {:<04.2}s\".format(n, interval * total)\n", + " for _ in trange(total, desc=text):\n", + " sleep(interval).auto\n", + " if n == 6:\n", + " tqdm.write(\"n == 6 completed.\")\n", + " tqdm.write(\"`tqdm.write()` is thread-safe in py3!\")\n", + "\n", + "if __name__ == '__main__':\n", + " with ThreadPoolExecutor() as p:\n", + " p.map(progresser, L)\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Hooks and callbacks\n", + "\n", + "`tqdm` can easily support callbacks/hooks and manual updates.\n", + "Here's an example with `urllib`:\n", + "\n", + "**`urllib.urlretrieve` documentation**\n", + "\n", + "> [...]\n", + "> If present, the hook function will be called once\n", + "> on establishment of the network connection and once after each block read\n", + "> thereafter. The hook will be passed three arguments; a count of blocks\n", + "> transferred so far, a block size in bytes, and the total size of the file.\n", + "> [...]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import urllib, os\n", + "from tqdm import tqdm\n", + "urllib = getattr(urllib, 'request', urllib)\n", + "\n", + "class TqdmUpTo(tqdm):\n", + " \"\"\"Provides `update_to(n)` which uses `tqdm.update(delta_n)`.\"\"\"\n", + " def update_to(self, b=1, bsize=1, tsize=None):\n", + " \"\"\"\n", + " b : int, optional\n", + " Number of blocks transferred so far [default: 1].\n", + " bsize : int, optional\n", + " Size of each block (in tqdm units) [default: 1].\n", + " tsize : int, optional\n", + " Total size (in tqdm units). If [default: None] remains unchanged.\n", + " \"\"\"\n", + " if tsize is not None:\n", + " self.total = tsize\n", + " return self.update(b * bsize - self.n) # also sets self.n = b * bsize\n", + "\n", + "eg_link = \"https://caspersci.uk.to/matryoshka.zip\"\n", + "with TqdmUpTo(unit='B', unit_scale=True, unit_divisor=1024, miniters=1,\n", + " desc=eg_link.split('/')[-1]) as t: # all optional kwargs\n", + " urllib.urlretrieve(eg_link, filename=os.devnull,\n", + " reporthook=t.update_to, data=None)\n", + " t.total = t.n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Inspired by [twine#242](https://github.com/pypa/twine/pull/242).\n", + "Functional alternative in\n", + "[examples/tqdm_wget.py](https://github.com/tqdm/tqdm/blob/master/examples/tqdm_wget.py).\n", + "\n", + "It is recommend to use `miniters=1` whenever there is potentially large\n", + "differences in iteration speed (e.g. downloading a file over a patchy\n", + "connection).\n", + "\n", + "**Wrapping read/write methods**\n", + "\n", + "To measure throughput through a file-like object's `read` or `write`\n", + "methods, use `CallbackIOWrapper`:\n", + "\n", + "```python\n", + "from tqdm import tqdm\n", + "from tqdm.utils import CallbackIOWrapper\n", + "\n", + "with tqdm(total=file_obj.size,\n", + " unit='B', unit_scale=True, unit_divisor=1024) as t:\n", + " fobj = CallbackIOWrapper(t.update, file_obj, \"read\")\n", + " while True:\n", + " chunk = fobj.read(chunk_size)\n", + " if not chunk:\n", + " break\n", + " t.reset()\n", + " # ... continue to use `t` for something else\n", + "```\n", + "\n", + "Alternatively, use the even simpler `wrapattr` convenience function,\n", + "which would condense both the `urllib` and `CallbackIOWrapper` examples\n", + "down to:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import urllib, os\n", + "from tqdm import tqdm\n", + "\n", + "eg_link = \"https://caspersci.uk.to/matryoshka.zip\"\n", + "response = getattr(urllib, 'request', urllib).urlopen(eg_link)\n", + "with tqdm.wrapattr(open(os.devnull, \"wb\"), \"write\",\n", + " miniters=1, desc=eg_link.split('/')[-1],\n", + " total=getattr(response, 'length', None)) as fout:\n", + " for chunk in response:\n", + " fout.write(chunk)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `requests` equivalent is nearly identical:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import requests, os\n", + "from tqdm import tqdm\n", + "\n", + "eg_link = \"https://caspersci.uk.to/matryoshka.zip\"\n", + "response = requests.get(eg_link, stream=True)\n", + "with tqdm.wrapattr(open(os.devnull, \"wb\"), \"write\",\n", + " miniters=1, desc=eg_link.split('/')[-1],\n", + " total=int(response.headers.get('content-length', 0))) as fout:\n", + " for chunk in response.iter_content(chunk_size=4096):\n", + " fout.write(chunk)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Pandas Integration\n", + "\n", + "Due to popular demand we've added support for `pandas` -- here's an example\n", + "for `DataFrame.progress_apply` and `DataFrameGroupBy.progress_apply`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import pandas as pd\n", + "import numpy as np\n", + "from tqdm import tqdm\n", + "\n", + "df = pd.DataFrame(np.random.randint(0, 100, (100000, 6)))\n", + "\n", + "# Register `pandas.progress_apply` and `pandas.Series.map_apply` with `tqdm`\n", + "# (can use `tqdm.gui.tqdm`, `tqdm.notebook.tqdm`, optional kwargs, etc.)\n", + "tqdm.pandas(desc=\"my bar!\")\n", + "\n", + "# Now you can use `progress_apply` instead of `apply`\n", + "# and `progress_map` instead of `map`\n", + "df.progress_apply(lambda x: x**2)\n", + "# can also groupby:\n", + "# df.groupby(0).progress_apply(lambda x: x**2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In case you're interested in how this works (and how to modify it for\n", + "your own callbacks), see the\n", + "[examples](https://github.com/tqdm/tqdm/tree/master/examples) folder or\n", + "import the module and run `help()`.\n", + "\n", + "### Keras Integration\n", + "\n", + "A `keras` callback is also available:\n", + "\n", + "```python\n", + "from tqdm.keras import TqdmCallback\n", + "\n", + "...\n", + "\n", + "model.fit(..., verbose=0, callbacks=[TqdmCallback()])\n", + "```\n", + "\n", + "### IPython/Jupyter Integration\n", + "\n", + "IPython/Jupyter is supported via the `tqdm.notebook` submodule:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from tqdm.notebook import trange, tqdm\n", + "from time import sleep\n", + "\n", + "for i in trange(3, desc='1st loop'):\n", + " for j in tqdm(range(100), desc='2nd loop'):\n", + " sleep(0.01)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In addition to `tqdm` features, the submodule provides a native Jupyter\n", + "widget (compatible with IPython v1-v4 and Jupyter), fully working nested\n", + "bars and colour hints (blue: normal, green: completed, red:\n", + "error/interrupt, light blue: no ETA); as demonstrated below.\n", + "\n", + "![Screenshot-Jupyter3](https://img.tqdm.ml/jupyter-3.gif)\n", + "\n", + "The `notebook` version supports percentage or pixels for overall width\n", + "(e.g.: `ncols='100%'` or `ncols='480px'`).\n", + "\n", + "It is also possible to let `tqdm` automatically choose between console\n", + "or notebook versions by using the `autonotebook` submodule:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from tqdm.autonotebook import tqdm\n", + "tqdm.pandas()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Note that this will issue a `TqdmExperimentalWarning` if run in a\n", + "notebook since it is not meant to be possible to distinguish between\n", + "`jupyter notebook` and `jupyter console`. Use `auto` instead of\n", + "`autonotebook` to suppress this warning.\n", + "\n", + "Note that notebooks will display the bar in the cell where it was\n", + "created. This may be a different cell from the one where it is used. If\n", + "this is not desired, the creation of the bar must be delayed/moved to\n", + "the cell where it is desired to be displayed.\n", + "\n", + "Another possibility is to have a single bar (near the top of the\n", + "notebook) which is constantly re-used (using `reset()` rather than\n", + "`close()`). For this reason, the notebook version (unlike the CLI\n", + "version) does not automatically call `close()` upon `Exception`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from tqdm.notebook import tqdm\n", + "pbar = tqdm()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# different cell\n", + "iterable = range(100)\n", + "pbar.reset(total=len(iterable)) # initialise with new `total`\n", + "for i in iterable:\n", + " pbar.update()\n", + "pbar.refresh() # force print final status but don't `close()`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Writing messages\n", + "\n", + "This is a work in progress (see\n", + "[#737](https://github.com/tqdm/tqdm/issues/737)).\n", + "\n", + "Since `tqdm` uses a simple printing mechanism to display progress bars,\n", + "you should not write any message in the terminal using `print()` while a\n", + "progressbar is open.\n", + "\n", + "To write messages in the terminal without any collision with `tqdm` bar\n", + "display, a `.write()` method is provided:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from tqdm.auto import tqdm, trange\n", + "from time import sleep\n", + "\n", + "bar = trange(10)\n", + "for i in bar:\n", + " # Print using tqdm class method .write()\n", + " sleep(0.1)\n", + " if not (i % 3):\n", + " tqdm.write(\"Done task %i\" % i)\n", + " # Can also use bar.write()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "By default, this will print to standard output `sys.stdout`. but you can\n", + "specify any file-like object using the `file` argument. For example,\n", + "this can be used to redirect the messages writing to a log file or class.\n", + "\n", + "[![README-Hits](https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&style=social&r=https://github.com/tqdm/tqdm&l=https://img.tqdm.ml/favicon.png&f=https://img.tqdm.ml/logo.gif)](https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&a=plot&r=https://github.com/tqdm/tqdm&l=https://img.tqdm.ml/favicon.png&f=https://img.tqdm.ml/logo.gif&style=social)|(Since 19 May 2016)\n", + "-|-" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Do your own experiments here 👇\n", + "\n", + "Try `tqdm` youself by adding your code below and running your own experiments." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import tqdm\n", + "\n", + "# your code here\n", + "tqdm." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython" + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} |