5-laziness.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Laziness\n",
    "\n",
    "*加速工作的效率和规模化*\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We build a somewhat large list of integers\n",
    "\n",
    "integers = range(10000000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这是10,000,000个数字。 不是很大，但也不小。 我们需要小心，这些整数需要蛮大的资源代价。\n",
    "\n",
    "素数计算测试是昂贵的。 让我们来做！\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sympy # don't worry if you don't have this\n",
    "from toolz import filter\n",
    "primes = filter(sympy.isprime, integers)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Q: *我认为测试素数计算是相当昂贵的。 为什么这么快就回来？\n",
    "\n",
    "A: The `toolz.filter` is lazy.  没有进行实际的工作。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<filter at 0x1fc3be0>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "primes  # not actually a list of primes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然而，我们仍然可以将变量primes看作一个素数列表。 让我们打印前几个"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "next(primes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "next(primes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "next(primes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "'filter' object is not subscriptable",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-8-d85f794975ab>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[1;31m# Note that primes is not a list like integers.  We can't index into it\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      2\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 3\u001b[1;33m \u001b[0mprimes\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;36m1000\u001b[0m\u001b[1;33m]\u001b[0m \u001b[1;31m# 1000th prime please\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mTypeError\u001b[0m: 'filter' object is not subscriptable"
     ]
    }
   ],
   "source": [
    "# Note that primes is not a list like integers.  We can't index into it\n",
    "\n",
    "primes[1000] # 1000th prime please"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "109\n"
     ]
    }
   ],
   "source": [
    "# But we *can* still iterate over it\n",
    "\n",
    "for p in primes:\n",
    "    print(p)\n",
    "    if p > 100:  # stop us from going through the whole list\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "请注意，前几个素数2，3，5不包含在此列表中。 惰性迭代器中的项目在使用时消耗，永远不会再被看到。 如果你真的想存储迭代器，那就调用它的`list`构造函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "primes = list(primes)  # fully evaluate the lazy iterator, store it in a list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Infinite iterators\n",
    "\n",
    "懒惰执行可以理解为 我们在谈工作，而没有真正在做它。 这种与现实的分离开辟了新的可能性。 例如，这里是所有斐波那契数字"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fib():\n",
    "    \"\"\" A generator for all Fibonacci numbers \"\"\"\n",
    "    a, b = 0, 1\n",
    "    while True:\n",
    "        yield a\n",
    "        a, b = b, a + b\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "1\n",
      "2\n"
     ]
    }
   ],
   "source": [
    "fibs = fib()  # make a lazy iterator, fibs\n",
    "print(next(fibs))\n",
    "print(next(fibs))\n",
    "print(next(fibs))\n",
    "print(next(fibs))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "'generator' object is not subscriptable",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-21-aaafa5ea4530>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mfibs\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;36m100\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mTypeError\u001b[0m: 'generator' object is not subscriptable"
     ]
    }
   ],
   "source": [
    "fibs[100]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Toolz functions\n",
    "\n",
    "Toolz提供了与惰性迭代器交互的函数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "from toolz import first, second, nth, drop, take"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "first(fib())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "second(fib())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "354224848179261915075"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nth(100, fib())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<itertools.islice at 0x7e9b9a8>"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "take(20, fib())  # Yet another iterator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0,\n",
       " 1,\n",
       " 1,\n",
       " 2,\n",
       " 3,\n",
       " 5,\n",
       " 8,\n",
       " 13,\n",
       " 21,\n",
       " 34,\n",
       " 55,\n",
       " 89,\n",
       " 144,\n",
       " 233,\n",
       " 377,\n",
       " 610,\n",
       " 987,\n",
       " 1597,\n",
       " 2584,\n",
       " 4181]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(take(20, fib()))  # But this one is finite, so we can expand it with list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "later_fibs = drop(100, fib())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "354224848179261915075"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "next(later_fibs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[55, 144, 377, 987, 2584]"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# These all depend on itertools.islice\n",
    "from itertools import islice\n",
    "list(islice(fib(), 10, 20, 2))  # fib()[10:20:1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 示例：将函数应用于文本\n",
    "\n",
    "Python文件对象是一个懒惰的迭代器。 当我打开一个大的文本文件时，Python实际上并没有一次读入所有的文本。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "book = open('data/tale-of-two-cities.txt')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each element of the book is a line of text.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 锘縏he Project Gutenberg EBook of A Tale of Two Cities, by Charles Dickens\n",
      "\n",
      "2 \n",
      "\n",
      "3 This eBook is for the use of anyone anywhere at no cost and with\n",
      "\n",
      "4 almost no restrictions whatsoever.  You may copy it, give it away or\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(1, next(book))\n",
    "print(2, next(book))\n",
    "print(3, next(book))\n",
    "print(4, next(book))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这是书Gutenberg的标题。 这大概是112行。 让我们开始翻书，然后放下标题。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'It was the best of times,\\n'"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "book = open('data/tale-of-two-cities.txt')\n",
    "book = drop(112, book)\n",
    "next(book)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这看起来很熟悉。 让我们从所有的行中删除 `\\r\\n`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from toolz import map  # toolz.map is lazy too"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'it was the worst of times,'"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "book = map(str.strip, book)\n",
    "next(book)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在语义上，就好像我们将“str.strip”应用于书中的每一行\n",
    "\n",
    "实际上我们没有做任何工作。\n",
    "\n",
    "相反，`map`返回了一个新的惰性迭代器，它从原始的`book`中绘制一个元素，应用`str.strip`，然后得到结果。 只有当我们要求时才这样做。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以将懒惰操作链接在一起。 剥去每一行后，我们将把重点放在那些包含单词“Mr.” 的行上，或带有 “Miss” 和 \"Mrs.\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'\"Yes, Mr. Lorry.\"'"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def good_line(line):\n",
    "    return \"Mr.\" in line or \"Miss\" in line or \"Mrs.\" in line\n",
    "\n",
    "book = filter(good_line, book)\n",
    "next(book)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 \"I know this messenger, guard,\" said Mr. Lorry, getting down into the\n",
      "1 like a larger dog-kennel. Mr. Lorry, the passenger, shaking himself out\n",
      "1 Mr. Lorry dropped off to sleep. The arrival of his breakfast roused him,\n",
      "1 time to-day. She may ask for Mr. Jarvis Lorry, or she may only ask for a\n",
      "1 When Mr. Lorry had finished his breakfast, he went out for a stroll on\n",
      "1 again charged with mist and vapour, Mr. Lorry's thoughts seemed to cloud\n",
      "1 Mr. Lorry had been idle a long time, and had just poured out his last\n",
      "1 In a very few minutes the waiter came in to announce that Miss Manette\n",
      "1 Miss Manette had taken some refreshment on the road, and required none\n",
      "1 wig at the ears, and follow the waiter to Miss Manette's apartment.\n"
     ]
    }
   ],
   "source": [
    "# Lets take 10 lines from this iterator and display them\n",
    "\n",
    "for line in take(10, book):\n",
    "    print(1, line)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Motivation\n",
    "\n",
    "懒惰主要有两个原因\n",
    "\n",
    "1.避免无用的工作：我们经常在数据集的开头附近找到我们需要的东西。 在整个数据集上计算是浪费的\n",
    "\n",
    "2.内存：只有少量的数据集需要留在任何一点的内存。 这使我们能够传输非常大的数据集。\n",
    "\n",
    "在每种情况下，我们都接近传统的Python语法。 我们经常编写懒惰的代码而不知道它。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}