regex.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 正则表达式"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 常见匹配模式"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| 模式| 描述|\n",
    "|----|----|\n",
    "| \\w\t| 匹配字母数字及下划线 |\n",
    "| \\W\t| 匹配非字母数字下划线 |\n",
    "| \\s\t| 匹配任意空白字符，等价于 [\\t\\n\\r\\f]. |\n",
    "| \\S\t| 匹配任意非空字符 |\n",
    "| \\d\t| 匹配任意数字，等价于 [0-9] |\n",
    "| \\D\t| 匹配任意非数字 |\n",
    "| \\A\t| 匹配字符串开始 |\n",
    "| \\Z\t| 匹配字符串结束，如果是存在换行，只匹配到换行前的结束字符串 |\n",
    "| \\z\t| 匹配字符串结束 |\n",
    "| \\G\t| 匹配最后匹配完成的位置 |\n",
    "| \\n | 匹配一个换行符 |\n",
    "| \\t | 匹配一个制表符 |\n",
    "| ^\t| 匹配字符串的开头 |\n",
    "| $\t| 匹配字符串的末尾。|\n",
    "| .\t| 匹配任意字符，除了换行符，当re.DOTALL标记被指定时，则可以匹配包括换行符的任意字符。|\n",
    "| [...]\t| 用来表示一组字符,单独列出：[amk] 匹配 'a'，'m'或'k' |\n",
    "| [^...]\t| 不在[]中的字符：[^abc] 匹配除了a,b,c之外的字符。| \n",
    "| *\t| 匹配0个或多个的表达式。|\n",
    "| +\t| 匹配1个或多个的表达式。|\n",
    "| ?\t| 匹配0个或1个由前面的正则表达式定义的片段，非贪婪方式| \n",
    "| {n}\t| 精确匹配n个前面表达式。|\n",
    "| {n, m} | 匹配 n 到 m 次由前面的正则表达式定义的片段，贪婪方式| \n",
    "| a&#124;b | 匹配a或b |\n",
    "| ( )\t| 匹配括号内的表达式，也表示一个组 |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## re.match\n",
    "re.match 尝试从字符串的起始位置匹配一个模式，如果不是起始位置匹配成功的话，match()就返回none。"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "re.match(pattern, string, flags=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 最常规的匹配"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "41\n",
      "<re.Match object; span=(0, 41), match='Hello 123 4567 World_This is a Regex Demo'>\n",
      "Hello 123 4567 World_This is a Regex Demo\n",
      "(0, 41)\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "content = 'Hello 123 4567 World_This is a Regex Demo'\n",
    "print(len(content))\n",
    "result = re.match('^Hello\\s\\d\\d\\d\\s\\d{4}\\s\\w{10}.*Demo$', content)\n",
    "print(result)\n",
    "print(result.group())\n",
    "print(result.span())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 泛匹配"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<re.Match object; span=(0, 41), match='Hello 123 4567 World_This is a Regex Demo'>\n",
      "Hello 123 4567 World_This is a Regex Demo\n",
      "(0, 41)\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "content = 'Hello 123 4567 World_This is a Regex Demo'\n",
    "result = re.match('^Hello.*Demo$', content)\n",
    "print(result)\n",
    "print(result.group())\n",
    "print(result.span())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 匹配目标"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<re.Match object; span=(0, 40), match='Hello 1234567 World_This is a Regex Demo'>\n",
      "1234567\n",
      "(0, 40)\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "content = 'Hello 1234567 World_This is a Regex Demo'\n",
    "result = re.match('^Hello\\s(\\d+)\\sWorld.*Demo$', content)\n",
    "print(result)\n",
    "print(result.group(1))\n",
    "print(result.span())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 贪婪匹配\n",
    "尽可能多的匹配 ``.*``会匹配尽可能多的字符"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<re.Match object; span=(0, 40), match='Hello 1234567 World_This is a Regex Demo'>\n",
      "7\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "content = 'Hello 1234567 World_This is a Regex Demo'\n",
    "result = re.match('^He.*(\\d+).*Demo$', content)\n",
    "print(result)\n",
    "print(result.group(1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 非贪婪匹配\n",
    "``?`` 会匹配0个或1个，可以用来限制贪婪匹配，（**常用**）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<re.Match object; span=(0, 40), match='Hello 1234567 World_This is a Regex Demo'>\n",
      "1234567\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "content = 'Hello 1234567 World_This is a Regex Demo'\n",
    "result = re.match('^He.*?(\\d+).*Demo$', content)\n",
    "print(result)\n",
    "print(result.group(1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 匹配模式\n",
    "``re.S`` 会匹配出换行符，若不加，则 ``.*`` 无法匹配 ``\\n``"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1234567\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "content = '''Hello 1234567 World_This\n",
    "is a Regex Demo\n",
    "'''\n",
    "result = re.match('^He.*?(\\d+).*?Demo$', content, re.S)\n",
    "print(result.group(1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 转义"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "content = 'price is $5.00'\n",
    "result = re.match('price is $5.00', content)\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<re.Match object; span=(0, 14), match='price is $5.00'>\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "content = 'price is $5.00'\n",
    "result = re.match('price is \\$5\\.00', content)\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "总结：尽量使用**泛匹配**、**使用括号**得到匹配目标、尽量使用**非贪婪模式**、有换行符就用**re.S**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## re.search\n",
    "re.search 扫描整个字符串并返回第一个成功的匹配。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "content = 'Extra stings Hello 1234567 World_This is a Regex Demo Extra stings'\n",
    "result = re.match('Hello.*?(\\d+).*?Demo', content)\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<re.Match object; span=(13, 53), match='Hello 1234567 World_This is a Regex Demo'>\n",
      "1234567\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "content = 'Extra stings Hello 1234567 World_This is a Regex Demo Extra stings'\n",
    "result = re.search('Hello.*?(\\d+).*?Demo', content)\n",
    "print(result)\n",
    "print(result.group(1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "总结：为匹配方便，能用 **search** 就不用 match"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 匹配演练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "齐秦 往事随风\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "html = '''<div id=\"songs-list\">\n",
    "    <h2 class=\"title\">经典老歌</h2>\n",
    "    <p class=\"introduction\">\n",
    "        经典老歌列表\n",
    "    </p>\n",
    "    <ul id=\"list\" class=\"list-group\">\n",
    "        <li data-view=\"2\">一路上有你</li>\n",
    "        <li data-view=\"7\">\n",
    "            <a href=\"/2.mp3\" singer=\"任贤齐\">沧海一声笑</a>\n",
    "        </li>\n",
    "        <li data-view=\"4\" class=\"active\">\n",
    "            <a href=\"/3.mp3\" singer=\"齐秦\">往事随风</a>\n",
    "        </li>\n",
    "        <li data-view=\"6\"><a href=\"/4.mp3\" singer=\"beyond\">光辉岁月</a></li>\n",
    "        <li data-view=\"5\"><a href=\"/5.mp3\" singer=\"陈慧琳\">记事本</a></li>\n",
    "        <li data-view=\"5\">\n",
    "            <a href=\"/6.mp3\" singer=\"邓丽君\"><i class=\"fa fa-user\"></i>但愿人长久</a>\n",
    "        </li>\n",
    "    </ul>\n",
    "</div>'''\n",
    "result = re.search('<li.*?active.*?singer=\"(.*?)\">(.*?)</a>', html, re.S)\n",
    "if result:\n",
    "    print(result.group(1), result.group(2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "任贤齐 沧海一声笑\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "html = '''<div id=\"songs-list\">\n",
    "    <h2 class=\"title\">经典老歌</h2>\n",
    "    <p class=\"introduction\">\n",
    "        经典老歌列表\n",
    "    </p>\n",
    "    <ul id=\"list\" class=\"list-group\">\n",
    "        <li data-view=\"2\">一路上有你</li>\n",
    "        <li data-view=\"7\">\n",
    "            <a href=\"/2.mp3\" singer=\"任贤齐\">沧海一声笑</a>\n",
    "        </li>\n",
    "        <li data-view=\"4\" class=\"active\">\n",
    "            <a href=\"/3.mp3\" singer=\"齐秦\">往事随风</a>\n",
    "        </li>\n",
    "        <li data-view=\"6\"><a href=\"/4.mp3\" singer=\"beyond\">光辉岁月</a></li>\n",
    "        <li data-view=\"5\"><a href=\"/5.mp3\" singer=\"陈慧琳\">记事本</a></li>\n",
    "        <li data-view=\"5\">\n",
    "            <a href=\"/6.mp3\" singer=\"邓丽君\">但愿人长久</a>\n",
    "        </li>\n",
    "    </ul>\n",
    "</div>'''\n",
    "result = re.search('<li.*?singer=\"(.*?)\">(.*?)</a>', html, re.S)\n",
    "if result:\n",
    "    print(result.group(1), result.group(2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "beyond 光辉岁月\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "html = '''<div id=\"songs-list\">\n",
    "    <h2 class=\"title\">经典老歌</h2>\n",
    "    <p class=\"introduction\">\n",
    "        经典老歌列表\n",
    "    </p>\n",
    "    <ul id=\"list\" class=\"list-group\">\n",
    "        <li data-view=\"2\">一路上有你</li>\n",
    "        <li data-view=\"7\">\n",
    "            <a href=\"/2.mp3\" singer=\"任贤齐\">沧海一声笑</a>\n",
    "        </li>\n",
    "        <li data-view=\"4\" class=\"active\">\n",
    "            <a href=\"/3.mp3\" singer=\"齐秦\">往事随风</a>\n",
    "        </li>\n",
    "        <li data-view=\"6\"><a href=\"/4.mp3\" singer=\"beyond\">光辉岁月</a></li>\n",
    "        <li data-view=\"5\"><a href=\"/5.mp3\" singer=\"陈慧琳\">记事本</a></li>\n",
    "        <li data-view=\"5\">\n",
    "            <a href=\"/6.mp3\" singer=\"邓丽君\">但愿人长久</a>\n",
    "        </li>\n",
    "    </ul>\n",
    "</div>'''\n",
    "result = re.search('<li.*?singer=\"(.*?)\">(.*?)</a>', html)\n",
    "if result:\n",
    "    print(result.group(1), result.group(2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## re.findall\n",
    "搜索字符串，以列表形式返回全部能匹配的子串。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[('/2.mp3', '任贤齐', '沧海一声笑'), ('/3.mp3', '齐秦', '往事随风'), ('/4.mp3', 'beyond', '光辉岁月'), ('/5.mp3', '陈慧琳', '记事本'), ('/6.mp3', '邓丽君', '但愿人长久')]\n",
      "<class 'list'>\n",
      "('/2.mp3', '任贤齐', '沧海一声笑')\n",
      "/2.mp3 任贤齐 沧海一声笑\n",
      "('/3.mp3', '齐秦', '往事随风')\n",
      "/3.mp3 齐秦 往事随风\n",
      "('/4.mp3', 'beyond', '光辉岁月')\n",
      "/4.mp3 beyond 光辉岁月\n",
      "('/5.mp3', '陈慧琳', '记事本')\n",
      "/5.mp3 陈慧琳 记事本\n",
      "('/6.mp3', '邓丽君', '但愿人长久')\n",
      "/6.mp3 邓丽君 但愿人长久\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "html = '''<div id=\"songs-list\">\n",
    "    <h2 class=\"title\">经典老歌</h2>\n",
    "    <p class=\"introduction\">\n",
    "        经典老歌列表\n",
    "    </p>\n",
    "    <ul id=\"list\" class=\"list-group\">\n",
    "        <li data-view=\"2\">一路上有你</li>\n",
    "        <li data-view=\"7\">\n",
    "            <a href=\"/2.mp3\" singer=\"任贤齐\">沧海一声笑</a>\n",
    "        </li>\n",
    "        <li data-view=\"4\" class=\"active\">\n",
    "            <a href=\"/3.mp3\" singer=\"齐秦\">往事随风</a>\n",
    "        </li>\n",
    "        <li data-view=\"6\"><a href=\"/4.mp3\" singer=\"beyond\">光辉岁月</a></li>\n",
    "        <li data-view=\"5\"><a href=\"/5.mp3\" singer=\"陈慧琳\">记事本</a></li>\n",
    "        <li data-view=\"5\">\n",
    "            <a href=\"/6.mp3\" singer=\"邓丽君\">但愿人长久</a>\n",
    "        </li>\n",
    "    </ul>\n",
    "</div>'''\n",
    "results = re.findall('<li.*?href=\"(.*?)\".*?singer=\"(.*?)\">(.*?)</a>', html, re.S)\n",
    "print(results)\n",
    "print(type(results))\n",
    "for result in results:\n",
    "    print(result)\n",
    "    print(result[0], result[1], result[2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[('', '一路上有你', ''), ('<a href=\"/2.mp3\" singer=\"任贤齐\">', '沧海一声笑', '</a>'), ('<a href=\"/3.mp3\" singer=\"齐秦\">', '往事随风', '</a>'), ('<a href=\"/4.mp3\" singer=\"beyond\">', '光辉岁月', '</a>'), ('<a href=\"/5.mp3\" singer=\"陈慧琳\">', '记事本', '</a>'), ('<a href=\"/6.mp3\" singer=\"邓丽君\">', '但愿人长久', '</a>')]\n",
      "一路上有你\n",
      "沧海一声笑\n",
      "往事随风\n",
      "光辉岁月\n",
      "记事本\n",
      "但愿人长久\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "html = '''<div id=\"songs-list\">\n",
    "    <h2 class=\"title\">经典老歌</h2>\n",
    "    <p class=\"introduction\">\n",
    "        经典老歌列表\n",
    "    </p>\n",
    "    <ul id=\"list\" class=\"list-group\">\n",
    "        <li data-view=\"2\">一路上有你</li>\n",
    "        <li data-view=\"7\">\n",
    "            <a href=\"/2.mp3\" singer=\"任贤齐\">沧海一声笑</a>\n",
    "        </li>\n",
    "        <li data-view=\"4\" class=\"active\">\n",
    "            <a href=\"/3.mp3\" singer=\"齐秦\">往事随风</a>\n",
    "        </li>\n",
    "        <li data-view=\"6\"><a href=\"/4.mp3\" singer=\"beyond\">光辉岁月</a></li>\n",
    "        <li data-view=\"5\"><a href=\"/5.mp3\" singer=\"陈慧琳\">记事本</a></li>\n",
    "        <li data-view=\"5\">\n",
    "            <a href=\"/6.mp3\" singer=\"邓丽君\">但愿人长久</a>\n",
    "        </li>\n",
    "    </ul>\n",
    "</div>'''\n",
    "results = re.findall('<li.*?>\\s*?(<a.*?>)?(\\w+)(</a>)?\\s*?</li>', html, re.S)\n",
    "print(results)\n",
    "for result in results:\n",
    "    print(result[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### re.sub\n",
    "替换字符串中每一个匹配的子串后返回替换后的字符串。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extra stings Hello  World_This is a Regex Demo Extra stings\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "content = 'Extra stings Hello 1234567 World_This is a Regex Demo Extra stings'\n",
    "content = re.sub('\\d+', '', content)\n",
    "print(content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extra stings Hello Replacement World_This is a Regex Demo Extra stings\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "content = 'Extra stings Hello 1234567 World_This is a Regex Demo Extra stings'\n",
    "content = re.sub('\\d+', 'Replacement', content)\n",
    "print(content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extra stings Hello 1234567 8910 World_This is a Regex Demo Extra stings\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "content = 'Extra stings Hello 1234567 World_This is a Regex Demo Extra stings'\n",
    "content = re.sub('(\\d+)', r'\\1 8910', content)\n",
    "print(content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "\n",
    "html = '''<div id=\"songs-list\">\n",
    "    <h2 class=\"title\">经典老歌</h2>\n",
    "    <p class=\"introduction\">\n",
    "        经典老歌列表\n",
    "    </p>\n",
    "    <ul id=\"list\" class=\"list-group\">\n",
    "        <li data-view=\"2\">一路上有你</li>\n",
    "        <li data-view=\"7\">\n",
    "            <a href=\"/2.mp3\" singer=\"任贤齐\">沧海一声笑</a>\n",
    "        </li>\n",
    "        <li data-view=\"4\" class=\"active\">\n",
    "            <a href=\"/3.mp3\" singer=\"齐秦\">往事随风</a>\n",
    "        </li>\n",
    "        <li data-view=\"6\"><a href=\"/4.mp3\" singer=\"beyond\">光辉岁月</a></li>\n",
    "        <li data-view=\"5\"><a href=\"/5.mp3\" singer=\"陈慧琳\">记事本</a></li>\n",
    "        <li data-view=\"5\">\n",
    "            <a href=\"/6.mp3\" singer=\"邓丽君\">但愿人长久</a>\n",
    "        </li>\n",
    "    </ul>\n",
    "</div>'''\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<div id=\"songs-list\">\n",
      "    <h2 class=\"title\">经典老歌</h2>\n",
      "    <p class=\"introduction\">\n",
      "        经典老歌列表\n",
      "    </p>\n",
      "    <ul id=\"list\" class=\"list-group\">\n",
      "        <li data-view=\"2\">一路上有你</li>\n",
      "        <li data-view=\"7\">\n",
      "            沧海一声笑\n",
      "        </li>\n",
      "        <li data-view=\"4\" class=\"active\">\n",
      "            往事随风\n",
      "        </li>\n",
      "        <li data-view=\"6\">光辉岁月</li>\n",
      "        <li data-view=\"5\">记事本</li>\n",
      "        <li data-view=\"5\">\n",
      "            但愿人长久\n",
      "        </li>\n",
      "    </ul>\n",
      "</div>\n",
      "['一路上有你', '\\n            沧海一声笑\\n        ', '\\n            往事随风\\n        ', '光辉岁月', '记事本', '\\n            但愿人长久\\n        ']\n",
      "一路上有你\n",
      "沧海一声笑\n",
      "往事随风\n",
      "光辉岁月\n",
      "记事本\n",
      "但愿人长久\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "html = '''<div id=\"songs-list\">\n",
    "    <h2 class=\"title\">经典老歌</h2>\n",
    "    <p class=\"introduction\">\n",
    "        经典老歌列表\n",
    "    </p>\n",
    "    <ul id=\"list\" class=\"list-group\">\n",
    "        <li data-view=\"2\">一路上有你</li>\n",
    "        <li data-view=\"7\">\n",
    "            <a href=\"/2.mp3\" singer=\"任贤齐\">沧海一声笑</a>\n",
    "        </li>\n",
    "        <li data-view=\"4\" class=\"active\">\n",
    "            <a href=\"/3.mp3\" singer=\"齐秦\">往事随风</a>\n",
    "        </li>\n",
    "        <li data-view=\"6\"><a href=\"/4.mp3\" singer=\"beyond\">光辉岁月</a></li>\n",
    "        <li data-view=\"5\"><a href=\"/5.mp3\" singer=\"陈慧琳\">记事本</a></li>\n",
    "        <li data-view=\"5\">\n",
    "            <a href=\"/6.mp3\" singer=\"邓丽君\">但愿人长久</a>\n",
    "        </li>\n",
    "    </ul>\n",
    "</div>'''\n",
    "html = re.sub('<a.*?>|</a>', '', html)\n",
    "print(html)\n",
    "results = re.findall('<li.*?>(.*?)</li>', html, re.S)\n",
    "print(results)\n",
    "for result in results:\n",
    "    print(result.strip())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## re.compile\n",
    "将正则字符串编译成正则表达式对象"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "将一个正则表达式串编译成正则对象，以便于复用该匹配模式"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<re.Match object; span=(0, 40), match='Hello 1234567 World_This\\nis a Regex Demo'>\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "content = '''Hello 1234567 World_This\n",
    "is a Regex Demo'''\n",
    "pattern = re.compile('Hello.*Demo', re.S)\n",
    "result = re.match(pattern, content)\n",
    "#result = re.match('Hello.*Demo', content, re.S)\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 实战练习"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "https://book.douban.com/subject/30197648/?icn=index-editionrecommend 时间杀手 [法]米歇尔·普西 2018-11\n",
      "https://book.douban.com/subject/30317420/?icn=index-editionrecommend 盛夏方程式 [日]东野圭吾 2018-11\n",
      "https://book.douban.com/subject/30359490/?icn=index-editionrecommend 裸眼观星 [美]鲍勃•金 2018-11\n",
      "https://book.douban.com/subject/30320912/?icn=index-editionrecommend 说服 [美]诺瓦·戈尔茨坦&nbsp;/&nbsp;[美]史蒂夫·马丁&nbsp;/&nbsp;[美]罗伯特·西奥迪尼 2018-12-1\n",
      "https://book.douban.com/subject/30354245/?icn=index-editionrecommend 注定一战 (美)格雷厄姆·艾利森(GrahamAllison) 2019-1\n",
      "https://book.douban.com/subject/28355275/?icn=index-latestbook-subject 摇摆时光 [英]扎迪·史密斯 2018-10\n",
      "https://book.douban.com/subject/30380222/?icn=index-latestbook-subject 欲望与尊严 肖索未 2018-10\n",
      "https://book.douban.com/subject/30297167/?icn=index-latestbook-subject 沉睡的记忆 [法]帕特里克·莫迪亚诺 2018-11\n",
      "https://book.douban.com/subject/30277179/?icn=index-latestbook-subject 打开一颗心 [英]斯蒂芬·韦斯塔比 2018-11\n",
      "https://book.douban.com/subject/30387994/?icn=index-latestbook-subject 剑桥中国经济史 万志英 2018-11-30\n",
      "https://book.douban.com/subject/30317421/?icn=index-latestbook-subject 禁断的魔术 [日]东野圭吾 2018-12\n",
      "https://book.douban.com/subject/30379527/?icn=index-latestbook-subject 世界观 [美]理查德·德威特 2018-11-15\n",
      "https://book.douban.com/subject/30385318/?icn=index-latestbook-subject 鲤 张悦然主编 2018-11\n",
      "https://book.douban.com/subject/30348025/?icn=index-latestbook-subject 文化理论与大众文化导论 [英]约翰·斯道雷 2019-1-1\n",
      "https://book.douban.com/subject/30325325/?icn=index-latestbook-subject 全能侦探社 [英]道格拉斯·亚当斯&nbsp;/&nbsp;DouglasAdams 2019-1\n",
      "https://book.douban.com/subject/30326080/?icn=index-latestbook-subject 历史写作简明指南 [美]理查德·马里厄斯（RICHARDMARIUS）&nbsp;/&nbsp;[美］梅尔文·E.佩吉（MELVINE.PAGE） 2018-12\n",
      "https://book.douban.com/subject/30314829/?icn=index-latestbook-subject 爱你 [美]贝奇·艾伯特利 2018-12\n",
      "https://book.douban.com/subject/30376180/?icn=index-latestbook-subject 激昂的幻梦 [美]李·斯特拉斯伯格著&nbsp;/&nbsp;[美]伊万杰琳·墨菲斯编辑整理 2018-12\n",
      "https://book.douban.com/subject/30364975/?icn=index-latestbook-subject 吃麻雀的少女 朱一叶 2018-12\n",
      "https://book.douban.com/subject/30383694/?icn=index-latestbook-subject 进击的智人 河森堡 2018-12\n",
      "https://book.douban.com/subject/30338153/?icn=index-latestbook-subject 景恒街 笛安 2019-1\n",
      "https://book.douban.com/subject/30318643/?icn=index-latestbook-subject 酗酒 [美]托马斯·林奇 2018-10\n",
      "https://book.douban.com/subject/30382687/?icn=index-latestbook-subject 度外 黄国峻 2018-12\n",
      "https://book.douban.com/subject/30223227/?icn=index-latestbook-subject 莎士比亚植物志 [英]玛格丽特·威尔斯 2019-1-1\n",
      "https://book.douban.com/subject/27614658/?icn=index-latestbook-subject 当代俄罗斯小说集 [俄]麦列霍夫等 2018-11\n",
      "https://book.douban.com/subject/30366602/?icn=index-latestbook-subject 认知迭代 【英】卡罗琳•威廉姆斯著 2018-12\n",
      "https://book.douban.com/subject/30344739/?icn=index-latestbook-subject 她跟我聊到枫树 [法]安东尼·帕耶 2018-11\n",
      "https://book.douban.com/subject/30368506/?icn=index-latestbook-subject 东南园墅 童寯 2018-11\n",
      "https://book.douban.com/subject/30318654/?icn=index-latestbook-subject 诗人 [美]迈克尔·康奈利 2018-11-2\n",
      "https://book.douban.com/subject/30373807/?icn=index-latestbook-subject 有滋有味 [美]露西·尼斯利 2019-1\n",
      "https://book.douban.com/subject/30331243/?icn=index-latestbook-subject 镖人 许先哲 2018-11\n",
      "https://book.douban.com/subject/30377493/?icn=index-latestbook-subject 周迅 周迅xLens 2018-11-27\n",
      "https://book.douban.com/subject/30368773/?icn=index-latestbook-subject 神奇动物 [英]J·K·罗琳 2018-11-16\n",
      "https://book.douban.com/subject/30258589/?icn=index-latestbook-subject 成为母亲 [英]蕾切尔·卡斯克(RachelCusk) 2019-1\n",
      "https://book.douban.com/subject/30276249/?icn=index-latestbook-subject 长路 [美]科马克·麦卡锡 2018-11\n",
      "https://book.douban.com/subject/30365536/?icn=index-latestbook-subject 清单 [英]肖恩•厄舍 2018-12\n",
      "https://book.douban.com/subject/30365850/?icn=index-latestbook-subject 喜欢就买单 闻珺里 2018-11-15\n",
      "https://book.douban.com/subject/30359547/?icn=index-latestbook-subject 写小说最重要的十件事 [美]厄休拉·勒古恩 2019-1\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "import re\n",
    "headers = {\n",
    "    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36'\n",
    "}\n",
    "content = requests.get('https://book.douban.com/',headers=headers).text\n",
    "#下面这个注释掉的单独匹配 url 没问题，单独匹配 title 就有问题了，放在一起会引起回溯灾难， CPU 狂飙 100% ,看来万能的 .*? 也不完美。参考于 http://www.cnblogs.com/study-everyday/p/7426862.html\n",
    "#pattern = re.compile('<li.*?cover.*?href=\"(.*?)\"title=\"(.*?)“',re.S)#.*?more-meta.*?author\">(.*?)</span>.*?year\">(.*?)</span>.*?</li>', re.S)\n",
    "pattern = re.compile('<li.*?cover.*?href=\"([a-zA-z]+://[^\\s]*)\" title=\"([\\u4e00-\\u9fa5]+).*?more-meta.*?author\">(.*?)</span>.*?year\">(.*?)</span>.*?</li>',re.S)\n",
    "results = re.findall(pattern, content)\n",
    "for result in results:\n",
    "    url, name, author, date = result\n",
    "    author = re.sub('\\s', '', author)\n",
    "    date = re.sub('\\s', '', date)\n",
    "    print(url, name, author, date)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}