Nobel API_2_1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Work in progress**\n",
    "\n",
    "\n",
    "# Nobel API 2.1 - check Wikidata and Nobel API\n",
    "\n",
    "The purpose of this Notebook is to have Nobelprize winners in Wikidata in synch with the prizewinners at Nobelprize... lesson learned is that we also find vandalization in Wikidata....\n",
    "\n",
    "\n",
    "* This [Jupyter Notebook](https://github.com/salgo60/open-data-examples/blob/master/Nobel%20API_2_1.ipynb)  using API 2.1 [api.nobelprize.org/2.1/laureates](https://api.nobelprize.org/2.1/laureates)\n",
    "  * old [Notebook](https://github.com/salgo60/open-data-examples/blob/master/Nobel%20API.ipynb)\n",
    "  * [notebook for Nobelprize motivation](https://github.com/salgo60/open-data-examples/blob/master/Nobel%20API_2_1-motivation.ipynb)\n",
    "\n",
    "\n",
    "  * Wikidata property [Property:P8024](https://www.wikidata.org/wiki/Property:P8024)\n",
    "  * [Developer zone nobelprize.org](https://www.nobelprize.org/about/developer-zone-2/)\n",
    "      * API 2.1 [api.nobelprize.org/2.1/laureates](https://api.nobelprize.org/2.1/laureates)\n",
    "  * [video](https://youtu.be/Iu1JtefueM8) explaining what we do\n",
    "  \n",
    "#### Misc\n",
    "Version history\n",
    "* 0.1 use API 2.1 --> Nobel API also includes Wikidata Q numbers\n",
    "##### See also\n",
    "* Kaggle [nobelfoundation/nobel-laureates](https://www.kaggle.com/nobelfoundation/nobel-laureates) and my [Kaggle Notebook](https://www.kaggle.com/magnusslg/wikidata-nobelprize-synch)\n",
    "\n",
    "#### Other sources we sync\n",
    "* Famous people on Uppsala old cemetery [Kulturpersoner Uppsalakyrkogård](https://github.com/salgo60/open-data-examples/blob/master/Check%20WD%20kulturpersoner%20uppsalakyrkogardar.ipynb)\n",
    "* Swedish Literature Bank - [Litteraturbanken](https://github.com/salgo60/open-data-examples/blob/master/Litteraturbanken%20Author.ipynb) \n",
    "  * WD property [P5101](https://www.wikidata.org/wiki/Property_talk:P5101) [P5123](https://www.wikidata.org/wiki/Property_talk:P5123)\n",
    "* [Nobelprize.org](https://github.com/salgo60/open-data-examples/blob/master/Nobel%20API.ipynb)\n",
    "  * WD [property 8024](https://www.wikidata.org/wiki/Property:P8024)\n",
    "* The Swedish National Archive [SBL](https://github.com/salgo60/open-data-examples/blob/master/SBL.ipynb) \n",
    "  * WD [property 3217](https://www.wikidata.org/wiki/Property:P3217) \n",
    "* The Biographical Dictionary of Swedish Women [SKBL](https://github.com/salgo60/open-data-examples/blob/master/Svenskt%20Kvinnobiografiskt%20lexikon%20part%203.ipynb)\n",
    "  * WD [property 4963](https://www.wikidata.org/wiki/Property:P4963)\n",
    "* Swedish Academy - [Svenska Akademien](https://github.com/salgo60/open-data-examples/blob/master/Svenska%20Akademien.ipynb) \n",
    "  * WD [property 5325](https://www.wikidata.org/wiki/Property:P5325) \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Last run:  2022-10-21 07:47:10.979146\n"
     ]
    }
   ],
   "source": [
    "from datetime import datetime\n",
    "start_time = datetime.now()\n",
    "print(\"Last run: \", datetime.now())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import urllib3, json\n",
    "import pandas as pd \n",
    "http = urllib3.PoolManager() \n",
    "from flatten_json import flatten \n",
    "url =\"https://api.nobelprize.org/2.1/laureates?offset=0&limit=1500\"\n",
    "dftot = pd.DataFrame()\n",
    "r = http.request('GET', url)\n",
    "data = json.loads(r.data)\n",
    "#dftot = dftot.append(pd.DataFrame(data[\"laureates\"]),sort=False)\n",
    "from flatten_json import flatten\n",
    "dftot = pd.json_normalize(data[\"laureates\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "#dfnorm.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 981 entries, 0 to 980\n",
      "Data columns (total 100 columns):\n",
      " #   Column                           Non-Null Count  Dtype \n",
      "---  ------                           --------------  ----- \n",
      " 0   id                               981 non-null    object\n",
      " 1   fileName                         981 non-null    object\n",
      " 2   gender                           954 non-null    object\n",
      " 3   sameAs                           981 non-null    object\n",
      " 4   links                            981 non-null    object\n",
      " 5   nobelPrizes                      981 non-null    object\n",
      " 6   knownName.en                     954 non-null    object\n",
      " 7   knownName.se                     954 non-null    object\n",
      " 8   givenName.en                     953 non-null    object\n",
      " 9   givenName.se                     953 non-null    object\n",
      " 10  familyName.en                    951 non-null    object\n",
      " 11  familyName.se                    951 non-null    object\n",
      " 12  fullName.en                      954 non-null    object\n",
      " 13  fullName.se                      954 non-null    object\n",
      " 14  birth.date                       954 non-null    object\n",
      " 15  birth.place.city.en              946 non-null    object\n",
      " 16  birth.place.city.no              946 non-null    object\n",
      " 17  birth.place.city.se              946 non-null    object\n",
      " 18  birth.place.country.en           951 non-null    object\n",
      " 19  birth.place.country.no           951 non-null    object\n",
      " 20  birth.place.country.se           951 non-null    object\n",
      " 21  birth.place.cityNow.en           946 non-null    object\n",
      " 22  birth.place.cityNow.no           946 non-null    object\n",
      " 23  birth.place.cityNow.se           946 non-null    object\n",
      " 24  birth.place.cityNow.sameAs       928 non-null    object\n",
      " 25  birth.place.countryNow.en        951 non-null    object\n",
      " 26  birth.place.countryNow.no        951 non-null    object\n",
      " 27  birth.place.countryNow.se        951 non-null    object\n",
      " 28  birth.place.countryNow.sameAs    946 non-null    object\n",
      " 29  birth.place.continent.en         951 non-null    object\n",
      " 30  birth.place.continent.no         951 non-null    object\n",
      " 31  birth.place.continent.se         951 non-null    object\n",
      " 32  birth.place.locationString.en    951 non-null    object\n",
      " 33  birth.place.locationString.no    951 non-null    object\n",
      " 34  birth.place.locationString.se    951 non-null    object\n",
      " 35  wikipedia.slug                   981 non-null    object\n",
      " 36  wikipedia.english                981 non-null    object\n",
      " 37  wikidata.id                      981 non-null    object\n",
      " 38  wikidata.url                     981 non-null    object\n",
      " 39  death.date                       650 non-null    object\n",
      " 40  death.place.city.en              632 non-null    object\n",
      " 41  death.place.city.no              632 non-null    object\n",
      " 42  death.place.city.se              632 non-null    object\n",
      " 43  death.place.country.en           638 non-null    object\n",
      " 44  death.place.country.no           638 non-null    object\n",
      " 45  death.place.country.se           638 non-null    object\n",
      " 46  death.place.country.sameAs       597 non-null    object\n",
      " 47  death.place.cityNow.en           632 non-null    object\n",
      " 48  death.place.cityNow.no           632 non-null    object\n",
      " 49  death.place.cityNow.se           632 non-null    object\n",
      " 50  death.place.cityNow.sameAs       619 non-null    object\n",
      " 51  death.place.countryNow.en        638 non-null    object\n",
      " 52  death.place.countryNow.no        638 non-null    object\n",
      " 53  death.place.countryNow.se        638 non-null    object\n",
      " 54  death.place.countryNow.sameAs    634 non-null    object\n",
      " 55  death.place.continent.en         638 non-null    object\n",
      " 56  death.place.continent.no         638 non-null    object\n",
      " 57  death.place.continent.se         638 non-null    object\n",
      " 58  death.place.locationString.en    650 non-null    object\n",
      " 59  death.place.locationString.no    650 non-null    object\n",
      " 60  death.place.locationString.se    650 non-null    object\n",
      " 61  acronym                          14 non-null     object\n",
      " 62  orgName.en                       27 non-null     object\n",
      " 63  orgName.no                       21 non-null     object\n",
      " 64  orgName.se                       27 non-null     object\n",
      " 65  founded.date                     26 non-null     object\n",
      " 66  founded.place.city.en            21 non-null     object\n",
      " 67  founded.place.city.no            21 non-null     object\n",
      " 68  founded.place.city.se            21 non-null     object\n",
      " 69  founded.place.country.en         23 non-null     object\n",
      " 70  founded.place.country.no         23 non-null     object\n",
      " 71  founded.place.country.se         23 non-null     object\n",
      " 72  founded.place.country.sameAs     23 non-null     object\n",
      " 73  founded.place.cityNow.en         21 non-null     object\n",
      " 74  founded.place.cityNow.no         21 non-null     object\n",
      " 75  founded.place.cityNow.se         21 non-null     object\n",
      " 76  founded.place.cityNow.sameAs     21 non-null     object\n",
      " 77  founded.place.countryNow.en      23 non-null     object\n",
      " 78  founded.place.countryNow.no      23 non-null     object\n",
      " 79  founded.place.countryNow.se      23 non-null     object\n",
      " 80  founded.place.countryNow.sameAs  23 non-null     object\n",
      " 81  founded.place.continent.en       23 non-null     object\n",
      " 82  founded.place.continent.no       23 non-null     object\n",
      " 83  founded.place.continent.se       23 non-null     object\n",
      " 84  founded.place.locationString.en  26 non-null     object\n",
      " 85  founded.place.locationString.no  26 non-null     object\n",
      " 86  founded.place.locationString.se  26 non-null     object\n",
      " 87  nativeName                       26 non-null     object\n",
      " 88  penName                          11 non-null     object\n",
      " 89  knownName.no                     5 non-null      object\n",
      " 90  givenName.no                     4 non-null      object\n",
      " 91  familyName.no                    4 non-null      object\n",
      " 92  fullName.no                      4 non-null      object\n",
      " 93  foundedCountry.en                1 non-null      object\n",
      " 94  foundedCountry.no                1 non-null      object\n",
      " 95  foundedCountry.se                1 non-null      object\n",
      " 96  foundedCountryNow.en             1 non-null      object\n",
      " 97  foundedCountryNow.no             1 non-null      object\n",
      " 98  foundedCountryNow.se             1 non-null      object\n",
      " 99  foundedContinent.en              1 non-null      object\n",
      "dtypes: object(100)\n",
      "memory usage: 766.5+ KB\n"
     ]
    }
   ],
   "source": [
    "dftot.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>fileName</th>\n",
       "      <th>gender</th>\n",
       "      <th>sameAs</th>\n",
       "      <th>links</th>\n",
       "      <th>nobelPrizes</th>\n",
       "      <th>knownName.en</th>\n",
       "      <th>knownName.se</th>\n",
       "      <th>givenName.en</th>\n",
       "      <th>givenName.se</th>\n",
       "      <th>...</th>\n",
       "      <th>givenName.no</th>\n",
       "      <th>familyName.no</th>\n",
       "      <th>fullName.no</th>\n",
       "      <th>foundedCountry.en</th>\n",
       "      <th>foundedCountry.no</th>\n",
       "      <th>foundedCountry.se</th>\n",
       "      <th>foundedCountryNow.en</th>\n",
       "      <th>foundedCountryNow.no</th>\n",
       "      <th>foundedCountryNow.se</th>\n",
       "      <th>foundedContinent.en</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>132</th>\n",
       "      <td>1020</td>\n",
       "      <td>center-for-civil-liberties</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[https://www.wikidata.org/wiki/Q114452906, htt...</td>\n",
       "      <td>[{'rel': 'laureate', 'href': 'https://api.nobe...</td>\n",
       "      <td>[{'awardYear': '2022', 'category': {'en': 'Pea...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95</th>\n",
       "      <td>1021</td>\n",
       "      <td>bernanke</td>\n",
       "      <td>male</td>\n",
       "      <td>[https://www.wikidata.org/wiki/Q201795, https:...</td>\n",
       "      <td>[{'rel': 'laureate', 'href': 'https://api.nobe...</td>\n",
       "      <td>[{'awardYear': '2022', 'category': {'en': 'Eco...</td>\n",
       "      <td>Ben Bernanke</td>\n",
       "      <td>Ben Bernanke</td>\n",
       "      <td>Ben</td>\n",
       "      <td>Ben</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>196</th>\n",
       "      <td>1022</td>\n",
       "      <td>diamond</td>\n",
       "      <td>male</td>\n",
       "      <td>[https://www.wikidata.org/wiki/Q185871, https:...</td>\n",
       "      <td>[{'rel': 'laureate', 'href': 'https://api.nobe...</td>\n",
       "      <td>[{'awardYear': '2022', 'category': {'en': 'Eco...</td>\n",
       "      <td>Douglas Diamond</td>\n",
       "      <td>Douglas Diamond</td>\n",
       "      <td>Douglas</td>\n",
       "      <td>Douglas</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>719</th>\n",
       "      <td>1023</td>\n",
       "      <td>dybvig</td>\n",
       "      <td>male</td>\n",
       "      <td>[https://www.wikidata.org/wiki/Q24565948, http...</td>\n",
       "      <td>[{'rel': 'laureate', 'href': 'https://api.nobe...</td>\n",
       "      <td>[{'awardYear': '2022', 'category': {'en': 'Eco...</td>\n",
       "      <td>Philip Dybvig</td>\n",
       "      <td>Philip Dybvig</td>\n",
       "      <td>Philip</td>\n",
       "      <td>Philip</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4 rows × 100 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       id                    fileName gender  \\\n",
       "132  1020  center-for-civil-liberties    NaN   \n",
       "95   1021                    bernanke   male   \n",
       "196  1022                     diamond   male   \n",
       "719  1023                      dybvig   male   \n",
       "\n",
       "                                                sameAs  \\\n",
       "132  [https://www.wikidata.org/wiki/Q114452906, htt...   \n",
       "95   [https://www.wikidata.org/wiki/Q201795, https:...   \n",
       "196  [https://www.wikidata.org/wiki/Q185871, https:...   \n",
       "719  [https://www.wikidata.org/wiki/Q24565948, http...   \n",
       "\n",
       "                                                 links  \\\n",
       "132  [{'rel': 'laureate', 'href': 'https://api.nobe...   \n",
       "95   [{'rel': 'laureate', 'href': 'https://api.nobe...   \n",
       "196  [{'rel': 'laureate', 'href': 'https://api.nobe...   \n",
       "719  [{'rel': 'laureate', 'href': 'https://api.nobe...   \n",
       "\n",
       "                                           nobelPrizes     knownName.en  \\\n",
       "132  [{'awardYear': '2022', 'category': {'en': 'Pea...              NaN   \n",
       "95   [{'awardYear': '2022', 'category': {'en': 'Eco...     Ben Bernanke   \n",
       "196  [{'awardYear': '2022', 'category': {'en': 'Eco...  Douglas Diamond   \n",
       "719  [{'awardYear': '2022', 'category': {'en': 'Eco...    Philip Dybvig   \n",
       "\n",
       "        knownName.se givenName.en givenName.se  ... givenName.no  \\\n",
       "132              NaN          NaN          NaN  ...          NaN   \n",
       "95      Ben Bernanke          Ben          Ben  ...          NaN   \n",
       "196  Douglas Diamond      Douglas      Douglas  ...          NaN   \n",
       "719    Philip Dybvig       Philip       Philip  ...          NaN   \n",
       "\n",
       "    familyName.no fullName.no foundedCountry.en foundedCountry.no  \\\n",
       "132           NaN         NaN               NaN               NaN   \n",
       "95            NaN         NaN               NaN               NaN   \n",
       "196           NaN         NaN               NaN               NaN   \n",
       "719           NaN         NaN               NaN               NaN   \n",
       "\n",
       "    foundedCountry.se foundedCountryNow.en foundedCountryNow.no  \\\n",
       "132               NaN                  NaN                  NaN   \n",
       "95                NaN                  NaN                  NaN   \n",
       "196               NaN                  NaN                  NaN   \n",
       "719               NaN                  NaN                  NaN   \n",
       "\n",
       "    foundedCountryNow.se foundedContinent.en  \n",
       "132                  NaN                 NaN  \n",
       "95                   NaN                 NaN  \n",
       "196                  NaN                 NaN  \n",
       "719                  NaN                 NaN  \n",
       "\n",
       "[4 rows x 100 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dftot[\"id\"] = dftot[\"id\"].astype(\"int\") \n",
    "dftot[\"id\"] = dftot[\"id\"].astype(\"int\") \n",
    "\n",
    "\n",
    "dftot.sort_values(by=['id'], ascending=[True], inplace=True)\n",
    "dftot.tail(4)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(981, 100)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Test flatten \n",
    "#from pandas.json_normalize import json_normalize\n",
    "#from pandas.io.json.normalize import nested_to_record\n",
    "from pandas import json_normalize\n",
    "#dftotPrize = json_normalize(data=data['laureates'], record_path='nobelPrizes', \n",
    "#                            meta=['id', 'filename','gender', 'wikidata'],errors='ignore') \n",
    "dftotPrize = json_normalize(data=data[\"laureates\"])\n",
    "dftotPrize.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>fileName</th>\n",
       "      <th>gender</th>\n",
       "      <th>sameAs</th>\n",
       "      <th>links</th>\n",
       "      <th>nobelPrizes</th>\n",
       "      <th>knownName.en</th>\n",
       "      <th>knownName.se</th>\n",
       "      <th>givenName.en</th>\n",
       "      <th>givenName.se</th>\n",
       "      <th>...</th>\n",
       "      <th>givenName.no</th>\n",
       "      <th>familyName.no</th>\n",
       "      <th>fullName.no</th>\n",
       "      <th>foundedCountry.en</th>\n",
       "      <th>foundedCountry.no</th>\n",
       "      <th>foundedCountry.se</th>\n",
       "      <th>foundedCountryNow.en</th>\n",
       "      <th>foundedCountryNow.no</th>\n",
       "      <th>foundedCountryNow.se</th>\n",
       "      <th>foundedContinent.en</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>760</th>\n",
       "      <td>554</td>\n",
       "      <td>tum</td>\n",
       "      <td>female</td>\n",
       "      <td>[https://www.wikidata.org/wiki/Q188620, https:...</td>\n",
       "      <td>[{'rel': 'laureate', 'href': 'https://api.nobe...</td>\n",
       "      <td>[{'awardYear': '1992', 'category': {'en': 'Pea...</td>\n",
       "      <td>Rigoberta Menchú Tum</td>\n",
       "      <td>Rigoberta Menchú Tum</td>\n",
       "      <td>Rigoberta</td>\n",
       "      <td>Rigoberta</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>564</th>\n",
       "      <td>686</td>\n",
       "      <td>kantorovich</td>\n",
       "      <td>male</td>\n",
       "      <td>[https://www.wikidata.org/wiki/Q107441, https:...</td>\n",
       "      <td>[{'rel': 'laureate', 'href': 'https://api.nobe...</td>\n",
       "      <td>[{'awardYear': '1975', 'category': {'en': 'Eco...</td>\n",
       "      <td>Leonid Vitaliyevich Kantorovich</td>\n",
       "      <td>Leonid Vitaliyevich Kantorovich</td>\n",
       "      <td>Leonid Vitaliyevich</td>\n",
       "      <td>Leonid Vitaliyevich</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>381</th>\n",
       "      <td>462</td>\n",
       "      <td>dunant</td>\n",
       "      <td>male</td>\n",
       "      <td>[https://www.wikidata.org/wiki/Q12091, https:/...</td>\n",
       "      <td>[{'rel': 'laureate', 'href': 'https://api.nobe...</td>\n",
       "      <td>[{'awardYear': '1901', 'category': {'en': 'Pea...</td>\n",
       "      <td>Henry Dunant</td>\n",
       "      <td>Henry Dunant</td>\n",
       "      <td>Henry</td>\n",
       "      <td>Henry</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>959</th>\n",
       "      <td>249</td>\n",
       "      <td>lipscomb</td>\n",
       "      <td>male</td>\n",
       "      <td>[https://www.wikidata.org/wiki/Q110935, https:...</td>\n",
       "      <td>[{'rel': 'laureate', 'href': 'https://api.nobe...</td>\n",
       "      <td>[{'awardYear': '1976', 'category': {'en': 'Che...</td>\n",
       "      <td>William Lipscomb</td>\n",
       "      <td>William Lipscomb</td>\n",
       "      <td>William</td>\n",
       "      <td>William</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>338</th>\n",
       "      <td>418</td>\n",
       "      <td>hounsfield</td>\n",
       "      <td>male</td>\n",
       "      <td>[https://www.wikidata.org/wiki/Q273769, https:...</td>\n",
       "      <td>[{'rel': 'laureate', 'href': 'https://api.nobe...</td>\n",
       "      <td>[{'awardYear': '1979', 'category': {'en': 'Phy...</td>\n",
       "      <td>Godfrey N. Hounsfield</td>\n",
       "      <td>Godfrey N. Hounsfield</td>\n",
       "      <td>Godfrey N.</td>\n",
       "      <td>Godfrey N.</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 100 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      id     fileName  gender  \\\n",
       "760  554          tum  female   \n",
       "564  686  kantorovich    male   \n",
       "381  462       dunant    male   \n",
       "959  249     lipscomb    male   \n",
       "338  418   hounsfield    male   \n",
       "\n",
       "                                                sameAs  \\\n",
       "760  [https://www.wikidata.org/wiki/Q188620, https:...   \n",
       "564  [https://www.wikidata.org/wiki/Q107441, https:...   \n",
       "381  [https://www.wikidata.org/wiki/Q12091, https:/...   \n",
       "959  [https://www.wikidata.org/wiki/Q110935, https:...   \n",
       "338  [https://www.wikidata.org/wiki/Q273769, https:...   \n",
       "\n",
       "                                                 links  \\\n",
       "760  [{'rel': 'laureate', 'href': 'https://api.nobe...   \n",
       "564  [{'rel': 'laureate', 'href': 'https://api.nobe...   \n",
       "381  [{'rel': 'laureate', 'href': 'https://api.nobe...   \n",
       "959  [{'rel': 'laureate', 'href': 'https://api.nobe...   \n",
       "338  [{'rel': 'laureate', 'href': 'https://api.nobe...   \n",
       "\n",
       "                                           nobelPrizes  \\\n",
       "760  [{'awardYear': '1992', 'category': {'en': 'Pea...   \n",
       "564  [{'awardYear': '1975', 'category': {'en': 'Eco...   \n",
       "381  [{'awardYear': '1901', 'category': {'en': 'Pea...   \n",
       "959  [{'awardYear': '1976', 'category': {'en': 'Che...   \n",
       "338  [{'awardYear': '1979', 'category': {'en': 'Phy...   \n",
       "\n",
       "                        knownName.en                     knownName.se  \\\n",
       "760             Rigoberta Menchú Tum             Rigoberta Menchú Tum   \n",
       "564  Leonid Vitaliyevich Kantorovich  Leonid Vitaliyevich Kantorovich   \n",
       "381                     Henry Dunant                     Henry Dunant   \n",
       "959                 William Lipscomb                 William Lipscomb   \n",
       "338            Godfrey N. Hounsfield            Godfrey N. Hounsfield   \n",
       "\n",
       "            givenName.en         givenName.se  ... givenName.no familyName.no  \\\n",
       "760            Rigoberta            Rigoberta  ...          NaN           NaN   \n",
       "564  Leonid Vitaliyevich  Leonid Vitaliyevich  ...          NaN           NaN   \n",
       "381                Henry                Henry  ...          NaN           NaN   \n",
       "959              William              William  ...          NaN           NaN   \n",
       "338           Godfrey N.           Godfrey N.  ...          NaN           NaN   \n",
       "\n",
       "    fullName.no foundedCountry.en foundedCountry.no foundedCountry.se  \\\n",
       "760         NaN               NaN               NaN               NaN   \n",
       "564         NaN               NaN               NaN               NaN   \n",
       "381         NaN               NaN               NaN               NaN   \n",
       "959         NaN               NaN               NaN               NaN   \n",
       "338         NaN               NaN               NaN               NaN   \n",
       "\n",
       "    foundedCountryNow.en foundedCountryNow.no foundedCountryNow.se  \\\n",
       "760                  NaN                  NaN                  NaN   \n",
       "564                  NaN                  NaN                  NaN   \n",
       "381                  NaN                  NaN                  NaN   \n",
       "959                  NaN                  NaN                  NaN   \n",
       "338                  NaN                  NaN                  NaN   \n",
       "\n",
       "    foundedContinent.en  \n",
       "760                 NaN  \n",
       "564                 NaN  \n",
       "381                 NaN  \n",
       "959                 NaN  \n",
       "338                 NaN  \n",
       "\n",
       "[5 rows x 100 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dftotPrize.sample(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(pen-name of Jacques Anatole Thibault)          1\n",
       "(pen-name of Lucila Godoy y Alcayaga)           1\n",
       "(pen-name of Giorgos Seferiadis)                1\n",
       "(pen-name of Grazia Madesani, née Deledda)      1\n",
       "(pen-name of Guan Moye)                         1\n",
       "(pen-name of Odysseus Alepoudhelis)             1\n",
       "(pen-name of Neftalí Ricardo Reyes Basoalto)    1\n",
       "(pen-name of Pearl Walsh, née Sydenstricker)    1\n",
       "(pen-name of Alexis Léger)                      1\n",
       "(pen-name of René François Armand Prudhomme)    1\n",
       "(pen-name of Rejment)                           1\n",
       "Name: penName, dtype: int64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dftotPrize[\"penName\"].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "USA               284\n",
       "United Kingdom     89\n",
       "Germany            84\n",
       "France             59\n",
       "Sweden             30\n",
       "                 ... \n",
       "Ghana               1\n",
       "Vietnam             1\n",
       "Croatia             1\n",
       "Azerbaijan          1\n",
       "Taiwan              1\n",
       "Name: birth.place.countryNow.en, Length: 84, dtype: int64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dftotPrize[\"birth.place.countryNow.en\"].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 981 entries, 0 to 980\n",
      "Data columns (total 100 columns):\n",
      " #   Column                           Non-Null Count  Dtype \n",
      "---  ------                           --------------  ----- \n",
      " 0   id                               981 non-null    object\n",
      " 1   fileName                         981 non-null    object\n",
      " 2   gender                           954 non-null    object\n",
      " 3   sameAs                           981 non-null    object\n",
      " 4   links                            981 non-null    object\n",
      " 5   nobelPrizes                      981 non-null    object\n",
      " 6   knownName.en                     954 non-null    object\n",
      " 7   knownName.se                     954 non-null    object\n",
      " 8   givenName.en                     953 non-null    object\n",
      " 9   givenName.se                     953 non-null    object\n",
      " 10  familyName.en                    951 non-null    object\n",
      " 11  familyName.se                    951 non-null    object\n",
      " 12  fullName.en                      954 non-null    object\n",
      " 13  fullName.se                      954 non-null    object\n",
      " 14  birth.date                       954 non-null    object\n",
      " 15  birth.place.city.en              946 non-null    object\n",
      " 16  birth.place.city.no              946 non-null    object\n",
      " 17  birth.place.city.se              946 non-null    object\n",
      " 18  birth.place.country.en           951 non-null    object\n",
      " 19  birth.place.country.no           951 non-null    object\n",
      " 20  birth.place.country.se           951 non-null    object\n",
      " 21  birth.place.cityNow.en           946 non-null    object\n",
      " 22  birth.place.cityNow.no           946 non-null    object\n",
      " 23  birth.place.cityNow.se           946 non-null    object\n",
      " 24  birth.place.cityNow.sameAs       928 non-null    object\n",
      " 25  birth.place.countryNow.en        951 non-null    object\n",
      " 26  birth.place.countryNow.no        951 non-null    object\n",
      " 27  birth.place.countryNow.se        951 non-null    object\n",
      " 28  birth.place.countryNow.sameAs    946 non-null    object\n",
      " 29  birth.place.continent.en         951 non-null    object\n",
      " 30  birth.place.continent.no         951 non-null    object\n",
      " 31  birth.place.continent.se         951 non-null    object\n",
      " 32  birth.place.locationString.en    951 non-null    object\n",
      " 33  birth.place.locationString.no    951 non-null    object\n",
      " 34  birth.place.locationString.se    951 non-null    object\n",
      " 35  wikipedia.slug                   981 non-null    object\n",
      " 36  wikipedia.english                981 non-null    object\n",
      " 37  wikidata.id                      981 non-null    object\n",
      " 38  wikidata.url                     981 non-null    object\n",
      " 39  death.date                       650 non-null    object\n",
      " 40  death.place.city.en              632 non-null    object\n",
      " 41  death.place.city.no              632 non-null    object\n",
      " 42  death.place.city.se              632 non-null    object\n",
      " 43  death.place.country.en           638 non-null    object\n",
      " 44  death.place.country.no           638 non-null    object\n",
      " 45  death.place.country.se           638 non-null    object\n",
      " 46  death.place.country.sameAs       597 non-null    object\n",
      " 47  death.place.cityNow.en           632 non-null    object\n",
      " 48  death.place.cityNow.no           632 non-null    object\n",
      " 49  death.place.cityNow.se           632 non-null    object\n",
      " 50  death.place.cityNow.sameAs       619 non-null    object\n",
      " 51  death.place.countryNow.en        638 non-null    object\n",
      " 52  death.place.countryNow.no        638 non-null    object\n",
      " 53  death.place.countryNow.se        638 non-null    object\n",
      " 54  death.place.countryNow.sameAs    634 non-null    object\n",
      " 55  death.place.continent.en         638 non-null    object\n",
      " 56  death.place.continent.no         638 non-null    object\n",
      " 57  death.place.continent.se         638 non-null    object\n",
      " 58  death.place.locationString.en    650 non-null    object\n",
      " 59  death.place.locationString.no    650 non-null    object\n",
      " 60  death.place.locationString.se    650 non-null    object\n",
      " 61  acronym                          14 non-null     object\n",
      " 62  orgName.en                       27 non-null     object\n",
      " 63  orgName.no                       21 non-null     object\n",
      " 64  orgName.se                       27 non-null     object\n",
      " 65  founded.date                     26 non-null     object\n",
      " 66  founded.place.city.en            21 non-null     object\n",
      " 67  founded.place.city.no            21 non-null     object\n",
      " 68  founded.place.city.se            21 non-null     object\n",
      " 69  founded.place.country.en         23 non-null     object\n",
      " 70  founded.place.country.no         23 non-null     object\n",
      " 71  founded.place.country.se         23 non-null     object\n",
      " 72  founded.place.country.sameAs     23 non-null     object\n",
      " 73  founded.place.cityNow.en         21 non-null     object\n",
      " 74  founded.place.cityNow.no         21 non-null     object\n",
      " 75  founded.place.cityNow.se         21 non-null     object\n",
      " 76  founded.place.cityNow.sameAs     21 non-null     object\n",
      " 77  founded.place.countryNow.en      23 non-null     object\n",
      " 78  founded.place.countryNow.no      23 non-null     object\n",
      " 79  founded.place.countryNow.se      23 non-null     object\n",
      " 80  founded.place.countryNow.sameAs  23 non-null     object\n",
      " 81  founded.place.continent.en       23 non-null     object\n",
      " 82  founded.place.continent.no       23 non-null     object\n",
      " 83  founded.place.continent.se       23 non-null     object\n",
      " 84  founded.place.locationString.en  26 non-null     object\n",
      " 85  founded.place.locationString.no  26 non-null     object\n",
      " 86  founded.place.locationString.se  26 non-null     object\n",
      " 87  nativeName                       26 non-null     object\n",
      " 88  penName                          11 non-null     object\n",
      " 89  knownName.no                     5 non-null      object\n",
      " 90  givenName.no                     4 non-null      object\n",
      " 91  familyName.no                    4 non-null      object\n",
      " 92  fullName.no                      4 non-null      object\n",
      " 93  foundedCountry.en                1 non-null      object\n",
      " 94  foundedCountry.no                1 non-null      object\n",
      " 95  foundedCountry.se                1 non-null      object\n",
      " 96  foundedCountryNow.en             1 non-null      object\n",
      " 97  foundedCountryNow.no             1 non-null      object\n",
      " 98  foundedCountryNow.se             1 non-null      object\n",
      " 99  foundedContinent.en              1 non-null      object\n",
      "dtypes: object(100)\n",
      "memory usage: 766.5+ KB\n"
     ]
    }
   ],
   "source": [
    "dftotPrize.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "#dftotPrize[\"death.place.country.sameAs\"].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>fileName</th>\n",
       "      <th>gender</th>\n",
       "      <th>sameAs</th>\n",
       "      <th>links</th>\n",
       "      <th>nobelPrizes</th>\n",
       "      <th>knownName.en</th>\n",
       "      <th>knownName.se</th>\n",
       "      <th>givenName.en</th>\n",
       "      <th>givenName.se</th>\n",
       "      <th>...</th>\n",
       "      <th>givenName.no</th>\n",
       "      <th>familyName.no</th>\n",
       "      <th>fullName.no</th>\n",
       "      <th>foundedCountry.en</th>\n",
       "      <th>foundedCountry.no</th>\n",
       "      <th>foundedCountry.se</th>\n",
       "      <th>foundedCountryNow.en</th>\n",
       "      <th>foundedCountryNow.no</th>\n",
       "      <th>foundedCountryNow.se</th>\n",
       "      <th>foundedContinent.en</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>941</th>\n",
       "      <td>224</td>\n",
       "      <td>libby</td>\n",
       "      <td>male</td>\n",
       "      <td>[https://www.wikidata.org/wiki/Q190486, https:...</td>\n",
       "      <td>[{'rel': 'laureate', 'href': 'https://api.nobe...</td>\n",
       "      <td>[{'awardYear': '1960', 'category': {'en': 'Che...</td>\n",
       "      <td>Willard F. Libby</td>\n",
       "      <td>Willard F. Libby</td>\n",
       "      <td>Willard F.</td>\n",
       "      <td>Willard F.</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>782</th>\n",
       "      <td>297</td>\n",
       "      <td>koch</td>\n",
       "      <td>male</td>\n",
       "      <td>[https://www.wikidata.org/wiki/Q37193, https:/...</td>\n",
       "      <td>[{'rel': 'laureate', 'href': 'https://api.nobe...</td>\n",
       "      <td>[{'awardYear': '1905', 'category': {'en': 'Phy...</td>\n",
       "      <td>Robert Koch</td>\n",
       "      <td>Robert Koch</td>\n",
       "      <td>Robert</td>\n",
       "      <td>Robert</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>936</th>\n",
       "      <td>361</td>\n",
       "      <td>forssmann</td>\n",
       "      <td>male</td>\n",
       "      <td>[https://www.wikidata.org/wiki/Q77152, https:/...</td>\n",
       "      <td>[{'rel': 'laureate', 'href': 'https://api.nobe...</td>\n",
       "      <td>[{'awardYear': '1956', 'category': {'en': 'Phy...</td>\n",
       "      <td>Werner Forssmann</td>\n",
       "      <td>Werner Forssmann</td>\n",
       "      <td>Werner</td>\n",
       "      <td>Werner</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>274</th>\n",
       "      <td>778</td>\n",
       "      <td>wilczek</td>\n",
       "      <td>male</td>\n",
       "      <td>[https://www.wikidata.org/wiki/Q107450, https:...</td>\n",
       "      <td>[{'rel': 'laureate', 'href': 'https://api.nobe...</td>\n",
       "      <td>[{'awardYear': '2004', 'category': {'en': 'Phy...</td>\n",
       "      <td>Frank Wilczek</td>\n",
       "      <td>Frank Wilczek</td>\n",
       "      <td>Frank</td>\n",
       "      <td>Frank</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4 rows × 100 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      id   fileName gender                                             sameAs  \\\n",
       "941  224      libby   male  [https://www.wikidata.org/wiki/Q190486, https:...   \n",
       "782  297       koch   male  [https://www.wikidata.org/wiki/Q37193, https:/...   \n",
       "936  361  forssmann   male  [https://www.wikidata.org/wiki/Q77152, https:/...   \n",
       "274  778    wilczek   male  [https://www.wikidata.org/wiki/Q107450, https:...   \n",
       "\n",
       "                                                 links  \\\n",
       "941  [{'rel': 'laureate', 'href': 'https://api.nobe...   \n",
       "782  [{'rel': 'laureate', 'href': 'https://api.nobe...   \n",
       "936  [{'rel': 'laureate', 'href': 'https://api.nobe...   \n",
       "274  [{'rel': 'laureate', 'href': 'https://api.nobe...   \n",
       "\n",
       "                                           nobelPrizes      knownName.en  \\\n",
       "941  [{'awardYear': '1960', 'category': {'en': 'Che...  Willard F. Libby   \n",
       "782  [{'awardYear': '1905', 'category': {'en': 'Phy...       Robert Koch   \n",
       "936  [{'awardYear': '1956', 'category': {'en': 'Phy...  Werner Forssmann   \n",
       "274  [{'awardYear': '2004', 'category': {'en': 'Phy...     Frank Wilczek   \n",
       "\n",
       "         knownName.se givenName.en givenName.se  ... givenName.no  \\\n",
       "941  Willard F. Libby   Willard F.   Willard F.  ...          NaN   \n",
       "782       Robert Koch       Robert       Robert  ...          NaN   \n",
       "936  Werner Forssmann       Werner       Werner  ...          NaN   \n",
       "274     Frank Wilczek        Frank        Frank  ...          NaN   \n",
       "\n",
       "    familyName.no fullName.no foundedCountry.en foundedCountry.no  \\\n",
       "941           NaN         NaN               NaN               NaN   \n",
       "782           NaN         NaN               NaN               NaN   \n",
       "936           NaN         NaN               NaN               NaN   \n",
       "274           NaN         NaN               NaN               NaN   \n",
       "\n",
       "    foundedCountry.se foundedCountryNow.en foundedCountryNow.no  \\\n",
       "941               NaN                  NaN                  NaN   \n",
       "782               NaN                  NaN                  NaN   \n",
       "936               NaN                  NaN                  NaN   \n",
       "274               NaN                  NaN                  NaN   \n",
       "\n",
       "    foundedCountryNow.se foundedContinent.en  \n",
       "941                  NaN                 NaN  \n",
       "782                  NaN                 NaN  \n",
       "936                  NaN                 NaN  \n",
       "274                  NaN                 NaN  \n",
       "\n",
       "[4 rows x 100 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dftotPrize.sample(4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Check Wikidata matches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# pip install sparqlwrapper\n",
    "# https://rdflib.github.io/sparqlwrapper/\n",
    "\n",
    "import sys\n",
    "from SPARQLWrapper import SPARQLWrapper, JSON\n",
    "\n",
    "endpoint_url = \"https://query.wikidata.org/sparql\"\n",
    "\n",
    "query = \"\"\"SELECT ?WDitem ?WDitemLabel ?WDNobelAPI ?WDdeath WHERE {\n",
    "  ?WDitem wdt:P8024 ?WDNobelAPI.\n",
    "  OPTIONAL {?WDitem wdt:P570 ?WDdeath}\n",
    "  SERVICE wikibase:label { bd:serviceParam wikibase:language \"en\". }\n",
    "} order by xsd:integer(?WDNobelAPI)\"\"\"\n",
    "\n",
    "\n",
    "def get_sparql_dataframe(endpoint_url, query):\n",
    "    \"\"\"\n",
    "    Helper function to convert SPARQL results into a Pandas data frame.\n",
    "    \"\"\"\n",
    "    user_agent = \"salgo60/%s.%s\" % (sys.version_info[0], sys.version_info[1])\n",
    " \n",
    "    sparql = SPARQLWrapper(endpoint_url, agent=user_agent)\n",
    "    sparql.setQuery(query)\n",
    "    sparql.setReturnFormat(JSON)\n",
    "    result = sparql.query()\n",
    "\n",
    "    processed_results = json.load(result.response)\n",
    "    cols = processed_results['head']['vars']\n",
    "    out = []\n",
    "    for row in processed_results['results']['bindings']:\n",
    "        item = []\n",
    "        for c in cols:\n",
    "            item.append(row.get(c, {}).get('value'))\n",
    "        out.append(item)\n",
    "\n",
    "    return pd.DataFrame(out, columns=cols)\n",
    "\n",
    "WDNobel = get_sparql_dataframe(endpoint_url, query)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 991 entries, 0 to 990\n",
      "Data columns (total 4 columns):\n",
      " #   Column       Non-Null Count  Dtype \n",
      "---  ------       --------------  ----- \n",
      " 0   WDitem       991 non-null    object\n",
      " 1   WDitemLabel  991 non-null    object\n",
      " 2   WDNobelAPI   991 non-null    int64 \n",
      " 3   WDdeath      661 non-null    object\n",
      "dtypes: int64(1), object(3)\n",
      "memory usage: 31.1+ KB\n"
     ]
    }
   ],
   "source": [
    "WDNobel[\"WDNobelAPI\"] = WDNobel[\"WDNobelAPI\"].astype(\"int\")\n",
    "WDNobel.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compare WD <-> Nobel "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 991 entries, 0 to 990\n",
      "Data columns (total 4 columns):\n",
      " #   Column       Non-Null Count  Dtype \n",
      "---  ------       --------------  ----- \n",
      " 0   WDitem       991 non-null    object\n",
      " 1   WDitemLabel  991 non-null    object\n",
      " 2   WDNobelAPI   991 non-null    int64 \n",
      " 3   WDdeath      661 non-null    object\n",
      "dtypes: int64(1), object(3)\n",
      "memory usage: 31.1+ KB\n"
     ]
    }
   ],
   "source": [
    "WDNobel.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "dftotPrize[\"id\"] = dftotPrize[\"id\"].astype(\"int\")\n",
    "\n",
    "#dftotPrize.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "dfmerge = pd.merge(WDNobel, dftotPrize,how='outer', left_on=\"WDNobelAPI\",right_on=\"id\",indicator=True)\n",
    "dfmerge['_merge'] = dfmerge['_merge'].str.replace('left_only','WD_only').str.replace('right_only','Nobel_only')\n",
    "dfmerge.rename(columns={\"_merge\": \"WD_Nobel\"},inplace = True)    \n",
    "#dfmerge[-10:]  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>WDitem</th>\n",
       "      <th>WDitemLabel</th>\n",
       "      <th>WDNobelAPI</th>\n",
       "      <th>WDdeath</th>\n",
       "      <th>id</th>\n",
       "      <th>fileName</th>\n",
       "      <th>gender</th>\n",
       "      <th>sameAs</th>\n",
       "      <th>links</th>\n",
       "      <th>nobelPrizes</th>\n",
       "      <th>...</th>\n",
       "      <th>familyName.no</th>\n",
       "      <th>fullName.no</th>\n",
       "      <th>foundedCountry.en</th>\n",
       "      <th>foundedCountry.no</th>\n",
       "      <th>foundedCountry.se</th>\n",
       "      <th>foundedCountryNow.en</th>\n",
       "      <th>foundedCountryNow.no</th>\n",
       "      <th>foundedCountryNow.se</th>\n",
       "      <th>foundedContinent.en</th>\n",
       "      <th>WD_Nobel</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>0 rows × 105 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [WDitem, WDitemLabel, WDNobelAPI, WDdeath, id, fileName, gender, sameAs, links, nobelPrizes, knownName.en, knownName.se, givenName.en, givenName.se, familyName.en, familyName.se, fullName.en, fullName.se, birth.date, birth.place.city.en, birth.place.city.no, birth.place.city.se, birth.place.country.en, birth.place.country.no, birth.place.country.se, birth.place.cityNow.en, birth.place.cityNow.no, birth.place.cityNow.se, birth.place.cityNow.sameAs, birth.place.countryNow.en, birth.place.countryNow.no, birth.place.countryNow.se, birth.place.countryNow.sameAs, birth.place.continent.en, birth.place.continent.no, birth.place.continent.se, birth.place.locationString.en, birth.place.locationString.no, birth.place.locationString.se, wikipedia.slug, wikipedia.english, wikidata.id, wikidata.url, death.date, death.place.city.en, death.place.city.no, death.place.city.se, death.place.country.en, death.place.country.no, death.place.country.se, death.place.country.sameAs, death.place.cityNow.en, death.place.cityNow.no, death.place.cityNow.se, death.place.cityNow.sameAs, death.place.countryNow.en, death.place.countryNow.no, death.place.countryNow.se, death.place.countryNow.sameAs, death.place.continent.en, death.place.continent.no, death.place.continent.se, death.place.locationString.en, death.place.locationString.no, death.place.locationString.se, acronym, orgName.en, orgName.no, orgName.se, founded.date, founded.place.city.en, founded.place.city.no, founded.place.city.se, founded.place.country.en, founded.place.country.no, founded.place.country.se, founded.place.country.sameAs, founded.place.cityNow.en, founded.place.cityNow.no, founded.place.cityNow.se, founded.place.cityNow.sameAs, founded.place.countryNow.en, founded.place.countryNow.no, founded.place.countryNow.se, founded.place.countryNow.sameAs, founded.place.continent.en, founded.place.continent.no, founded.place.continent.se, founded.place.locationString.en, founded.place.locationString.no, founded.place.locationString.se, nativeName, penName, knownName.no, givenName.no, familyName.no, fullName.no, foundedCountry.en, foundedCountry.no, foundedCountry.se, ...]\n",
       "Index: []\n",
       "\n",
       "[0 rows x 105 columns]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfmerge[dfmerge[\"WD_Nobel\"]!=\"both\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-19-148fcb106129>:1: FutureWarning: The default value of regex will change from True to False in a future version.\n",
      "  WDNobel['wikidata'] = WDNobel['WDitem'].str.replace('http://www.wikidata.org/entity/','')\n"
     ]
    }
   ],
   "source": [
    "WDNobel['wikidata'] = WDNobel['WDitem'].str.replace('http://www.wikidata.org/entity/','')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>WDitem</th>\n",
       "      <th>WDitemLabel</th>\n",
       "      <th>WDNobelAPI</th>\n",
       "      <th>WDdeath</th>\n",
       "      <th>wikidata</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>http://www.wikidata.org/entity/Q35149</td>\n",
       "      <td>Wilhelm Röntgen</td>\n",
       "      <td>1</td>\n",
       "      <td>1923-02-10T00:00:00Z</td>\n",
       "      <td>Q35149</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>http://www.wikidata.org/entity/Q41688</td>\n",
       "      <td>Hendrik Lorentz</td>\n",
       "      <td>2</td>\n",
       "      <td>1928-02-04T00:00:00Z</td>\n",
       "      <td>Q41688</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>http://www.wikidata.org/entity/Q79000</td>\n",
       "      <td>Pieter Zeeman</td>\n",
       "      <td>3</td>\n",
       "      <td>1943-10-09T00:00:00Z</td>\n",
       "      <td>Q79000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>http://www.wikidata.org/entity/Q41269</td>\n",
       "      <td>Henri Becquerel</td>\n",
       "      <td>4</td>\n",
       "      <td>1908-08-25T00:00:00Z</td>\n",
       "      <td>Q41269</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>http://www.wikidata.org/entity/Q37463</td>\n",
       "      <td>Pierre Curie</td>\n",
       "      <td>5</td>\n",
       "      <td>1906-04-19T00:00:00Z</td>\n",
       "      <td>Q37463</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                  WDitem      WDitemLabel  WDNobelAPI  \\\n",
       "0  http://www.wikidata.org/entity/Q35149  Wilhelm Röntgen           1   \n",
       "1  http://www.wikidata.org/entity/Q41688  Hendrik Lorentz           2   \n",
       "2  http://www.wikidata.org/entity/Q79000    Pieter Zeeman           3   \n",
       "3  http://www.wikidata.org/entity/Q41269  Henri Becquerel           4   \n",
       "4  http://www.wikidata.org/entity/Q37463     Pierre Curie           5   \n",
       "\n",
       "                WDdeath wikidata  \n",
       "0  1923-02-10T00:00:00Z   Q35149  \n",
       "1  1928-02-04T00:00:00Z   Q41688  \n",
       "2  1943-10-09T00:00:00Z   Q79000  \n",
       "3  1908-08-25T00:00:00Z   Q41269  \n",
       "4  1906-04-19T00:00:00Z   Q37463  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "WDNobel.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>WDitem</th>\n",
       "      <th>WDitemLabel</th>\n",
       "      <th>WDNobelAPI</th>\n",
       "      <th>WDdeath</th>\n",
       "      <th>wikidata</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>986</th>\n",
       "      <td>http://www.wikidata.org/entity/Q851563</td>\n",
       "      <td>Memorial</td>\n",
       "      <td>1019</td>\n",
       "      <td>None</td>\n",
       "      <td>Q851563</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>987</th>\n",
       "      <td>http://www.wikidata.org/entity/Q114452906</td>\n",
       "      <td>Center for Civil Liberties</td>\n",
       "      <td>1020</td>\n",
       "      <td>None</td>\n",
       "      <td>Q114452906</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>988</th>\n",
       "      <td>http://www.wikidata.org/entity/Q201795</td>\n",
       "      <td>Ben Bernanke</td>\n",
       "      <td>1021</td>\n",
       "      <td>None</td>\n",
       "      <td>Q201795</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>989</th>\n",
       "      <td>http://www.wikidata.org/entity/Q185871</td>\n",
       "      <td>Douglas Diamond</td>\n",
       "      <td>1022</td>\n",
       "      <td>None</td>\n",
       "      <td>Q185871</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>990</th>\n",
       "      <td>http://www.wikidata.org/entity/Q24565948</td>\n",
       "      <td>Philip H. Dybvig</td>\n",
       "      <td>1023</td>\n",
       "      <td>None</td>\n",
       "      <td>Q24565948</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                        WDitem                 WDitemLabel  \\\n",
       "986     http://www.wikidata.org/entity/Q851563                    Memorial   \n",
       "987  http://www.wikidata.org/entity/Q114452906  Center for Civil Liberties   \n",
       "988     http://www.wikidata.org/entity/Q201795                Ben Bernanke   \n",
       "989     http://www.wikidata.org/entity/Q185871             Douglas Diamond   \n",
       "990   http://www.wikidata.org/entity/Q24565948            Philip H. Dybvig   \n",
       "\n",
       "     WDNobelAPI WDdeath    wikidata  \n",
       "986        1019    None     Q851563  \n",
       "987        1020    None  Q114452906  \n",
       "988        1021    None     Q201795  \n",
       "989        1022    None     Q185871  \n",
       "990        1023    None   Q24565948  "
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "WDNobel.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "both          990\n",
       "WD_only         1\n",
       "Nobel_only      1\n",
       "Name: WD_Nobel, dtype: int64"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Merge on the Wikidata Qnumber at NobelTot  \n",
    "dfmergeWD = pd.merge(WDNobel, dftotPrize,how='outer', left_on=\"wikidata\",right_on=\"wikidata.id\",indicator=True)\n",
    "dfmergeWD['_merge'] = dfmergeWD['_merge'].str.replace('left_only','WD_only').str.replace('right_only','Nobel_only')\n",
    "dfmergeWD.rename(columns={\"_merge\": \"WD_Nobel\"},inplace = True)    \n",
    "dfmergeWD[\"WD_Nobel\"].value_counts() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>WDitem</th>\n",
       "      <th>WDitemLabel</th>\n",
       "      <th>WDNobelAPI</th>\n",
       "      <th>WDdeath</th>\n",
       "      <th>wikidata</th>\n",
       "      <th>id</th>\n",
       "      <th>fileName</th>\n",
       "      <th>gender</th>\n",
       "      <th>sameAs</th>\n",
       "      <th>links</th>\n",
       "      <th>...</th>\n",
       "      <th>familyName.no</th>\n",
       "      <th>fullName.no</th>\n",
       "      <th>foundedCountry.en</th>\n",
       "      <th>foundedCountry.no</th>\n",
       "      <th>foundedCountry.se</th>\n",
       "      <th>foundedCountryNow.en</th>\n",
       "      <th>foundedCountryNow.no</th>\n",
       "      <th>foundedCountryNow.se</th>\n",
       "      <th>foundedContinent.en</th>\n",
       "      <th>WD_Nobel</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>816</th>\n",
       "      <td>http://www.wikidata.org/entity/Q42037</td>\n",
       "      <td>Jean-Marie Gustave Le Clézio</td>\n",
       "      <td>832.0</td>\n",
       "      <td>None</td>\n",
       "      <td>Q42037</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>WD_only</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>991</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>832.0</td>\n",
       "      <td>clezio</td>\n",
       "      <td>male</td>\n",
       "      <td>[https://www.wikidata.org/wiki/Q7884119, https...</td>\n",
       "      <td>[{'rel': 'laureate', 'href': 'https://api.nobe...</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Nobel_only</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2 rows × 106 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                    WDitem                   WDitemLabel  \\\n",
       "816  http://www.wikidata.org/entity/Q42037  Jean-Marie Gustave Le Clézio   \n",
       "991                                    NaN                           NaN   \n",
       "\n",
       "     WDNobelAPI WDdeath wikidata     id fileName gender  \\\n",
       "816       832.0    None   Q42037    NaN      NaN    NaN   \n",
       "991         NaN     NaN      NaN  832.0   clezio   male   \n",
       "\n",
       "                                                sameAs  \\\n",
       "816                                                NaN   \n",
       "991  [https://www.wikidata.org/wiki/Q7884119, https...   \n",
       "\n",
       "                                                 links  ... familyName.no  \\\n",
       "816                                                NaN  ...           NaN   \n",
       "991  [{'rel': 'laureate', 'href': 'https://api.nobe...  ...           NaN   \n",
       "\n",
       "    fullName.no foundedCountry.en foundedCountry.no foundedCountry.se  \\\n",
       "816         NaN               NaN               NaN               NaN   \n",
       "991         NaN               NaN               NaN               NaN   \n",
       "\n",
       "    foundedCountryNow.en foundedCountryNow.no foundedCountryNow.se  \\\n",
       "816                  NaN                  NaN                  NaN   \n",
       "991                  NaN                  NaN                  NaN   \n",
       "\n",
       "    foundedContinent.en    WD_Nobel  \n",
       "816                 NaN     WD_only  \n",
       "991                 NaN  Nobel_only  \n",
       "\n",
       "[2 rows x 106 columns]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfmergeWD[dfmergeWD[\"WD_Nobel\"]!=\"both\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Check that Wikidata and api.nobelprize.org is in synch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "both    991\n",
       "Name: WD_Nobel, dtype: int64"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfmerge[\"WD_Nobel\"].value_counts()  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "Nobelonly = dfmerge[dfmerge[\"WD_Nobel\"]==\"Nobel_only\"].copy() \n",
    "\n",
    "from IPython.display import HTML\n",
    "#Nobelonly[\"Nobel\"] = \"<a href='https://api.nobelprize.org/v1/laureate.json?id=\" + Nobelonly[\"id\"] + \"'>link</a>\"\n",
    "#Nobelonly[\"Nobelhtml\"] = \"<a href='https://www.nobelprize.org/laureate/\" + Nobelonly[\"id\"] + \"'>html</a>\"\n",
    "#pd.set_option(\"display.max.columns\", None) \n",
    "#HTML(Nobelonly.to_html(escape=False))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "dfmerge.to_csv(\"Nobel_dfmerge_2_1.csv\")\n",
    "Nobelonly.to_csv(\"Nobel_only.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>WDitem</th>\n",
       "      <th>WDitemLabel</th>\n",
       "      <th>WDNobelAPI</th>\n",
       "      <th>WDdeath</th>\n",
       "      <th>id</th>\n",
       "      <th>fileName</th>\n",
       "      <th>gender</th>\n",
       "      <th>sameAs</th>\n",
       "      <th>links</th>\n",
       "      <th>nobelPrizes</th>\n",
       "      <th>...</th>\n",
       "      <th>familyName.no</th>\n",
       "      <th>fullName.no</th>\n",
       "      <th>foundedCountry.en</th>\n",
       "      <th>foundedCountry.no</th>\n",
       "      <th>foundedCountry.se</th>\n",
       "      <th>foundedCountryNow.en</th>\n",
       "      <th>foundedCountryNow.no</th>\n",
       "      <th>foundedCountryNow.se</th>\n",
       "      <th>foundedContinent.en</th>\n",
       "      <th>WD_Nobel</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>0 rows × 105 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [WDitem, WDitemLabel, WDNobelAPI, WDdeath, id, fileName, gender, sameAs, links, nobelPrizes, knownName.en, knownName.se, givenName.en, givenName.se, familyName.en, familyName.se, fullName.en, fullName.se, birth.date, birth.place.city.en, birth.place.city.no, birth.place.city.se, birth.place.country.en, birth.place.country.no, birth.place.country.se, birth.place.cityNow.en, birth.place.cityNow.no, birth.place.cityNow.se, birth.place.cityNow.sameAs, birth.place.countryNow.en, birth.place.countryNow.no, birth.place.countryNow.se, birth.place.countryNow.sameAs, birth.place.continent.en, birth.place.continent.no, birth.place.continent.se, birth.place.locationString.en, birth.place.locationString.no, birth.place.locationString.se, wikipedia.slug, wikipedia.english, wikidata.id, wikidata.url, death.date, death.place.city.en, death.place.city.no, death.place.city.se, death.place.country.en, death.place.country.no, death.place.country.se, death.place.country.sameAs, death.place.cityNow.en, death.place.cityNow.no, death.place.cityNow.se, death.place.cityNow.sameAs, death.place.countryNow.en, death.place.countryNow.no, death.place.countryNow.se, death.place.countryNow.sameAs, death.place.continent.en, death.place.continent.no, death.place.continent.se, death.place.locationString.en, death.place.locationString.no, death.place.locationString.se, acronym, orgName.en, orgName.no, orgName.se, founded.date, founded.place.city.en, founded.place.city.no, founded.place.city.se, founded.place.country.en, founded.place.country.no, founded.place.country.se, founded.place.country.sameAs, founded.place.cityNow.en, founded.place.cityNow.no, founded.place.cityNow.se, founded.place.cityNow.sameAs, founded.place.countryNow.en, founded.place.countryNow.no, founded.place.countryNow.se, founded.place.countryNow.sameAs, founded.place.continent.en, founded.place.continent.no, founded.place.continent.se, founded.place.locationString.en, founded.place.locationString.no, founded.place.locationString.se, nativeName, penName, knownName.no, givenName.no, familyName.no, fullName.no, foundedCountry.en, foundedCountry.no, foundedCountry.se, ...]\n",
       "Index: []\n",
       "\n",
       "[0 rows x 105 columns]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "WDonly = dfmerge[dfmerge[\"WD_Nobel\"]==\"WD_only\"].copy() \n",
    "WDonly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "End run:  2022-10-21 07:47:14.350743\n"
     ]
    }
   ],
   "source": [
    "print(\"End run: \", datetime.now())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time elapsed (hh:mm:ss.ms) 0:00:03.376840\n"
     ]
    }
   ],
   "source": [
    "print('Time elapsed (hh:mm:ss.ms) {}'.format(datetime.now() - start_time))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lesson learned\n",
    "Its a delay when the html page is linked after the announcement --> we do this list below to check when its ready and we can add it to Wikidata and the WIkipedia article"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Status 20201009 :\n",
    "* Ok all has html pages\n",
    "\n",
    "Templates supporting Wikidata Property 8024 = Wikidata [Q91652187](https://www.wikidata.org/wiki/Q91652187)\n",
    "* ar:Wikipedia [قالب:جائزة_نوبل](https://ar.wikipedia.org/wiki/%D9%82%D8%A7%D9%84%D8%A8:%D8%AC%D8%A7%D8%A6%D8%B2%D8%A9_%D9%86%D9%88%D8%A8%D9%84)\n",
    "* ca:wikipedia [Plantilla:Nobelprize](https://ca.wikipedia.org/wiki/Plantilla:Nobelprize)\n",
    "* en:Wikipedia [Template:Nobelprize](https://en.wikipedia.org/wiki/Template:Nobelprize)\n",
    "* ka:Wikipedia [თარგი:Nobelprize](https://ka.wikipedia.org/wiki/%E1%83%97%E1%83%90%E1%83%A0%E1%83%92%E1%83%98:Nobelprize)\n",
    "* nn:Wikipedia [Mal:Nobelprize](https://nn.wikipedia.org/wiki/Mal:Nobelprize)\n",
    "* pt:Wikipedia [Predefinição:Prémio Nobel](https://pt.wikipedia.org/wiki/Predefini%C3%A7%C3%A3o:Pr%C3%A9mio_Nobel)\n",
    "* sv:Wikipedia [Mall:Nobelprize](https://sv.wikipedia.org/wiki/Mall:Nobelprize)\n",
    "* sr:WIkipedia [Шаблон:Nobelprize](https://sr.wikipedia.org/wiki/%D0%A8%D0%B0%D0%B1%D0%BB%D0%BE%D0%BD:Nobelprize)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Articles using [template](https://www.wikidata.org/wiki/Q91652187) and view statistics this year and during the [Nobel week](https://www.nobelprize.org/press/#/publication/5f1719e411aab40004a17f36/552bd85dccc8e20c00e7f979?&sh=false)  5-12 October 2020 on those articles see also [video](https://youtu.be/Iu1JtefueM8)\n",
    "  * ar:Wikipedia [using](https://petscan.wmflabs.org/?psid=17573153) / not using\n",
    "    * [statistics](https://pageviews.toolforge.org/massviews/?platform=all-access&agent=user&source=category&range=this-year&subjectpage=0&subcategories=0&sort=views&direction=1&view=list&target=https://ar.wikipedia.org/wiki/%25D8%25AA%25D8%25B5%25D9%2586%25D9%258A%25D9%2581:%25D9%2582%25D8%25A7%25D9%2584%25D8%25A8_%25D8%25AC%25D8%25A7%25D8%25A6%25D8%25B2%25D8%25A9_%25D9%2586%25D9%2588%25D8%25A8%25D9%2584_%25D9%258A%25D8%25B3%25D8%25AA%25D8%25B9%25D9%2585%25D9%2584_%25D8%25AE%25D8%25A7%25D8%25B5%25D8%25A9_%25D9%2588%25D9%258A%25D9%2583%25D9%258A_%25D8%25A8%25D9%258A%25D8%25A7%25D9%2586%25D8%25A7%25D8%25AA_P8024) / [Nobel week](https://pageviews.toolforge.org/massviews/?platform=all-access&agent=user&source=category&start=2020-10-05&end=2020-10-12&subjectpage=0&subcategories=0&sort=views&direction=1&view=list&target=https://ar.wikipedia.org/wiki/%25D8%25AA%25D8%25B5%25D9%2586%25D9%258A%25D9%2581:%25D9%2582%25D8%25A7%25D9%2584%25D8%25A8_%25D8%25AC%25D8%25A7%25D8%25A6%25D8%25B2%25D8%25A9_%25D9%2586%25D9%2588%25D8%25A8%25D9%2584_%25D9%258A%25D8%25B3%25D8%25AA%25D8%25B9%25D9%2585%25D9%2584_%25D8%25AE%25D8%25A7%25D8%25B5%25D8%25A9_%25D9%2588%25D9%258A%25D9%2583%25D9%258A_%25D8%25A8%25D9%258A%25D8%25A7%25D9%2586%25D8%25A7%25D8%25AA_P8024)\n",
    "  * ca:Wikipedia miss track category [Q91672712](https://www.wikidata.org/wiki/Q91672712#sitelinks-wikipedia)\n",
    "  * en:Wikipedia [using](https://petscan.wmflabs.org/?psid=17573140) / [not using](https://petscan.wmflabs.org/?psid=17579263) \n",
    "    * [statistics](https://pageviews.toolforge.org/massviews/?platform=all-access&agent=user&source=category&range=this-year&subjectpage=0&subcategories=0&sort=views&direction=1&view=list&target=https://en.wikipedia.org/wiki/Category:Nobelprize_template_using_Wikidata_property_P8024) / [Nobel week](https://pageviews.toolforge.org/massviews/?platform=all-access&agent=user&source=category&start=2020-10-05&end=2020-10-12&subjectpage=0&subcategories=0&sort=views&direction=1&view=list&target=https://en.wikipedia.org/wiki/Category:Nobelprize_template_using_Wikidata_property_P8024) as a [chart](https://pageviews.toolforge.org/massviews/?platform=all-access&agent=user&source=category&start=2020-10-05&end=2020-10-10&subjectpage=0&subcategories=0&sort=views&direction=1&view=chart&target=https://en.wikipedia.org/wiki/Category:Nobelprize%20template%20using%20Wikidata%20property%20P8024)\n",
    "  * ka:Wikipedia miss track category [Q91672712](https://www.wikidata.org/wiki/Q91672712#sitelinks-wikipedia)\n",
    "  * nn:Wikipedia [using](https://petscan.wmflabs.org/?psid=17573155) / not using \n",
    "     * [statistics](https://pageviews.toolforge.org/massviews/?platform=all-access&agent=user&source=category&range=this-year&subjectpage=0&subcategories=0&sort=views&direction=1&view=list&target=https://nn.wikipedia.org/wiki/Kategori:Sider_som_nyttar_Mal:Nobelprize) / [Nobel week](https://pageviews.toolforge.org/massviews/?platform=all-access&agent=user&source=category&start=2020-10-05&end=2020-10-12&subjectpage=0&subcategories=0&sort=views&direction=1&view=list&target=https://nn.wikipedia.org/wiki/Kategori:Sider_som_nyttar_Mal:Nobelprize)\n",
    "  * pt:Wikipedia [using](https://petscan.wmflabs.org/?psid=17573158) / [not using](https://petscan.wmflabs.org/?psid=17579612) \n",
    "     * [statistics](https://pageviews.toolforge.org/massviews/?platform=all-access&agent=user&source=category&range=this-year&subjectpage=0&subcategories=0&sort=views&direction=1&view=list&target=https://pt.wikipedia.org/wiki/Categoria:Predefini%25C3%25A7%25C3%25A3o_sobre_pr%25C3%25A9mios_Nobel_que_usam_a_propriedade_do_Wikidata_P8024) / [Nobel week](https://pageviews.toolforge.org/massviews/?platform=all-access&agent=user&source=category&start=2020-10-05&end=2020-10-12&subjectpage=0&subcategories=0&sort=views&direction=1&view=list&target=https://pt.wikipedia.org/wiki/Categoria:Predefini%25C3%25A7%25C3%25A3o_sobre_pr%25C3%25A9mios_Nobel_que_usam_a_propriedade_do_Wikidata_P8024)\n",
    "  * sv:Wikipedia [using](https://petscan.wmflabs.org/?psid=17573129) / not using \n",
    "    * [statistics](https://pageviews.toolforge.org/massviews/?platform=all-access&agent=user&source=category&range=this-year&subjectpage=0&subcategories=0&sort=views&direction=1&view=list&target=https://sv.wikipedia.org/wiki/Kategori:Sidor_som_anv%25C3%25A4nder_Mall:Nobelprize) / [Nobel week](https://pageviews.toolforge.org/massviews/?platform=all-access&agent=user&source=category&start=2020-10-05&end=2020-10-12&subjectpage=0&subcategories=0&sort=views&direction=1&view=list&target=https://sv.wikipedia.org/wiki/Kategori:Sidor_som_anv%25C3%25A4nder_Mall:Nobelprize)\n",
    "  * sr:Wikipedia miss track category [Q91672712](https://www.wikidata.org/wiki/Q91672712#sitelinks-wikipedia)\n",
    "  \n",
    "Task [T251055](https://phabricator.wikimedia.org/T251055)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Next Nobelprize announcement\n",
    "see https://www.nobelprize.org/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}