The_art_of_prompt_engineering-20230607-1686096000.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"text-align:center\">\n",
    "    <a href=\"https://skills.network\" target=\"_blank\">\n",
    "    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/assets/logos/SN_web_lightmode.png\" width=\"200\" alt=\"Skills Network Logo\">\n",
    "    </a>\n",
    "</p>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# **The Art of Prompt Engineering**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Estimated time needed: **30** minutes\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center>\n",
    "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMSkillsNetwork-GPXX0TGVEN/images/art%20of%20prompt%20eng.jpeg\" width=\"500\" alt=\"Skills Network Logo\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This project delves into the concept of prompt engineering, a crucial aspect of AI that guides AI models to produce desired outputs. It's instrumental in AI tools like chatbots, text summarizers, and content generators, ensuring they communicate accurately, relevantly, and creatively. The project underscores the benefits of prompt engineering in business, from enhancing chatbot performance to guiding personalization systems, enabling accurate sentiment analysis, and optimizing productivity. It emphasizes the importance of clear instructions, examples, keywords, and feedback when working with large language models (LLMs) like ChatGPT. The project also outlines various techniques, including writing clear instructions, giving the model time to \"think\", and strategies like Chain-of-Thought (CoT) Prompting and Generated Knowledge Prompting. We also introduce Langchain library that can provide strong tools for prompt engineering.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __Table of Contents__\n",
    "\n",
    "\n",
    "1. [Introduction](#introduction)\n",
    "2. [Importance of Prompt Engineering in AI Tools](#importance-in-ai-tools)\n",
    "3. [Benefits of Prompt Engineering in Business](#benefits-in-business)\n",
    "4. [Prompt Engineering with LLM models like ChatGPT](#prompt-engineering-with-llm)\n",
    "5. [Prompt Engineering Techniques](#prompt-engineering-techniques)\n",
    "    1. [Principle 1: Write Clear and Specific Instructions](#principle-1)\n",
    "    2. [Principle 2: Give the Model Time to “Think”](#principle-2)\n",
    "    3. [Additional Strategies](#additional-strategies)\n",
    "6. [Python Project: Survival Analysis](#python-project)\n",
    "    1. [Understanding the Concept](#understanding-the-concept)\n",
    "    2. [Learning the Basics](#learning-the-basics)\n",
    "    3. [Python Implementation](#python-implementation)\n",
    "    4. [Asking Further Questions](#asking-further-questions)\n",
    "7. [Prompt Engineering with Langchain](#python-project)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Objectives\n",
    "\n",
    "After completing this lab you will be able to:\n",
    "\n",
    "1. Learn to communicate effectively with AI models like ChatGPT.\n",
    "2. Gain hands-on experience in prompt engineering in implementing survival analysis in Python.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "Prompt engineering is key in AI, especially in natural language processing. It guides AI models to produce desired outputs like answering questions or generating summaries.\n",
    "\n",
    "## Importance in AI Tools\n",
    "Prompt engineering is vital in AI tools like chatbots, text summarizers, and content generators. It ensures clear communication with the model for accurate, relevant, and creative outputs. For example, in the case of chatbots, it is essential to have suitable inquiries and replies that keep conversations natural and captivating. Likewise, when it comes to text summarizers, they need to present key points and adhere to specified summary length, whereas content generators rely on prompts regarding topic, genre, and tone.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Benefits of Prompt Engineering in Business:\n",
    "\n",
    "Prompt engineering can have a positive impact on productivity and bring various benefits to businesses. Several studies have highlighted the advantages of prompt engineering in improving business outcomes:\n",
    "\n",
    "| Area | Importance of Prompt Engineering | Statistics | Source |\n",
    "| --- | --- | --- | --- |\n",
    "| Chatbot Performance | Ensures accurate and relevant responses, enhancing customer satisfaction and reducing the need for human intervention. | Chatbots will save businesses an estimated $11 billion annually by 2023. | Juniper Research. “Chatbots: Retail, eCommerce, Banking & Healthcare 2018–2023.” |\n",
    "| Content Generation | Boosts the efficiency and quality of content generation processes. | Businesses that prioritize content marketing experience six times higher conversion rates. | HubSpot. “The Ultimate List of Marketing Statistics for 2021.” |\n",
    "| Personalization and Recommendation Systems | Guides systems to understand user preferences and provide relevant suggestions. | Personalization can deliver 40% more revenue and lift revenue by 10 to 15 percent or more. | McKinsey & Company. “The CEO guide to personalization.” |\n",
    "| Sentiment Analysis and Brand Reputation | Enables AI systems to accurately analyze sentiment and identify trends in customer feedback and reviews. | 88% of consumers are more likely to recommend a company to others after a positive customer experience. | Temkin Group. “ROI of Customer Experience, 2018.” |\n",
    "| Productivity and Process Optimization | Enhances internal processes, increasing efficiency and productivity within organizations. | Automation technologies could increase productivity by 0.8% to 1.4% annually, saving almost 15 trillion in wages. | McKinsey Global Institute. “A future that works: Automation, employment, and productivity.” |\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prompt Engineering with LLM Model Like ChatGPT\n",
    "\n",
    "Prompt engineering is a key factor in harnessing the full potential of AI models like ChatGPT. A well-crafted prompt can guide the model to generate accurate, relevant, and coherent responses, while minimizing errors and misunderstandings. Conversely, poorly constructed prompts can lead to irrelevant, ambiguous, or even incorrect outputs. Therefore, investing time in crafting efficient prompts is crucial for obtaining the best results from ChatGPT. Using following aspects you can format your prompt, properly.\n",
    "\n",
    "\n",
    "| Aspect | Description | Example |\n",
    "| --- | --- | --- |\n",
    "| Clear and Specific Instructions | Provide explicit instructions on the desired task and how the model should approach it. | \"Compose a four-line poem on the subject of love, utilizing the ABAB rhyme scheme.\" |\n",
    "| Examples and Templates | Illustrate the desired output by including examples or templates. | \"Summary: [one sentence capturing the article’s main idea].\" |\n",
    "| Keywords and Cues | Employ keywords and cues to guide the model’s attention toward relevant information and context. | \"Alice: [a young witch eager to learn magic]. Bob: [a wise wizard instructing Alice]. Dialogue:\" |\n",
    "| Feedback and Refinement | Evaluate the model’s output and provide feedback for refinement. | \"Slogan for a new toothpaste that whitens teeth and freshens breath. Output: [suggested slogan]. Feedback: [too long/too boring/too generic, etc.]. Refinement: [suggestion for improvement].\" |\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Prompt Engineering Techniques\n",
    "\n",
    "We define principles and strategies to consider creating prompts. Before we delve into them, here is the summary table:\n",
    "\n",
    "\n",
    "\n",
    "| Principle | Strategies | Description | Example |\n",
    "|---|---|---|---|\n",
    "| **Principle 1: Write Clear and Specific Instructions** | Strategy 1: Use Delimiters to Clearly Indicate Distinct Parts of the Input | Delimiters help avoid potential interference from misleading user input. | `prompt = \"\"\"Summarize the text delimited by triple backticks into a single sentence.```{text}```\"` |\n",
    "| | Strategy 2: Ask for Structured Output | This approach helps make model outputs directly usable for programs, such as JSON outputs that can be read and converted into dictionary format by Python programs. | `prompt = \"\"\"Generate a list of three made-up book titles along with their authors and genres. Provide them in JSON format with the following keys: book_id, title, author, genre.\"\"\"` |\n",
    "| | Strategy 3: Check Whether Conditions are Satisfied | If the completion of the task has preconditions that must be met, we should require the model to check these conditions first and instruct it to stop trying if they are not met. | `prompt = \"\"\"You will be provided with text delimited by triple quotes. If it contains a sequence of instructions, re-write those instructions in the following format: Step 1 - ... Step 2 - … … Step N - … If the text does not contain a sequence of instructions, then simply write \"No steps provided.\"\\\"\\\"\\\"{text_1}\\\"\\\"\\\"\"\"\"` |\n",
    "| | Strategy 4: “N-shot” Prompting | Providing the model with one or more sample prompts helps clarify the expected output. | `prompt = \"\"\"Your task is to answer in a consistent style.<child>: Teach me about patience.<grandparent>: The river that carves the deepest valley flows from a modest spring; the grandest symphony originates from a single note; the most intricate tapestry begins with a solitary thread.<child>: Teach me about resilience.\"\"\"` |\n",
    "| **Principle 2: Give the Model Time to “Think”** | Strategy 1: Specify the Steps Required to Complete a Task | By providing the necessary steps, the model can reference the results of previous steps and improve the accuracy of the output. | `prompt = \"\"\"Your task is to perform the following actions: 1 - Summarize the following text delimited by <> with 1 sentence. 2 - Translate the summary into French. 3 - List each name in the French summary. 4 - Output a json object that contains the following keys: french_summary, num_names.\"\"\"` |\n",
    "| | Strategy 2: Instruct the Model to Work Out Its Own Solution Before Rushing to a Conclusion | If the task is too complicated or the description is too little, then the model can only draw conclusions by guessing. So, in this case, we can instruct the model to take longer to think about the problem. | `prompt = \"\"\"Your task is to determine if the student's solution is correct or not. To solve the problem do the following: - First, work out your own solution to the problem. - Then compare your solution to the student's solution and evaluate if the student's solution is correct or not. Don't decide if the student's solution is correct until you have done the problem yourself.\"\"\"` |\n",
    "| **Additional Strategies** | Chain-of-Thought (CoT) Prompting | CoT prompting prompts the model to produce intermediate reasoning steps before giving the final answer to a multi-step problem. |  `prompt = \"\"\" Your task is to solve the following math problem: 2x^2 - 3x + 1 = 0. Break down the problem into simpler subproblems, solve each one in sequence, and then combine the solutions to solve the original problem.\"\"\"` |\n",
    "| | Generated Knowledge Prompting | The idea behind the generated knowledge prompting is to ask the AI to generate potentially useful information about a given question/prompt, and then leverage that provided knowledge as additional input for generating a final | `prompt: \"\"\"Before writing an article about cybersecurity, particularly cookie theft, generate some dangers and protections against cookie theft. \"\"\"prompt = f\"\"\" Now, using the generated knowledge, write an article about cybersecurity with a focus on cookie theft. Dangers: {response['dangers']} Protections: {response['protections']} \"\"\"` In this example, the AI is first asked to generate some dangers and protections against cookie theft. Then, it is asked to use this generated knowledge to write an article about cybersecurity.  |\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Principle 1: Write Clear and Specific Instructions\n",
    "\n",
    "Ensure your prompts are clear and concise to help the model understand the intent and desired output. Avoid ambiguous language or phrasing that could lead to multiple interpretations. This can be accomplished with strategies such as:\n",
    "\n",
    "#### Strategy 1: Use Delimiters to Clearly Indicate Distinct Parts of the Input\n",
    "\n",
    "Delimiters help avoid potential interference from misleading user input. Examples of delimiters include:\n",
    "\n",
    "- Triple quotes: `\"\"\"`\n",
    "- Triple backticks: ```` ``` ````\n",
    "- Triple dashes: `---`\n",
    "- Angle brackets: `<>`\n",
    "- XML tags: `<tag>`\n",
    "\n",
    "Example:\n",
    "\n",
    "```python\n",
    "text = \"\"\"\n",
    "You should express what you want a model to do by providing instructions that are as clear and specific as you can possibly make them. This will guide the model towards the desired output, and reduce the chances of receiving irrelevant or incorrect responses. Don't confuse writing a clear prompt with writing a short prompt. In many cases, longer prompts provide more clarity and context for the model, which can lead to more detailed and relevant outputs.\n",
    "\"\"\"\n",
    "prompt = \"\"\"\n",
    "Summarize the text delimited by triple backticks into a single sentence.\n",
    "```{text}```\n",
    "\"\"\"\n",
    "response = get_completion(prompt)\n",
    "print(response)\n",
    "```\n",
    "\n",
    "\n",
    "\n",
    "#### Strategy 2: Ask for Structured Output\n",
    "\n",
    "This approach helps make model outputs directly usable for programs, such as JSON outputs that can be read and converted into dictionary format by Python programs.\n",
    "\n",
    "Example:\n",
    "\n",
    "```python\n",
    "prompt = \"\"\"\n",
    "Generate a list of three made-up book titles along with their authors and genres. Provide them in JSON format with the following keys: book_id, title, author, genre.\n",
    "\"\"\"\n",
    "response = get_completion(prompt)\n",
    "print(response)\n",
    "```\n",
    "\n",
    "<mark> Result:\n",
    "```json\n",
    "[\n",
    "  {\n",
    "    \"book_id\": 1,\n",
    "    \"title\": \"The Lost City of Zorath\",\n",
    "    \"author\": \"Aria Blackwood\",\n",
    "    \"genre\": \"Fantasy\"\n",
    "  },\n",
    "  {\n",
    "    \"book_id\": 2,\n",
    "    \"title\": \"The Last Survivors\",\n",
    "    \"author\": \"Ethan Stone\",\n",
    "    \"genre\": \"Science Fiction\"\n",
    "  },\n",
    "]\n",
    "\n",
    "```\n",
    "\n",
    "#### Strategy 3: Check Whether Conditions are Satisfied\n",
    "\n",
    "If the completion of the task has preconditions that must be met, we should require the model to check these conditions first and instruct it to stop trying if they are not met.\n",
    "\n",
    "Example:\n",
    "\n",
    "```python\n",
    "text_1 = \"\"\"\n",
    "Making a cup of tea is easy! First, you need to get some water boiling. While that's happening, grab a cup and put a tea bag in it. Once the water is hot enough, just pour it over the tea bag. Let it sit for a bit so the tea can steep. After a few minutes, take out the tea bag. If you like, you can add some sugar or milk to taste. And that's it! You've got yourself a delicious cup of tea to enjoy.\n",
    "\"\"\"\n",
    "prompt = \"\"\"\n",
    "You will be provided with text delimited by triple quotes. If it contains a sequence of instructions, re-write those instructions in the following format:\n",
    "Step 1 - ...\n",
    "Step 2 - …\n",
    "…\n",
    "Step N - …\n",
    "If the text does not contain a sequence of instructions, then simply write \"No steps provided.\"\n",
    "\\\"\\\"\\\"{text_1}\\\"\\\"\\\"\n",
    "\"\"\"\n",
    "response = get_completion(prompt)\n",
    "print(\"Completion for Text 1:\")\n",
    "print(response)\n",
    "```\n",
    "\n",
    "<mark> Output:\n",
    "```\n",
    "Step 1 - Get some water boiling.\n",
    "Step 2 - Grab a cup and put a tea bag in it.\n",
    "Step 3 - Once the water is hot enough, pour it over the tea bag.\n",
    "Step 4 - Let it sit for a bit so the tea can steep.\n",
    "Step 5 - After a few minutes, take out the tea bag.\n",
    "Step 6 - If you like, you can add some sugar or milk to taste.\n",
    "```\n",
    "\n",
    "\n",
    "\n",
    "#### Strategy 4: “N-shot” Prompting\n",
    "\n",
    "Providing the model with one or more sample prompts helps clarify the expected output.\n",
    "Providing the model with one or more sample prompts helps clarify the expected output.\n",
    "N-shot prompting, including Zero-shot and Few-shot prompting, refers to the number of \"training\" examples or clues given to the model to make predictions.\n",
    "\n",
    "| Type | Description | Example |\n",
    "| --- | --- | --- |\n",
    "| Zero-shot prompting | The model makes predictions without any additional training. This works for common straightforward problems like classification or text transformation. | classification (i.e. sentiment analysis, spam classification => \"Is this email spam or not? ==> <i>PASTE THE EMAIL CONTENT</i> \"), text transformation (i.e. translation, summarizing, expanding), and simple text generation on which the LLM has been largely trained  |\n",
    "| Few-shot prompting | Uses a small amount of data (typically between two and five) to adapt its output based on these small examples. These examples are meant to steer the model to better performance for a more context-specific problem. | \"Translate the following English sentences to French: 'Hello, how are you?' 'I am fine, thank you.'\" |\n",
    "\n",
    "\n",
    "Example:\n",
    "\n",
    "```python\n",
    "prompt = \"\"\"\n",
    "Your task is to answer in a consistent style.\n",
    "<child>: Teach me about patience.\n",
    "<grandparent>: The river that carves the deepest valley flows from a modest spring; the grandest symphony originates from a single note; the most intricate tapestry begins with a solitary thread.\n",
    "<child>: Teach me about resilience.\n",
    "\"\"\"\n",
    "response = get_completion(prompt)\n",
    "print(response)\n",
    "```\n",
    "<mark> Output:\n",
    "```\n",
    "<grandparent>: Resilience is like a tree that bends but does not break in the face of a storm. It's about facing life's challenges and bouncing back stronger than before.\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Principle 2: Give the Model Time to “Think”\n",
    "\n",
    "This principle utilizes the idea of a thought chain, breaking complex tasks into N sequential subtasks, allowing the model to think step-by-step and produce more accurate outputs.\n",
    "\n",
    "#### Strategy 1: Specify the Steps Required to Complete a Task\n",
    "\n",
    "Here’s an example involving summarizing text, translating it into French, listing names in the French summary, and finally outputting data in JSON format. By providing the necessary steps, the model can reference the results of previous steps and improve the accuracy of the output.\n",
    "\n",
    "```python\n",
    "prompt = \"\"\"\n",
    "Your task is to perform the following actions: \n",
    "1 - Summarize the following text delimited by <> with 1 sentence.\n",
    "2 - Translate the summary into French.\n",
    "3 - List each name in the French summary.\n",
    "4 - Output a json object that contains the following keys: french_summary, num_names.\n",
    "\n",
    "Use the following format:\n",
    "Text: <text to summarize>\n",
    "Summary: <summary>\n",
    "Translation: <summary translation>\n",
    "Names: <list of names in French summary>\n",
    "Output JSON: <json with summary and num_names>\n",
    "\n",
    "Text: <{text}>\n",
    "\"\"\"\n",
    "response = get_completion(prompt)\n",
    "print(response)\n",
    "```\n",
    "<mark> Outcome example:\n",
    "```\n",
    "Text: <John and Mary went to the park. They played football and had a picnic.>\n",
    "Summary: <John and Mary enjoyed a day at the park playing football and having a picnic.>\n",
    "Translation: <John et Mary ont passé une journée agréable au parc en jouant au football et en pique-niquant.>\n",
    "Names: <['John', 'Mary']>\n",
    "Output JSON: <{\"french_summary\": \"John et Mary ont passé une journée agréable au parc en jouant au football et en pique-niquant.\", \"num_names\": 2}>\n",
    "\n",
    "```\n",
    "\n",
    "#### Strategy 2: Instruct the Model to Work Out Its Own Solution Before Rushing to a Conclusion\n",
    "\n",
    "If the task is too complicated or the description is too little, then the model can only draw conclusions by guessing. So, in this case, we can instruct the model to take longer to think about the problem.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Additional Strategies\n",
    "### Chain-of-Thought (CoT) Prompting\n",
    "\n",
    "Introduced by Google researchers in 2022, CoT prompting prompts the model to produce intermediate reasoning steps before giving the final answer to a multi-step problem. This method enables models to decompose multi-step problems into intermediate steps, enabling them to solve complex reasoning problems that are not solvable with standard prompting methods.\n",
    "\n",
    "Absolutely, here are examples for each type of Chain-of-Thought (CoT) prompting:\n",
    "\n",
    "| Type | Description | Example |\n",
    "| --- | --- | --- |\n",
    "| Self-consistency prompting | Sample multiple diverse reasoning paths and select the most consistent answers. | If the task is to predict the weather, the model might generate multiple predictions like \"It will rain\", \"It will be cloudy\", and \"It will be sunny\". The model then selects the most consistent answer. |\n",
    "| Least-to-Most prompting (LtM) | Specify the chain of thought to first break a problem into a series of simpler subproblems and then solve them in sequence. | If the task is to solve a complex math problem, the model first breaks it down into simpler subproblems, solves each one in sequence, and then combines the solutions to solve the original problem. |\n",
    "| Active Prompting | Determine which questions are the most important and helpful ones for human annotation. | If the task is to write a detailed report, the model first identifies the most uncertain areas (e.g., specific data points or arguments), asks for human annotation on these areas, and then incorporates the annotated information into the final report. |\n",
    "\n",
    "### Generated Knowledge Prompting\n",
    "\n",
    "The idea behind the generated knowledge prompting is to ask the AI to generate potentially useful information about a given question/prompt, and then leverage that provided knowledge as additional input for generating a final response.\n",
    "\n",
    "For example, say you want to write an article about cybersecurity, particularly cookie theft. Before asking the AI to write the article, you can ask it to generate some dangers and protections against cookie theft. This will help the AI write a more informative blog post.\n",
    "\n",
    "In addition to these techniques, a new method called \"Prompt Tuning\" has been introduced, which involves learning \"soft prompts\" to condition frozen language models to perform specific downstream tasks. Unlike the discrete text prompts used by GPT-3, soft prompts are learned through backpropagation and can be tuned to incorporate signal from any number of labeled examples. This approach outperforms GPT-3's \"few-shot\" learning by a large margin and becomes more competitive with scale. As models exceed billions of parameters, this method matches the strong performance of model tuning (where all model weights are tuned) [source](https://arxiv.org/abs/2104.08691).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Put Into Practice With Python Project\n",
    "\n",
    "Let's imagine you're interested in learning survival analysis in Python, but you have limited knowledge about this technique and its practical applications. We can leverage the power of ChatGPT and effective prompt engineering to explore and understand survival analysis together.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "let's embark on this learning journey together. We'll use the principles of prompt engineering to guide our exploration of survival analysis in Python. Here's how we might structure our inquiry:\n",
    "\n",
    "1. **Understanding the Concept**: First, we need to understand what survival analysis is. We can ask ChatGPT to provide a definition and explanation of survival analysis.\n",
    "\n",
    "```python\n",
    "prompt = \"\"\"\n",
    "Please provide a detailed explanation of survival analysis, including its definition, purpose, and applications.\n",
    "\"\"\"\n",
    "```\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>Click here to see the outcome from chatGPT</summary>\n",
    "\n",
    "## Person: \"Please provide a detailed explanation of survival analysis, including its definition, purpose, and applications.\"\n",
    "## ChatGPT:\n",
    "\n",
    "Survival analysis is a branch of statistics that deals with the analysis of time-to-event data. These are situations where the outcome of interest is not just whether an event happened, but when it happened. The \"event\" in question is often death (hence the term \"survival analysis\"), but it can be any event of interest: the failure of a machine, the recovery of a patient, the relapse of a disease, the churn of a customer, etc.\n",
    "\n",
    "Here are some key points about survival analysis:\n",
    "\n",
    "**Definition and Purpose**\n",
    "\n",
    "Survival analysis is used to analyze the time until the occurrence of an event of interest. The goal is to estimate and interpret survivor functions and hazard functions. \n",
    "\n",
    "- The survivor function, denoted S(t), is the probability that the event of interest has not yet occurred by time t.\n",
    "- The hazard function, denoted h(t), is the event rate at time t conditional on survival until time t (i.e., the risk of the event per time unit).\n",
    "\n",
    "Survival analysis takes into account censoring, which occurs when we have some information about individual survival time, but we don’t know the survival time exactly (for example, if a patient is still alive at the end of the study, or if a customer has not yet churned by the time data were collected).\n",
    "\n",
    "**Applications**\n",
    "\n",
    "Survival analysis has applications in many fields:\n",
    "\n",
    "1. **Medicine**: This is the field where survival analysis is most used. It's often used to measure the fraction of patients living for a certain amount of time after treatment. In this context, the event of interest is often death, but can also be the occurrence of a disease, or recovery from surgery.\n",
    "\n",
    "2. **Biology**: Survival analysis can be used to measure lifetimes of species.\n",
    "\n",
    "3. **Engineering**: In reliability engineering, it can be used to analyze time-to-failure data.\n",
    "\n",
    "4. **Marketing and Business**: Survival analysis can be used to predict customer churn and understand customer retention.\n",
    "\n",
    "5. **Social Sciences**: It can be used to analyze the time until events like marriage, divorce, or retirement.\n",
    "\n",
    "**Methods**\n",
    "\n",
    "There are several methods used in survival analysis, including the Kaplan-Meier estimator, the Nelson-Aalen estimator, and Cox proportional hazards models. These methods have different strengths and assumptions, and the choice of method depends on the specifics of the problem at hand.\n",
    "\n",
    "- The Kaplan-Meier estimator is a non-parametric statistic used to estimate the survival function from lifetime data. It allows for varying survival times and can handle censored data.\n",
    "\n",
    "- The Nelson-Aalen estimator is another non-parametric statistic used to estimate the cumulative hazard function in survival analysis.\n",
    "\n",
    "- Cox proportional hazards model (or Cox regression model) is a semi-parametric model that allows the inclusion of multiple covariates and assumes that the effects of the covariates are multiplicative with respect to the hazard rate and are constant over time.\n",
    "\n",
    "In conclusion, survival analysis is a powerful tool for analyzing time-to-event data, with applications in many fields. It's particularly useful when dealing with censored data, which are common in many research areas.\n",
    "\n",
    "\n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. **Learning the Basics**: Next, we can ask ChatGPT to explain the basic principles and techniques used in survival analysis.\n",
    "\n",
    "```python\n",
    "prompt = \"\"\"\n",
    "Could you explain the basic principles and techniques used in survival analysis?\n",
    "\"\"\"\n",
    "\n",
    "```\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>Click here to see the outcome from chatGPT</summary>\n",
    "\n",
    "## Person: \"Could you explain the basic principles and techniques used in survival analysis?\"\n",
    "## ChatGPT:\n",
    "\n",
    "Sure, I'd be happy to explain the basic principles and techniques used in survival analysis. \n",
    "\n",
    "**Basic Principles**\n",
    "\n",
    "1. **Survival Function (S(t))**: This is the probability that a subject survives from the time origin (t=0) to a specified future time t. Survival function is defined as S(t) = Pr(T > t), where T is the random variable denoting the time of death.\n",
    "\n",
    "2. **Hazard Function (h(t))**: This is the event rate at time t conditional on survival until time t or later (i.e., T ≥ t). It can be interpreted as the risk of dying at time t. It is formally defined as the derivative of the survival function.\n",
    "\n",
    "3. **Censoring**: This is a type of missing data problem common in survival analysis. Right censoring is the most common type, which happens if a patient is still alive (or the event has not occurred) at the end of the observation period. The exact survival time for this patient is not known, but it is known to be longer than a certain time.\n",
    "\n",
    "**Techniques**\n",
    "\n",
    "1. **Kaplan-Meier Estimator**: This is a non-parametric statistic used to estimate the survival function from lifetime data. In medical research, it is often used to measure the fraction of patients living for a certain amount of time after treatment. The estimator is named after Edward L. Kaplan and Paul Meier, who each submitted similar manuscripts to the Journal of the American Statistical Association. The journal editor, John Tukey, convinced them to combine their work into one paper, which has since been cited over 50,000 times.\n",
    "\n",
    "2. **Log-Rank Test**: This is a hypothesis test to compare the survival distributions of two samples. It is used to test the null hypothesis that the survival distributions are the same across the groups. The test statistic is a sum of the observed and expected number of events in one of the groups at each observed event time.\n",
    "\n",
    "3. **Cox Proportional Hazards Model**: This is a semi-parametric model developed by Sir David Cox in 1972. It allows us to estimate the hazard (or risk) of death, based on the values of covariates for each individual. The model's name is derived from the proportional hazards assumption, which implies that the hazard functions of individuals are proportional and the ratio of hazard functions is constant over time.\n",
    "\n",
    "4. **Accelerated Failure Time Model (AFT)**: This is a parametric model that provides an alternative to the Cox model. Instead of modelling the hazard function, the AFT model focuses on the survival time. The model assumes that covariates have a multiplicative effect on the survival time.\n",
    "\n",
    "5. **Parametric Survival Models**: These are models that assume a particular distribution for the survival times (e.g., exponential, Weibull, Gompertz). They can provide a better fit to data than the Cox model if the distributional assumption is correct, and they can provide more interpretable summaries of effects.\n",
    "\n",
    "These are just a few of the basic principles and techniques used in survival analysis. The choice of technique depends on the specifics of the problem at hand, including the nature of the data and the research question.\n",
    "\n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. **Python Implementation**: Once we understand the basics, we can ask ChatGPT to guide us on how to implement survival analysis in Python. This could include asking for recommended libraries, sample code, and resources for further learning.\n",
    "\n",
    "```python\n",
    "prompt = \"\"\"\n",
    "What are the recommended Python libraries for performing survival analysis? Could you provide a simple example of how to implement survival analysis using Python?\n",
    "\"\"\"\n",
    "\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  <mark> ChatGPT response:\n",
    "\n",
    "\n",
    "> Sure, I can provide a simple example of how to perform survival analysis using Python. The most commonly used Python libraries for survival analysis are `lifelines` and `scikit-survival`.\n",
    ">\n",
    "> Here is a simple example of how to perform survival analysis using the `lifelines` library. We will use the Kaplan-Meier estimator, which is a non-parametric method used to estimate the survival function from lifetime data.\n",
    ">\n",
    "> First, we need to install the necessary library. Let's install `lifelines`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting lifelines==0.27.7\n",
      "  Downloading lifelines-0.27.7-py3-none-any.whl (409 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m409.4/409.4 kB\u001b[0m \u001b[31m33.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: numpy>=1.14.0 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from lifelines==0.27.7) (1.21.6)\n",
      "Requirement already satisfied: scipy>=1.2.0 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from lifelines==0.27.7) (1.7.3)\n",
      "Requirement already satisfied: pandas>=1.0.0 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from lifelines==0.27.7) (1.3.5)\n",
      "Requirement already satisfied: matplotlib>=3.0 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from lifelines==0.27.7) (3.5.3)\n",
      "Collecting autograd>=1.5 (from lifelines==0.27.7)\n",
      "  Downloading autograd-1.6.2-py3-none-any.whl (49 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.3/49.3 kB\u001b[0m \u001b[31m8.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting autograd-gamma>=0.3 (from lifelines==0.27.7)\n",
      "  Downloading autograd-gamma-0.5.0.tar.gz (4.0 kB)\n",
      "  Preparing metadata (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25hCollecting formulaic>=0.2.2 (from lifelines==0.27.7)\n",
      "  Downloading formulaic-1.0.1-py3-none-any.whl (94 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m94.2/94.2 kB\u001b[0m \u001b[31m18.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting future>=0.15.2 (from autograd>=1.5->lifelines==0.27.7)\n",
      "  Downloading future-1.0.0-py3-none-any.whl (491 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m491.3/491.3 kB\u001b[0m \u001b[31m49.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: astor>=0.8 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from formulaic>=0.2.2->lifelines==0.27.7) (0.8.1)\n",
      "Collecting cached-property>=1.3.0 (from formulaic>=0.2.2->lifelines==0.27.7)\n",
      "  Downloading cached_property-1.5.2-py2.py3-none-any.whl (7.6 kB)\n",
      "Collecting graphlib-backport>=1.0.0 (from formulaic>=0.2.2->lifelines==0.27.7)\n",
      "  Downloading graphlib_backport-1.0.3-py3-none-any.whl (5.1 kB)\n",
      "Collecting interface-meta>=1.2.0 (from formulaic>=0.2.2->lifelines==0.27.7)\n",
      "  Downloading interface_meta-1.3.0-py3-none-any.whl (14 kB)\n",
      "Requirement already satisfied: typing-extensions>=4.2.0 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from formulaic>=0.2.2->lifelines==0.27.7) (4.5.0)\n",
      "Requirement already satisfied: wrapt>=1.0 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from formulaic>=0.2.2->lifelines==0.27.7) (1.14.1)\n",
      "Requirement already satisfied: cycler>=0.10 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from matplotlib>=3.0->lifelines==0.27.7) (0.11.0)\n",
      "Requirement already satisfied: fonttools>=4.22.0 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from matplotlib>=3.0->lifelines==0.27.7) (4.38.0)\n",
      "Requirement already satisfied: kiwisolver>=1.0.1 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from matplotlib>=3.0->lifelines==0.27.7) (1.4.4)\n",
      "Requirement already satisfied: packaging>=20.0 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from matplotlib>=3.0->lifelines==0.27.7) (23.1)\n",
      "Requirement already satisfied: pillow>=6.2.0 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from matplotlib>=3.0->lifelines==0.27.7) (8.1.0)\n",
      "Requirement already satisfied: pyparsing>=2.2.1 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from matplotlib>=3.0->lifelines==0.27.7) (3.0.9)\n",
      "Requirement already satisfied: python-dateutil>=2.7 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from matplotlib>=3.0->lifelines==0.27.7) (2.8.2)\n",
      "Requirement already satisfied: pytz>=2017.3 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from pandas>=1.0.0->lifelines==0.27.7) (2023.3)\n",
      "Requirement already satisfied: six>=1.5 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from python-dateutil>=2.7->matplotlib>=3.0->lifelines==0.27.7) (1.16.0)\n",
      "Building wheels for collected packages: autograd-gamma\n",
      "  Building wheel for autograd-gamma (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for autograd-gamma: filename=autograd_gamma-0.5.0-py3-none-any.whl size=4031 sha256=a0db475ece5505ff29bbc6a35a7be3c88d63fa6eae383ee8df3ed98bc0c7af37\n",
      "  Stored in directory: /home/jupyterlab/.cache/pip/wheels/9f/01/ee/1331593abb5725ff7d8c1333aee93a50a1c29d6ddda9665c9f\n",
      "Successfully built autograd-gamma\n",
      "Installing collected packages: cached-property, interface-meta, graphlib-backport, future, autograd, formulaic, autograd-gamma, lifelines\n",
      "Successfully installed autograd-1.6.2 autograd-gamma-0.5.0 cached-property-1.5.2 formulaic-1.0.1 future-1.0.0 graphlib-backport-1.0.3 interface-meta-1.3.0 lifelines-0.27.7\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "pip install lifelines==0.27.7"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Now, let's import the necessary libraries and load some data. We will use the built-in dataset from `lifelines` called `lifelines.datasets.load_dd`. This dataset is a sample of individuals who had undergone a certain type of liver disease surgery. The dataset contains the following columns:\n",
    ">\n",
    "> - ctryname: Country name\n",
    "> - cowcode2: Country code\n",
    "> - politycode: Polity code\n",
    "> - un_region_name: United Nations region name\n",
    "> - un_continent_name: United Nations continent name\n",
    "> - ehead: Executive head\n",
    "> - leaderspellreg: Leader's spell regime\n",
    "> - democracy: Democracy status\n",
    "> - regime: Regime type\n",
    "> - start_year: Start year of the regime\n",
    "> - duration: Duration of the regime\n",
    "> - observed: Observation status\n",
    ">\n",
    ">\n",
    "> Let's load the data and take a look at the first few rows.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ctryname</th>\n",
       "      <th>cowcode2</th>\n",
       "      <th>politycode</th>\n",
       "      <th>un_region_name</th>\n",
       "      <th>un_continent_name</th>\n",
       "      <th>ehead</th>\n",
       "      <th>leaderspellreg</th>\n",
       "      <th>democracy</th>\n",
       "      <th>regime</th>\n",
       "      <th>start_year</th>\n",
       "      <th>duration</th>\n",
       "      <th>observed</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>700</td>\n",
       "      <td>700.0</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Asia</td>\n",
       "      <td>Mohammad Zahir Shah</td>\n",
       "      <td>Mohammad Zahir Shah.Afghanistan.1946.1952.Mona...</td>\n",
       "      <td>Non-democracy</td>\n",
       "      <td>Monarchy</td>\n",
       "      <td>1946</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>700</td>\n",
       "      <td>700.0</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Asia</td>\n",
       "      <td>Sardar Mohammad Daoud</td>\n",
       "      <td>Sardar Mohammad Daoud.Afghanistan.1953.1962.Ci...</td>\n",
       "      <td>Non-democracy</td>\n",
       "      <td>Civilian Dict</td>\n",
       "      <td>1953</td>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>700</td>\n",
       "      <td>700.0</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Asia</td>\n",
       "      <td>Mohammad Zahir Shah</td>\n",
       "      <td>Mohammad Zahir Shah.Afghanistan.1963.1972.Mona...</td>\n",
       "      <td>Non-democracy</td>\n",
       "      <td>Monarchy</td>\n",
       "      <td>1963</td>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>700</td>\n",
       "      <td>700.0</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Asia</td>\n",
       "      <td>Sardar Mohammad Daoud</td>\n",
       "      <td>Sardar Mohammad Daoud.Afghanistan.1973.1977.Ci...</td>\n",
       "      <td>Non-democracy</td>\n",
       "      <td>Civilian Dict</td>\n",
       "      <td>1973</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>700</td>\n",
       "      <td>700.0</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Asia</td>\n",
       "      <td>Nur Mohammad Taraki</td>\n",
       "      <td>Nur Mohammad Taraki.Afghanistan.1978.1978.Civi...</td>\n",
       "      <td>Non-democracy</td>\n",
       "      <td>Civilian Dict</td>\n",
       "      <td>1978</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      ctryname  cowcode2  politycode un_region_name un_continent_name  \\\n",
       "0  Afghanistan       700       700.0  Southern Asia              Asia   \n",
       "1  Afghanistan       700       700.0  Southern Asia              Asia   \n",
       "2  Afghanistan       700       700.0  Southern Asia              Asia   \n",
       "3  Afghanistan       700       700.0  Southern Asia              Asia   \n",
       "4  Afghanistan       700       700.0  Southern Asia              Asia   \n",
       "\n",
       "                   ehead                                     leaderspellreg  \\\n",
       "0    Mohammad Zahir Shah  Mohammad Zahir Shah.Afghanistan.1946.1952.Mona...   \n",
       "1  Sardar Mohammad Daoud  Sardar Mohammad Daoud.Afghanistan.1953.1962.Ci...   \n",
       "2    Mohammad Zahir Shah  Mohammad Zahir Shah.Afghanistan.1963.1972.Mona...   \n",
       "3  Sardar Mohammad Daoud  Sardar Mohammad Daoud.Afghanistan.1973.1977.Ci...   \n",
       "4    Nur Mohammad Taraki  Nur Mohammad Taraki.Afghanistan.1978.1978.Civi...   \n",
       "\n",
       "       democracy         regime  start_year  duration  observed  \n",
       "0  Non-democracy       Monarchy        1946         7         1  \n",
       "1  Non-democracy  Civilian Dict        1953        10         1  \n",
       "2  Non-democracy       Monarchy        1963        10         1  \n",
       "3  Non-democracy  Civilian Dict        1973         5         0  \n",
       "4  Non-democracy  Civilian Dict        1978         1         0  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from lifelines.datasets import load_dd\n",
    "\n",
    "data = load_dd()\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">Now that we have loaded the data, let's perform survival analysis using the Kaplan-Meier estimator. This estimator is a non-parametric statistic used to estimate the survival function from lifetime data.\n",
    ">\n",
    "> We will create an instance of the `KaplanMeierFitter` class and then fit it to our data using the `fit` method. The `fit` method takes two arguments: the durations (length of the political regime) and the event observed (1 if the end of the regime was observed, 0 otherwise).\n",
    ">\n",
    "> After fitting the model, we can plot the survival function using the `plot_survival_function` method. This will provide us with a visual representation of the estimated survival function for political regimes, where the y-axis represents the probability of a regime surviving after a given amount of time (x-axis).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:xlabel='timeline'>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGwCAYAAAB7MGXBAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA0sUlEQVR4nO3de3RUVYLv8d9JVVJJCAlCICQQIMhDhFYgqBMEFdQI0o60uoxLR1TE7twBbYxoi0wrMC6jjjrKIKitQDutgm/tnoimfQCKegHBRnHaVtCgBmLoJgkEKvU49w9vSmNSqVOxUjuP72etWos6Z1ednVNZ1C/7adm2bQsAAMCQBNMVAAAA3RthBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGuU1XwIlgMKhvvvlGPXv2lGVZpqsDAAAcsG1bdXV1ysnJUUJC+PaPThFGvvnmG+Xm5pquBgAAaIO9e/dq4MCBYc93ijDSs2dPSd/9MOnp6YZrAwAAnKitrVVubm7oezycThFGGrtm0tPTCSMAAHQykYZYMIAVAAAYRRgBAABGEUYAAIBRnWLMCADAjGAwqIaGBtPVQAeVmJgol8v1k9+HMAIAaFFDQ4P27NmjYDBouirowHr16qX+/fv/pHXACCMAgGZs21ZlZaVcLpdyc3NbXbAK3ZNt26qvr1dVVZUkKTs7u83vRRgBADTj9/tVX1+vnJwcpaammq4OOqiUlBRJUlVVlfr169fmLhuiLgCgmUAgIElKSkoyXBN0dI1h1efztfk9CCMAgLDYDwyRxOJ3hDACAACMijqMbNy4Ueedd55ycnJkWZZefPHFiK/ZsGGD8vPzlZycrKFDh+qhhx5qS10BAEAXFHUYOXz4sE488UQtX77cUfk9e/bo3HPP1eTJk7V9+3bdcsstuu666/Tcc89FXVkAALqyxYsXa+zYsaarEXdRh5Hp06fr9ttv1wUXXOCo/EMPPaRBgwbp/vvv16hRozRnzhzNnj1b99xzT9SVjSXbtlXf4Hf8sG3baH0BAJFdeeWVmjlzZpNjzz77rJKTk3X33Xdr8eLFsixL06ZNa/bau+++W5Zl6YwzzohLXVvqXViwYIFef/31dr92Rws97T61991331VhYWGTY+ecc44ee+wx+Xw+JSYmNnuN1+uV1+sNPa+trY15vY74Ajr+1lcdlx+b20sv/OtEBnMBQCfy6KOPau7cuXrwwQc1Z84cLV68WNnZ2XrzzTf11VdfaeDAgaGyq1ev1qBBgwzWVkpLS1NaWprROpjQ7gNY9+3bp6ysrCbHsrKy5Pf7VV1d3eJrSktLlZGREXrk5ua2dzUj2rH3oCoPHjVdDQAwItrW5Fg+2toyfffdd2vevHl68sknNWfOnNDxfv36qbCwUL///e9DxzZv3qzq6mrNmDEjqmusXr1ao0aNUnJyso477jitWLEidK6hoUHz5s1Tdna2kpOTNWTIEJWWlkqShgwZIkn6xS9+IcuyQs9/3GLR2NJzxx13KCsrS7169dKSJUvk9/t14403qnfv3ho4cKBWrVrVpF6/+c1vNGLECKWmpmro0KH67W9/G5p6u2bNGi1ZskQffvihLMuSZVlas2aNJKmmpka//OUv1a9fP6Wnp2vq1Kn68MMPo7onbRGXRc9+3JrQ+IsVrpVh4cKFKikpCT2vra2NeSBJSXRp19JzVFPvU4M//FLHR3wBTXtgkyTJF2BJZADdU7StybG0a+k5Sk2K7uvq5ptv1oMPPqg//elPOuuss5qdnz17tm666SYtWrRIkrRq1SpddtllUV3jd7/7nW677TYtX75c48aN0/bt23XNNdeoR48euuKKK7Rs2TK9/PLLevrppzVo0CDt3btXe/fulSRt2bJF/fr10+rVqzVt2rRWFwt74403NHDgQG3cuFHvvPOOrr76ar377rs67bTT9P7772vdunUqLi7W2WefHfqu7Nmzp9asWaOcnBzt3LlT11xzjXr27KmbbrpJRUVF+uijj7R+/Xr9+c9/liRlZGTItm3NmDFDvXv3VllZmTIyMvTwww/rzDPP1KeffqrevXtHdX+i0e5hpH///tq3b1+TY1VVVXK73erTp0+Lr/F4PPJ4PO1aL8uylJrkjvgLXt/gb9d6AABi65VXXtFLL72k119/XVOnTm2xzM9//nMVFxdr48aNys/P19NPP6233367WQtDa/793/9d9957b2gMZV5ennbt2qWHH35YV1xxhSoqKjR8+HBNmjRJlmVp8ODBodf27dtX0vf7urSmd+/eWrZsmRISEjRy5Ejdfffdqq+v1y233CLpuz/g77zzTr3zzju65JJLJEn/9m//Fnr9kCFDdMMNN2jdunW66aablJKSorS0NLnd7ibXfuONN7Rz505VVVWFvoPvuecevfjii3r22Wf1y1/+0vG9iVa7h5GCggL98Y9/bHLstdde04QJE1ocLwIA6HgaW5NNXTsaJ5xwgqqrq3XrrbfqpJNOUs+ePZuVSUxM1L/8y79o9erV2r17t0aMGKETTjjB8TW+/fZb7d27V1dffbWuueaa0HG/36+MjAxJ33WxnH322Ro5cqSmTZumn//8583GUDoxevToJnsDZWVlacyYMaHnLpdLffr0Ce0RI303aPf+++/XZ599pkOHDsnv9ys9Pb3V62zbtk2HDh1q1lBw5MgRff7551HXOxpRh5FDhw7ps88+Cz3fs2ePduzYod69e2vQoEFauHChvv76az3++OOSpOLiYi1fvlwlJSW65ppr9O677+qxxx7TU089FbufAgDQrhpbkzuDAQMG6LnnntOUKVM0bdo0rV+/vsVAMnv2bJ1yyin66KOPNHv27Kiu0biT8e9+9zudcsopTc41drmMHz9ee/bs0SuvvKI///nPuvjii3XWWWfp2WefjepaP/7D3bKsFo811um9997TJZdcoiVLluicc85RRkaG1q5dq3vvvTfiz5Sdna233nqr2blevXpFVedoRf2btXXrVk2ZMiX0vHFsxxVXXKE1a9aosrJSFRUVofN5eXkqKyvT9ddfrwcffFA5OTlatmyZLrzwwhhUHwCA5gYNGqQNGzZoypQpKiws1KuvvtqsZWD06NEaPXq0/vKXv+jSSy+N6v2zsrI0YMAA7d69u9WxJunp6SoqKlJRUZEuuugiTZs2TX//+9/Vu3dvJSYmhvYAiqV33nlHgwcPDo2HkaQvv/yySZmkpKRm1x4/frz27dsnt9sdGlAbL1GHkTPOOKPVkc2NI3J/6PTTT9cHH3wQ7aUAAGizgQMH6q233moSSH7sjTfekM/na9Nf/osXL9Z1112n9PR0TZ8+XV6vV1u3btU//vEPlZSU6D//8z+VnZ2tsWPHKiEhQc8884z69+8futaQIUP0+uuv69RTT5XH49ExxxzzE3/i7wwbNkwVFRVau3atTjrpJP3P//yPXnjhhSZlhgwZEurZGDhwoHr27KmzzjpLBQUFmjlzpu666y6NHDlS33zzjcrKyjRz5kxNmDAhJvVrCXvTAAC6rAEDBmjDhg06ePCgzj77bB08eLDJ+R49erS5C2LOnDl69NFHtWbNGv3sZz/T6aefrjVr1igvL0/Sd2uG3HXXXZowYYJOOukkffHFFyorKwuN/7j33ntVXl6u3NxcjRs37qf8mE2cf/75uv766zVv3jyNHTtWmzdv1m9/+9smZS688EJNmzZNU6ZMUd++ffXUU0/JsiyVlZXptNNO0+zZszVixAhdcskl+uKLL5ot0RFrlt0Jlhatra1VRkaGampqIg7AibX6Bn9oOtuGBWdocGaPuF4fAEw4evSo9uzZo7y8PCUnJ5uuDjqw1n5XnH5/0zICAACMIowAANCCxqXZW3ps2rTJdPW6lM4xTwsAgDjbsWNH2HMDBgyIX0W6AcIIAAAtGDZsmOkqdBt00wAAwuoEcxxgWONiaz8FLSMAgGYSExNlWZa+/fZb9e3bN+zGpui+bNtWQ0ODvv32WyUkJCgpKanN70UYAQA043K5NHDgQH311Vf64osvTFcHHVhqaqoGDRrUZP+caBFGAAAtSktL0/Dhw+Xz+UxXBR2Uy+WS2+3+yS1nhBEAQFgulyu08RvQXhjACgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIwijAAAAKPaFEZWrFihvLw8JScnKz8/X5s2bWq1/BNPPKETTzxRqampys7O1lVXXaUDBw60qcIAAKBriTqMrFu3TvPnz9eiRYu0fft2TZ48WdOnT1dFRUWL5d9++23NmjVLV199tT7++GM988wz2rJli+bMmfOTKw8AADq/qMPIfffdp6uvvlpz5szRqFGjdP/99ys3N1crV65ssfx7772nIUOG6LrrrlNeXp4mTZqkX/3qV9q6dWvYa3i9XtXW1jZ5AACArimqMNLQ0KBt27apsLCwyfHCwkJt3ry5xddMnDhRX331lcrKymTbtvbv369nn31WM2bMCHud0tJSZWRkhB65ubnRVBMAAHQiUYWR6upqBQIBZWVlNTmelZWlffv2tfiaiRMn6oknnlBRUZGSkpLUv39/9erVS//1X/8V9joLFy5UTU1N6LF3795oqgkAADqRNg1gtSyryXPbtpsda7Rr1y5dd911uvXWW7Vt2zatX79ee/bsUXFxcdj393g8Sk9Pb/IAAABdkzuawpmZmXK5XM1aQaqqqpq1ljQqLS3VqaeeqhtvvFGSdMIJJ6hHjx6aPHmybr/9dmVnZ7ex6gAAoCuIqmUkKSlJ+fn5Ki8vb3K8vLxcEydObPE19fX1SkhoehmXyyXpuxYVAADQvUXdTVNSUqJHH31Uq1at0ieffKLrr79eFRUVoW6XhQsXatasWaHy5513np5//nmtXLlSu3fv1jvvvKPrrrtOJ598snJycmL3kwAAgE4pqm4aSSoqKtKBAwe0dOlSVVZWasyYMSorK9PgwYMlSZWVlU3WHLnyyitVV1en5cuX64YbblCvXr00depU3XXXXbH7KQAAQKdl2Z2gr6S2tlYZGRmqqamJ+2DW+ga/jr/1VUnShgVnaHBmj7heHwCAzsrp9zd70wAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADAq6o3yurMjvoAOef2tlnEnWEpOdMWpRgAAdH6EkSjs/PqgPt1f12qZVI9bk4ZlEkgAAHCIMBIFtytBaZ7EsOe9/oDqvX75gx1+I2QAADoMwkgUkl0upSS13uLhCwTjVBsAALoGBrACAACjCCMAAMAoummi4PUHddQXCH/eF5BtM14EAIBoEEaiMP/pHRHLDM3soamjstq/MgAAdBF000SQkujSuEG9HJffXX1YR1ppPQEAAE3RMhKBZVl6fPbJWv9RpdKSEpUcZjaN1x9U8R+2xbl2AAB0foQRByzLksftkifRxWJmAADEGN00AADAKMIIAAAwijACAACMYsxIFLz+1tcYAQAA0SOMOOBOsJTqcave6w+798wPg4rXx/40AAA4RRhxIDnRpUnDMlvdjbe+wR/6d5BdewEAcIww4lCkKb0JVpwqAgBAF8MAVgAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGCU23QFuqIjvoAOef1hz7sTLCUnuuJYIwAAOi7CSDvY+fVBfbq/Luz5VI9bk4ZlEkgAABBhpF24XQlK8yS2eM7rD6je65c/aMe5VgAAdEyEkXaQ7HIpJSl8q4cvEIxjbQAA6NgYwAoAAIwijAAAAKPaFEZWrFihvLw8JScnKz8/X5s2bWq1vNfr1aJFizR48GB5PB4de+yxWrVqVZsqDAAAupaox4ysW7dO8+fP14oVK3Tqqafq4Ycf1vTp07Vr1y4NGjSoxddcfPHF2r9/vx577DENGzZMVVVV8vvDT33t7Lz+oI76Ai2f8wVk2wxeBQCgkWVH+c14yimnaPz48Vq5cmXo2KhRozRz5kyVlpY2K79+/Xpdcskl2r17t3r37t2mStbW1iojI0M1NTVKT09v03u0t/oGv46/9VVHZYdm9tBL805Vz+SWZ9wAANAVOP3+jqqbpqGhQdu2bVNhYWGT44WFhdq8eXOLr3n55Zc1YcIE3X333RowYIBGjBihBQsW6MiRI2Gv4/V6VVtb2+TR0aUkujRuUC9HZXdXH9aRMC0nAAB0N1F101RXVysQCCgrK6vJ8aysLO3bt6/F1+zevVtvv/22kpOT9cILL6i6ulr/+q//qr///e9hx42UlpZqyZIl0VTNOMuy9Pjsk7X+o0qlJSUquYWpvV5/UMV/2GagdgAAdFxtGsBqWVaT57ZtNzvWKBgMyrIsPfHEEzr55JN17rnn6r777tOaNWvCto4sXLhQNTU1ocfevXvbUs24syxLHrdLnkSXklt4eNxMXgIA4MeiahnJzMyUy+Vq1gpSVVXVrLWkUXZ2tgYMGKCMjIzQsVGjRsm2bX311VcaPnx4s9d4PB55PJ5oqgYAADqpqP5UT0pKUn5+vsrLy5scLy8v18SJE1t8zamnnqpvvvlGhw4dCh379NNPlZCQoIEDB7ahygAAoCuJut+gpKREjz76qFatWqVPPvlE119/vSoqKlRcXCzpuy6WWbNmhcpfeuml6tOnj6666irt2rVLGzdu1I033qjZs2crJSUldj8JAADolKJeZ6SoqEgHDhzQ0qVLVVlZqTFjxqisrEyDBw+WJFVWVqqioiJUPi0tTeXl5br22ms1YcIE9enTRxdffLFuv/322P0UAACg04p6nRETOsM6I5J0yOvXG5/sV6IrQR53C7NpfAH9nyc/kCT930Vnql/P5HhXEQCAuHH6/c2uvTHkTrCU6nGr3utvcWder//7tUW8PnbuBQBAIozEVHKiS5OGZcofbLmxqb7h+yXwg2HKAADQ3RBGYiw5sXn3TKOElpdiAQCgW2MVLgAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEaxN40hR3wBHfL6Wy3jTrBa3esGAICugDBiyM6vD+rT/XWtlkn1uDVpWCaBBADQpRFGDHG7EpTmSQx73usPqN7rlz9ox7FWAADEH2HEkGSXSylJrbd4+ALBONUGAABzGMAKAACMIowAAACjCCMAAMAoxowY4vUHddQXCH/eF5BtM3gVAND1EUYMmf/0johlhmb20NRRWe1fGQAADKKbJo5SEl0aN6iX4/K7qw/rSCutJwAAdAW0jMSRZVl6fPbJWv9RpdKSEpUcZmqv1x9U8R+2xbl2AACYQRiJM8uy5HG75El0sbIqAACimwYAABhGGAEAAEYRRgAAgFGEEQAAYBQDWA3x+ltf8AwAgO6CMBJn7gRLqR636r3+sLvy/jCoeH3s3AsA6NoII3GWnOjSpGGZ8gfDL/Ve3+AP/TvYSjkAALoCwogBkdYXSbDiVBEAADoABrACAACjCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwir1pOrgjvoAOef1hz7sTrIh73QAA0JERRjq4nV8f1Kf768KeT/W4NWlYJoEEANBpEUY6OLcrQWmexBbPef0B1Xv98gftONcKAIDYIYx0cMkul1KSwrd6+ALBONYGAIDYYwArAAAwijACAACMIowAAACjGDPSwXn9QR31BVo+5wvIthm8CgDo3AgjHdz8p3e0en5oZg9NHZUVn8oAANAO6KbpgFISXRo3qJejsrurD+tImJYTAAA6A1pGOiDLsvT47JO1/qNKpSUlKrmFqb1ef1DFf9hmoHYAAMRWm1pGVqxYoby8PCUnJys/P1+bNm1y9Lp33nlHbrdbY8eObctluxXLsuRxu+RJdCm5hYfHTaMWAKBriPobbd26dZo/f74WLVqk7du3a/LkyZo+fboqKipafV1NTY1mzZqlM888s82VBQAAXU/UYeS+++7T1VdfrTlz5mjUqFG6//77lZubq5UrV7b6ul/96le69NJLVVBQ0ObKAgCArieqMNLQ0KBt27apsLCwyfHCwkJt3rw57OtWr16tzz//XLfddpuj63i9XtXW1jZ5AACArimqMFJdXa1AIKCsrKZTSbOysrRv374WX/O3v/1NN998s5544gm53c7Gy5aWliojIyP0yM3NjaaaAACgE2nTKEjLspo8t2272TFJCgQCuvTSS7VkyRKNGDHC8fsvXLhQNTU1ocfevXvbUs0uwesP6EhD88fRBqbzAgC6hqim9mZmZsrlcjVrBamqqmrWWiJJdXV12rp1q7Zv36558+ZJkoLBoGzbltvt1muvvaapU6c2e53H45HH44mmal2OO8FSqseteq+/xZ15vf7vw4jXx869AIDOK6owkpSUpPz8fJWXl+sXv/hF6Hh5ebnOP//8ZuXT09O1c+fOJsdWrFihN954Q88++6zy8vLaWO2uLznRpUnDMuUPtrzce32DP/TvYJgyAAB0BlEvelZSUqLLL79cEyZMUEFBgR555BFVVFSouLhY0nddLF9//bUef/xxJSQkaMyYMU1e369fPyUnJzc7juaSE5svdtYooXmvGAAAnVLUYaSoqEgHDhzQ0qVLVVlZqTFjxqisrEyDBw+WJFVWVkZccwQAAKCRZXeCbV9ra2uVkZGhmpoapaenm65Oh1Df4Nfxt74qSdqw4AwNzuxhuEYAADTl9PubNcUBAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFGEEQAAYBRhBAAAGBX1omfoeI74Ajrk9bdaxp1gtbqiKwAAphBGuoCdXx/Up/vrWi2T6nFr0rBMAgkAoMMhjHQBbleC0jyJYc97/QHVe/1hN90DAMAkwkgXkOxyKSWp9RYPXyAYp9oAABAdBrACAACjCCMAAMAoumm6AK8/qKO+QPjzvoA6webMAIBuijDSBcx/ekfEMkMze2jqqKz2rwwAAFGim6aTSkl0adygXo7L764+rCOttJ4AAGAKLSOdlGVZenz2yVr/UaXSkhKVHGY2jdcfVPEftsW5dgAAOEcY6cQsy5LH7ZIn0cViZgCATotuGgAAYBRhBAAAGEUYAQAARhFGAACAUQxg7QK8/tYXPAMAoCMjjHRi7gRLqR636r3+sBvh/TCoeH1slgcA6HgII51YcqJLk4Zlyh8Mv9R7fYM/9O9gK+UAADCFMNLJRVpfJMH6/t9HfAEd8vrDF9Z3rS2sWQIAiCfCSDey8+uD+nR/XatlUj1uTRqWSSABAMQNYaQbcbsSlOZJDHve6w+o3utvtdsHAIBYI4x0I8kul1LC7GHTKNxAWAAA2gvrjAAAAKMIIwAAwCi6aboRrz+oo60sgub1BWTbjBcBAMQXYaQbmf/0johlhmb20NRRWe1fGQAA/j/CSBeXkujSuEG9tL3ioKPyu6sP6x+HfbIsK2wZ1iIBAMQSYaSLsyxLj88+Wes/qlRaUqKSw8ym8fqDKv7DNknS+3uq5XGHDxusRQIAiCXCSDdgWZY8bpc8iS5HASItKVGeMOVYiwQAEGuEETSTnNR6aGEtEgBALDG1FwAAGEUYAQAARtFNg2a8/vDdMKxFAgCINcJIN+L1t77gWaPGWTXhsBYJACCWCCPdgDvBUqrHrXqvP+zgU9u2NTSzh3ZXH474frurD+uIL6CeyeF3AAYAwCnCSDeQnOjSpGGZEafjFhybqQ2fVoVdj+SHa5EAABArhJFuwukCZdGsRwIAQCwwmwYAABhFywja5GhDUIe8/lbLsIcNAMAJwgjaJNL+NRJ72AAAnCGMoE1a279GYg8bAIBzhBE0E249kh+uRRJp/xqJPWwAAM4QRhASaT2SH4YUnz9I9wsAICYIIwiJtB5JfcP3A1YDdL8AAGKEMIImWmvtSLC+/7fXH9RRX+vLy9u2rSMN4ctIzLgBABBG0Ebzn94RsczgPqmSJMuywpZhxg0AgDACx1ISXRo3qJe2Vxx0VP7LA/VKcrnCzrphxg0AQCKMIAqWZenx2Sdr/UeVYfevkZruYRNp1g0zbgAAhBFExbIs9q8BAMQUe9MAAACj2hRGVqxYoby8PCUnJys/P1+bNm0KW/b555/X2Wefrb59+yo9PV0FBQV69dVX21xhdC6Ns25aejTOuAEAdG9Rd9OsW7dO8+fP14oVK3Tqqafq4Ycf1vTp07Vr1y4NGjSoWfmNGzfq7LPP1h133KFevXpp9erVOu+88/T+++9r3LhxMfkh0HE1jh0JZ2hmD00dlRWn2gAAOiLLjvJP01NOOUXjx4/XypUrQ8dGjRqlmTNnqrS01NF7jB49WkVFRbr11ltbPO/1euX1ekPPa2trlZubq5qaGqWnp0dTXcTYIa9fb3yyX4muhLAb5dm2rdL1/6vPqg45es+NN05R77SkVsuwHgkAdD61tbXKyMiI+P0dVctIQ0ODtm3bpptvvrnJ8cLCQm3evNnRewSDQdXV1al3795hy5SWlmrJkiXRVA1xEmnJ+EbXTjlWB4/4lJ2eokR3897AH864YQdgAOjeogoj1dXVCgQCyspq2qyelZWlffv2OXqPe++9V4cPH9bFF18ctszChQtVUlISet7YMgLzIi0Z3+hIQ0Dvfl4ttyshYoBgB2AA6N7aNLX3xytq2rbd6iqbjZ566iktXrxYL730kvr16xe2nMfjkcfjaUvVEAexbp1gB2AA6N6iCiOZmZlyuVzNWkGqqqqatZb82Lp163T11VfrmWee0VlnnRV9TQEAQJcU1dTepKQk5efnq7y8vMnx8vJyTZw4MezrnnrqKV155ZV68sknNWPGjLbVFJ2S1x/QkYbmj6MRNtADAHQfUXfTlJSU6PLLL9eECRNUUFCgRx55RBUVFSouLpb03XiPr7/+Wo8//rik74LIrFmz9MADD+if/umfQq0qKSkpysjIiOGPgo4k0kBXr//7MHLoqL/V92I9EgDo2qIOI0VFRTpw4ICWLl2qyspKjRkzRmVlZRo8eLAkqbKyUhUVFaHyDz/8sPx+v+bOnau5c+eGjl9xxRVas2bNT/8J0CFFGuha3/B9ALl27faI7zc0s4cKjs2MWI4pwADQ+US9zogJTucpo/OwbVsXrNzseAdgSfqPi37GFGAA6ETaZZ0RIFbasgMwU4ABoGsijMCYaHcAZgowAHRNhBF0OUcizNRhXAkAdCyEEXQaXn/rrR7+QFCHvD69+3l1q+UYVwIAHQthBMb9cJpvs3O+789F2gFYkob3S9PN044LuyIw40oAoOMhjMAYJ5vu2batoZk9tLv6sKP3/FvVISVE6IbxBYIRu3Ia60frCQC0P8IIjHG66V7BsZna8GmV41k3rXElWDrc4I/YlSPRnQMA8UIYgVFOv+ijmXUTaWxJdnqKIvXS0J0DAPFDGEGXE6mFZGRWT9123vERd5pmmjAAxEdUG+UBHZXHnaCRWT0dlf3r/rqIrScAgPihZQRdgmVZuu2841sNGU7HlfwQa5YAQPsjjKDTaG0KcCNXgqUkd+QGP9YsAYCOgzCCDs/JFOBGhxv8yslIjRhIWLMEADoOwgg6PKdTgI80BPTu59UKhCnXOK7kr/vrHF2XNUsAID4II+gUovkib60756ZzRioQtJXYSstJe61ZMn7QMa222BBYAHRXhBF0GU67c5x25USS5E5QTkZq2JaYRr5gUN/WHdXbf/u21XKMPwHQXRFG0GU46c6J1JXzY5EGunrcCRFDTYpcSna7Wr1m4/iTuqORx6DQggKgqyGMoEuJ9Zd0rBZQixRYXAmWvqmpZ5l6AN0SYQTdUmvjSmzb1rB+afqs6lDE9/nr/jrVHvXLEyFseNwJrQYWp10+zOAB0BURRtCtOB1Xcu2UY3XwiE/Z6SktDnb94SBXJ4NdnbSgOB3DwgweAF0NYQTdSrTThN2uhBa/0KOdJuykBSVS64nErsMAuibCCLqdWHw5O1l+XoquBWVwn1QtPm90xGtHs+twpAGxtJ4A6AgII0AbWVbkL/JoWlC+PFCvq9ZsiVjOSZeP0wGxTtY/kQgtANoXYQRoRaT9cCLtheO0BWXxHz/WlwfqHdWpcdfh1sKBkwGxTtc/kejyAdC+CCNAC2K5gJqTFpTSX/wsqi6fSGWlyGugOFn/5Ltr0eUDoH0RRoAWRLOAWn2DX4Fg61/CTlpQovkij+cMnmi6fGg9AdAWhBEgjEhfqtHuJty7h0eJCeG//CMFFlMzeJx0+bD+CYCfwrJtu8P/71FbW6uMjAzV1NQoPT3ddHWAkKO+QMQv4AZ/UB9U/EP1Xn+r5Zx0+di2HVV3TiROZ/BECi1HGgI65PWp4NhMpSRFDnG0ngDdg9Pvb1pGgJ/A0ZeqR3Ht8jExgyfWOxhLhBagOyGMAHEQ6y6f1lpQ2msGT6QuHyfrn0Q7gydSaCGwAF0D3TRAB+Gky6exBSXNkxixOySSWHf5ON00sMEfjDiDxxcM6u+HveqR1PrfS7SyAB0b3TRAJxPNl2Wk9U+k2MzgiabLJ5pNAyOFByfTjmllAboOWkaATuSoL6C3P6uOOBhWcjYg1olILSjRtJ5IzltQnKCVBejYaBkBuqBoN/qLNCA2UuuJFLkFpT2mHDe+r5Npx5GYaGWRCC1ANGgZAbogpy0oTtY/kSKHlliPP5GcTzt2wkmwiWUri0TXECA5//4mjABdVKQBsU7XP5Fis2ibbdta8sddjltQYslJsHESWCS6hoBoEEYARNQRF22Topt2HCuxHMsiRQ4ttLKgO2DMCICIYr1oW6QWA6d78DjZONApp8HG6VgWp2K1UaHT8SzsDYTOjDACICKnX3CxmHIsRb9xYGsiBZsfjmWJZkxLJLHaqNBJaHG6s7JECwo6JsIIgJ8s1psGSs5CixOxng3kVKw2KpQihxanOytLtKCgY2LMCICYiOX4E8l5aHEiFrOBnDKxUaHkbGCt1x+QLxB0tKGhE7SyIBIGsALokGIdWpyIxWwgp9pj1lCsZgM1+IP6pqbe0aBZJ2hl6TqONARiElB/jDACoFNzElqciOVsIKectLTEesZQLPcGcqKxlWXqqCyleejx78xq6n3aVVmr47PTlZGaGNP3ZjYNgE4tZn9tRzEbKNKKtdGI1NLidMZQrGcDOdkbyClfIKgjDZEHLaNjO3ikQVLjAPTYhhGnCCMAurxIwSaaAbhOxaprKNazgWK1noorwdLhBr+jQbPo+FI9biUkxGaNnbYgjADo9pzu+ePUD7uGvArfcuCka8jU3kBOZKenKEa3DAYd/f9Tw02O/WHMCAC0g0hjXhq7hhJdCfK4W/8SMLE3kBOxXrUWZhxpCOiQ19cu438YMwIABsWyaygWXT7tsZ7KX/fXyesPMpsGPxlhBAAMcNo1FKsuH8uydNt5x8dkPZUftrLEan0WmOP1BWS6k4QwAgCGxHpvICezgWK1nkqjWHf9wIyhmT00dVSWsesTRgCgg4t3l08ktm1rWL80fVZ1qM3vgY5ld/VhHfEF1DOZqb0AgDaIdZePE9dOOVYHj/iUnZ6ixBi2tCC+2mNgc1sQRgCgC4hVl49TjV1DblcCA1jxkxFGAKAbiXVw+G7VTnRWXl/H+PwIIwCAqLXHqrWIvx+GSa/P3OdIGAEARC3Wq9bCjPqG7zeQDBr8LNsURlasWKH/+I//UGVlpUaPHq37779fkydPDlt+w4YNKikp0ccff6ycnBzddNNNKi4ubnOlAQDmMVak8+uR5NKGBWdod/VhJSeaG4gc9ZXXrVun+fPna9GiRdq+fbsmT56s6dOnq6KiosXye/bs0bnnnqvJkydr+/btuuWWW3Tdddfpueee+8mVBwAAbWdZllKSXEpOdBld1j/qvWlOOeUUjR8/XitXrgwdGzVqlGbOnKnS0tJm5X/zm9/o5Zdf1ieffBI6VlxcrA8//FDvvvuuo2uyNw0AAO2jqvaoPv/2sI7t20P90pNj+t5Ov7+jahlpaGjQtm3bVFhY2OR4YWGhNm/e3OJr3n333WblzznnHG3dulU+n6/F13i9XtXW1jZ5AACA2GvcqDHSho3tKaowUl1drUAgoKyspkvGZmVlad++fS2+Zt++fS2W9/v9qq6ubvE1paWlysjICD1yc3OjqSYAAHAoIzVRY3N7KSPVzOqrUhvGjEhq1q9k23arfU0tlW/peKOFCxeqpqYm9Ni7d29bqgkAABxISTI7GDmq2TSZmZlyuVzNWkGqqqqatX406t+/f4vl3W63+vTp0+JrPB6PPB5PNFUDAACdVFQtI0lJScrPz1d5eXmT4+Xl5Zo4cWKLrykoKGhW/rXXXtOECROUmGiuSQgAAHQMUXfTlJSU6NFHH9WqVav0ySef6Prrr1dFRUVo3ZCFCxdq1qxZofLFxcX68ssvVVJSok8++USrVq3SY489pgULFsTupwAAAJ1W1IueFRUV6cCBA1q6dKkqKys1ZswYlZWVafDgwZKkysrKJmuO5OXlqaysTNdff70efPBB5eTkaNmyZbrwwgtj91MAAIBOK+p1RkxgnREAADqfdllnBAAAINYIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMinoFVhMa12Wrra01XBMAAOBU4/d2pPVVO0UYqaurkyTl5uYargkAAIhWXV2dMjIywp7vFMvBB4NBffPNN+rZs6csy4rZ+9bW1io3N1d79+5lmXkDuP9mcf/N4v6bxf2PD9u2VVdXp5ycHCUkhB8Z0ilaRhISEjRw4MB2e//09HR+GQ3i/pvF/TeL+28W97/9tdYi0ogBrAAAwCjCCAAAMKpbhxGPx6PbbrtNHo/HdFW6Je6/Wdx/s7j/ZnH/O5ZOMYAVAAB0Xd26ZQQAAJhHGAEAAEYRRgAAgFGEEQAAYFS3DiMrVqxQXl6ekpOTlZ+fr02bNpmuUpe0ceNGnXfeecrJyZFlWXrxxRebnLdtW4sXL1ZOTo5SUlJ0xhln6OOPPzZT2S6mtLRUJ510knr27Kl+/fpp5syZ+utf/9qkDPe//axcuVInnHBCaGGtgoICvfLKK6Hz3Pv4Ki0tlWVZmj9/fugYn0HH0G3DyLp16zR//nwtWrRI27dv1+TJkzV9+nRVVFSYrlqXc/jwYZ144olavnx5i+fvvvtu3XfffVq+fLm2bNmi/v376+yzzw7tSYS227Bhg+bOnav33ntP5eXl8vv9Kiws1OHDh0NluP/tZ+DAgbrzzju1detWbd26VVOnTtX5558f+rLj3sfPli1b9Mgjj+iEE05ocpzPoIOwu6mTTz7ZLi4ubnLsuOOOs2+++WZDNeoeJNkvvPBC6HkwGLT79+9v33nnnaFjR48etTMyMuyHHnrIQA27tqqqKluSvWHDBtu2uf8mHHPMMfajjz7KvY+juro6e/jw4XZ5ebl9+umn27/+9a9t2+b3vyPpli0jDQ0N2rZtmwoLC5scLyws1ObNmw3Vqnvas2eP9u3b1+Sz8Hg8Ov300/ks2kFNTY0kqXfv3pK4//EUCAS0du1aHT58WAUFBdz7OJo7d65mzJihs846q8lxPoOOo1NslBdr1dXVCgQCysrKanI8KytL+/btM1Sr7qnxfrf0WXz55ZcmqtRl2batkpISTZo0SWPGjJHE/Y+HnTt3qqCgQEePHlVaWppeeOEFHX/88aEvO+59+1q7dq0++OADbdmypdk5fv87jm4ZRhpZltXkuW3bzY4hPvgs2t+8efP0l7/8RW+//Xazc9z/9jNy5Ejt2LFDBw8e1HPPPacrrrhCGzZsCJ3n3refvXv36te//rVee+01JScnhy3HZ2Bet+ymyczMlMvlatYKUlVV1Swho331799fkvgs2tm1116rl19+WW+++aYGDhwYOs79b39JSUkaNmyYJkyYoNLSUp144ol64IEHuPdxsG3bNlVVVSk/P19ut1tut1sbNmzQsmXL5Ha7Q/eZz8C8bhlGkpKSlJ+fr/Ly8ibHy8vLNXHiREO16p7y8vLUv3//Jp9FQ0ODNmzYwGcRA7Zta968eXr++ef1xhtvKC8vr8l57n/82bYtr9fLvY+DM888Uzt37tSOHTtCjwkTJuiyyy7Tjh07NHToUD6DDqLbdtOUlJTo8ssv14QJE1RQUKBHHnlEFRUVKi4uNl21LufQoUP67LPPQs/37NmjHTt2qHfv3ho0aJDmz5+vO+64Q8OHD9fw4cN1xx13KDU1VZdeeqnBWncNc+fO1ZNPPqmXXnpJPXv2DP0FmJGRoZSUlNCaC9z/9nHLLbdo+vTpys3NVV1dndauXau33npL69ev597HQc+ePUPjoxr16NFDffr0CR3nM+ggzE3kMe/BBx+0Bw8ebCclJdnjx48PTXdEbL355pu2pGaPK664wrbt76bX3XbbbXb//v1tj8djn3baafbOnTvNVrqLaOm+S7JXr14dKsP9bz+zZ88O/R/Tt29f+8wzz7Rfe+210Hnuffz9cGqvbfMZdBSWbdu2oRwEAADQPceMAACAjoMwAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAR9566y1ZlqWDBw+263XWrFmjXr16hZ4vXrxYY8eObddrAjCLMAKgRWeccYbmz58fej5x4kRVVlYqIyMjrvVYsGCBXn/99bheE0B8dduN8gBEJykpKbTtfTylpaUpLS0t7tcFED+0jABo5sorr9SGDRv0wAMPyLIsWZalNWvWNOmmaexO+dOf/qSRI0cqNTVVF110kQ4fPqzf//73GjJkiI455hhde+21CgQCofduaGjQTTfdpAEDBqhHjx465ZRT9NZbb4Wty4+7aa688krNnDlT99xzj7Kzs9WnTx/NnTtXPp+vzdcAYBYtIwCaeeCBB/Tpp59qzJgxWrp0qSTp448/blauvr5ey5Yt09q1a1VXV6cLLrhAF1xwgXr16qWysjLt3r1bF154oSZNmqSioiJJ0lVXXaUvvvhCa9euVU5Ojl544QVNmzZNO3fu1PDhwx3V780331R2drbefPNNffbZZyoqKtLYsWN1zTXXxOwaAOKHMAKgmYyMDCUlJSk1NTXUNfO///u/zcr5fD6tXLlSxx57rCTpoosu0n//939r//79SktL0/HHH68pU6bozTffVFFRkT7//HM99dRT+uqrr5STkyPpuzEh69ev1+rVq3XHHXc4qt8xxxyj5cuXy+Vy6bjjjtOMGTP0+uuv65prronZNQDED2EEQJulpqaGgogkZWVlaciQIU3GeGRlZamqqkqS9MEHH8i2bY0YMaLJ+3i9XvXp08fxdUePHi2XyxV6np2drZ07d8b0GgDihzACoM0SExObPLcsq8VjwWBQkhQMBuVyubRt27YmYUJSVINU43ENAPFDGAHQoqSkpCYDT2Nh3LhxCgQCqqqq0uTJk2P63vG8BoDYYjYNgBYNGTJE77//vr744gtVV1eHWh5+ihEjRuiyyy7TrFmz9Pzzz2vPnj3asmWL7rrrLpWVlcWg1vG5BoDYIowAaNGCBQvkcrl0/PHHq2/fvqqoqIjJ+65evVqzZs3SDTfcoJEjR+qf//mf9f777ys3Nzcm7x+vawCIHcu2bdt0JQAAQPdFywgAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACj/h/qwwBylke67gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from lifelines.datasets import load_dd\n",
    "from lifelines import KaplanMeierFitter\n",
    "\n",
    "# Load the dataset\n",
    "data = load_dd()\n",
    "\n",
    "# Create an instance of KaplanMeierFitter\n",
    "kmf = KaplanMeierFitter()\n",
    "\n",
    "# Fit the data into the model\n",
    "kmf.fit(durations = data['duration'], event_observed = data['observed'])\n",
    "\n",
    "# Create an estimate of the survival function\n",
    "kmf.plot_survival_function()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Using \"N-shot\" Prompting strategy to Ask Further Questions\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "Prompt= \"\"\"\n",
    "Calculate and interpret the median survival time from the Kaplan-Meier survival analysis. Explain what the median survival time represents and how it is calculated. Provide Python code for this analysis\n",
    "\"\"\"\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <mark> ChatGPT Response:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> The median survival time is the time at which the survival probability drops to 50%. In other words, it is the time at which the estimated survival function S(t) equals 0.5. It represents the time by which half of the population has experienced the event of interest.\n",
    ">\n",
    "> The median survival time is a commonly used summary statistic in survival analysis because it is easy to interpret and does not depend on the entire survival distribution. It provides a measure of the central tendency of the survival times.\n",
    ">\n",
    "> In the Kaplan-Meier survival analysis, the median survival time can be calculated using the `median_survival_time_` attribute of the fitted `KaplanMeierFitter` object.\n",
    ">\n",
    "> Here is how you can calculate the median survival time in Python using the `lifelines` library:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The median survival time is: 4.0\n"
     ]
    }
   ],
   "source": [
    "from lifelines import KaplanMeierFitter\n",
    "\n",
    "# Create an instance of KaplanMeierFitter\n",
    "kmf = KaplanMeierFitter()\n",
    "\n",
    "# Fit the data into the model\n",
    "kmf.fit(durations = data['duration'], event_observed = data['observed'])\n",
    "\n",
    "# Calculate the median survival time\n",
    "median_survival_time = kmf.median_survival_time_\n",
    "\n",
    "print(\"The median survival time is:\", median_survival_time)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> This code will print the median survival time of the political regimes in the dataset. Please note that you need to have the `lifelines` library installed in your Python environment to run this code. You can install it using the command `!pip install lifelines`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prompt Engineering with Langchain\n",
    "\n",
    "LangChain is a powerful library that leverages the potential of prompts for working with LLMs. It offers a set of objects called PromptTemplates that let you customize and optimize your prompts for different tasks and scenarios.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting langchain==0.0.18\n",
      "  Downloading langchain-0.0.18-py3-none-any.whl (95 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m95.9/95.9 kB\u001b[0m \u001b[31m18.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting pydantic (from langchain==0.0.18)\n",
      "  Downloading pydantic-2.5.3-py3-none-any.whl (381 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m381.9/381.9 kB\u001b[0m \u001b[31m45.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: sqlalchemy in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from langchain==0.0.18) (1.3.24)\n",
      "Requirement already satisfied: numpy in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from langchain==0.0.18) (1.21.6)\n",
      "Requirement already satisfied: requests in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from langchain==0.0.18) (2.29.0)\n",
      "Requirement already satisfied: pyyaml in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from langchain==0.0.18) (6.0)\n",
      "Collecting annotated-types>=0.4.0 (from pydantic->langchain==0.0.18)\n",
      "  Downloading annotated_types-0.5.0-py3-none-any.whl (11 kB)\n",
      "Requirement already satisfied: importlib-metadata in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from pydantic->langchain==0.0.18) (4.11.4)\n",
      "Collecting pydantic-core==2.14.6 (from pydantic->langchain==0.0.18)\n",
      "  Downloading pydantic_core-2.14.6-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.1 MB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.1/2.1 MB\u001b[0m \u001b[31m81.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting typing-extensions>=4.6.1 (from pydantic->langchain==0.0.18)\n",
      "  Downloading typing_extensions-4.7.1-py3-none-any.whl (33 kB)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from requests->langchain==0.0.18) (3.1.0)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from requests->langchain==0.0.18) (3.4)\n",
      "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from requests->langchain==0.0.18) (1.26.15)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from requests->langchain==0.0.18) (2023.5.7)\n",
      "Requirement already satisfied: zipp>=0.5 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from importlib-metadata->pydantic->langchain==0.0.18) (3.15.0)\n",
      "Installing collected packages: typing-extensions, pydantic-core, annotated-types, pydantic, langchain\n",
      "  Attempting uninstall: typing-extensions\n",
      "    Found existing installation: typing_extensions 4.5.0\n",
      "    Uninstalling typing_extensions-4.5.0:\n",
      "      Successfully uninstalled typing_extensions-4.5.0\n",
      "Successfully installed annotated-types-0.5.0 langchain-0.0.18 pydantic-2.5.3 pydantic-core-2.14.6 typing-extensions-4.7.1\n",
      "Collecting pydantic==1.10.9\n",
      "  Downloading pydantic-1.10.9-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.1/3.1 MB\u001b[0m \u001b[31m69.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: typing-extensions>=4.2.0 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from pydantic==1.10.9) (4.7.1)\n",
      "Installing collected packages: pydantic\n",
      "  Attempting uninstall: pydantic\n",
      "    Found existing installation: pydantic 2.5.3\n",
      "    Uninstalling pydantic-2.5.3:\n",
      "      Successfully uninstalled pydantic-2.5.3\n",
      "Successfully installed pydantic-1.10.9\n",
      "Collecting huggingface-hub\n",
      "  Downloading huggingface_hub-0.16.4-py3-none-any.whl (268 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m268.8/268.8 kB\u001b[0m \u001b[31m29.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting filelock (from huggingface-hub)\n",
      "  Downloading filelock-3.12.2-py3-none-any.whl (10 kB)\n",
      "Requirement already satisfied: fsspec in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from huggingface-hub) (2023.1.0)\n",
      "Requirement already satisfied: requests in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from huggingface-hub) (2.29.0)\n",
      "Requirement already satisfied: tqdm>=4.42.1 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from huggingface-hub) (4.60.0)\n",
      "Requirement already satisfied: pyyaml>=5.1 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from huggingface-hub) (6.0)\n",
      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from huggingface-hub) (4.7.1)\n",
      "Requirement already satisfied: packaging>=20.9 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from huggingface-hub) (23.1)\n",
      "Requirement already satisfied: importlib-metadata in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from huggingface-hub) (4.11.4)\n",
      "Requirement already satisfied: zipp>=0.5 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from importlib-metadata->huggingface-hub) (3.15.0)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from requests->huggingface-hub) (3.1.0)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from requests->huggingface-hub) (3.4)\n",
      "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from requests->huggingface-hub) (1.26.15)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from requests->huggingface-hub) (2023.5.7)\n",
      "Installing collected packages: filelock, huggingface-hub\n",
      "Successfully installed filelock-3.12.2 huggingface-hub-0.16.4\n"
     ]
    }
   ],
   "source": [
    "!pip install langchain==0.0.18\n",
    "!pip install pydantic==1.10.9\n",
    "#!pip install OpenAI\n",
    "!pip install huggingface-hub"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once you have installed that, please **restart your kernel**. You can do that by navigating to the button shown below:\n",
    "\n",
    "<center> <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/build-a-hotdog-not-hotdog-classifier-guided-project/images/Restarting_the_Kernel.png\" width=\"60%\" alt=\"Restart kernel\"> </center>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You also need to insert your LLM API key. As an example, here's a demonstration to show you how to get your hugging face API key (which is free).\n",
    "\n",
    "Initialize HuggingFace API key from your account by following steps: \n",
    "\n",
    "1. Go to the https://huggingface.co/\n",
    "2. Log in to your account (or sign up free if it is your first time)\n",
    "3. Go to Settings -> Access Tokens -> click on New Token (image blew)\n",
    "4. Select either read or write option and copy the token\n",
    "\n",
    "<div style=\"text-align:center\">\n",
    "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMSkillsNetwork-GPXX0DTPEN/images/HF%20key%201.png\n",
    "\" width=\"600\" alt=\"\">\n",
    "</div>\n",
    "\n",
    "<div style=\"text-align:center\">\n",
    "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMSkillsNetwork-GPXX0DTPEN/images/HF%20key%202.png\n",
    "\" width=\"600\" alt=\"\">\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "To read a book effectively, you should start by setting aside a specific time and place for reading. This will help you stay focused and avoid distractions. You should also choose a book that interests you and that you are excited to read. Additionally, try to read in a comfortable position and take breaks as needed. Finally, try to engage with the material by asking questions and making connections to what you already know.\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "#from langchain.llms import OpenAI\n",
    "from langchain.llms import HuggingFaceHub\n",
    "\n",
    "# set up the environment with respected API key\n",
    "#os.environ[\"OPENAI_API_KEY\"] = \"\"\n",
    "\n",
    "os.environ[\"HUGGINGFACEHUB_API_TOKEN\"] = \"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\"\n",
    "\n",
    "# you can choose between different llm models\n",
    "\n",
    "# The \"temperature\" is a hyperparameter that controls the randomness of the model's output. A lower value (like 0.1) makes the output more deterministic, while a higher value makes it more random.\n",
    "# \"max_new_tokens\" parameter sets a limit on the maximum number of new tokens (words/characters) that the model can generate as output.\n",
    "\n",
    "llm = HuggingFaceHub(repo_id=\"tiiuae/falcon-7b-instruct\",model_kwargs={\"temperature\": 0.1, \"max_new_tokens\": 600})\n",
    "\n",
    "\n",
    "# you can use OpenAI GPT models\n",
    "#llm = OpenAI(model_name=\"gpt-3.5-turbo\")\n",
    "\n",
    "\n",
    "text = \"How read book effectively?\"\n",
    "\n",
    "print(llm(text))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>Click here to see the sample of openAI outcome</summary>\n",
    "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMSkillsNetwork-GPXX0TGVEN/images/langchain_sample_notebook.png\" alt=\"output\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prompt Template in Langchain\n",
    "\n",
    "A prompt template in LangChain is a class that allows you to create and format prompts for LLMs with dynamic inputs. A prompt template has input variables, a template string, and an optional output parser. You can use different types of prompt templates for different tasks and scenarios, such as few-shot learning, chatbots, question answering, etc. For more information visit 🦜🔗 [LangChain.](https://python.langchain.com/en/latest/modules/prompts/prompt_templates.html)\n",
    "\n",
    "\n",
    "For example, here is a prompt template for step by step instruction a text and present it in table-like format. This prompt can then be passed to an LLM to generate the desired output.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "1. Sit on the ground with your legs straight in front of you.\n",
      "2. Place your hands on either side of your body, with your palms facing the ground.\n",
      "3. Inhale deeply and press your feet into the ground to lift your body off the ground.\n",
      "4. Keep your legs straight and your body in a straight line as you flip over.\n",
      "5. Land back on the ground with your body in a straight line.\n",
      "6. Exhale and take a few deep breaths to catch your breath.\n",
      "7. Stretch and massage your body to release any tension.\n",
      "\n",
      "Remember to always consult with a professional before attempting any new physical activity.\n"
     ]
    }
   ],
   "source": [
    "from langchain import PromptTemplate\n",
    "\n",
    "\n",
    "# Define the template\n",
    "template = \"\"\"\n",
    "Give me step by step instruction in table format:\n",
    "\n",
    "{text}\n",
    "\"\"\"\n",
    "\n",
    "# Create the prompt template object\n",
    "summary_prompt = PromptTemplate(\n",
    "    input_variables=[\"text\"], # The name of the input variable\n",
    "    template=template # The template string\n",
    ")\n",
    "\n",
    "# Format the prompt with some text\n",
    "text = \"I want to backflip\"\n",
    "formatted_prompt = summary_prompt.format(text=text)\n",
    "\n",
    "# Print the formatted prompt\n",
    "print(llm(formatted_prompt))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>Click here to see the outcome (the output is from the OpenAI llm)</summary>\n",
    "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMSkillsNetwork-GPXX0TGVEN/images/langchain_notebook_prompt_output.png\" alt=\"output\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prompt Engineering With IBM Watsonx\n",
    "\n",
    "<center> <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMSkillsNetwork-GPXX0TGVEN/images/watsonX.jpeg\" width=\"60%\" alt=\"Restart kernel\"> </center>\n",
    "\n",
    "IBM has a special offer for watsonx.ai, a studio for new foundation models, generative AI and machine learning. To take advantage of this offer visit [watsonx.ai homepage](https://dataplatform.cloud.ibm.com/registration/stepone?utm_source=skills_network&utm_content=in_lab_content_link&utm_id=Lab-IBMSkillsNetwork-GPXX0TGVEN&context=wx&apps=data_science_experience%2Cwatson_data_platform%2Ccos). \n",
    "\n",
    "\n",
    "In the Prompt Lab in [IBM watsonx.ai](https://dataplatform.cloud.ibm.com/registration/stepone?utm_source=skills_network&utm_content=in_lab_content_link&utm_id=Lab-IBMSkillsNetwork-GPXX0TGVEN&context=wx&apps=data_science_experience%2Cwatson_data_platform%2Ccos), you can experiment with prompting different foundation models, explore sample prompts, as well as save and share your best prompts. \n",
    "\n",
    "\n",
    "<iframe width=\"480\" height=\"270\" src=\"https://video.ibm.com/embed/recorded/132848516\" scrolling=\"no\" allowfullscreen webkitallowfullscreen frameborder=\"0\" style=\"border: 0 none transparent;\"></iframe>\n",
    "\n",
    "\n",
    "The IBM watsonx prompt lab is a graphical interface that allows you to experiment with prompting different foundation models, explore sample prompts, as well as save and share your best prompts¹. The prompt lab helps you craft effective prompts by providing different modes, parameters, and feedback. You can also learn from documented samples and tips for writing foundation model prompts². The prompt lab is part of the IBM watsonx platform, which is a cloud-based service that enables you to access, analyze, and build with LLM models³.\n",
    "\n",
    "\n",
    "\n",
    "1. Prompt Lab - IBM. https://www.ibm.com/docs/en/watsonx?topic=models-prompt-lab.\n",
    "2. Tips for writing foundation model prompts: prompt engineering - IBM. https://www.ibm.com/docs/en/watsonx?topic=models-prompt-tips.\n",
    "3. Prompt Lab | IBM watsonx. https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-prompt-lab.html?context=wx.\n",
    "\n",
    "\n",
    "Here is the example of how WatsonX platform facilitate prompt engineering: \n",
    "\n",
    "> **Before:**\n",
    "\n",
    "> In this image, you can see a prompt with the original, simple instruction. This prompt doesn't produce great results.\n",
    "\n",
    "<center> <img src=\"https://www.ibm.com/docs/en/SSYOK8/wsj/analyze-data/images/fm-prompt-lab-prompt-too-simple.png\" width=\"80%\" alt=\"watsonX\"> </center>\n",
    "\n",
    "\n",
    "> Example prompt text with just a simple instruction\n",
    "\n",
    "> **After:**\n",
    "<center> <img src=\"https://www.ibm.com/docs/en/SSYOK8/wsj/analyze-data/images/fm-prompt-lab-prompt-components.png\" width=\"80%\" alt=\"watsonX\"> </center>\n",
    "\n",
    "> In this image, you can see all the prompt components: instruction (complete with descriptive details), context, example, and cue. This prompt produces a much better result.\n",
    "\n",
    "\n",
    "You can experiment with this prompt in the Prompt Lab yourself.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "Writing efficient ChatGPT prompts is critical to getting high-quality, relevant, and contextually appropriate responses. By applying the techniques discussed in this article, you can optimize the use of ChatGPT in various applications, such as chatbots, language translation, and content generation. Mastering the art of prompt engineering can truly unlock the power of AI.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Authors\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| - | - |\n",
    "|---|---|\n",
    "| **Sina Nazeri (PhD.)**  | [linkedin](https://www.linkedin.com/in/sina-nazeri) |\n",
    "| ![Sina](https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMSkillsNetwork-GPXX0Q8REN/images/sina_low_qual.jpeg) | As a data scientist in IBM, I have always been passionate about sharing my knowledge and helping others learn about the field. I believe that everyone should have the opportunity to learn about data science, regardless of their background or experience level. This belief has inspired me to become a learning content provider, creating and sharing educational materials that are accessible and engaging for everyone. |\n",
    "Sina Nazeri\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "JOSEPH SANTARCANGELO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Other Contributors\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Change Log\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "|Date (YYYY-MM-DD)|Version|Changed By|Change Description|\n",
    "|-|-|-|-|\n",
    "|2023-06-17|0.1|Sina Nazeri|Create Lab |\n",
    "|2023-09-07|0.2|Sina Nazeri|changed openAI api to huggingface api|\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright © 2022 IBM Corporation. All rights reserved.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python",
   "language": "python",
   "name": "conda-env-python-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}