00_tensorflow_fundamentals.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "00_tensorflow_fundamentals.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "mount_file_id": "1mvRa0gAlwaoEiesyqq4p7ANU8yQyRP3y",
      "authorship_tag": "ABX9TyP3K3kqrfNknqW6na125F+N",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/rcortezk9/tensorflow_lessons/blob/main/00_tensorflow_fundamentals.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "STizKQCFWPQp"
      },
      "source": [
        "# In this notebook, we're going to cover some of the most fundamental concepts of tensors using TensorFlow\n",
        "\n",
        "More specifically, we're going to cover:\n",
        "* Intro to tensors\n",
        "* Getting info from tensors\n",
        "* Manuipulating Tensor\n",
        "* Tensors & NumPy\n",
        "* Using @tf.functions (a way to speed up your regular Python functions )\n",
        "* Using GPUs with TensorFlow (or TPUs)\n",
        "* Exercises to try for yourself"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CjPEHAN_Wvah"
      },
      "source": [
        "## Introduction to Tensors"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3ziajr7gXgAz",
        "outputId": "3ad9f08e-69f4-442d-b104-b2c9714400f8"
      },
      "source": [
        "# Import TensorFlow\n",
        "\n",
        "import tensorflow as tf\n",
        "print(tf.__version__)"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2.5.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9-6dMiI_YL0t",
        "outputId": "d214e304-7181-49d0-b0bb-808a18674b9e"
      },
      "source": [
        "# Create tensors with tf.constant()\n",
        "scalar = tf.constant(7) \n",
        "scalar"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=7>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GPtYtof-9rM5"
      },
      "source": [
        "A scalar is known as a rank 0 tensor. Because it has no dimensions (it's just a number).\n",
        "\n",
        ">🔑 **Note:** For now, you don't need to know too much about the different ranks of tensors (but we will see more on this later). The importnat point is knowing tensors can have an unlimited range of dimensions (the exact maount will depend on what data you're respresenting)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Wbra1Dv1YLn3",
        "outputId": "f0601183-e0af-4baf-d1ad-9deb48b7dde2"
      },
      "source": [
        "# Check the number of dimensions of a tensor (ndim stands for number of dimensions)\n",
        "\n",
        "scalar.ndim"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zB46ZNw9YxAX",
        "outputId": "02ede376-09cf-4249-ad33-efa0e617706e"
      },
      "source": [
        "# Create a vector\n",
        "vector = tf.constant([10, 10])\n",
        "vector "
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([10, 10], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "at24lx1FZiTO",
        "outputId": "89adf2d8-26ea-45a3-dec4-1fde5135c611"
      },
      "source": [
        "# check the dimension of our vector\n",
        "vector.ndim"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "X2Vs7mpJZyJP",
        "outputId": "6dbeab2f-897c-4366-c458-3cb05d285dc1"
      },
      "source": [
        "# Create a matrix (has more than 1 dimension)\n",
        "\n",
        "matrix = tf.constant([[10, 7],\n",
        "                      [7, 10]])\n",
        "matrix "
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 7, 10]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "l7WJXApRaUqJ",
        "outputId": "3a821b76-3c86-44bf-f919-f32884284242"
      },
      "source": [
        "matrix.ndim"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VqxH2ERB-uSp"
      },
      "source": [
        "By default, TensorFlow creates tensors with either an int32 or float32 datatype.\n",
        "\n",
        "This is known as [32-bit](https://en.wikipedia.org/wiki/Precision_(computer_science) precision (the higher the number, the more precise the number, the more space it takes up on your computer)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nFOa3fw5acaI",
        "outputId": "69248de2-38b1-48e8-a625-77a519c4b1e3"
      },
      "source": [
        "# Create another matrix\n",
        "another_matrix = tf.constant([[10., 7],\n",
        "                              [3., 2.],\n",
        "                              [8., 0.]], dtype=tf.float16) # Specify the data type with dtype parameter\n",
        "another_matrix"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=float16, numpy=\n",
              "array([[10.,  7.],\n",
              "       [ 3.,  2.],\n",
              "       [ 8.,  0.]], dtype=float16)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rkeaaGfNbg8R",
        "outputId": "8e162ca5-491d-4680-d32b-8cd719477751"
      },
      "source": [
        "# What's the number dimensions of another_matrix?\n",
        "another_matrix"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=float16, numpy=\n",
              "array([[10.,  7.],\n",
              "       [ 3.,  2.],\n",
              "       [ 8.,  0.]], dtype=float16)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PRuTUcNECF87",
        "outputId": "c301642f-e3c4-41f4-bbbd-1f675d21110f"
      },
      "source": [
        " # How about a tensor? (more than 2 dimensions, although, all of the above items \n",
        " # are also technically tensors)\n",
        "\n",
        " tensor = tf.constant([[[1, 2, 3],\n",
        "                        [4, 5, 6]],\n",
        "                       [[7, 8, 9],\n",
        "                        [10, 11, 12]],\n",
        "                       [[13, 14, 15],\n",
        "                       [16, 17, 18]]])\n",
        "tensor "
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2, 3), dtype=int32, numpy=\n",
              "array([[[ 1,  2,  3],\n",
              "        [ 4,  5,  6]],\n",
              "\n",
              "       [[ 7,  8,  9],\n",
              "        [10, 11, 12]],\n",
              "\n",
              "       [[13, 14, 15],\n",
              "        [16, 17, 18]]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "f7Kwhiw_Dd5x",
        "outputId": "7cf860f8-cae6-44cf-cca0-b29dc3858cf8"
      },
      "source": [
        "tensor.ndim"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p6IVx8b3Fdg7"
      },
      "source": [
        "All of the above variables we've created are actually tensors. But you may also hear them referred to as their different names (the ones we gave them):\n",
        "\n",
        "* Scalar: a single number\n",
        "* Vector: a number with direction (e.g. wind speed and directions)\n",
        "* Matrix: 2-dimensional array of numbers\n",
        "* Tensor: an n-dimensional array of numbers (when n can be any number, a 0-dimensional tensor is a scalar, a 1-dimensional t ensor is a vector)\n",
        "\n",
        "For more on the mathematical difference between scalars, vectors and matrices see the [visual algebra post by Math is Fun.](https://www.mathsisfun.com/algebra/scalar-vector-matrix.html)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kNSvAc4fGHWG",
        "outputId": "80a2610e-5821-40ab-b01c-d503f7fb06f6"
      },
      "source": [
        " tf.Variable"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensorflow.python.ops.variables.Variable"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_suxjKknG31e"
      },
      "source": [
        "### Creating tensors with `tf.Variable` \n",
        "\n",
        "You can also (although you likely rarely will, because often, when working with data, tensors are created for you automatically) create tensors using [tf.Variable](https://www.tensorflow.org/api_docs/python/tf/Variable)\n",
        "\n",
        "The difference between `tf.Variable()` and `tf.constant()` is tensors created with tf.constant() are **immutable** (can't be changed, can only be used to create a new tensor), where as, tensors created with `tf.Variable()` are **mutable** (can be changed)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dN4CrkO-G2e1",
        "outputId": "50beb9ed-1539-4f63-8f5c-bb068cbf7710"
      },
      "source": [
        "# Create the same tensors with tf.Variable() and tf.constant()\n",
        "changeable_tensor = tf.Variable([10, 7])\n",
        "unchangeable_tensor = tf.constant([10, 7])\n",
        "changeable_tensor, unchangeable_tensor"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Variable 'Variable:0' shape=(2,) dtype=int32, numpy=array([10,  7], dtype=int32)>,\n",
              " <tf.Tensor: shape=(2,), dtype=int32, numpy=array([10,  7], dtype=int32)>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-ipRCzjXIMV4",
        "outputId": "599c2061-4857-4708-cb49-c640b406d16a"
      },
      "source": [
        "# Let's try change one fo the elements in our changeable tensor\n",
        "# changeable_tensor[0] = 7\n",
        "changeable_tensor[0].assign(7)\n",
        "changeable_tensor"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Variable 'Variable:0' shape=(2,) dtype=int32, numpy=array([7, 7], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3dl6ZkDFIyqO"
      },
      "source": [
        "# Let's try change our unchagable tensor\n",
        "\n",
        "# Will error (can't change tf.constant())\n",
        "\n",
        "# unchangeable_tensor[0].assign(7)\n",
        "# unchangeable_tensor\n"
      ],
      "execution_count": 15,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "88zenw-eLFuz"
      },
      "source": [
        "🔑  **Note**: Rarely in practice will you need to decide whether to use `tf.constant` or `tf.Variable` to create tensors, as TensorFlow does this for you. However, if in doubt, use `tf.constant` and change it later if needed."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-H9ukPqpK4OX"
      },
      "source": [
        "### Creating random tensors\n",
        "\n",
        "Random tensors are tensors of some abitrary size which contain random numbers.\n",
        "\n",
        "Why would you want to create random tensors?\n",
        "\n",
        "This is what neural networks use to intialize their weights (patterns) that they're trying to learn in the data.\n",
        "\n",
        "For example, the process of a neural network learning often involves taking a random n-dimensional array of numbers and refining them until they represent some kind of pattern (a compressed way to represent the original data)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Da4sAs51L5_-",
        "outputId": "061d9af1-18d4-44a6-86f8-d27503c689e7"
      },
      "source": [
        "# Create two random (but the same) tensor\n",
        "random_1 = tf.random.Generator.from_seed(42) # set seed for reproducibility\n",
        "random_1 = random_1.normal(shape=(3, 2)) # create tensor from a normal distribution\n",
        "random_2 = tf.random.Generator.from_seed(42)\n",
        "random_2 = random_2.normal(shape=(3, 2))\n",
        "\n",
        "# Are they equal\n",
        "random_1, random_2, random_1 == random_2"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              " array([[-0.7565803 , -0.06854702],\n",
              "        [ 0.07595026, -1.2573844 ],\n",
              "        [-0.23193765, -1.8107855 ]], dtype=float32)>,\n",
              " <tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              " array([[-0.7565803 , -0.06854702],\n",
              "        [ 0.07595026, -1.2573844 ],\n",
              "        [-0.23193765, -1.8107855 ]], dtype=float32)>,\n",
              " <tf.Tensor: shape=(3, 2), dtype=bool, numpy=\n",
              " array([[ True,  True],\n",
              "        [ True,  True],\n",
              "        [ True,  True]])>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "np0U0aMAWBNq"
      },
      "source": [
        "The random tensors we've made are actually [pseudorandom numbers](https://www.computerhope.com/jargon/p/pseudo-random.htm) (they appear as random, but really aren't).\n",
        "\n",
        "If we set a seed we'll get the same random numbers (if you've ever used NumPy, this is similar to np.random.seed(42)).\n",
        "\n",
        "Setting the seed says, \"hey, create some random numbers, but flavour them with X\" (X is the seed).\n",
        "\n",
        "What do you think will happen when we change the seed?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7HrDu35HWPMs",
        "outputId": "cd10f034-b240-4c20-872e-88b93e4f77bf"
      },
      "source": [
        "# Create two random (and different) tensors\n",
        "random_3 = tf.random.Generator.from_seed(42)\n",
        "random_3 = random_3.normal(shape=(3, 2))\n",
        "random_4 = tf.random.Generator.from_seed(11)\n",
        "random_4 = random_4.normal(shape=(3, 2))\n",
        "\n",
        "# Check the tensors and see if they are equal\n",
        "random_3, random_4, random_1 == random_3, random_3 == random_4"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              " array([[-0.7565803 , -0.06854702],\n",
              "        [ 0.07595026, -1.2573844 ],\n",
              "        [-0.23193765, -1.8107855 ]], dtype=float32)>,\n",
              " <tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              " array([[ 0.2730574 , -0.29925638],\n",
              "        [-0.3652325 ,  0.61883307],\n",
              "        [-1.0130816 ,  0.2829171 ]], dtype=float32)>,\n",
              " <tf.Tensor: shape=(3, 2), dtype=bool, numpy=\n",
              " array([[ True,  True],\n",
              "        [ True,  True],\n",
              "        [ True,  True]])>,\n",
              " <tf.Tensor: shape=(3, 2), dtype=bool, numpy=\n",
              " array([[False, False],\n",
              "        [False, False],\n",
              "        [False, False]])>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Pt5SrHqcLnpi"
      },
      "source": [
        "### Shuffle the order of elements in a tensor\n",
        "\n",
        "Why would we want to shuffle the order of a tensor?\n",
        "\n",
        "The order could effect how a neural network learns (it may overfit by learning the order of the data), instead, it might be a good idea to move your data around."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Do9rli1sqBAk",
        "outputId": "05756350-f1d6-4cfe-c547-3eb3703c266d"
      },
      "source": [
        "# shuffle a tensor (valuable for when you want to shuffle your \n",
        "# datat so the inherent order doesn't affect learning)\n",
        "not_shuffled = tf.constant([[10, 7], [3, 4], [2, 5]])\n",
        "\n",
        "# Shuffle our non-shuffled tensor. It will also get us \n",
        "# different results each time \n",
        "tf.random.shuffle(not_shuffled, seed=42)"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[ 2,  5],\n",
              "       [ 3,  4],\n",
              "       [10,  7]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "75TFzNlIrSAP",
        "outputId": "ce724643-ac0c-469f-b979-2d008a9f4a63"
      },
      "source": [
        "not_shuffled"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 3,  4],\n",
              "       [ 2,  5]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pyEG1juC4dsA"
      },
      "source": [
        "🛠   **Exercise:** Read through TensorFlow documentation on random seed generation:\n",
        "https://www.tensorflow.org/api_docs/python/tf/random/set_seed and practice writing 5 reandom test\n",
        "\n",
        "It looks like if we want our shuffled tensors to be in the same orderm, we've got to be in the same order, we've got to use the global level random seed as well as the operation level random seed:\n",
        "\n",
        "> Rule 4: if both the global and the operation seed are set: Both seeds are used in conjuction to determine the random sequence."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "g4EeKb4a4s3p",
        "outputId": "ced93290-8d8f-4ae8-a368-1f0202d88a24"
      },
      "source": [
        "tf.random.set_seed(42) # global level random seed\n",
        "tf.random.shuffle(not_shuffled, seed=42) # operation level random seed"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 3,  4],\n",
              "       [ 2,  5]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ESyYumSM7l0_"
      },
      "source": [
        "## Other way to make tensors\n",
        "\n",
        "Though you might rarely use these (remember, many tensor operations are done behind the scenes for you), you can use `tf.ones()` to create a tensor of all ones and tf.zeros() to create a tensor of all zeros."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RQ8BM60w7u9U",
        "outputId": "c0fbe059-1f6c-4526-d913-95f160c951f0"
      },
      "source": [
        "# Create a tensor of all ones\n",
        "tf.ones([10, 7])"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(10, 7), dtype=float32, numpy=\n",
              "array([[1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qqgzA4qv8Awh",
        "outputId": "ba42f1f1-4c30-47b0-ca8b-955565160d3f"
      },
      "source": [
        "# Create a tensor of all zeros\n",
        "tf.zeros(shape=(3, 4))"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 4), dtype=float32, numpy=\n",
              "array([[0., 0., 0., 0.],\n",
              "       [0., 0., 0., 0.],\n",
              "       [0., 0., 0., 0.]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Yp9c74MP8R8M"
      },
      "source": [
        "### Turn NumPy arrays into tensors\n",
        "\n",
        "The main differnece between NumPy arrays and TensorFlow tensors is that tensors can be run on a GPU (much faster for numerical computing).\n",
        "\n",
        ">🔑 **Note:** A matrix or tensor is typically represented by a capital letter (e.g. X or A) where as a vector is typically represented by a lowercase letter (e.g. y or b)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YIztDIoF8JvY",
        "outputId": "abe9bd9e-4a88-44e2-c632-58fede38e1f4"
      },
      "source": [
        "# You can also turn NumPy arrays into tensors\n",
        "import numpy as np\n",
        "numpy_A = np.arange(1, 25, dtype=np.int32)\n",
        "numpy_A\n",
        "\n",
        "# X = tf.constant(some_matrix) # capital for matrix or tensor\n",
        "# y = tf.constant(vector) # non-capital for vector"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,\n",
              "       18, 19, 20, 21, 22, 23, 24], dtype=int32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SXmNGhwg9hNx",
        "outputId": "52e1900b-d805-4ba9-827a-b389fc5421de"
      },
      "source": [
        "A = tf.constant(numpy_A, shape=(2, 3, 4))\n",
        "B = tf.constant(numpy_A)\n",
        "A, B"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2, 3, 4), dtype=int32, numpy=\n",
              " array([[[ 1,  2,  3,  4],\n",
              "         [ 5,  6,  7,  8],\n",
              "         [ 9, 10, 11, 12]],\n",
              " \n",
              "        [[13, 14, 15, 16],\n",
              "         [17, 18, 19, 20],\n",
              "         [21, 22, 23, 24]]], dtype=int32)>,\n",
              " <tf.Tensor: shape=(24,), dtype=int32, numpy=\n",
              " array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,\n",
              "        18, 19, 20, 21, 22, 23, 24], dtype=int32)>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2JMyGhA998C9",
        "outputId": "2d086339-cd06-4362-bcaf-967e372173af"
      },
      "source": [
        "2 * 3 * 4"
      ],
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "24"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_koOkgTh-lnS",
        "outputId": "dc1e3362-a0ad-49c5-999c-a72d56eb8e7c"
      },
      "source": [
        "A.ndim"
      ],
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-bGexX-N-bZI"
      },
      "source": [
        "### Getting information form tensors"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xZCNIgp4Js02"
      },
      "source": [
        "#### Getting information form tensors\n",
        "\n",
        "When dealing with tensors you probably want to be aware of the following attributes:\n",
        "\n",
        "* **Shape:** The length (number of elements) of each of the dimensions of a tensor.\n",
        "* **Rank:** The number of tensor dimensions. A scalar has rank 0, a vector has rank 1, a matrix is rank 2, a tensor has rank n.\n",
        "* **Axis** or **dimension:** A particular dimension of a tensor.\n",
        "* **Size:** The total number of items in the tensor.\n",
        "\n",
        "You'll use these especially when you're trying to line up the shapes of your data to the shapes of your model. For example, making sure the shape of your image tensors are the same shape as your models input layer.\n",
        "\n",
        "We've already seen one of these before using the `ndim` attribute. Let's see the rest."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_d0PUZJn-ep3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "8aa74dfc-b79c-4f7b-c830-3139fb419a0e"
      },
      "source": [
        "# Create a rank 4 tensor (4 dimensions)\n",
        "rank_4_tensor = tf.zeros(shape=[2, 3, 4, 5])\n",
        "rank_4_tensor"
      ],
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 3, 4, 5), dtype=float32, numpy=\n",
              "array([[[[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]]],\n",
              "\n",
              "\n",
              "       [[[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]]]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "a0UraDc5LPwz",
        "outputId": "3b4d9869-5eef-4056-a0a8-01129766b4a2"
      },
      "source": [
        "rank_4_tensor.shape, rank_4_tensor.ndim, tf.size(rank_4_tensor)"
      ],
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(TensorShape([2, 3, 4, 5]), 4, <tf.Tensor: shape=(), dtype=int32, numpy=120>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "C9KX6IXYLc85",
        "outputId": "ebd1c823-dc5a-4e7b-b2fe-8854912037f4"
      },
      "source": [
        "# Get vaious attributes of our tensor\n",
        "print(\"Datatype of every elemetn:\", rank_4_tensor.dtype)\n",
        "print(\"Number of dimensions (rank)\", rank_4_tensor.ndim)\n",
        "print(\"Shape of tensor:\", rank_4_tensor.shape)\n",
        "print(\"Elements along the 0 axis:\", rank_4_tensor.shape[0])\n",
        "print(\"Elements along the last axis:\", rank_4_tensor.shape[-1])\n",
        "print(\"Total number of elements in our tensor:\", tf.size(rank_4_tensor).numpy()) # we get it in a single integer"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Datatype of every elemetn: <dtype: 'float32'>\n",
            "Number of dimensions (rank) 4\n",
            "Shape of tensor: (2, 3, 4, 5)\n",
            "Elements along the 0 axis: 2\n",
            "Elements along the last axis: 5\n",
            "Total number of elements in our tensor: 120\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kXaauPtUM1Rp"
      },
      "source": [
        "## Indexing tensors\n",
        "\n",
        "Tensors can be indexed just like Python list"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "i99rgGQ2OQYf",
        "outputId": "39b7e1b4-f140-49a5-c13a-da096c7c9d9f"
      },
      "source": [
        "some_list = [1, 2, 3, 4]\n",
        "some_list[:2]"
      ],
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[1, 2]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Bhi5Ha_MN0eI",
        "outputId": "d385d088-20cc-4c5e-9c5e-8417d1af39dc"
      },
      "source": [
        "# Get the first2 elements of each dimension\n",
        "rank_4_tensor[:2, :2, :2, :2]"
      ],
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2, 2, 2), dtype=float32, numpy=\n",
              "array([[[[0., 0.],\n",
              "         [0., 0.]],\n",
              "\n",
              "        [[0., 0.],\n",
              "         [0., 0.]]],\n",
              "\n",
              "\n",
              "       [[[0., 0.],\n",
              "         [0., 0.]],\n",
              "\n",
              "        [[0., 0.],\n",
              "         [0., 0.]]]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zM8VQeUGOk5M",
        "outputId": "38cf6083-e1a7-425a-bdf3-63a2af7e3f08"
      },
      "source": [
        "# Get the first element each dimension from each \n",
        "# dimension from each index except for the final one\n",
        "rank_4_tensor[:1, :1, :1]"
      ],
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 1, 1, 5), dtype=float32, numpy=array([[[[0., 0., 0., 0., 0.]]]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "l5t-xUCnO4cW",
        "outputId": "3e95f36e-7cc8-4659-f795-9154055b52a3"
      },
      "source": [
        "# Create a rank 2 tensor (2 dimensions)\n",
        "rank_2_tensor = tf.constant([[10, 7],\n",
        "                             [3, 4]])\n",
        "rank_2_tensor.shape, rank_2_tensor.ndim"
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(TensorShape([2, 2]), 2)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SZTMFuzBQZJc",
        "outputId": "c9a28e53-bd63-47db-8a9d-1a5f1e04338b"
      },
      "source": [
        "rank_2_tensor"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 3,  4]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WwDydvsTP52C",
        "outputId": "d2011887-e820-4995-aec7-f959e4212624"
      },
      "source": [
        "# Get the item of each of our rank 2 tensor\n",
        "rank_2_tensor[:, -1]"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([7, 4], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uZUCZ9Sma4Cm"
      },
      "source": [
        "You can also add dimensions to your tensor whilst keeping the same information present using `tf.newaxis`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XBGBhIH0QeaY",
        "outputId": "f301e0fc-4e13-48f5-ca88-cdfc4d37193c"
      },
      "source": [
        "# Add in extra dimension to our rank 2 tensor\n",
        "# in Python \"...\" means \"all dimensions prior to\"\n",
        "rank_3_tensor = rank_2_tensor[..., tf.newaxis]\n",
        "rank_3_tensor"
      ],
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2, 1), dtype=int32, numpy=\n",
              "array([[[10],\n",
              "        [ 7]],\n",
              "\n",
              "       [[ 3],\n",
              "        [ 4]]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 36
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pH6RgQRQbOVY"
      },
      "source": [
        "You can achieve the same using `tf.expand_dims()` Documentation [here](https://www.tensorflow.org/api_docs/python/tf/expand_dims)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lGha_1M6RWWW",
        "outputId": "616efab9-7e23-45cf-f176-ef9af135d0de"
      },
      "source": [
        "# alternative to tf.newaxis\n",
        "tf.expand_dims(rank_2_tensor, axis=-1) # \"-1\" means expand the final axisa"
      ],
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2, 1), dtype=int32, numpy=\n",
              "array([[[10],\n",
              "        [ 7]],\n",
              "\n",
              "       [[ 3],\n",
              "        [ 4]]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 37
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2YEu-bgoRlMo"
      },
      "source": [
        "### Manipulating tensors (tensor operations)\n",
        "\n",
        "**Basic operations**\n",
        "\n",
        "`+`, `-`, `*`, `/`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Y6iUQArmnhHG",
        "outputId": "83eba7db-84af-43ee-f70d-f6afb8b18400"
      },
      "source": [
        "# you can add values to a tensor using the addition operator\n",
        "tensor = tf.constant([[10, 7], [3, 4]])\n",
        "tensor + 10"
      ],
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[20, 17],\n",
              "       [13, 14]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 38
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Rrr6zPVhrqN6",
        "outputId": "515ed94b-8c8f-45d0-aa6d-671ccc25df87"
      },
      "source": [
        "# original tensor is unchanged\n",
        "tensor"
      ],
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 3,  4]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cJHxNeN9rxLi",
        "outputId": "9a732c42-8c7e-4288-ee4b-f81bb232abed"
      },
      "source": [
        "tensor = tensor + 10\n",
        "tensor "
      ],
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[20, 17],\n",
              "       [13, 14]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 40
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nfTRj_XHr4iS",
        "outputId": "d7d3255c-a5e1-4389-82a1-6dfdb96a51f1"
      },
      "source": [
        "# Multiplication also works\n",
        "tensor * 10"
      ],
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[200, 170],\n",
              "       [130, 140]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "g-LwZBESr9Z2",
        "outputId": "6b43aa17-0418-402e-d5db-6c9ea36b94a7"
      },
      "source": [
        "# Substraction if you want\n",
        "tensor - 10"
      ],
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 3,  4]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 42
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9s0ETie7sjRo",
        "outputId": "eadaf893-ce93-4ee2-850a-36dd892a8bd6"
      },
      "source": [
        "# We can use the tensorflow built-in function\n",
        "tf.multiply(tensor, 10)"
      ],
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[200, 170],\n",
              "       [130, 140]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nWZu_aU-tGdd"
      },
      "source": [
        "**Matrix multiplication**\n",
        "\n",
        "In machine learning, matrix multiplication is one of the most tensor operations.\n",
        "\n",
        "There are two rules our tensors (or matrices) need to fulfil if we're going to matrix multiply them:\n",
        "\n",
        "1. The inner dimensions must match\n",
        "2. The resulting matrix has the shape of the outer dimensions\n",
        "\n",
        "One of the most common operations in machine learning algorithms is [matrix multiplication](https://www.mathsisfun.com/algebra/matrix-multiplying.html).\n",
        "\n",
        "TensorFlow implements this matrix multiplication functionality in the `tf.matmul()` method.\n",
        "\n",
        "The main two rules for matrix multiplication to remember are:\n",
        "\n",
        "The inner dimensions must match:\n",
        "* `(3, 5) @ (3, 5)` won't work\n",
        "* `(5, 3) @ (3, 5)`will work\n",
        "* `(3, 5) @ (5, 3)` will work\n",
        "\n",
        "The resulting matrix has the shape of the inner dimensions:\n",
        "* `(5, 3) @ (3, 5) -> (3, 3)`\n",
        "* `(3, 5) @ (5, 3) -> (5, 5)`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pDRHC0zptb9O",
        "outputId": "2a4f5095-ce88-46a4-f31a-677636670bbb"
      },
      "source": [
        "# Matrix multiplication in tensor\n",
        "print(tensor)\n",
        "tf.matmul(tensor, tensor)"
      ],
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tf.Tensor(\n",
            "[[20 17]\n",
            " [13 14]], shape=(2, 2), dtype=int32)\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[621, 578],\n",
              "       [442, 417]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 44
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HbYGxIsQu2i0",
        "outputId": "72a8034e-9d1d-4b9b-a7e8-94a1071d23ac"
      },
      "source": [
        "# Matrix mutliplication with python operator \"@\"\n",
        "tensor @ tensor"
      ],
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[621, 578],\n",
              "       [442, 417]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 45
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "svup2tTavex5",
        "outputId": "ae3c3c38-ad8a-4065-9dd6-97af012ef529"
      },
      "source": [
        "tensor.shape"
      ],
      "execution_count": 46,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TensorShape([2, 2])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 46
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "O3hOr77ivh6c",
        "outputId": "541e5608-0ca8-475d-c7f9-104e78adfaae"
      },
      "source": [
        "# create a tensor (3, 2) tensor\n",
        "X = tf.constant([[1, 2],\n",
        "                 [3, 4],\n",
        "                 [5, 6]])\n",
        "# Creat another (3, 3) tensor\n",
        "Y = tf.constant([[7, 8],\n",
        "                 [9, 10],\n",
        "                 [11, 12]])\n",
        "X, Y"
      ],
      "execution_count": 47,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[1, 2],\n",
              "        [3, 4],\n",
              "        [5, 6]], dtype=int32)>, <tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[ 7,  8],\n",
              "        [ 9, 10],\n",
              "        [11, 12]], dtype=int32)>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 47
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gDJuDbRwfXUl"
      },
      "source": [
        "Trying to matrix multiply two tensors with the shape (3, 2) errors because the inner dimensions don't match.\n",
        "\n",
        "We need to either:\n",
        "* Reshape X to (2, 3) so it's (2, 3) @ (3, 2).\n",
        "* Reshape Y to (3, 2) so it's (3, 2) @ (2, 3).\n",
        "\n",
        "We can do this with either:\n",
        "* tf.reshape() - allows us to reshape a tensor into a defined shape.\n",
        "* tf.transpose() - switches the dimensions of a given tensor."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ECa_NbNtxYyX",
        "outputId": "9ff18427-d536-4872-c58a-1a31e56ca29b"
      },
      "source": [
        "# Let's change the shape of Y\n",
        "tf.reshape(Y, shape=(2, 3))"
      ],
      "execution_count": 48,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
              "array([[ 7,  8,  9],\n",
              "       [10, 11, 12]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KPjfhWxE0Rj-",
        "outputId": "507b9873-d527-4f56-f05a-f9d5213c0387"
      },
      "source": [
        "X.shape, tf.reshape(Y, shape=(2, 3)).shape"
      ],
      "execution_count": 49,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(TensorShape([3, 2]), TensorShape([2, 3]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8nG2Co4nz5A3",
        "outputId": "e17f1d16-1da9-45c0-9a0a-fa50004b1209"
      },
      "source": [
        "# Try to multiply X by reshaped Y\n",
        "X @ tf.reshape(Y, shape=(2, 3))"
      ],
      "execution_count": 50,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 27,  30,  33],\n",
              "       [ 61,  68,  75],\n",
              "       [ 95, 106, 117]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 50
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hWmEVcU-0naP",
        "outputId": "ad651a23-51f0-49f9-85df-9af56a650428"
      },
      "source": [
        "# Also\n",
        "tf.matmul(X, tf.reshape(Y, shape=(2, 3)))"
      ],
      "execution_count": 51,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 27,  30,  33],\n",
              "       [ 61,  68,  75],\n",
              "       [ 95, 106, 117]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 51
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "o1RgrVuh07YM",
        "outputId": "f477edb6-b3d9-4097-bdee-2e725683f441"
      },
      "source": [
        "# Try change the shape of X instead of Y\n",
        "tf.matmul(tf.reshape(X, shape=(2, 3)), Y)"
      ],
      "execution_count": 52,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[ 58,  64],\n",
              "       [139, 154]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 52
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tM7UTS-a2E6d",
        "outputId": "b37fd7a3-3eba-4d10-fc08-da0dcd9c08aa"
      },
      "source": [
        "# Can do the same with transpose\n",
        "X, tf.transpose(X), tf.reshape(X, shape=(2, 3))"
      ],
      "execution_count": 53,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[1, 2],\n",
              "        [3, 4],\n",
              "        [5, 6]], dtype=int32)>, <tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
              " array([[1, 3, 5],\n",
              "        [2, 4, 6]], dtype=int32)>, <tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
              " array([[1, 2, 3],\n",
              "        [4, 5, 6]], dtype=int32)>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 53
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EKtX5baJ2xWl",
        "outputId": "9ca822c1-363d-4221-f21a-68d995fda12b"
      },
      "source": [
        "# Try matrix multiplication with transpose rather than reshape\n",
        "tf.matmul(tf.transpose(X), Y)"
      ],
      "execution_count": 54,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[ 89,  98],\n",
              "       [116, 128]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 54
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vn3QJ8bdwyWx"
      },
      "source": [
        "📖 **Resource:** info and example of matrix multiplication: https://www.mathsisfun.com/algebra/matrix-multiplying.html"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CtDZ6NxMatKG"
      },
      "source": [
        "**The dot product**\n",
        "\n",
        "Matrix multiplications  is also referred to as the dot product.\n",
        "\n",
        "You can perform matrix multiplication using:\n",
        "* `tf.matmul()`\n",
        "* `tf.tesnordot()`\n",
        "\n",
        "More about [tesnordot](https://www.tensorflow.org/api_docs/python/tf/tensordot)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hQeQrw0Gbrzg",
        "outputId": "4f2aef92-e1aa-4e33-ef4e-169b857a90a7"
      },
      "source": [
        "X, Y"
      ],
      "execution_count": 55,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[1, 2],\n",
              "        [3, 4],\n",
              "        [5, 6]], dtype=int32)>, <tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[ 7,  8],\n",
              "        [ 9, 10],\n",
              "        [11, 12]], dtype=int32)>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 55
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3Elbng8iadje",
        "outputId": "5ec69465-08df-408e-ad31-b094d6871b75"
      },
      "source": [
        "# Perform the dot product on X and Y (requires X or Y to be transposed)\n",
        "tf. tensordot(tf.transpose(X), Y, axes=1)"
      ],
      "execution_count": 56,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[ 89,  98],\n",
              "       [116, 128]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 56
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4KD5_Ju3b-fr",
        "outputId": "60186833-ae6b-4364-bb2f-41cc7b4f816c"
      },
      "source": [
        "# Perform matrix mutliplication between X and Y (transposed)\n",
        "tf.matmul(X, tf.transpose(Y))"
      ],
      "execution_count": 57,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 23,  29,  35],\n",
              "       [ 53,  67,  81],\n",
              "       [ 83, 105, 127]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 57
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6sh33QxdcTvm",
        "outputId": "a57883ac-0fec-4830-ef30-18a5d6bb9e78"
      },
      "source": [
        "# Perform matrix multiplication between X and Y (reshaped)\n",
        "tf.matmul(X, tf.reshape(Y, shape=(2, 3)))"
      ],
      "execution_count": 58,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 27,  30,  33],\n",
              "       [ 61,  68,  75],\n",
              "       [ 95, 106, 117]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 58
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WaALqnXhcsO2",
        "outputId": "5eb308fb-6cdd-4cea-9635-84e882d9d9a2"
      },
      "source": [
        "# Check the values of Y, reshape Y and transposed Y\n",
        "print(\"Normal Y:\")\n",
        "print(Y, \"\\n\") \n",
        "\n",
        "print(\"Y reshaped to (2, 3):\")\n",
        "print(tf.reshape(Y, (2, 3)), \"\\n\")\n",
        "\n",
        "print(\"Y transposed:\")\n",
        "print(tf.transpose(Y))"
      ],
      "execution_count": 59,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Normal Y:\n",
            "tf.Tensor(\n",
            "[[ 7  8]\n",
            " [ 9 10]\n",
            " [11 12]], shape=(3, 2), dtype=int32) \n",
            "\n",
            "Y reshaped to (2, 3):\n",
            "tf.Tensor(\n",
            "[[ 7  8  9]\n",
            " [10 11 12]], shape=(2, 3), dtype=int32) \n",
            "\n",
            "Y transposed:\n",
            "tf.Tensor(\n",
            "[[ 7  9 11]\n",
            " [ 8 10 12]], shape=(2, 3), dtype=int32)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "W1AEt5YUdqDc",
        "outputId": "6efff123-dfaa-4c94-f6fb-441b4aba6f5b"
      },
      "source": [
        "tf.matmul(X, tf.transpose(Y))"
      ],
      "execution_count": 60,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 23,  29,  35],\n",
              "       [ 53,  67,  81],\n",
              "       [ 83, 105, 127]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 60
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XpQxCqzxd04R"
      },
      "source": [
        "Generally , when performing matrix multiplication on two tensors and one of the axes doesn't line up, you will tranpose (rather than reshape) one of the tensors to satisify the matrix multiplication rules\n",
        "\n",
        "* `tf.reshape()` - change the shape of the given tensor (first) and then insert values in order they appear (in our case, 7, 8, 9, 10, 11, 12).\n",
        "* `tf.transpose()` - swap the order of the axes, by default the last axis becomes the first, however the order can be changed using the [perm](https://www.tensorflow.org/api_docs/python/tf/transpose) parameter."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g2jVD8R3epNk"
      },
      "source": [
        "### Changing the datatype of a tensor"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "sfAlsZVZfoR4",
        "outputId": "caa36815-c15b-4e3c-a7bb-5915dc32304c"
      },
      "source": [
        "tf.__version__"
      ],
      "execution_count": 61,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'2.5.0'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 61
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6gm46v0RfIpK",
        "outputId": "82116f52-d881-4c61-dd35-dba6e699e525"
      },
      "source": [
        "# Create a new tensor with default datatype (float32)\n",
        "B = tf.constant([1.7, 7.4])\n",
        "B, B.dtype"
      ],
      "execution_count": 62,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2,), dtype=float32, numpy=array([1.7, 7.4], dtype=float32)>,\n",
              " tf.float32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 62
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "v5NW19G8fZvv",
        "outputId": "b9dfc149-b36a-4c38-a9c3-6bfc574477ed"
      },
      "source": [
        "C = tf.constant([7, 10])\n",
        "C, C.dtype"
      ],
      "execution_count": 63,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2,), dtype=int32, numpy=array([ 7, 10], dtype=int32)>,\n",
              " tf.int32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 63
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fpnd0RAIgA5w",
        "outputId": "899e96f5-df45-46ad-e8b7-f4086e42c946"
      },
      "source": [
        "# Change form float 32 to float16 (reduced precision)\n",
        "D = tf.cast(B, dtype=tf.float16)\n",
        "D, D.dtype"
      ],
      "execution_count": 64,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2,), dtype=float16, numpy=array([1.7, 7.4], dtype=float16)>,\n",
              " tf.float16)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 64
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jnHYtRzog7PT",
        "outputId": "5958d102-8200-4913-caab-e7702429d8d3"
      },
      "source": [
        "# change form int32 to float32\n",
        "E = tf.cast(C, dtype=tf.float32)\n",
        "E "
      ],
      "execution_count": 65,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=float32, numpy=array([ 7., 10.], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 65
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LCewDAhbhY1t",
        "outputId": "8b05c625-d5a6-4b35-f38f-1308a4e46c86"
      },
      "source": [
        "E_float16 = tf.cast(E, dtype=tf.float16)\n",
        "E_float16"
      ],
      "execution_count": 66,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=float16, numpy=array([ 7., 10.], dtype=float16)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 66
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Op9mBUvQhn24"
      },
      "source": [
        "## Aggregating Tensors\n",
        "\n",
        "Aggregating tensors = condensing them from multiple values down to a smaller amount of values.\n",
        "\n",
        "To get the absolute value [tf.abs](https://www.tensorflow.org/api_docs/python/tf/math/abs)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sfAgqufrhzNF",
        "outputId": "b56cb6eb-0347-4748-b53d-22714ed44be8"
      },
      "source": [
        "# Get the absolute values\n",
        "D = tf.constant([-7, -10])\n",
        "D"
      ],
      "execution_count": 67,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([ -7, -10], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 67
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LDgArxQXjrxR",
        "outputId": "edbba0e7-29d6-434e-d1ff-3ee7b978d415"
      },
      "source": [
        "# Get the absolute values\n",
        "tf.abs(D)"
      ],
      "execution_count": 68,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([ 7, 10], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 68
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Bo7oegxQltUA"
      },
      "source": [
        "Let's go through the following forms of aggregation:\n",
        "* `tf.reduce_min()` - find the minimum value in a tensor.\n",
        "* `tf.reduce_max()` - find the maximum value in a tensor (helpful for when you want to find the highest prediction probability).\n",
        "* `tf.reduce_mean()` - find the mean of all elements in a tensor.\n",
        "* `tf.reduce_sum()` - find the sum of all elements in a tensor.\n",
        "* **Note:** typically, each of these is under the math module, e.g. `tf.math.reduce_min()` but you can use the alias tf.`reduce_min()`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "q9_TApAnljWG",
        "outputId": "e2e5d876-f11d-48ea-f88b-5a0daca5bc2a"
      },
      "source": [
        "# Creating a random tensor with a values between 0 and 100 of size 50\n",
        "E = tf.constant(np.random.randint(0, 100, size=50))\n",
        "E"
      ],
      "execution_count": 69,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(50,), dtype=int64, numpy=\n",
              "array([78, 58, 63, 73, 79, 10, 92, 55, 91, 67, 30, 10, 68,  0, 77, 94, 89,\n",
              "       15, 94, 51,  7, 29, 35, 63, 42, 56, 25, 30, 31, 51, 75, 52, 56, 26,\n",
              "       31, 74, 68,  4, 67, 20, 18, 64, 12, 96, 25,  9, 21, 57, 50, 16])>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 69
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YXh3-K4Bmoq8",
        "outputId": "cf3b157c-51e0-41bd-b970-c26cea2d217c"
      },
      "source": [
        "tf.size(E), E.shape, E.ndim"
      ],
      "execution_count": 70,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(), dtype=int32, numpy=50>, TensorShape([50]), 1)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 70
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YQxJ_2YRmx3_",
        "outputId": "a02efca9-1fe9-46d0-f96d-34e373af3ac5"
      },
      "source": [
        "# Find the minimum\n",
        "tf.reduce_min(E)"
      ],
      "execution_count": 71,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 71
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HiCGRGFNnEra",
        "outputId": "0f35258f-e553-4566-a4d0-d915c56a2dc5"
      },
      "source": [
        "# Find the max\n",
        "tf.reduce_max(E)"
      ],
      "execution_count": 72,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=96>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 72
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XkiP8K5vnKu1",
        "outputId": "bebaf7ea-5ccd-46a3-e5fc-a90859af2139"
      },
      "source": [
        "# Find t he mean \n",
        "tf.reduce_mean(E)"
      ],
      "execution_count": 73,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=48>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 73
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZgoqCOkvnUKo",
        "outputId": "380fa4c6-32cf-4185-891f-06926633f3ea"
      },
      "source": [
        "# Find the sum\n",
        "tf.reduce_sum(E)"
      ],
      "execution_count": 74,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=2404>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 74
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LkKyJw-lnESX"
      },
      "source": [
        "🛠 **Exercise:** With what we've just learned, find the variance and standard deviation of our `E` tensor using TensorFlow methods."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pg4cZKbUn8k7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4588a4a8-fa94-4bfc-b816-a920f57ea826"
      },
      "source": [
        "# To find the variance of our tensor, we need access to tensorflow_probability\n",
        "import tensorflow_probability as tfp\n",
        "tfp.stats.variance(E)"
      ],
      "execution_count": 75,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=776>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 75
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "g4HrmFbwRh1_",
        "outputId": "4a31d93b-9d32-4b5a-ff43-109a5b47ef05"
      },
      "source": [
        "# Find the standard deviation\n",
        "tf.math.reduce_std(tf.cast(E, dtype=tf.float32))"
      ],
      "execution_count": 76,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=27.872452>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 76
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6hmscgEdS21S",
        "outputId": "cdcbf335-29c2-4abf-85a8-8be29b3466aa"
      },
      "source": [
        "# Find the variance of our `E` tensor\n",
        "tf.math.reduce_variance(tf.cast(E, dtype=tf.float32))"
      ],
      "execution_count": 77,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=776.8736>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 77
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IjF7eih2SCqG"
      },
      "source": [
        "### Find the positional maximum and minimum"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4-yjCVvKSjhD",
        "outputId": "2cee885c-a2c8-44f4-82e1-be09f54ff2bc"
      },
      "source": [
        "# Create a new tensor for finding positional minimum and maxium\n",
        "tf.random.set_seed(42)\n",
        "F = tf.random.uniform(shape=[50])\n",
        "F "
      ],
      "execution_count": 78,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(50,), dtype=float32, numpy=\n",
              "array([0.6645621 , 0.44100678, 0.3528825 , 0.46448255, 0.03366041,\n",
              "       0.68467236, 0.74011743, 0.8724445 , 0.22632635, 0.22319686,\n",
              "       0.3103881 , 0.7223358 , 0.13318717, 0.5480639 , 0.5746088 ,\n",
              "       0.8996835 , 0.00946367, 0.5212307 , 0.6345445 , 0.1993283 ,\n",
              "       0.72942245, 0.54583454, 0.10756552, 0.6767061 , 0.6602763 ,\n",
              "       0.33695042, 0.60141766, 0.21062577, 0.8527372 , 0.44062173,\n",
              "       0.9485276 , 0.23752594, 0.81179297, 0.5263394 , 0.494308  ,\n",
              "       0.21612847, 0.8457197 , 0.8718841 , 0.3083862 , 0.6868038 ,\n",
              "       0.23764038, 0.7817228 , 0.9671384 , 0.06870162, 0.79873943,\n",
              "       0.66028714, 0.5871513 , 0.16461694, 0.7381023 , 0.32054043],\n",
              "      dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 78
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iDbPXqweUVuC",
        "outputId": "e911c338-19d3-4ebc-c846-ca55ea33ca54"
      },
      "source": [
        "# Find the positional max\n",
        "tf.argmax(F)"
      ],
      "execution_count": 79,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=42>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 79
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "STE7BDgHUpkW",
        "outputId": "f2570dad-c750-429f-ee9e-3bddce129934"
      },
      "source": [
        "# Index on our largest value position\n",
        "F[tf.argmax(F)]"
      ],
      "execution_count": 80,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.9671384>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 80
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kSK-JwOKU1GY",
        "outputId": "e7bed866-3bd5-4c31-96ed-73db98b59fb7"
      },
      "source": [
        "# Find the max value of F\n",
        "tf.reduce_max(F)"
      ],
      "execution_count": 81,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.9671384>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 81
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "obNyO5dyVA5i",
        "outputId": "0f941041-bcc8-4e96-9822-132c0efb6986"
      },
      "source": [
        "# Check for equality\n",
        "F[tf.argmax(F)] == tf.reduce_max(F)"
      ],
      "execution_count": 82,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=bool, numpy=True>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 82
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zz-hPKTAVQDt",
        "outputId": "6f388841-0556-4d75-9630-eb5969c3d161"
      },
      "source": [
        "# Find the positional min\n",
        "tf.argmin(F)"
      ],
      "execution_count": 83,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=16>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 83
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "cellView": "code",
        "id": "taStVlK-VlZz",
        "outputId": "b24c1db0-9697-47b7-b311-4a430212d389"
      },
      "source": [
        "#@title Default title text\n",
        "# Find the min using the positional min index\n",
        "F[tf.argmin(F)]"
      ],
      "execution_count": 84,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.009463668>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 84
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fXCo1un4bYl-"
      },
      "source": [
        "### Squeezing a tensor (removing all single dimensions)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "R5uJWBnHbijW",
        "outputId": "9f1e13cc-3718-4636-9284-7944508f87b6"
      },
      "source": [
        " # Create a tensor to get started \n",
        " tf.random.set_seed(42)\n",
        " G = tf.constant(tf.random.uniform(shape=[50]), shape=(1, 1, 1, 1, 50))\n",
        " G"
      ],
      "execution_count": 85,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 1, 1, 1, 50), dtype=float32, numpy=\n",
              "array([[[[[0.6645621 , 0.44100678, 0.3528825 , 0.46448255, 0.03366041,\n",
              "           0.68467236, 0.74011743, 0.8724445 , 0.22632635, 0.22319686,\n",
              "           0.3103881 , 0.7223358 , 0.13318717, 0.5480639 , 0.5746088 ,\n",
              "           0.8996835 , 0.00946367, 0.5212307 , 0.6345445 , 0.1993283 ,\n",
              "           0.72942245, 0.54583454, 0.10756552, 0.6767061 , 0.6602763 ,\n",
              "           0.33695042, 0.60141766, 0.21062577, 0.8527372 , 0.44062173,\n",
              "           0.9485276 , 0.23752594, 0.81179297, 0.5263394 , 0.494308  ,\n",
              "           0.21612847, 0.8457197 , 0.8718841 , 0.3083862 , 0.6868038 ,\n",
              "           0.23764038, 0.7817228 , 0.9671384 , 0.06870162, 0.79873943,\n",
              "           0.66028714, 0.5871513 , 0.16461694, 0.7381023 , 0.32054043]]]]],\n",
              "      dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 85
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZMA9E5jHcWvF",
        "outputId": "27b5252d-6de3-42f8-edea-00cb970a0b95"
      },
      "source": [
        "G.shape"
      ],
      "execution_count": 86,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TensorShape([1, 1, 1, 1, 50])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 86
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WRr3R2UScZAd",
        "outputId": "cbb87952-2982-40fd-d008-3ffea52dfc52"
      },
      "source": [
        "G_squeezed = tf.squeeze(G)\n",
        "G_squeezed, G_squeezed.shape"
      ],
      "execution_count": 87,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(50,), dtype=float32, numpy=\n",
              " array([0.6645621 , 0.44100678, 0.3528825 , 0.46448255, 0.03366041,\n",
              "        0.68467236, 0.74011743, 0.8724445 , 0.22632635, 0.22319686,\n",
              "        0.3103881 , 0.7223358 , 0.13318717, 0.5480639 , 0.5746088 ,\n",
              "        0.8996835 , 0.00946367, 0.5212307 , 0.6345445 , 0.1993283 ,\n",
              "        0.72942245, 0.54583454, 0.10756552, 0.6767061 , 0.6602763 ,\n",
              "        0.33695042, 0.60141766, 0.21062577, 0.8527372 , 0.44062173,\n",
              "        0.9485276 , 0.23752594, 0.81179297, 0.5263394 , 0.494308  ,\n",
              "        0.21612847, 0.8457197 , 0.8718841 , 0.3083862 , 0.6868038 ,\n",
              "        0.23764038, 0.7817228 , 0.9671384 , 0.06870162, 0.79873943,\n",
              "        0.66028714, 0.5871513 , 0.16461694, 0.7381023 , 0.32054043],\n",
              "       dtype=float32)>, TensorShape([50]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 87
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uG7P_gRORyXq"
      },
      "source": [
        "### One-hot encoding tensors"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "C-jD61-tRxI9",
        "outputId": "2df12227-32ff-4e11-fe44-3ae85826ede1"
      },
      "source": [
        "# Create a list of indecies\n",
        "some_list = [0, 1, 2, 3] # could be red, green, blue, purple\n",
        "\n",
        "# One hot encode our list of indeces\n",
        "tf.one_hot(some_list, depth=4)"
      ],
      "execution_count": 88,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4, 4), dtype=float32, numpy=\n",
              "array([[1., 0., 0., 0.],\n",
              "       [0., 1., 0., 0.],\n",
              "       [0., 0., 1., 0.],\n",
              "       [0., 0., 0., 1.]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 88
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "T82ZSgKNS3sC",
        "outputId": "c89a4cb9-b0be-4603-ce13-aec6f602eaef"
      },
      "source": [
        "# Specify custom values for one hot encoding\n",
        "tf.one_hot(some_list, depth=4, on_value=\"Yo I love deep learning\", off_value=\"I also like to dance\")"
      ],
      "execution_count": 89,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4, 4), dtype=string, numpy=\n",
              "array([[b'Yo I love deep learning', b'I also like to dance',\n",
              "        b'I also like to dance', b'I also like to dance'],\n",
              "       [b'I also like to dance', b'Yo I love deep learning',\n",
              "        b'I also like to dance', b'I also like to dance'],\n",
              "       [b'I also like to dance', b'I also like to dance',\n",
              "        b'Yo I love deep learning', b'I also like to dance'],\n",
              "       [b'I also like to dance', b'I also like to dance',\n",
              "        b'I also like to dance', b'Yo I love deep learning']],\n",
              "      dtype=object)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 89
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UrUNf2uOTjD6"
      },
      "source": [
        "### Squaring, log, square root"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "896WkNqFUF48",
        "outputId": "c1b72a49-1f67-412d-e2a8-24825632a199"
      },
      "source": [
        "# Create a new tensor\n",
        "H = tf.range(1, 10)\n",
        "H"
      ],
      "execution_count": 90,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=int32, numpy=array([1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 90
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TQu2ii8-UQMp",
        "outputId": "ca9b5b04-9d64-4dee-f20f-a200c0a99794"
      },
      "source": [
        "# Square it\n",
        "tf.square(H)"
      ],
      "execution_count": 91,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=int32, numpy=array([ 1,  4,  9, 16, 25, 36, 49, 64, 81], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 91
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0tMwu41tUd5_",
        "outputId": "d4772bae-a527-4655-fb9d-748aa183b37d"
      },
      "source": [
        "# Finde the square root\n",
        "tf.sqrt(tf.cast(H, dtype=tf.float32)) # will require to be a float type"
      ],
      "execution_count": 92,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=float32, numpy=\n",
              "array([1.       , 1.4142135, 1.7320508, 2.       , 2.236068 , 2.4494898,\n",
              "       2.6457512, 2.828427 , 3.       ], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 92
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8jdQgKgtU_so",
        "outputId": "58a08dbe-0aeb-4649-abe0-7c5aa17a320f"
      },
      "source": [
        "# Find log\n",
        "tf.math.log(tf.cast(H, dtype=tf.float32))"
      ],
      "execution_count": 93,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=float32, numpy=\n",
              "array([0.       , 0.6931472, 1.0986123, 1.3862944, 1.609438 , 1.7917595,\n",
              "       1.9459102, 2.0794415, 2.1972246], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 93
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mmSiceTVVQUm"
      },
      "source": [
        "### Tensor and NumPy\n",
        "\n",
        "Tensorflow interacts beautifully with NumPy arrays.\n",
        "\n",
        "Tensors can also be converted to NumPy arrays using:\n",
        "* `np.array()` - pass a tensor to convert to an ndarray (NumPy's main datatype).\n",
        "* `tensor.numpy()` - call on a tensor to convert to an ndarray.\n",
        "\n",
        "Doing this is helpful as it makes tensors iterable as well as allows us to use any of NumPy's methods on them.\n",
        "\n",
        "🔑  **Note:** One of the main differences between a TensorFlow tensor and a NumPy array is that a TensorFlow tensor can be run on a GPU or TPU (for faster numerical processing)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7XBCigJZXDor",
        "outputId": "94de3ffe-49d9-4dcb-9ab5-0ecd7d10b3c3"
      },
      "source": [
        "# Creates a tensor directly from a NumPy Array\n",
        "J = tf.constant(np.array([3., 7., 10.]))\n",
        "J "
      ],
      "execution_count": 94,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3,), dtype=float64, numpy=array([ 3.,  7., 10.])>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 94
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "C9P8g-vJXk-M",
        "outputId": "d00c50c9-b542-4a85-f7a0-050296fc48ac"
      },
      "source": [
        "# Convert our tensor back to a NumPy array\n",
        "np.array(J), type(np.array(J))"
      ],
      "execution_count": 95,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([ 3.,  7., 10.]), numpy.ndarray)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 95
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SdJvYvSnXsmH",
        "outputId": "090d05d8-0d7f-4b6d-e2ea-b95bb49ac748"
      },
      "source": [
        "# Convert Tensor J to a NumPy array\n",
        "J.numpy(), type(J.numpy())"
      ],
      "execution_count": 96,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([ 3.,  7., 10.]), numpy.ndarray)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 96
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BqrlvcTuYARC",
        "outputId": "9a72fd5d-6a54-4bfe-db1a-e715e0066620"
      },
      "source": [
        "J = tf.constant([3.])\n",
        "J.numpy()[0]"
      ],
      "execution_count": 97,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3.0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 97
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "27aob7r0YLsp",
        "outputId": "b448b5e0-0535-424f-d115-96cb4764f2b2"
      },
      "source": [
        "# The default  types of each are slightly different\n",
        "numpy_J = tf.constant(np.array([3., 7., 10.]))\n",
        "tensor_J = tf.constant([3., 7., 10.]) \n",
        "# Check the datatypes of each \n",
        "numpy_J.dtype, tensor_J.dtype"
      ],
      "execution_count": 98,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tf.float64, tf.float32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 98
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O9kFxF7uxHfY"
      },
      "source": [
        "### Finding the usage of GPU"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bn_JHwa6xLP0",
        "outputId": "f9831110-726b-4369-eb00-0dcb453fef4e"
      },
      "source": [
        "import tensorflow as tf \n",
        "tf.config.list_physical_devices()"
      ],
      "execution_count": 99,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU'),\n",
              " PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 99
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yu4Fk9f7xgVp",
        "outputId": "620bf0c3-fbd1-4469-da5a-ac6d8945ad09"
      },
      "source": [
        "!nvidia-smi"
      ],
      "execution_count": 100,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Mon Jul 19 16:17:41 2021       \n",
            "+-----------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 470.42.01    Driver Version: 460.32.03    CUDA Version: 11.2     |\n",
            "|-------------------------------+----------------------+----------------------+\n",
            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                               |                      |               MIG M. |\n",
            "|===============================+======================+======================|\n",
            "|   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   51C    P0    27W /  70W |    224MiB / 15109MiB |      0%      Default |\n",
            "|                               |                      |                  N/A |\n",
            "+-------------------------------+----------------------+----------------------+\n",
            "                                                                               \n",
            "+-----------------------------------------------------------------------------+\n",
            "| Processes:                                                                  |\n",
            "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
            "|        ID   ID                                                   Usage      |\n",
            "|=============================================================================|\n",
            "|  No running processes found                                                 |\n",
            "+-----------------------------------------------------------------------------+\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xjaryxyeyxSx"
      },
      "source": [
        "> 🔑 **Note:** If you have access to a CUDA-enabled GPU, TensorFlow will automatically use it whenever possible."
      ]
    }
  ]
}