Skip to content

Latest commit

 

History

History
223 lines (155 loc) · 7.46 KB

感知器算法.md

File metadata and controls

223 lines (155 loc) · 7.46 KB

感知器算法

Agenda

Note

簡介

本章重點是如何解決分類問題，要學的第一個算法是perception algorithm。

分類問題

67

  • 要如何知道student 3是否會錄取?

68

  • 根據之前的錄取資料，我們可以推測出student 3應該是會被錄取。

如何根據之前的錄取資料，畫出正確的界線呢?

線性界線

69

  • 將分類問題轉化成數學式，邊界是一個線性函數，將資料切分為二。

70

  • 換成更正式的數學寫法。

更高維度

71

  • 延伸到3維空間，但數學算式仍是相同。

72

  • 延伸到高維空間，但數學算式仍是相同。

感知器

73

  • 將前面的錄取問題，轉化成感知器(這也是神經網路的基礎)。

74

75

  • 兩種不同的表示法，把bias是否有抽離出去，但都是一樣的結果。

76

77

  • 透過step function，將linear function結果轉成分類(1,0)。

作為邏輯運算符的感知器

78

79

  • AND, OR感知器的設計。

AND 感知器的权重和偏差是什么？

import pandas as pd

# TODO: Set weight1, weight2, and bias
weight1 = 0.5
weight2 = 0.5
bias = -1.0


# DON'T CHANGE ANYTHING BELOW
# Inputs and outputs
test_inputs = [(0, 0), (0, 1), (1, 0), (1, 1)]
correct_outputs = [False, False, False, True]
outputs = []

# Generate and check output
for test_input, correct_output in zip(test_inputs, correct_outputs):
    linear_combination = weight1 * test_input[0] + weight2 * test_input[1] + bias
    output = int(linear_combination >= 0)
    is_correct_string = 'Yes' if output == correct_output else 'No'
    outputs.append([test_input[0], test_input[1], linear_combination, output, is_correct_string])

# Print output
num_wrong = len([output[4] for output in outputs if output[4] == 'No'])
output_frame = pd.DataFrame(outputs, columns=['Input 1', '  Input 2', '  Linear Combination', '  Activation Output', '  Is Correct'])
if not num_wrong:
    print('Nice!  You got it all correct.\n')
else:
    print('You got {} wrong.  Keep trying!\n'.format(num_wrong))
print(output_frame.to_string(index=False))

NOT 感知器的权重和偏差是什么？

  • 和我们刚刚研究的其他感知器不一样，NOT 运算仅关心一个输入。如果输入是 1，则运算返回 0，如果输入是 0，则返回 1。感知器的其他输入被忽略了。
import pandas as pd

# TODO: Set weight1, weight2, and bias
weight1 = 0.0
weight2 = -1.0
bias = 0.5


# DON'T CHANGE ANYTHING BELOW
# Inputs and outputs
test_inputs = [(0, 0), (0, 1), (1, 0), (1, 1)]
correct_outputs = [True, False, True, False]
outputs = []

# Generate and check output
for test_input, correct_output in zip(test_inputs, correct_outputs):
    linear_combination = weight1 * test_input[0] + weight2 * test_input[1] + bias
    output = int(linear_combination >= 0)
    is_correct_string = 'Yes' if output == correct_output else 'No'
    outputs.append([test_input[0], test_input[1], linear_combination, output, is_correct_string])

# Print output
num_wrong = len([output[4] for output in outputs if output[4] == 'No'])
output_frame = pd.DataFrame(outputs, columns=['Input 1', '  Input 2', '  Linear Combination', '  Activation Output', '  Is Correct'])
if not num_wrong:
    print('Nice!  You got it all correct.\n')
else:
    print('You got {} wrong.  Keep trying!\n'.format(num_wrong))
print(output_frame.to_string(index=False))

构建一个 XOR 多层感知器

81

80

感知器技巧

82

  • 如何透過感知器，找到這條直線，盡可能將紅點跟藍點區分出來??

83

84

  • 利用愈靠近的點座標來修改原本直線方程式(搭配學習率)，讓直線慢慢地靠近點。

感知器算法

85

86

import numpy as np
# Setting the random seed, feel free to change it and see different solutions.
np.random.seed(42)

def stepFunction(t):
    if t >= 0:
        return 1
    return 0

def prediction(X, W, b):
    return stepFunction((np.matmul(X,W)+b)[0])

# TODO: Fill in the code below to implement the perceptron trick.
# The function should receive as inputs the data X, the labels y,
# the weights W (as an array), and the bias b,
# update the weights and bias W, b, according to the perceptron algorithm,
# and return W and b.
def perceptronStep(X, y, W, b, learn_rate = 0.01):
    # Fill in code
    for i in range(len(X)):
        y_hat = prediction(X[i],W,b)
        if y[i]-y_hat == 1: # y[i] = 1, y_hat = 0
            W[0] += X[i][0]*learn_rate
            W[1] += X[i][1]*learn_rate
            b += learn_rate
        elif y[i]-y_hat == -1: # y[i] = 0, y_hat = 1
            W[0] -= X[i][0]*learn_rate
            W[1] -= X[i][1]*learn_rate
            b -= learn_rate
    return W, b
    
# This function runs the perceptron algorithm repeatedly on the dataset,
# and returns a few of the boundary lines obtained in the iterations,
# for plotting purposes.
# Feel free to play with the learning rate and the num_epochs,
# and see your results plotted below.
def trainPerceptronAlgorithm(X, y, learn_rate = 0.01, num_epochs = 25):
    x_min, x_max = min(X.T[0]), max(X.T[0])
    y_min, y_max = min(X.T[1]), max(X.T[1])
    W = np.array(np.random.rand(2,1))
    b = np.random.rand(1)[0] + x_max
    # These are the solution lines that get plotted below.
    boundary_lines = []
    for i in range(num_epochs):
        # In each epoch, we apply the perceptron step.
        W, b = perceptronStep(X, y, W, b, learn_rate)
        boundary_lines.append((-W[0]/W[1], -b/W[1]))
    return boundary_lines