tensorflow_tutorials_02_linear_regression

"""Simple tutorial for using TensorFlow to compute a linear regression.
Parag K. Mital, Jan. 2016"""
# %% imports
import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt

• 創建toy數據，xs，ys均爲一維向量。
• 用plt.ion()和plt.draw()實現不中斷程序的可視化。
• plt.draw()：Redraw the current figure.
• fig.show()和plt.draw()都去掉也可以，why added？

# %% Let's create some toy data
plt.ion()
n_observations = 100
fig, ax = plt.subplots(1, 1)
xs = np.linspace(-3, 3, n_observations)
ys = np.sin(xs) + np.random.uniform(-0.5, 0.5, n_observations)
ax.scatter(xs, ys)
fig.show()
plt.draw()

• 原理：$\underset{(w,b)}{\operatorname{argmin}}||(X * w + b) - y || ^ 2$
• tf.placeholder()用於網絡的輸入和輸出。佔位符是我們準備計算圖形時需要填寫的變量。
• tf.Variable()構造函數需要給變量提供初始值，它可以是任何類型和形狀的“Tensor”。初始值定義了變量的類型和形狀，定義後變量的類型和形狀是不變的，通過賦值可以改變變量的值。
• 損失函數 cost 將衡量觀測值與預測值之間的差距，並取平均值。
• tf.reduce_sum() 壓縮求和（全部求和、按維度求和）
• tf.random_normal()或tf.random.normal()函數用於從服從指定正太分佈的數值中取出指定個數的值。

  tf.random_normal(shape, mean=0.0, stddev=1.0, dtype=tf.float32, seed=None, name=None)
  

• tf.truncated_normal()截斷的正態分佈函數。生成的值遵循一個正態分佈，但不會大於平均值2個標準差。

# %% tf.placeholders for the input and output of the network. Placeholders are
# variables which we need to fill in when we are ready to compute the graph.
X = tf.placeholder(tf.float32)
Y = tf.placeholder(tf.float32)

# %% We will try to optimize min_(W,b) ||(X*w + b) - y||^2
# The `Variable()` constructor requires an initial value for the variable,
# which can be a `Tensor` of any type and shape. The initial value defines the
# type and shape of the variable. After construction, the type and shape of
# the variable are fixed. The value can be changed using one of the assign
# methods.
W = tf.Variable(tf.random_normal([1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')
Y_pred = tf.add(tf.multiply(X, W), b)

# %% Loss function will measure the distance between our observations
# and predictions and average over them.
cost = tf.reduce_sum(tf.pow(Y_pred - Y, 2)) / (n_observations - 1)

• 如果我們想添加正則化，我們可以在成本函數中添加其他術語，例如：嶺迴歸可以看作是線性迴歸的一種增強版，實現的時候加入一個二範數正則化項，但是應當注意，這裏只對於權重進行懲罰，偏置項是不加入正則化的。
• tf.global_norm() 計算多個張量的全局範數.
• tf.mul tf.sub tf.neg 已經廢棄，分別可用tf.multiply tf.subtract tf.negative替代.

# %% if we wanted to add regularization, we could add other terms to the cost,
# e.g. ridge regression has a parameter controlling the amount of shrinkage
# over the norm of activations. the larger the shrinkage, the more robust
# to collinearity.
# cost = tf.add(cost, tf.mul(1e-6, tf.global_norm([W])))

• 使用梯度下降來優化w，b，通過負梯度執行單個步驟。
• with tf.Session() as sess：創建一個session來使用圖graph。
• sess.run(tf.global_variables_initializer())告訴tensorflow我們想要初始化圖中的所有變量，以便我們可以使用它們。
• alpha表示透明度，取值0到1之間，1表示不透明。

# %% Use gradient descent to optimize W,b
# Performs a single step in the negative gradient
learning_rate = 0.01
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)

# %% We create a session to use the graph
n_epochs = 1000
with tf.Session() as sess:
    # Here we tell tensorflow that we want to initialize all
    # the variables in the graph so we can use them
    sess.run(tf.global_variables_initializer())

    # Fit all training data
    prev_training_cost = 0.0
    for epoch_i in range(n_epochs):
        for (x, y) in zip(xs, ys):
            sess.run(optimizer, feed_dict={X: x, Y: y})

        training_cost = sess.run(cost, feed_dict={X: xs, Y: ys})
        print(training_cost)

        if epoch_i % 20 == 0:
            ax.plot(xs, 
            		Y_pred.eval(feed_dict={X: xs}, session=sess), 
            		'k', 
            		alpha=epoch_i / n_epochs)
            fig.show()
            plt.draw()

        # Allow the training to quit if we've reached a minimum
        if np.abs(prev_training_cost - training_cost) < 0.000001:
            break
        prev_training_cost = training_cost
fig.show()
plt.waitforbuttonpress()

# 控制檯
...
0.25716764
0.2571666
0.25716555
0.25716457

plt.waitforbuttonpress()會使圖片無法截圖，一按快捷鍵或鼠標就消失了。採用

plt.ioff()
plt.show()

代替最後兩行代碼即可。