beginner/examples_tensor/polynomial_numpy

Run in Google Colab

Colab

Download Notebook

Notebook

View on GitHub

GitHub

Warm-up: numpy

Created On: Dec 03, 2020 | Last Updated: Dec 03, 2020 | Last Verified: Nov 05, 2024

A third order polynomial, trained to predict \(y=\sin(x)\) from \(-\pi\) to \(pi\) by minimizing squared Euclidean distance.

This implementation uses numpy to manually compute the forward pass, loss, and backward pass.

A numpy array is a generic n-dimensional array; it does not know anything about deep learning or gradients or computational graphs, and is just a way to perform generic numeric computations.

99 3977.1822667730075
199 2632.9665842995732
299 1744.0846284304607
399 1156.2971417835356
499 767.6125744884587
599 510.58777809337755
699 340.62509859074476
799 228.23372187140396
899 153.912412770934
999 104.76567294065885
1099 72.26613763768472
1199 50.774934080497985
1299 36.56324970119874
1399 27.165330950060202
1499 20.95064443773275
1599 16.840961913821346
1699 14.123278066001996
1799 12.326099136386503
1899 11.1376364540267
1999 10.3517102732154
Result: y = -0.001502297203477509 + 0.8186860605374503 x + 0.00025917126593323424 x^2 + -0.0879174276087672 x^3

import numpy as np
import math

# Create random input and output data
x = np.linspace(-math.pi, math.pi, 2000)
y = np.sin(x)

# Randomly initialize weights
a = np.random.randn()
b = np.random.randn()
c = np.random.randn()
d = np.random.randn()

learning_rate = 1e-6
for t in range(2000):
    # Forward pass: compute predicted y
    # y = a + b x + c x^2 + d x^3
    y_pred = a + b * x + c * x ** 2 + d * x ** 3

    # Compute and print loss
    loss = np.square(y_pred - y).sum()
    if t % 100 == 99:
        print(t, loss)

    # Backprop to compute gradients of a, b, c, d with respect to loss
    grad_y_pred = 2.0 * (y_pred - y)
    grad_a = grad_y_pred.sum()
    grad_b = (grad_y_pred * x).sum()
    grad_c = (grad_y_pred * x ** 2).sum()
    grad_d = (grad_y_pred * x ** 3).sum()

    # Update weights
    a -= learning_rate * grad_a
    b -= learning_rate * grad_b
    c -= learning_rate * grad_c
    d -= learning_rate * grad_d

print(f'Result: y = {a} + {b} x + {c} x^2 + {d} x^3')