| Title: | Simple Multilayer Perceptron Neural Network |
|---|---|
| Description: | Create and train a multilayer perceptron, a type of feedforward, fully connected neural network. Features 2 ReLU hidden layers. Learn more about about the activation functions and backpropagation used by this network in Goodfellow et al. (2016, ISBN: 9780262035613) "Deep Learning". |
| Authors: | Cullen Pu [aut, cre] |
| Maintainer: | Cullen Pu <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 1.0.0 |
| Built: | 2024-11-10 04:45:11 UTC |
| Source: | https://github.com/cullenpu/simplemlp |
Runs a backwards pass through the network.
backprop(model, error, forward_pass)backprop(model, error, forward_pass)
model |
list of all the weights and biases |
error |
gradients to the output of the network |
forward_pass |
intermediate values from the forward pass |
list of derivatives after the backwards pass
Evaluates the performance of a model on a given dataset.
evaluate(inputs, target, model)evaluate(inputs, target, model)
inputs |
set of inputs to the model |
target |
set of targets in one-hot encoded form |
model |
list of weights and biases |
accuracy of the model
## Not run: evaluate(train_data, train_target, mlp_model) ## End(Not run)## Not run: evaluate(train_data, train_target, mlp_model) ## End(Not run)
Runs a forward pass through the network.
forwardprop(model, x)forwardprop(model, x)
model |
list of all the weights and biases |
x |
input to the network |
list of all intermediate values
Initialize 3 layer fully connected neural network, also known as multilayer perceptron, setting biases to 0 and using the Xavier initialization method for weights.
init_nn(num_inputs, num_hidden_1, num_hidden_2, num_outputs)init_nn(num_inputs, num_hidden_1, num_hidden_2, num_outputs)
num_inputs |
dimension of inputs |
|
dimension of first hidden layer |
|
|
dimension of second hidden layer |
|
num_outputs |
dimension of output |
list containing weight and bias matrices in each layer of the network
mlp_model <- init_nn(784, 100, 50, 10)mlp_model <- init_nn(784, 100, 50, 10)
Loads MNIST training, validation, and test data and generates one hot encodings for the targets. The test set proportion is not specified and is instead the remainder from the test and validation proportions.
load_mnist(train_prop = 0.8, validate_prop = 0.1)load_mnist(train_prop = 0.8, validate_prop = 0.1)
train_prop |
proportion of the data used for the training set |
validate_prop |
proportion of the data used for the validation set |
list of training and validation data and targets
mnist <- load_mnist(0.8, 0.1) train_data <- mnist[1] train_target <- mnist[2] validate_data <- mnist[3] validate_target <- mnist[4] test_data <- mnist[5] test_target <- mnist[6]mnist <- load_mnist(0.8, 0.1) train_data <- mnist[1] train_target <- mnist[2] validate_data <- mnist[3] validate_target <- mnist[4] test_data <- mnist[5] test_target <- mnist[6]
Creates a one hot encoding matrix with the specified number of categories for the targets. Target must be the first column of the data_raw input.
one_hot_encoding(data_raw, ncat = 10)one_hot_encoding(data_raw, ncat = 10)
data_raw |
data input to create encoding; target must be first column |
ncat |
number of categories to use for the encoding |
targets in a one hot encoding matrix
Plot the training and validation accuracy.
plot_accuracy(accuracy_train, accuracy_validate)plot_accuracy(accuracy_train, accuracy_validate)
accuracy_train |
list of training accuracy |
accuracy_validate |
list of validation accuracy |
Train the network with specified hyperparameters and return the trained model.
train_nn( train_data, train_target, validate_data, validate_target, model, alpha, epochs, batch_size = nrow(train_data), plot_acc = TRUE )train_nn( train_data, train_target, validate_data, validate_target, model, alpha, epochs, batch_size = nrow(train_data), plot_acc = TRUE )
train_data |
set of training data |
train_target |
set of training data targets in one-hot encoded form |
validate_data |
set of validation data targets in one-hot encoded form |
validate_target |
set of targets in |
model |
list of weights and biases |
alpha |
learning rate |
epochs |
number of epochs |
batch_size |
mini-batch size |
plot_acc |
whether or not to plot training and validation accuracy |
list of weights and biases after training
## Not run: mlp_model <- init_nn(784, 100, 50, 10) mnist <- load_mnist() train_data <- mnist[1] train_target <- mnist[2] validate_data <- mnist[3] validate_target <- mnist[4] mlp_model <- train_nn(train_data, train_target, validate_data, validate_target, mlp_model, 0.01, 1, 64) ## End(Not run)## Not run: mlp_model <- init_nn(784, 100, 50, 10) mnist <- load_mnist() train_data <- mnist[1] train_target <- mnist[2] validate_data <- mnist[3] validate_target <- mnist[4] mlp_model <- train_nn(train_data, train_target, validate_data, validate_target, mlp_model, 0.01, 1, 64) ## End(Not run)
Updates the model using derivatives from a backward pass.
update(model, back_pass, alpha)update(model, back_pass, alpha)
model |
list of all the weights and biases |
back_pass |
derivatives from a backwards pass through the network |
alpha |
learning rate |
updated list of the weights and biases