README.md

description of the project

The objective of this project is to develop a classification model that will be able to recognize the class of given images. To do so, we should:

1. Load a dataset which contains the images and their classes

2. Split the dataset into a training set and test set

3. Train our model with training data

4. Test the model with the test set

5. Evaluate the model

description of the dataset

The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images. The dataset is divided into five training batches and one test batch, each with 10000 color images of size 32x32 divided into 10 classes(plane, car, bird, cat, ...).

Here are the classes in the dataset, as well as 10 random images from each:

the dataset can be obtained at this adress: https://www.cs.toronto.edu/~kriz/cifar.html

In our project we developed the model using the Knn algorithm and the MLP neural network

K-nearest neighbors

This algorithm consists of calculating a Euclidean distance between the training data and the test data and choose the majority class out of K labels that have the smallest distance with the train data

Usage

To understand the Knn algorithm with a simple example visit the link: https://www.tutorialspoint.com/machine_learning_with_python/machine_learning_with_python_knn_algorithm_finding_nearest_neighbors.htm#:~:text=With%20the%20help%20of%20KNN,Image%20Recognition%20and%20Video%20Recognition.

The precision of our Knn Model can change according to the value of K

The following figure shows the accuracy of our model based on K

Multi Layer Perceptron Neural Network

The objective here is to develop a classifier based on a multilayer perceptron (MLP) neural network.

We will focus on the backpropagation of the gradient.

We first initialize the network weights and biases and precise our data and desired output

Usage of initializing network weights and biases

import numpy as np

N = 30  # number of input data
d_in = 3  # input dimension
d_h = 3  # number of neurons in the hidden layer
d_out = 2  # output dimension (number of neurons of the output layer)

# Random initialization of the network weights and biaises
w1 = 2 * np.random.rand(d_in, d_h) - 1  # first layer weights
b1 = np.zeros((1, d_h))  # first layer biaises
w2 = 2 * np.random.rand(d_h, d_out) - 1  # second layer weights
b2 = np.zeros((1, d_out))  # second layer biaises

data = np.random.rand(N, d_in)  # create a random data
targets = np.random.rand(N, d_out)  # create a random targets

To develop and train our MLP neural network model we repeat this 4 steps

1. Forward propagation

2. Compute the loss that separates the output from the desired target

3. backpropagation

4. Gradient descent

Forward propagation

We circulate the data from the input to the output

Usage

# Forward propagation
a0 = data # the data are the input of the first layer
z1 = np.matmul(a0, w1) + b1  # input of the hidden layer
a1 = 1 / (1 + np.exp(-z1))  # output of the hidden layer (sigmoid activation function)
z2 = np.matmul(a1, w2) + b2  # input of the output layer
a2 = 1 / (1 + np.exp(-z2))  # output of the output layer (sigmoid activation function)
predictions = a2  # the predicted values are the outputs of the output layer

Compute the loss

We compute the loss that separates the output from the desired target

# Compute loss (MSE)
loss = np.mean(np.square(predictions - targets))
print(loss)

Backpropagation

we compute the variation of this loss function in each layer starting from the last layer to the first layer

Gradient descent

We correct the model parameters as the weights and biases

Usage

# updated weights of the network

w1 = w1 - learning_rate * var_loss_w1
w2 = w2 - learning_rate * var_loss_w2

# updated biaises of the network

b1 = b1 - learning_rate * var_loss_b1
b2 = b2 - learning_rate * var_loss_b2

I trained the model 100 times, the following picture shows the variation of the accuracy along the training

The variation of the loss: