The goal is to apply a Convolutional Neural Net (CNN) model on the CIFAR10 image dataset and test the accuracy of the model on the basis of image classification. Compare the Accuracy VS the neural network implemented during TD1.
The goal is to apply a Convolutional Neural Net (CNN) model on the CIFAR10 image dataset and test the accuracy of the model on the basis of image classification. Compare the Accuracy VS the neural network implemented during TD1.
Have a look at the following documentation to be familiar with PyTorch.
Have a look at the following documentation to be familiar with PyTorch.
"Validation loss decreased ({:.6f} --> {:.6f}). Saving model ...".format(
"Validation loss decreased ({:.6f} --> {:.6f}). Saving model ...".format(
valid_loss_min,valid_loss
valid_loss_min,valid_loss
)
)
)
)
torch.save(model.state_dict(),"model_cifar.pt")# our model saved locally
torch.save(model.state_dict(),"model_cifar.pt")# our model saved locally
valid_loss_min=valid_loss
valid_loss_min=valid_loss
```
```
%% Output
%% Output
Epoch: 0 Training Loss: 42.684944 Validation Loss: 38.051762
Epoch: 0 Training Loss: 42.684944 Validation Loss: 38.051762
Validation loss decreased (inf --> 38.051762). Saving model ...
Validation loss decreased (inf --> 38.051762). Saving model ...
Epoch: 1 Training Loss: 34.362200 Validation Loss: 32.100003
Epoch: 1 Training Loss: 34.362200 Validation Loss: 32.100003
Validation loss decreased (38.051762 --> 32.100003). Saving model ...
Validation loss decreased (38.051762 --> 32.100003). Saving model ...
Epoch: 2 Training Loss: 30.699758 Validation Loss: 29.829906
Epoch: 2 Training Loss: 30.699758 Validation Loss: 29.829906
Validation loss decreased (32.100003 --> 29.829906). Saving model ...
Validation loss decreased (32.100003 --> 29.829906). Saving model ...
Epoch: 3 Training Loss: 28.636639 Validation Loss: 27.644334
Epoch: 3 Training Loss: 28.636639 Validation Loss: 27.644334
Validation loss decreased (29.829906 --> 27.644334). Saving model ...
Validation loss decreased (29.829906 --> 27.644334). Saving model ...
Epoch: 4 Training Loss: 27.035070 Validation Loss: 26.414261
Epoch: 4 Training Loss: 27.035070 Validation Loss: 26.414261
Validation loss decreased (27.644334 --> 26.414261). Saving model ...
Validation loss decreased (27.644334 --> 26.414261). Saving model ...
Epoch: 5 Training Loss: 25.561310 Validation Loss: 25.290589
Epoch: 5 Training Loss: 25.561310 Validation Loss: 25.290589
Validation loss decreased (26.414261 --> 25.290589). Saving model ...
Validation loss decreased (26.414261 --> 25.290589). Saving model ...
Epoch: 6 Training Loss: 24.435097 Validation Loss: 24.265817
Epoch: 6 Training Loss: 24.435097 Validation Loss: 24.265817
Validation loss decreased (25.290589 --> 24.265817). Saving model ...
Validation loss decreased (25.290589 --> 24.265817). Saving model ...
Epoch: 7 Training Loss: 23.348866 Validation Loss: 23.822086
Epoch: 7 Training Loss: 23.348866 Validation Loss: 23.822086
Validation loss decreased (24.265817 --> 23.822086). Saving model ...
Validation loss decreased (24.265817 --> 23.822086). Saving model ...
Epoch: 8 Training Loss: 22.444620 Validation Loss: 23.803609
Epoch: 8 Training Loss: 22.444620 Validation Loss: 23.803609
Validation loss decreased (23.822086 --> 23.803609). Saving model ...
Validation loss decreased (23.822086 --> 23.803609). Saving model ...
Epoch: 9 Training Loss: 21.623235 Validation Loss: 22.938253
Epoch: 9 Training Loss: 21.623235 Validation Loss: 22.938253
Validation loss decreased (23.803609 --> 22.938253). Saving model ...
Validation loss decreased (23.803609 --> 22.938253). Saving model ...
Epoch: 10 Training Loss: 20.838078 Validation Loss: 22.255618
Epoch: 10 Training Loss: 20.838078 Validation Loss: 22.255618
Validation loss decreased (22.938253 --> 22.255618). Saving model ...
Validation loss decreased (22.938253 --> 22.255618). Saving model ...
Epoch: 11 Training Loss: 20.030158 Validation Loss: 22.129453
Epoch: 11 Training Loss: 20.030158 Validation Loss: 22.129453
Validation loss decreased (22.255618 --> 22.129453). Saving model ...
Validation loss decreased (22.255618 --> 22.129453). Saving model ...
Epoch: 12 Training Loss: 19.361079 Validation Loss: 21.821713
Epoch: 12 Training Loss: 19.361079 Validation Loss: 21.821713
Validation loss decreased (22.129453 --> 21.821713). Saving model ...
Validation loss decreased (22.129453 --> 21.821713). Saving model ...
Epoch: 13 Training Loss: 18.696286 Validation Loss: 22.176465
Epoch: 13 Training Loss: 18.696286 Validation Loss: 22.176465
Epoch: 14 Training Loss: 18.010778 Validation Loss: 21.106473
Epoch: 14 Training Loss: 18.010778 Validation Loss: 21.106473
Validation loss decreased (21.821713 --> 21.106473). Saving model ...
Validation loss decreased (21.821713 --> 21.106473). Saving model ...
Epoch: 15 Training Loss: 17.438658 Validation Loss: 21.717994
Epoch: 15 Training Loss: 17.438658 Validation Loss: 21.717994
Epoch: 16 Training Loss: 16.805439 Validation Loss: 21.671730
Epoch: 16 Training Loss: 16.805439 Validation Loss: 21.671730
Epoch: 17 Training Loss: 16.255950 Validation Loss: 22.948138
Epoch: 17 Training Loss: 16.255950 Validation Loss: 22.948138
Epoch: 18 Training Loss: 15.736671 Validation Loss: 23.720485
Epoch: 18 Training Loss: 15.736671 Validation Loss: 23.720485
Epoch: 19 Training Loss: 15.186564 Validation Loss: 21.873747
Epoch: 19 Training Loss: 15.186564 Validation Loss: 21.873747
Epoch: 20 Training Loss: 14.735234 Validation Loss: 21.490069
Epoch: 20 Training Loss: 14.735234 Validation Loss: 21.490069
Epoch: 21 Training Loss: 14.149223 Validation Loss: 22.076123
Epoch: 21 Training Loss: 14.149223 Validation Loss: 22.076123
Epoch: 22 Training Loss: 13.684295 Validation Loss: 23.688867
Epoch: 22 Training Loss: 13.684295 Validation Loss: 23.688867
Epoch: 23 Training Loss: 13.208945 Validation Loss: 23.035013
Epoch: 23 Training Loss: 13.208945 Validation Loss: 23.035013
Epoch: 24 Training Loss: 12.753977 Validation Loss: 23.947329
Epoch: 24 Training Loss: 12.753977 Validation Loss: 23.947329
Epoch: 25 Training Loss: 12.327804 Validation Loss: 23.363292
Epoch: 25 Training Loss: 12.327804 Validation Loss: 23.363292
Epoch: 26 Training Loss: 11.929456 Validation Loss: 24.268032
Epoch: 26 Training Loss: 11.929456 Validation Loss: 24.268032
Epoch: 27 Training Loss: 11.526298 Validation Loss: 24.391640
Epoch: 27 Training Loss: 11.526298 Validation Loss: 24.391640
Epoch: 28 Training Loss: 11.076694 Validation Loss: 24.709161
Epoch: 28 Training Loss: 11.076694 Validation Loss: 24.709161
Epoch: 29 Training Loss: 10.715857 Validation Loss: 26.162475
Epoch: 29 Training Loss: 10.715857 Validation Loss: 26.162475
%% Cell type:markdown id:13e1df74 tags:
%% Cell type:markdown id:13e1df74 tags:
Does overfit occur? If so, do an early stopping.
Does overfit occur? If so, do an early stopping.
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
<p><spanstyle = "color : orange;">Answer :</span> Yes, Overfitting has been happening since the 15th epoch, the plot below show the increase of the loss.</p>
<p><spanstyle = "color : orange;">Answer :</span> Yes, Overfitting has been happening since the 15th epoch, the plot below show the increase of the loss.</p>
<p>we're talking about validation, because the train loss can't demonstrate if the model can generalize.</p>
<p>we're talking about validation, because the train loss can't demonstrate if the model can generalize.</p>
For each class, compare the classification test accuracy of the initial model and the quantized model. Also give the overall test accuracy for both models.
For each class, compare the classification test accuracy of the initial model and the quantized model. Also give the overall test accuracy for both models.
%% Cell type:markdown id:a0a34b90 tags:
%% Cell type:markdown id:a0a34b90 tags:
Try training aware quantization to mitigate the impact on the accuracy (doc available here https://pytorch.org/docs/stable/quantization.html#torch.quantization.quantize_dynamic)
Try training aware quantization to mitigate the impact on the accuracy (doc available here https://pytorch.org/docs/stable/quantization.html#torch.quantization.quantize_dynamic)
%% Cell type:markdown id:201470f9 tags:
%% Cell type:markdown id:201470f9 tags:
## Exercise 3: working with pre-trained models.
## Exercise 3: working with pre-trained models.
PyTorch offers several pre-trained models https://pytorch.org/vision/0.8/models.html
PyTorch offers several pre-trained models https://pytorch.org/vision/0.8/models.html
We will use ResNet50 trained on ImageNet dataset (https://www.image-net.org/index.php). Use the following code with the files `imagenet-simple-labels.json` that contains the imagenet labels and the image dog.png that we will use as test.
We will use ResNet50 trained on ImageNet dataset (https://www.image-net.org/index.php). Use the following code with the files `imagenet-simple-labels.json` that contains the imagenet labels and the image dog.png that we will use as test.
%% Cell type:code id:b4d13080 tags:
%% Cell type:code id:b4d13080 tags:
``` python
``` python
importjson
importjson
fromPILimportImage
fromPILimportImage
# Choose an image to pass through the model
# Choose an image to pass through the model
test_image="dog.png"
test_image="dog.png"
# Configure matplotlib for pretty inline plots
# Configure matplotlib for pretty inline plots
#%matplotlib inline
#%matplotlib inline
#%config InlineBackend.figure_format = 'retina'
#%config InlineBackend.figure_format = 'retina'
# Prepare the labels
# Prepare the labels
withopen("imagenet-simple-labels.json")asf:
withopen("imagenet-simple-labels.json")asf:
labels=json.load(f)
labels=json.load(f)
# First prepare the transformations: resize the image to what the model was trained on and convert it to a tensor
# First prepare the transformations: resize the image to what the model was trained on and convert it to a tensor
# Download the model if it's not there already. It will take a bit on the first run, after that it's fast
# Download the model if it's not there already. It will take a bit on the first run, after that it's fast
model=models.resnet50(pretrained=True)
model=models.resnet50(pretrained=True)
# Send the model to the GPU
# Send the model to the GPU
# model.cuda()
# model.cuda()
# Set layers such as dropout and batchnorm in evaluation mode
# Set layers such as dropout and batchnorm in evaluation mode
model.eval()
model.eval()
# Get the 1000-dimensional model output
# Get the 1000-dimensional model output
out=model(image)
out=model(image)
# Find the predicted class
# Find the predicted class
print("Predicted class is: {}".format(labels[out.argmax()]))
print("Predicted class is: {}".format(labels[out.argmax()]))
```
```
%% Cell type:markdown id:184cfceb tags:
%% Cell type:markdown id:184cfceb tags:
Experiments:
Experiments:
Study the code and the results obtained. Possibly add other images downloaded from the internet.
Study the code and the results obtained. Possibly add other images downloaded from the internet.
What is the size of the model? Quantize it and then check if the model is still able to correctly classify the other images.
What is the size of the model? Quantize it and then check if the model is still able to correctly classify the other images.
Experiment with other pre-trained CNN models.
Experiment with other pre-trained CNN models.
%% Cell type:markdown id:5d57da4b tags:
%% Cell type:markdown id:5d57da4b tags:
## Exercise 4: Transfer Learning
## Exercise 4: Transfer Learning
For this work, we will use a pre-trained model (ResNet18) as a descriptor extractor and will refine the classification by training only the last fully connected layer of the network. Thus, the output layer of the pre-trained network will be replaced by a layer adapted to the new classes to be recognized which will be in our case ants and bees.
For this work, we will use a pre-trained model (ResNet18) as a descriptor extractor and will refine the classification by training only the last fully connected layer of the network. Thus, the output layer of the pre-trained network will be replaced by a layer adapted to the new classes to be recognized which will be in our case ants and bees.
Download and unzip in your working directory the dataset available at the address :
Download and unzip in your working directory the dataset available at the address :
plt.pause(0.001)# pause a bit so that plots are updated
plt.pause(0.001)# pause a bit so that plots are updated
plt.show()
plt.show()
# Get a batch of training data
# Get a batch of training data
inputs,classes=next(iter(dataloaders["train"]))
inputs,classes=next(iter(dataloaders["train"]))
# Make a grid from batch
# Make a grid from batch
out=torchvision.utils.make_grid(inputs)
out=torchvision.utils.make_grid(inputs)
imshow(out,title=[class_names[x]forxinclasses])
imshow(out,title=[class_names[x]forxinclasses])
```
```
%% Cell type:markdown id:bbd48800 tags:
%% Cell type:markdown id:bbd48800 tags:
Now, execute the following code which uses a pre-trained model ResNet18 having replaced the output layer for the ants/bees classification and performs the model training by only changing the weights of this output layer.
Now, execute the following code which uses a pre-trained model ResNet18 having replaced the output layer for the ants/bees classification and performs the model training by only changing the weights of this output layer.
%% Cell type:code id:572d824c tags:
%% Cell type:code id:572d824c tags:
``` python
``` python
importcopy
importcopy
importos
importos
importtime
importtime
importmatplotlib.pyplotasplt
importmatplotlib.pyplotasplt
importnumpyasnp
importnumpyasnp
importtorch
importtorch
importtorch.nnasnn
importtorch.nnasnn
importtorch.optimasoptim
importtorch.optimasoptim
importtorchvision
importtorchvision
fromtorch.optimimportlr_scheduler
fromtorch.optimimportlr_scheduler
fromtorchvisionimportdatasets,transforms
fromtorchvisionimportdatasets,transforms
# Data augmentation and normalization for training
# Data augmentation and normalization for training
# Just normalization for validation
# Just normalization for validation
data_transforms={
data_transforms={
"train":transforms.Compose(
"train":transforms.Compose(
[
[
transforms.RandomResizedCrop(
transforms.RandomResizedCrop(
224
224
),# ImageNet models were trained on 224x224 images
),# ImageNet models were trained on 224x224 images
transforms.RandomHorizontalFlip(),# flip horizontally 50% of the time - increases train set variability
transforms.RandomHorizontalFlip(),# flip horizontally 50% of the time - increases train set variability
transforms.ToTensor(),# convert it to a PyTorch tensor
transforms.ToTensor(),# convert it to a PyTorch tensor
Modify the code and add an "eval_model" function to allow
Modify the code and add an "eval_model" function to allow
the evaluation of the model on a test set (different from the learning and validation sets used during the learning phase). Study the results obtained.
the evaluation of the model on a test set (different from the learning and validation sets used during the learning phase). Study the results obtained.
Now modify the code to replace the current classification layer with a set of two layers using a "relu" activation function for the middle layer, and the "dropout" mechanism for both layers. Renew the experiments and study the results obtained.
Now modify the code to replace the current classification layer with a set of two layers using a "relu" activation function for the middle layer, and the "dropout" mechanism for both layers. Renew the experiments and study the results obtained.
Apply ther quantization (post and quantization aware) and evaluate impact on model size and accuracy.
Apply ther quantization (post and quantization aware) and evaluate impact on model size and accuracy.
%% Cell type:markdown id:04a263f0 tags:
%% Cell type:markdown id:04a263f0 tags:
## Optional
## Optional
Try this at home!!
Try this at home!!
Pytorch offers a framework to export a given CNN to your selfphone (either android or iOS). Have a look at the tutorial https://pytorch.org/mobile/home/
Pytorch offers a framework to export a given CNN to your selfphone (either android or iOS). Have a look at the tutorial https://pytorch.org/mobile/home/
The Exercise consists in deploying the CNN of Exercise 4 in your phone and then test it on live.
The Exercise consists in deploying the CNN of Exercise 4 in your phone and then test it on live.
