The goal is to apply a Convolutional Neural Net (CNN) model on the CIFAR10 image dataset and test the accuracy of the model on the basis of image classification. Compare the Accuracy VS the neural network implemented during TD1.
The goal is to apply a Convolutional Neural Net (CNN) model on the CIFAR10 image dataset and test the accuracy of the model on the basis of image classification. Compare the Accuracy VS the neural network implemented during TD1.
Have a look at the following documentation to be familiar with PyTorch.
Have a look at the following documentation to be familiar with PyTorch.
print("The size of the original model has been divided by %.2f compared to the Quantized model" % (size_model / size_quantized))
```
```
%% Cell type:markdown id:7b108e17 tags:
%% Cell type:markdown id:7b108e17 tags:
For each class, compare the classification test accuracy of the initial model and the quantized model. Also give the overall test accuracy for both models.
For each class, compare the classification test accuracy of the initial model and the quantized model. Also give the overall test accuracy for both models.
%% Cell type:markdown id:a0a34b90 tags:
%% Cell type:markdown id:a0a34b90 tags:
Try training aware quantization to mitigate the impact on the accuracy (doc available here https://pytorch.org/docs/stable/quantization.html#torch.quantization.quantize_dynamic)
Try training aware quantization to mitigate the impact on the accuracy (doc available here https://pytorch.org/docs/stable/quantization.html#torch.quantization.quantize_dynamic)
PyTorch offers several pre-trained models https://pytorch.org/vision/0.8/models.html
PyTorch offers several pre-trained models https://pytorch.org/vision/0.8/models.html
We will use ResNet50 trained on ImageNet dataset (https://www.image-net.org/index.php). Use the following code with the files `imagenet-simple-labels.json` that contains the imagenet labels and the image dog.png that we will use as test.
We will use ResNet50 trained on ImageNet dataset (https://www.image-net.org/index.php). Use the following code with the files `imagenet-simple-labels.json` that contains the imagenet labels and the image dog.png that we will use as test.
%% Cell type:code id:b4d13080 tags:
%% Cell type:code id:b4d13080 tags:
```python
```
import json
import json
from PIL import Image
from PIL import Image
# Choose an image to pass through the model
# Choose an image to pass through the model
test_image = "dog.png"
test_image = "dog.png"
# Configure matplotlib for pretty inline plots
# Configure matplotlib for pretty inline plots
#%matplotlib inline
#%matplotlib inline
#%config InlineBackend.figure_format = 'retina'
#%config InlineBackend.figure_format = 'retina'
# Prepare the labels
# Prepare the labels
with open("imagenet-simple-labels.json") as f:
with open("imagenet-simple-labels.json") as f:
labels = json.load(f)
labels = json.load(f)
# First prepare the transformations: resize the image to what the model was trained on and convert it to a tensor
# First prepare the transformations: resize the image to what the model was trained on and convert it to a tensor
print("The size of the original ResNet50 model has been divided by %.2f compared to the Quantized ResNet50 model" % (size_resnet50 /size_resnet50_quantized))
print("The size of the original GoogleNet model has been divided by %.2f compared to the Quantized GoogleNet model" % (size_resnet50 /size_resnet50_quantized))
```
%% Cell type:markdown id:5d57da4b tags:
%% Cell type:markdown id:5d57da4b tags:
## Exercise 4: Transfer Learning
## Exercise 4: Transfer Learning
For this work, we will use a pre-trained model (ResNet18) as a descriptor extractor and will refine the classification by training only the last fully connected layer of the network. Thus, the output layer of the pre-trained network will be replaced by a layer adapted to the new classes to be recognized which will be in our case ants and bees.
For this work, we will use a pre-trained model (ResNet18) as a descriptor extractor and will refine the classification by training only the last fully connected layer of the network. Thus, the output layer of the pre-trained network will be replaced by a layer adapted to the new classes to be recognized which will be in our case ants and bees.
Download and unzip in your working directory the dataset available at the address :
Download and unzip in your working directory the dataset available at the address :
imshow(out, title=[class_names[x] for x in classes])
imshow(out, title=[class_names[x] for x in classes])
```
```
%% Cell type:markdown id:bbd48800 tags:
%% Cell type:markdown id:bbd48800 tags:
Now, execute the following code which uses a pre-trained model ResNet18 having replaced the output layer for the ants/bees classification and performs the model training by only changing the weights of this output layer.
Now, execute the following code which uses a pre-trained model ResNet18 having replaced the output layer for the ants/bees classification and performs the model training by only changing the weights of this output layer.
%% Cell type:code id:572d824c tags:
%% Cell type:code id:572d824c tags:
```python
```
import copy
import copy
import os
import os
import time
import time
import matplotlib.pyplot as plt
import matplotlib.pyplot as plt
import numpy as np
import numpy as np
import torch
import torch
import torch.nn as nn
import torch.nn as nn
import torch.optim as optim
import torch.optim as optim
import torchvision
import torchvision
from torch.optim import lr_scheduler
from torch.optim import lr_scheduler
from torchvision import datasets, transforms
from torchvision import datasets, transforms
# Data augmentation and normalization for training
# Data augmentation and normalization for training
# Just normalization for validation
# Just normalization for validation
data_transforms = {
data_transforms = {
"train": transforms.Compose(
"train": transforms.Compose(
[
[
transforms.RandomResizedCrop(
transforms.RandomResizedCrop(
224
224
), # ImageNet models were trained on 224x224 images
), # ImageNet models were trained on 224x224 images
transforms.RandomHorizontalFlip(), # flip horizontally 50% of the time - increases train set variability
transforms.RandomHorizontalFlip(), # flip horizontally 50% of the time - increases train set variability
transforms.ToTensor(), # convert it to a PyTorch tensor
transforms.ToTensor(), # convert it to a PyTorch tensor
Modify the code and add an "eval_model" function to allow
Modify the code and add an "eval_model" function to allow
the evaluation of the model on a test set (different from the learning and validation sets used during the learning phase). Study the results obtained.
the evaluation of the model on a test set (different from the learning and validation sets used during the learning phase). Study the results obtained.
Now modify the code to replace the current classification layer with a set of two layers using a "relu" activation function for the middle layer, and the "dropout" mechanism for both layers. Renew the experiments and study the results obtained.
Now modify the code to replace the current classification layer with a set of two layers using a "relu" activation function for the middle layer, and the "dropout" mechanism for both layers. Renew the experiments and study the results obtained.
Apply ther quantization (post and quantization aware) and evaluate impact on model size and accuracy.
Apply ther quantization (post and quantization aware) and evaluate impact on model size and accuracy.
"\nThe size of the resNet18 model is %.2fMB, which is %.0f times bigger than the resNet18_quantized model"
% (
size_resNet18 / 1000000,
size_resNet18 / size_resNet18_quantized
)
)
```
%% Cell type:markdown id:04a263f0 tags:
%% Cell type:markdown id:04a263f0 tags:
## Optional
## Optional
Try this at home!!
Try this at home!!
Pytorch offers a framework to export a given CNN to your selfphone (either android or iOS). Have a look at the tutorial https://pytorch.org/mobile/home/
Pytorch offers a framework to export a given CNN to your selfphone (either android or iOS). Have a look at the tutorial https://pytorch.org/mobile/home/
The Exercise consists in deploying the CNN of Exercise 4 in your phone and then test it on live.
The Exercise consists in deploying the CNN of Exercise 4 in your phone and then test it on live.
%% Cell type:markdown id:fe954ce4 tags:
%% Cell type:markdown id:fe954ce4 tags:
## Author
## Author
Alberto BOSIO - Ph. D.
Alberto BOSIO - Ph. D.
%% Cell type:markdown id:7edf7168 tags:
%% Cell type:markdown id:7edf7168 tags:
# TD2: Deep learning
# TD2: Deep learning
%% Cell type:markdown id:fbb8c8df tags:
%% Cell type:markdown id:fbb8c8df tags:
In this TD, you must modify this notebook to answer the questions. To do this,
In this TD, you must modify this notebook to answer the questions. To do this,
1. Fork this repository
1. Fork this repository
2. Clone your forked repository on your local computer
2. Clone your forked repository on your local computer
3. Answer the questions
3. Answer the questions
4. Commit and push regularly
4. Commit and push regularly
The last commit is due on Sunday, December 1, 11:59 PM. Later commits will not be taken into account.
The last commit is due on Sunday, December 1, 11:59 PM. Later commits will not be taken into account.
%% Cell type:markdown id:3d167a29 tags:
%% Cell type:markdown id:3d167a29 tags:
Install and test PyTorch from https://pytorch.org/get-started/locally.
Install and test PyTorch from https://pytorch.org/get-started/locally.
%% Cell type:markdown id:QCeGtWXR9J0y tags:
%% Cell type:code id:330a42f5 tags:
%% Cell type:code id:330a42f5 tags:
```python
```
%pip install torch torchvision
%pip install torch torchvision
%pwd
```
```
%% Output
Requirement already satisfied: torch in /usr/local/lib/python3.10/dist-packages (2.1.0+cu118)
Requirement already satisfied: torchvision in /usr/local/lib/python3.10/dist-packages (0.16.0+cu118)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch) (3.13.1)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch) (4.5.0)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch) (1.12)
Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch) (3.2.1)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch) (3.1.2)
Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch) (2023.6.0)
Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch) (2.1.0)
Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from torchvision) (1.23.5)
Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from torchvision) (2.31.0)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.10/dist-packages (from torchvision) (9.4.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch) (2.1.3)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->torchvision) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->torchvision) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->torchvision) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->torchvision) (2023.7.22)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch) (1.3.0)
The goal is to apply a Convolutional Neural Net (CNN) model on the CIFAR10 image dataset and test the accuracy of the model on the basis of image classification. Compare the Accuracy VS the neural network implemented during TD1.
The goal is to apply a Convolutional Neural Net (CNN) model on the CIFAR10 image dataset and test the accuracy of the model on the basis of image classification. Compare the Accuracy VS the neural network implemented during TD1.
Have a look at the following documentation to be familiar with PyTorch.
Have a look at the following documentation to be familiar with PyTorch.
print("The size of the original model has been divided by %.2f compared to the Quantized model" % (size_model / size_quantized))
```
```
%% Cell type:markdown id:7b108e17 tags:
%% Cell type:markdown id:7b108e17 tags:
For each class, compare the classification test accuracy of the initial model and the quantized model. Also give the overall test accuracy for both models.
For each class, compare the classification test accuracy of the initial model and the quantized model. Also give the overall test accuracy for both models.
%% Cell type:markdown id:a0a34b90 tags:
%% Cell type:markdown id:a0a34b90 tags:
Try training aware quantization to mitigate the impact on the accuracy (doc available here https://pytorch.org/docs/stable/quantization.html#torch.quantization.quantize_dynamic)
Try training aware quantization to mitigate the impact on the accuracy (doc available here https://pytorch.org/docs/stable/quantization.html#torch.quantization.quantize_dynamic)
PyTorch offers several pre-trained models https://pytorch.org/vision/0.8/models.html
PyTorch offers several pre-trained models https://pytorch.org/vision/0.8/models.html
We will use ResNet50 trained on ImageNet dataset (https://www.image-net.org/index.php). Use the following code with the files `imagenet-simple-labels.json` that contains the imagenet labels and the image dog.png that we will use as test.
We will use ResNet50 trained on ImageNet dataset (https://www.image-net.org/index.php). Use the following code with the files `imagenet-simple-labels.json` that contains the imagenet labels and the image dog.png that we will use as test.
%% Cell type:code id:b4d13080 tags:
%% Cell type:code id:b4d13080 tags:
```python
```
import json
import json
from PIL import Image
from PIL import Image
# Choose an image to pass through the model
# Choose an image to pass through the model
test_image = "dog.png"
test_image = "dog.png"
# Configure matplotlib for pretty inline plots
# Configure matplotlib for pretty inline plots
#%matplotlib inline
#%matplotlib inline
#%config InlineBackend.figure_format = 'retina'
#%config InlineBackend.figure_format = 'retina'
# Prepare the labels
# Prepare the labels
with open("imagenet-simple-labels.json") as f:
with open("imagenet-simple-labels.json") as f:
labels = json.load(f)
labels = json.load(f)
# First prepare the transformations: resize the image to what the model was trained on and convert it to a tensor
# First prepare the transformations: resize the image to what the model was trained on and convert it to a tensor
print("The size of the original ResNet50 model has been divided by %.2f compared to the Quantized ResNet50 model" % (size_resnet50 /size_resnet50_quantized))
print("The size of the original GoogleNet model has been divided by %.2f compared to the Quantized GoogleNet model" % (size_resnet50 /size_resnet50_quantized))
```
%% Cell type:markdown id:5d57da4b tags:
%% Cell type:markdown id:5d57da4b tags:
## Exercise 4: Transfer Learning
## Exercise 4: Transfer Learning
For this work, we will use a pre-trained model (ResNet18) as a descriptor extractor and will refine the classification by training only the last fully connected layer of the network. Thus, the output layer of the pre-trained network will be replaced by a layer adapted to the new classes to be recognized which will be in our case ants and bees.
For this work, we will use a pre-trained model (ResNet18) as a descriptor extractor and will refine the classification by training only the last fully connected layer of the network. Thus, the output layer of the pre-trained network will be replaced by a layer adapted to the new classes to be recognized which will be in our case ants and bees.
Download and unzip in your working directory the dataset available at the address :
Download and unzip in your working directory the dataset available at the address :
imshow(out, title=[class_names[x] for x in classes])
imshow(out, title=[class_names[x] for x in classes])
```
```
%% Cell type:markdown id:bbd48800 tags:
%% Cell type:markdown id:bbd48800 tags:
Now, execute the following code which uses a pre-trained model ResNet18 having replaced the output layer for the ants/bees classification and performs the model training by only changing the weights of this output layer.
Now, execute the following code which uses a pre-trained model ResNet18 having replaced the output layer for the ants/bees classification and performs the model training by only changing the weights of this output layer.
%% Cell type:code id:572d824c tags:
%% Cell type:code id:572d824c tags:
```python
```
import copy
import copy
import os
import os
import time
import time
import matplotlib.pyplot as plt
import matplotlib.pyplot as plt
import numpy as np
import numpy as np
import torch
import torch
import torch.nn as nn
import torch.nn as nn
import torch.optim as optim
import torch.optim as optim
import torchvision
import torchvision
from torch.optim import lr_scheduler
from torch.optim import lr_scheduler
from torchvision import datasets, transforms
from torchvision import datasets, transforms
# Data augmentation and normalization for training
# Data augmentation and normalization for training
# Just normalization for validation
# Just normalization for validation
data_transforms = {
data_transforms = {
"train": transforms.Compose(
"train": transforms.Compose(
[
[
transforms.RandomResizedCrop(
transforms.RandomResizedCrop(
224
224
), # ImageNet models were trained on 224x224 images
), # ImageNet models were trained on 224x224 images
transforms.RandomHorizontalFlip(), # flip horizontally 50% of the time - increases train set variability
transforms.RandomHorizontalFlip(), # flip horizontally 50% of the time - increases train set variability
transforms.ToTensor(), # convert it to a PyTorch tensor
transforms.ToTensor(), # convert it to a PyTorch tensor
Modify the code and add an "eval_model" function to allow
Modify the code and add an "eval_model" function to allow
the evaluation of the model on a test set (different from the learning and validation sets used during the learning phase). Study the results obtained.
the evaluation of the model on a test set (different from the learning and validation sets used during the learning phase). Study the results obtained.
Now modify the code to replace the current classification layer with a set of two layers using a "relu" activation function for the middle layer, and the "dropout" mechanism for both layers. Renew the experiments and study the results obtained.
Now modify the code to replace the current classification layer with a set of two layers using a "relu" activation function for the middle layer, and the "dropout" mechanism for both layers. Renew the experiments and study the results obtained.
Apply ther quantization (post and quantization aware) and evaluate impact on model size and accuracy.
Apply ther quantization (post and quantization aware) and evaluate impact on model size and accuracy.
"\nThe size of the resNet18 model is %.2fMB, which is %.0f times bigger than the resNet18_quantized model"
% (
size_resNet18 / 1000000,
size_resNet18 / size_resNet18_quantized
)
)
```
%% Cell type:markdown id:04a263f0 tags:
%% Cell type:markdown id:04a263f0 tags:
## Optional
## Optional
Try this at home!!
Try this at home!!
Pytorch offers a framework to export a given CNN to your selfphone (either android or iOS). Have a look at the tutorial https://pytorch.org/mobile/home/
Pytorch offers a framework to export a given CNN to your selfphone (either android or iOS). Have a look at the tutorial https://pytorch.org/mobile/home/
The Exercise consists in deploying the CNN of Exercise 4 in your phone and then test it on live.
The Exercise consists in deploying the CNN of Exercise 4 in your phone and then test it on live.
