The goal is to apply a Convolutional Neural Net (CNN) model on the CIFAR10 image dataset and test the accuracy of the model on the basis of image classification. Compare the Accuracy VS the neural network implemented during TD1.
Have a look at the following documentation to be familiar with PyTorch.
"Validation loss decreased ({:.6f} --> {:.6f}). Saving model ...".format(
valid_loss_min,valid_loss
)
)
torch.save(model.state_dict(),"model_cifar.pt")
valid_loss_min=valid_loss
else:
i+=1
ifi==5:
break
```
%% Output
C:\Users\thoma\anaconda3\lib\site-packages\torch\nn\functional.py:1538: UserWarning: dropout2d: Received a 2-D input to dropout2d, which is deprecated and will result in an error in a future release. To retain the behavior and silence this warning, please use dropout instead. Note that dropout2d exists to provide channel-wise dropout on inputs with 2 spatial dimensions, a channel dimension, and an optional batch dimension (i.e. 3D or 4D inputs).
# forward pass: compute predicted outputs by passing inputs to the model
output=model(data)
# calculate the batch loss
loss=criterion(output,target)
# update test loss
test_loss+=loss.item()*data.size(0)
# convert output probabilities to predicted class
_,pred=torch.max(output,1)
# compare predictions to true label
correct_tensor=pred.eq(target.data.view_as(pred))
correct=(
np.squeeze(correct_tensor.numpy())
ifnottrain_on_gpu
elsenp.squeeze(correct_tensor.cpu().numpy())
)
# calculate test accuracy for each object class
foriinrange(batch_size):
label=target.data[i]
class_correct[label]+=correct[i].item()
class_total[label]+=1
# average test loss
test_loss=test_loss/len(test_loader)
print("Test Loss: {:.6f}\n".format(test_loss))
foriinrange(10):
ifclass_total[i]>0:
print(
"Test Accuracy of %5s: %2d%% (%2d/%2d)"
%(
classes[i],
100*class_correct[i]/class_total[i],
np.sum(class_correct[i]),
np.sum(class_total[i]),
)
)
else:
print("Test Accuracy of %5s: N/A (no training examples)"%(classes[i]))
print(
"\nTest Accuracy (Overall): %2d%% (%2d/%2d)"
%(
100.0*np.sum(class_correct)/np.sum(class_total),
np.sum(class_correct),
np.sum(class_total),
)
)
```
%% Output
C:\Users\thoma\AppData\Local\Temp/ipykernel_39460/3291884398.py:1: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
For each class, compare the classification test accuracy of the initial model and the quantized model. Also give the overall test accuracy for both models.
%% Cell type:markdown id:a0a34b90 tags:
Try training aware quantization to mitigate the impact on the accuracy (doc available here https://pytorch.org/docs/stable/quantization.html#torch.quantization.quantize_dynamic)
# forward pass: compute predicted outputs by passing inputs to the model
output=model(data)
# calculate the batch loss
loss=criterion(output,target)
# update test loss
test_loss+=loss.item()*data.size(0)
# convert output probabilities to predicted class
_,pred=torch.max(output,1)
# compare predictions to true label
correct_tensor=pred.eq(target.data.view_as(pred))
correct=(
np.squeeze(correct_tensor.numpy())
ifnottrain_on_gpu
elsenp.squeeze(correct_tensor.cpu().numpy())
)
# calculate test accuracy for each object class
foriinrange(batch_size):
label=target.data[i]
class_correct[label]+=correct[i].item()
class_total[label]+=1
# average test loss
test_loss=test_loss/len(test_loader)
print("Test Loss: {:.6f}\n".format(test_loss))
foriinrange(10):
ifclass_total[i]>0:
print(
"Test Accuracy of %5s: %2d%% (%2d/%2d)"
%(
classes[i],
100*class_correct[i]/class_total[i],
np.sum(class_correct[i]),
np.sum(class_total[i]),
)
)
else:
print("Test Accuracy of %5s: N/A (no training examples)"%(classes[i]))
print(
"\nTest Accuracy (Overall): %2d%% (%2d/%2d)\n"
%(
100.0*np.sum(class_correct)/np.sum(class_total),
np.sum(class_correct),
np.sum(class_total),
)
)
test_loss=0.0
class_correct=list(0.0foriinrange(10))
class_total=list(0.0foriinrange(10))
quantized_model.eval()
# iterate over test data
fordata,targetintest_loader:
# move tensors to GPU if CUDA is available
iftrain_on_gpu:
data,target=data.cuda(),target.cuda()
# forward pass: compute predicted outputs by passing inputs to the model
output=quantized_model(data)
# calculate the batch loss
loss=criterion(output,target)
# update test loss
test_loss+=loss.item()*data.size(0)
# convert output probabilities to predicted class
_,pred=torch.max(output,1)
# compare predictions to true label
correct_tensor=pred.eq(target.data.view_as(pred))
correct=(
np.squeeze(correct_tensor.numpy())
ifnottrain_on_gpu
elsenp.squeeze(correct_tensor.cpu().numpy())
)
# calculate test accuracy for each object class
foriinrange(batch_size):
label=target.data[i]
class_correct[label]+=correct[i].item()
class_total[label]+=1
# average test loss
test_loss=test_loss/len(test_loader)
print("Quantized test Loss: {:.6f}\n".format(test_loss))
foriinrange(10):
ifclass_total[i]>0:
print(
"Quantized test Accuracy of %5s: %2d%% (%2d/%2d)"
%(
classes[i],
100*class_correct[i]/class_total[i],
np.sum(class_correct[i]),
np.sum(class_total[i]),
)
)
else:
print("Quantized test Accuracy of %5s: N/A (no training examples)"%(classes[i]))
print(
"\nQuantized test Accuracy (Overall): %2d%% (%2d/%2d)"
%(
100.0*np.sum(class_correct)/np.sum(class_total),
np.sum(class_correct),
np.sum(class_total),
)
)
```
%% Output
C:\Users\thoma\AppData\Local\Temp/ipykernel_39460/681464573.py:1: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
Quantized test Accuracy of airplane: 77% (779/1000)
Quantized test Accuracy of automobile: 88% (881/1000)
Quantized test Accuracy of bird: 58% (582/1000)
Quantized test Accuracy of cat: 47% (479/1000)
Quantized test Accuracy of deer: 74% (743/1000)
Quantized test Accuracy of dog: 59% (599/1000)
Quantized test Accuracy of frog: 73% (739/1000)
Quantized test Accuracy of horse: 79% (790/1000)
Quantized test Accuracy of ship: 81% (811/1000)
Quantized test Accuracy of truck: 74% (749/1000)
Quantized test Accuracy (Overall): 71% (7152/10000)
%% Cell type:markdown id:84fe7b31 tags:
The two tests are almost equally performant, so the quantization doesn't have any impact on the porformance although it weights way less.
%% Cell type:markdown id:201470f9 tags:
## Exercise 3: working with pre-trained models.
PyTorch offers several pre-trained models https://pytorch.org/vision/0.8/models.html
We will use ResNet50 trained on ImageNet dataset (https://www.image-net.org/index.php). Use the following code with the files `imagenet-simple-labels.json` that contains the imagenet labels and the image dog.png that we will use as test.
%% Cell type:code id:b4d13080 tags:
``` python
importjson
fromPILimportImage
definitialize_model():
print_size_of_model(model,"fp32")
# Send the model to the GPU
# model.cuda()
# Set layers such as dropout and batchnorm in evaluation mode
print("For the test, predicted class is: {}".format(labels[out.argmax()]))
print("For the quantized test, predicted class is: {}".format(labels[quantized_out.argmax()]))
model=models.resnet50(pretrained=True)
print('Resnet')
initialize_model()
classify("dog.png")
classify("airplane.jpg")
classify("automobile.jpeg")
classify("ship.jpg")
model=models.alexnet(pretrained=True)
print('Alexnet')
initialize_model()
classify("dog.png")
classify("airplane.jpg")
classify("automobile.jpeg")
classify("ship.jpg")
model=models.vgg16(pretrained=True)
print('Vgg16')
initialize_model()
classify("dog.png")
classify("airplane.jpg")
classify("automobile.jpeg")
classify("ship.jpg")
```
%% Output
Resnet
model: fp32 Size (KB): 102523.238
model: int8 Size (KB): 96379.996
dog.png
For the test, predicted class is: Golden Retriever
For the quantized test, predicted class is: Golden Retriever
airplane.jpg
For the test, predicted class is: airliner
For the quantized test, predicted class is: airliner
automobile.jpeg
For the test, predicted class is: sports car
For the quantized test, predicted class is: sports car
ship.jpg
For the test, predicted class is: motorboat
For the quantized test, predicted class is: motorboat
C:\Users\thoma\anaconda3\lib\site-packages\torchvision\models\_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=AlexNet_Weights.IMAGENET1K_V1`. You can also use `weights=AlexNet_Weights.DEFAULT` to get the most up-to-date weights.
warnings.warn(msg)
Downloading: "https://download.pytorch.org/models/alexnet-owt-7be5be79.pth" to C:\Users\thoma/.cache\torch\hub\checkpoints\alexnet-owt-7be5be79.pth
For the test, predicted class is: Golden Retriever
For the quantized test, predicted class is: Golden Retriever
airplane.jpg
For the test, predicted class is: airliner
For the quantized test, predicted class is: airliner
automobile.jpeg
For the test, predicted class is: station wagon
For the quantized test, predicted class is: sports car
ship.jpg
For the test, predicted class is: motorboat
For the quantized test, predicted class is: motorboat
C:\Users\thoma\anaconda3\lib\site-packages\torchvision\models\_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=VGG16_Weights.IMAGENET1K_V1`. You can also use `weights=VGG16_Weights.DEFAULT` to get the most up-to-date weights.
warnings.warn(msg)
Downloading: "https://download.pytorch.org/models/vgg16-397923af.pth" to C:\Users\thoma/.cache\torch\hub\checkpoints\vgg16-397923af.pth
For the test, predicted class is: Golden Retriever
For the quantized test, predicted class is: Golden Retriever
airplane.jpg
For the test, predicted class is: airliner
For the quantized test, predicted class is: airliner
automobile.jpeg
For the test, predicted class is: sports car
For the quantized test, predicted class is: sports car
ship.jpg
For the test, predicted class is: motorboat
For the quantized test, predicted class is: motorboat
%% Cell type:markdown id:184cfceb tags:
Experiments:
Study the code and the results obtained. Possibly add other images downloaded from the internet.
What is the size of the model? Quantize it and then check if the model is still able to correctly classify the other images.
Experiment with other pre-trained CNN models.
We can see similar performance with all models, wheither it's quantized or not, except for Alexnet which predict wrong of automobile.jpeg, but rigth with its quantized model.
%% Cell type:markdown id:5d57da4b tags:
## Exercise 4: Transfer Learning
For this work, we will use a pre-trained model (ResNet18) as a descriptor extractor and will refine the classification by training only the last fully connected layer of the network. Thus, the output layer of the pre-trained network will be replaced by a layer adapted to the new classes to be recognized which will be in our case ants and bees.
Download and unzip in your working directory the dataset available at the address :
plt.pause(0.001)# pause a bit so that plots are updated
plt.show()
# Get a batch of training data
inputs,classes=next(iter(dataloaders["train"]))
# Make a grid from batch
out=torchvision.utils.make_grid(inputs)
# imshow(out, title=[class_names[x] for x in classes])
```
%% Cell type:markdown id:bbd48800 tags:
Now, execute the following code which uses a pre-trained model ResNet18 having replaced the output layer for the ants/bees classification and performs the model training by only changing the weights of this output layer.
%% Cell type:code id:572d824c tags:
``` python
importcopy
importos
importtime
importmatplotlib.pyplotasplt
importnumpyasnp
importtorch
importtorch.nnasnn
importtorch.optimasoptim
importtorchvision
fromtorch.optimimportlr_scheduler
fromtorchvisionimportdatasets,transforms
# Data augmentation and normalization for training
# Just normalization for validation
data_transforms={
"train":transforms.Compose(
[
transforms.RandomResizedCrop(
224
),# ImageNet models were trained on 224x224 images
transforms.RandomHorizontalFlip(),# flip horizontally 50% of the time - increases train set variability
transforms.ToTensor(),# convert it to a PyTorch tensor
C:\Users\thoma\anaconda3\Lib\site-packages\torchvision\models\_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
warnings.warn(
C:\Users\thoma\anaconda3\Lib\site-packages\torchvision\models\_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet18_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet18_Weights.DEFAULT` to get the most up-to-date weights.
warnings.warn(msg)
Epoch 1/10
----------
C:\Users\thoma\anaconda3\Lib\site-packages\torch\optim\lr_scheduler.py:224: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`. Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
Modify the code and add an "eval_model" function to allow
the evaluation of the model on a test set (different from the learning and validation sets used during the learning phase). Study the results obtained.
The accuracy is 0.9231 so the model is still performant. The test set is made by pictures downloaded from google.
Now modify the code to replace the current classification layer with a set of two layers using a "relu" activation function for the middle layer, and the "dropout" mechanism for both layers. Renew the experiments and study the results obtained.
The validation is equivalent, but the accuraccy on the test data set is not 1.
Apply ther quantization (post and quantization aware) and evaluate impact on model size and accuracy.
The model is a bit less heavy, but not significaly. The accuracy on the testing set is the same.
%% Cell type:markdown id:04a263f0 tags:
## Optional
Try this at home!!
Pytorch offers a framework to export a given CNN to your selfphone (either android or iOS). Have a look at the tutorial https://pytorch.org/mobile/home/
The Exercise consists in deploying the CNN of Exercise 4 in your phone and then test it on live.
%% Cell type:markdown id:fe954ce4 tags:
## Author
Alberto BOSIO - Ph. D.
%% Cell type:markdown id:7edf7168 tags:
# TD2: Deep learning
%% Cell type:markdown id:fbb8c8df tags:
In this TD, you must modify this notebook to answer the questions. To do this,
1. Fork this repository
2. Clone your forked repository on your local computer
3. Answer the questions
4. Commit and push regularly
The last commit is due on Wednesday, December 4, 11:59 PM. Later commits will not be taken into account.
%% Cell type:markdown id:3d167a29 tags:
Install and test PyTorch from https://pytorch.org/get-started/locally.
%% Cell type:code id:330a42f5 tags:
``` python
%pipinstalltorchtorchvision
```
%% Output
Requirement already satisfied: torch in c:\users\thoma\anaconda3\lib\site-packages (2.5.1)
Requirement already satisfied: torchvision in c:\users\thoma\anaconda3\lib\site-packages (0.20.1)
Requirement already satisfied: filelock in c:\users\thoma\anaconda3\lib\site-packages (from torch) (3.3.1)
Requirement already satisfied: typing-extensions>=4.8.0 in c:\users\thoma\anaconda3\lib\site-packages (from torch) (4.12.2)
Requirement already satisfied: networkx in c:\users\thoma\anaconda3\lib\site-packages (from torch) (2.6.3)
Requirement already satisfied: jinja2 in c:\users\thoma\anaconda3\lib\site-packages (from torch) (2.11.3)
Requirement already satisfied: fsspec in c:\users\thoma\anaconda3\lib\site-packages (from torch) (2021.10.1)
Requirement already satisfied: sympy==1.13.1 in c:\users\thoma\anaconda3\lib\site-packages (from torch) (1.13.1)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\users\thoma\anaconda3\lib\site-packages (from sympy==1.13.1->torch) (1.2.1)
Requirement already satisfied: numpy in c:\users\thoma\anaconda3\lib\site-packages (from torchvision) (1.20.3)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in c:\users\thoma\anaconda3\lib\site-packages (from torchvision) (9.3.0)
Requirement already satisfied: MarkupSafe>=0.23 in c:\users\thoma\anaconda3\lib\site-packages (from jinja2->torch) (1.1.1)
Note: you may need to restart the kernel to use updated packages.
WARNING: Ignoring invalid distribution -illow (c:\users\thoma\anaconda3\lib\site-packages)
WARNING: Error parsing dependencies of pyodbc: Invalid version: '4.0.0-unsupported'
WARNING: Ignoring invalid distribution -illow (c:\users\thoma\anaconda3\lib\site-packages)
ERROR: Exception:
Traceback (most recent call last):
File "C:\Users\thoma\anaconda3\lib\site-packages\pip\_internal\cli\base_command.py", line 105, in _run_wrapper
status = _inner_run()
File "C:\Users\thoma\anaconda3\lib\site-packages\pip\_internal\cli\base_command.py", line 96, in _inner_run
return self.run(options, args)
File "C:\Users\thoma\anaconda3\lib\site-packages\pip\_internal\cli\req_command.py", line 67, in wrapper
return func(self, options, args)
File "C:\Users\thoma\anaconda3\lib\site-packages\pip\_internal\commands\install.py", line 483, in run
The goal is to apply a Convolutional Neural Net (CNN) model on the CIFAR10 image dataset and test the accuracy of the model on the basis of image classification. Compare the Accuracy VS the neural network implemented during TD1.
Have a look at the following documentation to be familiar with PyTorch.
"Validation loss decreased ({:.6f} --> {:.6f}). Saving model ...".format(
valid_loss_min,valid_loss
)
)
torch.save(model.state_dict(),"model_cifar.pt")
valid_loss_min=valid_loss
else:
i+=1
ifi==5:
break
```
%% Output
C:\Users\thoma\anaconda3\lib\site-packages\torch\nn\functional.py:1538: UserWarning: dropout2d: Received a 2-D input to dropout2d, which is deprecated and will result in an error in a future release. To retain the behavior and silence this warning, please use dropout instead. Note that dropout2d exists to provide channel-wise dropout on inputs with 2 spatial dimensions, a channel dimension, and an optional batch dimension (i.e. 3D or 4D inputs).
# forward pass: compute predicted outputs by passing inputs to the model
output=model(data)
# calculate the batch loss
loss=criterion(output,target)
# update test loss
test_loss+=loss.item()*data.size(0)
# convert output probabilities to predicted class
_,pred=torch.max(output,1)
# compare predictions to true label
correct_tensor=pred.eq(target.data.view_as(pred))
correct=(
np.squeeze(correct_tensor.numpy())
ifnottrain_on_gpu
elsenp.squeeze(correct_tensor.cpu().numpy())
)
# calculate test accuracy for each object class
foriinrange(batch_size):
label=target.data[i]
class_correct[label]+=correct[i].item()
class_total[label]+=1
# average test loss
test_loss=test_loss/len(test_loader)
print("Test Loss: {:.6f}\n".format(test_loss))
foriinrange(10):
ifclass_total[i]>0:
print(
"Test Accuracy of %5s: %2d%% (%2d/%2d)"
%(
classes[i],
100*class_correct[i]/class_total[i],
np.sum(class_correct[i]),
np.sum(class_total[i]),
)
)
else:
print("Test Accuracy of %5s: N/A (no training examples)"%(classes[i]))
print(
"\nTest Accuracy (Overall): %2d%% (%2d/%2d)"
%(
100.0*np.sum(class_correct)/np.sum(class_total),
np.sum(class_correct),
np.sum(class_total),
)
)
```
%% Output
C:\Users\thoma\AppData\Local\Temp/ipykernel_39460/3291884398.py:1: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
For each class, compare the classification test accuracy of the initial model and the quantized model. Also give the overall test accuracy for both models.
%% Cell type:markdown id:a0a34b90 tags:
Try training aware quantization to mitigate the impact on the accuracy (doc available here https://pytorch.org/docs/stable/quantization.html#torch.quantization.quantize_dynamic)
# forward pass: compute predicted outputs by passing inputs to the model
output=model(data)
# calculate the batch loss
loss=criterion(output,target)
# update test loss
test_loss+=loss.item()*data.size(0)
# convert output probabilities to predicted class
_,pred=torch.max(output,1)
# compare predictions to true label
correct_tensor=pred.eq(target.data.view_as(pred))
correct=(
np.squeeze(correct_tensor.numpy())
ifnottrain_on_gpu
elsenp.squeeze(correct_tensor.cpu().numpy())
)
# calculate test accuracy for each object class
foriinrange(batch_size):
label=target.data[i]
class_correct[label]+=correct[i].item()
class_total[label]+=1
# average test loss
test_loss=test_loss/len(test_loader)
print("Test Loss: {:.6f}\n".format(test_loss))
foriinrange(10):
ifclass_total[i]>0:
print(
"Test Accuracy of %5s: %2d%% (%2d/%2d)"
%(
classes[i],
100*class_correct[i]/class_total[i],
np.sum(class_correct[i]),
np.sum(class_total[i]),
)
)
else:
print("Test Accuracy of %5s: N/A (no training examples)"%(classes[i]))
print(
"\nTest Accuracy (Overall): %2d%% (%2d/%2d)\n"
%(
100.0*np.sum(class_correct)/np.sum(class_total),
np.sum(class_correct),
np.sum(class_total),
)
)
test_loss=0.0
class_correct=list(0.0foriinrange(10))
class_total=list(0.0foriinrange(10))
quantized_model.eval()
# iterate over test data
fordata,targetintest_loader:
# move tensors to GPU if CUDA is available
iftrain_on_gpu:
data,target=data.cuda(),target.cuda()
# forward pass: compute predicted outputs by passing inputs to the model
output=quantized_model(data)
# calculate the batch loss
loss=criterion(output,target)
# update test loss
test_loss+=loss.item()*data.size(0)
# convert output probabilities to predicted class
_,pred=torch.max(output,1)
# compare predictions to true label
correct_tensor=pred.eq(target.data.view_as(pred))
correct=(
np.squeeze(correct_tensor.numpy())
ifnottrain_on_gpu
elsenp.squeeze(correct_tensor.cpu().numpy())
)
# calculate test accuracy for each object class
foriinrange(batch_size):
label=target.data[i]
class_correct[label]+=correct[i].item()
class_total[label]+=1
# average test loss
test_loss=test_loss/len(test_loader)
print("Quantized test Loss: {:.6f}\n".format(test_loss))
foriinrange(10):
ifclass_total[i]>0:
print(
"Quantized test Accuracy of %5s: %2d%% (%2d/%2d)"
%(
classes[i],
100*class_correct[i]/class_total[i],
np.sum(class_correct[i]),
np.sum(class_total[i]),
)
)
else:
print("Quantized test Accuracy of %5s: N/A (no training examples)"%(classes[i]))
print(
"\nQuantized test Accuracy (Overall): %2d%% (%2d/%2d)"
%(
100.0*np.sum(class_correct)/np.sum(class_total),
np.sum(class_correct),
np.sum(class_total),
)
)
```
%% Output
C:\Users\thoma\AppData\Local\Temp/ipykernel_39460/681464573.py:1: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
Quantized test Accuracy of airplane: 77% (779/1000)
Quantized test Accuracy of automobile: 88% (881/1000)
Quantized test Accuracy of bird: 58% (582/1000)
Quantized test Accuracy of cat: 47% (479/1000)
Quantized test Accuracy of deer: 74% (743/1000)
Quantized test Accuracy of dog: 59% (599/1000)
Quantized test Accuracy of frog: 73% (739/1000)
Quantized test Accuracy of horse: 79% (790/1000)
Quantized test Accuracy of ship: 81% (811/1000)
Quantized test Accuracy of truck: 74% (749/1000)
Quantized test Accuracy (Overall): 71% (7152/10000)
%% Cell type:markdown id:84fe7b31 tags:
The two tests are almost equally performant, so the quantization doesn't have any impact on the porformance although it weights way less.
%% Cell type:markdown id:201470f9 tags:
## Exercise 3: working with pre-trained models.
PyTorch offers several pre-trained models https://pytorch.org/vision/0.8/models.html
We will use ResNet50 trained on ImageNet dataset (https://www.image-net.org/index.php). Use the following code with the files `imagenet-simple-labels.json` that contains the imagenet labels and the image dog.png that we will use as test.
%% Cell type:code id:b4d13080 tags:
``` python
importjson
fromPILimportImage
definitialize_model():
print_size_of_model(model,"fp32")
# Send the model to the GPU
# model.cuda()
# Set layers such as dropout and batchnorm in evaluation mode
print("For the test, predicted class is: {}".format(labels[out.argmax()]))
print("For the quantized test, predicted class is: {}".format(labels[quantized_out.argmax()]))
model=models.resnet50(pretrained=True)
print('Resnet')
initialize_model()
classify("dog.png")
classify("airplane.jpg")
classify("automobile.jpeg")
classify("ship.jpg")
model=models.alexnet(pretrained=True)
print('Alexnet')
initialize_model()
classify("dog.png")
classify("airplane.jpg")
classify("automobile.jpeg")
classify("ship.jpg")
model=models.vgg16(pretrained=True)
print('Vgg16')
initialize_model()
classify("dog.png")
classify("airplane.jpg")
classify("automobile.jpeg")
classify("ship.jpg")
```
%% Output
Resnet
model: fp32 Size (KB): 102523.238
model: int8 Size (KB): 96379.996
dog.png
For the test, predicted class is: Golden Retriever
For the quantized test, predicted class is: Golden Retriever
airplane.jpg
For the test, predicted class is: airliner
For the quantized test, predicted class is: airliner
automobile.jpeg
For the test, predicted class is: sports car
For the quantized test, predicted class is: sports car
ship.jpg
For the test, predicted class is: motorboat
For the quantized test, predicted class is: motorboat
C:\Users\thoma\anaconda3\lib\site-packages\torchvision\models\_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=AlexNet_Weights.IMAGENET1K_V1`. You can also use `weights=AlexNet_Weights.DEFAULT` to get the most up-to-date weights.
warnings.warn(msg)
Downloading: "https://download.pytorch.org/models/alexnet-owt-7be5be79.pth" to C:\Users\thoma/.cache\torch\hub\checkpoints\alexnet-owt-7be5be79.pth
For the test, predicted class is: Golden Retriever
For the quantized test, predicted class is: Golden Retriever
airplane.jpg
For the test, predicted class is: airliner
For the quantized test, predicted class is: airliner
automobile.jpeg
For the test, predicted class is: station wagon
For the quantized test, predicted class is: sports car
ship.jpg
For the test, predicted class is: motorboat
For the quantized test, predicted class is: motorboat
C:\Users\thoma\anaconda3\lib\site-packages\torchvision\models\_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=VGG16_Weights.IMAGENET1K_V1`. You can also use `weights=VGG16_Weights.DEFAULT` to get the most up-to-date weights.
warnings.warn(msg)
Downloading: "https://download.pytorch.org/models/vgg16-397923af.pth" to C:\Users\thoma/.cache\torch\hub\checkpoints\vgg16-397923af.pth
For the test, predicted class is: Golden Retriever
For the quantized test, predicted class is: Golden Retriever
airplane.jpg
For the test, predicted class is: airliner
For the quantized test, predicted class is: airliner
automobile.jpeg
For the test, predicted class is: sports car
For the quantized test, predicted class is: sports car
ship.jpg
For the test, predicted class is: motorboat
For the quantized test, predicted class is: motorboat
%% Cell type:markdown id:184cfceb tags:
Experiments:
Study the code and the results obtained. Possibly add other images downloaded from the internet.
What is the size of the model? Quantize it and then check if the model is still able to correctly classify the other images.
Experiment with other pre-trained CNN models.
We can see similar performance with all models, wheither it's quantized or not, except for Alexnet which predict wrong of automobile.jpeg, but rigth with its quantized model.
%% Cell type:markdown id:5d57da4b tags:
## Exercise 4: Transfer Learning
For this work, we will use a pre-trained model (ResNet18) as a descriptor extractor and will refine the classification by training only the last fully connected layer of the network. Thus, the output layer of the pre-trained network will be replaced by a layer adapted to the new classes to be recognized which will be in our case ants and bees.
Download and unzip in your working directory the dataset available at the address :
plt.pause(0.001)# pause a bit so that plots are updated
plt.show()
# Get a batch of training data
inputs,classes=next(iter(dataloaders["train"]))
# Make a grid from batch
out=torchvision.utils.make_grid(inputs)
# imshow(out, title=[class_names[x] for x in classes])
```
%% Cell type:markdown id:bbd48800 tags:
Now, execute the following code which uses a pre-trained model ResNet18 having replaced the output layer for the ants/bees classification and performs the model training by only changing the weights of this output layer.
%% Cell type:code id:572d824c tags:
``` python
importcopy
importos
importtime
importmatplotlib.pyplotasplt
importnumpyasnp
importtorch
importtorch.nnasnn
importtorch.optimasoptim
importtorchvision
fromtorch.optimimportlr_scheduler
fromtorchvisionimportdatasets,transforms
# Data augmentation and normalization for training
# Just normalization for validation
data_transforms={
"train":transforms.Compose(
[
transforms.RandomResizedCrop(
224
),# ImageNet models were trained on 224x224 images
transforms.RandomHorizontalFlip(),# flip horizontally 50% of the time - increases train set variability
transforms.ToTensor(),# convert it to a PyTorch tensor
C:\Users\thoma\anaconda3\Lib\site-packages\torchvision\models\_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
warnings.warn(
C:\Users\thoma\anaconda3\Lib\site-packages\torchvision\models\_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet18_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet18_Weights.DEFAULT` to get the most up-to-date weights.
warnings.warn(msg)
Epoch 1/10
----------
C:\Users\thoma\anaconda3\Lib\site-packages\torch\optim\lr_scheduler.py:224: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`. Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
Modify the code and add an "eval_model" function to allow
the evaluation of the model on a test set (different from the learning and validation sets used during the learning phase). Study the results obtained.
The accuracy is 0.9231 so the model is still performant. The test set is made by pictures downloaded from google.
Now modify the code to replace the current classification layer with a set of two layers using a "relu" activation function for the middle layer, and the "dropout" mechanism for both layers. Renew the experiments and study the results obtained.
The validation is equivalent, but the accuraccy on the test data set is not 1.
Apply ther quantization (post and quantization aware) and evaluate impact on model size and accuracy.
The model is a bit less heavy, but not significaly. The accuracy on the testing set is the same.
%% Cell type:markdown id:04a263f0 tags:
## Optional
Try this at home!!
Pytorch offers a framework to export a given CNN to your selfphone (either android or iOS). Have a look at the tutorial https://pytorch.org/mobile/home/
The Exercise consists in deploying the CNN of Exercise 4 in your phone and then test it on live.
