diff --git a/TD2 Deep Learning.ipynb b/TD2 Deep Learning.ipynb
index 61e2cd374f463b9d3e7b8be2c83f90004b0f2aea..2a9c12e040653f255f0d41441084e530c58122b9 100644
--- a/TD2 Deep Learning.ipynb
+++ b/TD2 Deep Learning.ipynb
@@ -1369,20 +1369,6 @@
"The result is that the test accuracy is almost the same for the initial model and for the quantized model."
]
},
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### 3. Aware quantization"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": []
- },
{
"cell_type": "markdown",
"id": "201470f9",
@@ -1887,65 +1873,89 @@
},
{
"cell_type": "code",
- "execution_count": 35,
+ "execution_count": 79,
"id": "572d824c",
"metadata": {},
"outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "c:\\Users\\lucil\\anaconda3\\Lib\\site-packages\\torchvision\\models\\_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.\n",
+ " warnings.warn(\n",
+ "c:\\Users\\lucil\\anaconda3\\Lib\\site-packages\\torchvision\\models\\_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet18_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet18_Weights.DEFAULT` to get the most up-to-date weights.\n",
+ " warnings.warn(msg)\n"
+ ]
+ },
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/10\n",
- "----------\n",
- "train Loss: 0.5453 Acc: 0.7008\n",
- "val Loss: 0.2199 Acc: 0.9281\n",
+ "----------\n"
+ ]
+ },
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "c:\\Users\\lucil\\anaconda3\\Lib\\site-packages\\torch\\optim\\lr_scheduler.py:136: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`. Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate\n",
+ " warnings.warn(\"Detected call of `lr_scheduler.step()` before `optimizer.step()`. \"\n"
+ ]
+ },
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "train Loss: 0.6057 Acc: 0.6598\n",
+ "val Loss: 0.2570 Acc: 0.9346\n",
"\n",
"Epoch 2/10\n",
"----------\n",
- "train Loss: 0.6431 Acc: 0.6885\n",
- "val Loss: 0.1848 Acc: 0.9477\n",
+ "train Loss: 0.4204 Acc: 0.8033\n",
+ "val Loss: 0.2251 Acc: 0.9216\n",
"\n",
"Epoch 3/10\n",
"----------\n",
- "train Loss: 0.5283 Acc: 0.7377\n",
- "val Loss: 0.1921 Acc: 0.9477\n",
+ "train Loss: 0.5382 Acc: 0.7459\n",
+ "val Loss: 0.2880 Acc: 0.8693\n",
"\n",
"Epoch 4/10\n",
"----------\n",
- "train Loss: 0.4287 Acc: 0.8197\n",
- "val Loss: 0.2242 Acc: 0.9281\n",
+ "train Loss: 0.5375 Acc: 0.7787\n",
+ "val Loss: 0.1798 Acc: 0.9412\n",
"\n",
"Epoch 5/10\n",
"----------\n",
- "train Loss: 0.6329 Acc: 0.7254\n",
- "val Loss: 0.1941 Acc: 0.9412\n",
+ "train Loss: 0.6131 Acc: 0.7623\n",
+ "val Loss: 0.1591 Acc: 0.9542\n",
"\n",
"Epoch 6/10\n",
"----------\n",
- "train Loss: 0.4771 Acc: 0.7869\n",
- "val Loss: 0.3280 Acc: 0.8693\n",
+ "train Loss: 0.3371 Acc: 0.8689\n",
+ "val Loss: 0.1554 Acc: 0.9477\n",
"\n",
"Epoch 7/10\n",
"----------\n",
- "train Loss: 0.3549 Acc: 0.8402\n",
- "val Loss: 0.1751 Acc: 0.9542\n",
+ "train Loss: 0.3477 Acc: 0.8607\n",
+ "val Loss: 0.1815 Acc: 0.9477\n",
"\n",
"Epoch 8/10\n",
"----------\n",
- "train Loss: 0.3525 Acc: 0.8279\n",
- "val Loss: 0.1632 Acc: 0.9542\n",
+ "train Loss: 0.3790 Acc: 0.8115\n",
+ "val Loss: 0.1819 Acc: 0.9346\n",
"\n",
"Epoch 9/10\n",
"----------\n",
- "train Loss: 0.3126 Acc: 0.8648\n",
- "val Loss: 0.1709 Acc: 0.9542\n",
+ "train Loss: 0.3273 Acc: 0.8689\n",
+ "val Loss: 0.1686 Acc: 0.9477\n",
"\n",
"Epoch 10/10\n",
"----------\n",
- "train Loss: 0.3212 Acc: 0.8689\n",
- "val Loss: 0.1691 Acc: 0.9477\n",
+ "train Loss: 0.2768 Acc: 0.8934\n",
+ "val Loss: 0.1883 Acc: 0.9412\n",
"\n",
- "Training complete in 8m 5s\n",
+ "Training complete in 9m 5s\n",
"Best val Acc: 0.954248\n"
]
}
@@ -2150,104 +2160,89 @@
{
"cell_type": "markdown",
"metadata": {},
- "source": []
+ "source": [
+ "### 1. Adding an \"eval_model\" function"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Before adding this function, it is necessary to add an \"eval\" folder in the directory \"hymenoptera data\". To do that, it is necessary to search and save random pictures of ants and bees on Internet. This way, we'll be able to evaluate the training of the model."
+ ]
},
{
"cell_type": "code",
- "execution_count": 46,
+ "execution_count": 85,
"metadata": {},
"outputs": [
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "c:\\Users\\lucil\\anaconda3\\Lib\\site-packages\\torchvision\\models\\_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.\n",
- " warnings.warn(\n",
- "c:\\Users\\lucil\\anaconda3\\Lib\\site-packages\\torchvision\\models\\_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet18_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet18_Weights.DEFAULT` to get the most up-to-date weights.\n",
- " warnings.warn(msg)\n"
- ]
- },
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/10\n",
- "----------\n"
- ]
- },
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "c:\\Users\\lucil\\anaconda3\\Lib\\site-packages\\torch\\optim\\lr_scheduler.py:136: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`. Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate\n",
- " warnings.warn(\"Detected call of `lr_scheduler.step()` before `optimizer.step()`. \"\n"
- ]
- },
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "train Loss: 0.5158 Acc: 0.7336\n",
- "val Loss: 0.4568 Acc: 0.7778\n",
+ "----------\n",
+ "train Loss: 0.6263 Acc: 0.6475\n",
+ "val Loss: 0.3515 Acc: 0.8431\n",
"\n",
"Epoch 2/10\n",
"----------\n",
- "train Loss: 0.4562 Acc: 0.7869\n",
- "val Loss: 0.2000 Acc: 0.9542\n",
+ "train Loss: 0.5289 Acc: 0.7295\n",
+ "val Loss: 0.7287 Acc: 0.6667\n",
"\n",
"Epoch 3/10\n",
"----------\n",
- "train Loss: 0.5090 Acc: 0.7828\n",
- "val Loss: 0.1858 Acc: 0.9477\n",
+ "train Loss: 0.4102 Acc: 0.8033\n",
+ "val Loss: 0.1947 Acc: 0.9477\n",
"\n",
"Epoch 4/10\n",
"----------\n",
- "train Loss: 0.4266 Acc: 0.7992\n",
- "val Loss: 0.2278 Acc: 0.9477\n",
+ "train Loss: 0.5403 Acc: 0.7541\n",
+ "val Loss: 0.2066 Acc: 0.9216\n",
"\n",
"Epoch 5/10\n",
"----------\n",
- "train Loss: 0.5189 Acc: 0.7705\n",
- "val Loss: 0.2774 Acc: 0.9020\n",
+ "train Loss: 0.5845 Acc: 0.7418\n",
+ "val Loss: 0.1745 Acc: 0.9608\n",
"\n",
"Epoch 6/10\n",
"----------\n",
- "train Loss: 0.4524 Acc: 0.8074\n",
- "val Loss: 0.2509 Acc: 0.9216\n",
+ "train Loss: 0.4366 Acc: 0.8238\n",
+ "val Loss: 0.1962 Acc: 0.9412\n",
"\n",
"Epoch 7/10\n",
"----------\n",
- "train Loss: 0.4077 Acc: 0.8115\n",
- "val Loss: 0.1791 Acc: 0.9412\n",
+ "train Loss: 0.3217 Acc: 0.8238\n",
+ "val Loss: 0.1911 Acc: 0.9346\n",
"\n",
"Epoch 8/10\n",
"----------\n",
- "train Loss: 0.4013 Acc: 0.8402\n",
- "val Loss: 0.1973 Acc: 0.9412\n",
+ "train Loss: 0.2924 Acc: 0.8852\n",
+ "val Loss: 0.1924 Acc: 0.9216\n",
"\n",
"Epoch 9/10\n",
"----------\n",
- "train Loss: 0.3460 Acc: 0.8689\n",
- "val Loss: 0.1996 Acc: 0.9412\n",
+ "train Loss: 0.3513 Acc: 0.8238\n",
+ "val Loss: 0.2039 Acc: 0.9281\n",
"\n",
"Epoch 10/10\n",
"----------\n",
- "train Loss: 0.2655 Acc: 0.8893\n",
- "val Loss: 0.1862 Acc: 0.9477\n",
+ "train Loss: 0.3813 Acc: 0.8279\n",
+ "val Loss: 0.2065 Acc: 0.9216\n",
"\n",
- "Training complete in 8m 60s\n",
- "Best val Acc: 0.954248\n",
+ "Training complete in 7m 23s\n",
+ "Best val Acc: 0.960784\n",
"Evaluation of the model :\n",
- "eval Loss: 0.2000 Acc: 0.9542\n"
+ "eval Loss: 0.0442 Acc: 1.0000\n"
]
},
{
"data": {
"text/plain": [
- "(0.19995229715615317, tensor(0.9542, dtype=torch.float64))"
+ "(0.04420919088380677, tensor(1., dtype=torch.float64))"
]
},
- "execution_count": 46,
+ "execution_count": 85,
"metadata": {},
"output_type": "execute_result"
}
@@ -2487,11 +2482,13 @@
{
"cell_type": "markdown",
"metadata": {},
- "source": []
+ "source": [
+ "### 2. Modify the model by replacing the current classification layer"
+ ]
},
{
"cell_type": "code",
- "execution_count": 48,
+ "execution_count": 81,
"metadata": {},
"outputs": [
{
@@ -2500,67 +2497,67 @@
"text": [
"Epoch 1/10\n",
"----------\n",
- "train Loss: 0.7083 Acc: 0.5984\n",
- "val Loss: 0.5109 Acc: 0.7124\n",
+ "train Loss: 0.8205 Acc: 0.5164\n",
+ "val Loss: 0.5431 Acc: 0.6471\n",
"\n",
"Epoch 2/10\n",
"----------\n",
- "train Loss: 0.5679 Acc: 0.6967\n",
- "val Loss: 0.3589 Acc: 0.8758\n",
+ "train Loss: 0.6203 Acc: 0.6270\n",
+ "val Loss: 0.4018 Acc: 0.9216\n",
"\n",
"Epoch 3/10\n",
"----------\n",
- "train Loss: 0.5573 Acc: 0.6762\n",
- "val Loss: 0.3162 Acc: 0.8693\n",
+ "train Loss: 0.6033 Acc: 0.6598\n",
+ "val Loss: 0.5706 Acc: 0.6078\n",
"\n",
"Epoch 4/10\n",
"----------\n",
- "train Loss: 0.6000 Acc: 0.6762\n",
- "val Loss: 0.3620 Acc: 0.8497\n",
+ "train Loss: 0.5956 Acc: 0.6926\n",
+ "val Loss: 0.3167 Acc: 0.9216\n",
"\n",
"Epoch 5/10\n",
"----------\n",
- "train Loss: 0.5870 Acc: 0.6721\n",
- "val Loss: 0.2739 Acc: 0.9216\n",
+ "train Loss: 0.6396 Acc: 0.6189\n",
+ "val Loss: 0.3525 Acc: 0.9346\n",
"\n",
"Epoch 6/10\n",
"----------\n",
- "train Loss: 0.4931 Acc: 0.6844\n",
- "val Loss: 0.2464 Acc: 0.9216\n",
+ "train Loss: 0.5478 Acc: 0.6926\n",
+ "val Loss: 0.3266 Acc: 0.9020\n",
"\n",
"Epoch 7/10\n",
"----------\n",
- "train Loss: 0.4539 Acc: 0.7787\n",
- "val Loss: 0.2249 Acc: 0.9281\n",
+ "train Loss: 0.5412 Acc: 0.7049\n",
+ "val Loss: 0.3001 Acc: 0.9216\n",
"\n",
"Epoch 8/10\n",
"----------\n",
- "train Loss: 0.5093 Acc: 0.7295\n",
- "val Loss: 0.2308 Acc: 0.9477\n",
+ "train Loss: 0.4871 Acc: 0.7336\n",
+ "val Loss: 0.2641 Acc: 0.9477\n",
"\n",
"Epoch 9/10\n",
"----------\n",
- "train Loss: 0.4690 Acc: 0.7500\n",
- "val Loss: 0.2339 Acc: 0.9346\n",
+ "train Loss: 0.5152 Acc: 0.7049\n",
+ "val Loss: 0.2713 Acc: 0.9346\n",
"\n",
"Epoch 10/10\n",
"----------\n",
- "train Loss: 0.4816 Acc: 0.7172\n",
- "val Loss: 0.2278 Acc: 0.9281\n",
+ "train Loss: 0.4989 Acc: 0.7254\n",
+ "val Loss: 0.2615 Acc: 0.9477\n",
"\n",
- "Training complete in 8m 6s\n",
+ "Training complete in 7m 27s\n",
"Best val Acc: 0.947712\n",
"Evaluation of the new model :\n",
- "eval Loss: 0.2308 Acc: 0.9477\n"
+ "eval Loss: 0.2224 Acc: 0.9714\n"
]
},
{
"data": {
"text/plain": [
- "(0.23079453324716465, tensor(0.9477, dtype=torch.float64))"
+ "(0.2223798223904201, tensor(0.9714, dtype=torch.float64))"
]
},
- "execution_count": 48,
+ "execution_count": 81,
"metadata": {},
"output_type": "execute_result"
}
@@ -2598,9 +2595,23 @@
"eval_model(model, criterion)"
]
},
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 3. Quantization of the model"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "First we calculate the model size."
+ ]
+ },
{
"cell_type": "code",
- "execution_count": 49,
+ "execution_count": 82,
"metadata": {},
"outputs": [
{
@@ -2616,7 +2627,7 @@
"45304250"
]
},
- "execution_count": 49,
+ "execution_count": 82,
"metadata": {},
"output_type": "execute_result"
}
@@ -2628,11 +2639,13 @@
{
"cell_type": "markdown",
"metadata": {},
- "source": []
+ "source": [
+ "Now we quantize the model and calculate its size."
+ ]
},
{
"cell_type": "code",
- "execution_count": 50,
+ "execution_count": 83,
"metadata": {},
"outputs": [
{
@@ -2648,7 +2661,7 @@
"44911014"
]
},
- "execution_count": 50,
+ "execution_count": 83,
"metadata": {},
"output_type": "execute_result"
}
@@ -2663,35 +2676,37 @@
{
"cell_type": "markdown",
"metadata": {},
- "source": []
+ "source": [
+ "The difference in sizes in minimal."
+ ]
},
{
"cell_type": "code",
- "execution_count": 51,
+ "execution_count": 84,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
- "Evaluation of the quatized model :\n",
- "eval Loss: 0.2306 Acc: 0.9477\n"
+ "Evaluation of the quantized model :\n",
+ "eval Loss: 0.2229 Acc: 0.9714\n"
]
},
{
"data": {
"text/plain": [
- "(0.23059894524368585, tensor(0.9477, dtype=torch.float64))"
+ "(0.22287905045918055, tensor(0.9714, dtype=torch.float64))"
]
},
- "execution_count": 51,
+ "execution_count": 84,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Now we can evaluate the model\n",
- "print('Evaluation of the quatized model :')\n",
+ "print('Evaluation of the quantized model :')\n",
"eval_model(quantized_model, criterion)"
]
},