results

.gitignore

@@ -3,4 +3,5 @@ image/*
 _pycache_/*
 plotting.py
 image-classification/
+cifar.PNG
 

README.md

-# Image Classification
+# Image Classification Project
 
+This project aims to implement an image classification program using two successive models: k-nearest neighbors (KNN) and artificial neural networks (ANN).
 
+## CIFAR-10 Dataset
 
-## Getting started
+The CIFAR-10 dataset is a commonly used database in computer vision for image classification. It consists of 60,000 color images of 32x32 pixels, distributed across 10 distinct classes, representing different objects or animals.
 
-To make it easy for you to get started with GitLab, here's a list of recommended next steps.
+![Semantic description of image](cifar.PNG)
+### CIFAR-10 Dataset Classes:
 
-Already a pro? Just edit this README.md and make it your own. Want to make it easy? [Use the template at the bottom](#editing-this-readme)!
+1. airplane
+2. automobile
+3. bird
+4. cat
+5. deer
+6. dog
+7. frog
+8. horse
+9. ship
+10. truck
 
-## Add your files
+## Accomplished Steps
 
-- [ ] [Create](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#create-a-file) or [upload](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#upload-a-file) files
-- [ ] [Add files using the command line](https://docs.gitlab.com/ee/gitlab-basics/add-file.html#add-a-file-using-the-command-line) or push an existing Git repository with the following command:
+### CIFAR Database Preparation
 
-```
-cd existing_repo
-git remote add origin https://gitlab.ec-lyon.fr/selalimi/image-classification.git
-git branch -M main
-git push -uf origin main
-```
+#### Database Download:
+The CIFAR-10 database was downloaded from [Dataset_Cifar-10](https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz).
 
-## Integrate with your tools
+#### Creating the data Folder:
+A folder named data was created to store the CIFAR-10 database files.
 
-- [ ] [Set up project integrations](https://gitlab.ec-lyon.fr/selalimi/image-classification/-/settings/integrations)
+#### Writing the read_cifar.py Script:
+A Python file named read_cifar.py was created, including the following functions:
+- `read_cifar_batch`
+- `read_cifar`
+- `split_dataset`
 
-## Collaborate with your team
+## Implementation of k-nearest neighbors (KNN)
 
-- [ ] [Invite team members and collaborators](https://docs.gitlab.com/ee/user/project/members/)
-- [ ] [Create a new merge request](https://docs.gitlab.com/ee/user/project/merge_requests/creating_merge_requests.html)
-- [ ] [Automatically close issues from merge requests](https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically)
-- [ ] [Enable merge request approvals](https://docs.gitlab.com/ee/user/project/merge_requests/approvals/)
-- [ ] [Set auto-merge](https://docs.gitlab.com/ee/user/project/merge_requests/merge_when_pipeline_succeeds.html)
+### Writing the knn.py Script:
+A Python file named knn.py was created, including the following functions:
+- `distance_matrix`
+- `knn_predict`
+- `evaluate_knn`
+- `plot_KNN`
 
-## Test and Deploy
+### Performance Study
+The effectiveness of the KNN algorithm was evaluated based on the number of neighbors (k) for `split_factor=0.9`.
 
-Use the built-in continuous integration in GitLab.
+### Running the KNN Code
 
-- [ ] [Get started with GitLab CI/CD](https://docs.gitlab.com/ee/ci/quick_start/index.html)
-- [ ] [Analyze your code for known vulnerabilities with Static Application Security Testing(SAST)](https://docs.gitlab.com/ee/user/application_security/sast/)
-- [ ] [Deploy to Kubernetes, Amazon EC2, or Amazon ECS using Auto Deploy](https://docs.gitlab.com/ee/topics/autodevops/requirements.html)
-- [ ] [Use pull-based deployments for improved Kubernetes management](https://docs.gitlab.com/ee/user/clusters/agent/)
-- [ ] [Set up protected environments](https://docs.gitlab.com/ee/ci/environments/protected_environments.html)
-
-***
+1. Run the script to split data
+```bash
+import read_cifar as rc
+X, y = rc.read_cifar('data') 
+# Split the Dataset
+X_train, y_train, X_test, y_test = rc.split_dataset(X, y, split=0.9) 
+```
 
-# Editing this README
+## Results :
+### Generating the Graph
+1. Results using KNN:
 
-When you're ready to make this README your own, just edit this file and use the handy template below (or feel free to structure it however you want - this is just a starting point!). Thank you to [makeareadme.com](https://www.makeareadme.com/) for this template.
+A graph showing the accuracy variation with k was generated using matplotlib and saved as "knn.png" in the "Results" folder. :
 
-## Suggestions for a good README
-Every project is different, so consider which of these sections apply to yours. The sections used in the template are suggestions for most open source projects. Also keep in mind that while a README can be too long and detailed, too long is better than too short. If you think your README is too long, consider utilizing another form of documentation rather than cutting out information.
 
-## Name
-Choose a self-explaining name for your project.
+![Semantic description of image](Results/knn.png)
 
-## Description
-Let people know what your project can do specifically. Provide context and add a link to any reference visitors might be unfamiliar with. A list of Features or a Background subsection can also be added here. If there are alternatives to your project, this is a good place to list differentiating factors.
+2. Results using ANN :
 
-## Badges
-On some READMEs, you may see small images that convey metadata, such as whether or not all the tests are passing for the project. You can use Shields to add some to your README. Many services also have instructions for adding a badge.
+A graph showing the accuracy variation with the number of epochs was generated using matplotlib and saved as "mlp.png" in the "Results" folder.
 
-## Visuals
-Depending on what you are making, it can be a good idea to include screenshots or even a video (you'll frequently see GIFs rather than actual videos). Tools like ttygif can help, but check out Asciinema for a more sophisticated method.
+![Semantic description of image](Results/mlp.png)
 
-## Installation
-Within a particular ecosystem, there may be a common way of installing things, such as using Yarn, NuGet, or Homebrew. However, consider the possibility that whoever is reading your README is a novice and would like more guidance. Listing specific steps helps remove ambiguity and gets people to using your project as quickly as possible. If it only runs in a specific context like a particular programming language version or operating system or has dependencies that have to be installed manually, also add a Requirements subsection.
+## Analysis of KNN Results
+Unfortunately, the performance of the KNN algorithm was disappointing, with accuracy ranging between 0.33 and 0.34 for different values of k (up to k=20). Several reasons may explain these mixed results:
 
-## Usage
-Use examples liberally, and show the expected output if you can. It's helpful to have inline the smallest example of usage that you can demonstrate, while providing links to more sophisticated examples if they are too long to reasonably include in the README.
+1. **High Dimensionality of Data**: CIFAR-10 dataset images are 32x32 pixels, resulting in high-dimensional data. This can make Euclidean distance less discriminative, affecting KNN's performance.
 
-## Support
-Tell people where they can go to for help. It can be any combination of an issue tracker, a chat room, an email address, etc.
+2. **Scale Sensitivity**: KNN is sensitive to different feature scales. Pixels in an image can have different values, and KNN may be influenced by these disparities.
 
-## Roadmap
-If you have ideas for releases in the future, it is a good idea to list them in the README.
+3. **Choice of k**: The choice of the number of neighbors (k) can significantly influence results. An inappropriate k value can lead to underestimation or overestimation of the model's complexity.
 
-## Contributing
-State if you are open to contributions and what your requirements are for accepting them.
+4. **Lack of Feature Abstraction**: KNN directly uses pixels as features. More advanced feature extraction techniques could improve performance
 
-For people who want to make changes to your project, it's helpful to have some documentation on how to get started. Perhaps there is a script that they should run or some environment variables that they need to set. Make these steps explicit. These instructions could also be useful to your future self.
+ ## Analysis of ANN Results
+The deep learning algorithm (ANN) used for our dataset has relatively low performance, with test set accuracy plateauing around 0.098 over 100 epochs.
 
-You can also document commands to lint the code or run tests. These steps help to ensure high code quality and reduce the likelihood that the changes inadvertently break something. Having instructions for running tests is especially helpful if it requires external setup, such as starting a Selenium server for testing in a browser.
+These results suggest that adjustments to certain aspects of the model, such as complexity, hyperparameters, or weight initialization, may be necessary to improve its ability to generalize to new data. Further exploration of these aspects could be beneficial in optimizing model performance.
 
-## Authors and acknowledgment
-Show your appreciation to those who have contributed to the project.
+ ## Conculsion 
+ The best accuracy is achieved with the KNN model, reaching 36%. However, it could be further improved by using Convolutional Neural Networks (CNN) instead of Artificial Neural Networks (ANN). CNNs are particularly recognized for their effectiveness in image recognition, analysis, and classification of images and videos.
 
-## License
-For open source projects, say how it is licensed.
+## Author
+Sara EL ALIMI
 
-## Project status
-If you have run out of energy or time for your project, put a note at the top of the README saying that development has slowed down or stopped completely. Someone may choose to fork your project or volunteer to step in as a maintainer or owner, allowing your project to keep going. You can also make an explicit request for maintainers.
+## Licence
+Ce projet est sous licence MIT.

mlp.py

@@ -32,7 +32,7 @@ def sigmoid(x,derivate):
 # Define the softmax activation function
 def softmax(x,derivate):
     if derivate == False :
-      return np.exp(x) / np.exp(np.array(x)).sum(axis=-1, keepdims=True)
+      return np.exp(x) / np.exp(np.array(x)).sum(axis=1, keepdims=True)
     else :
         return x*(1-x)
     
@@ -94,7 +94,7 @@ def learn_once_mse(W1, b1, W2, b2, data, targets, learning_rate):
 
     # Update weights and biases of the output layer
     W2 = W2 - learning_rate * np.dot(hidden_layer_output.T, output_layer_gradients) / data.shape[0]
-    b2 = b2 - learning_rate * (1 / hidden_layer_output.shape[1]) * output_layer_gradients.sum(axis=0, keepdims=True)
+    b2 = b2 - learning_rate * (1 / hidden_layer_output.shape[1]) * output_layer_gradients.sum(axis=0)
 
     # Calculate the error at the hidden layer
     hidden_layer_error = np.dot(output_layer_gradients, W2.T)
@@ -104,7 +104,7 @@ def learn_once_mse(W1, b1, W2, b2, data, targets, learning_rate):
 
     # Update weights and biases of the hidden layer
     W1 = W1 - learning_rate * np.dot(data.T, hidden_layer_gradients) / data.shape[0]
-    b1 = b1 - learning_rate * (1 / data.shape[1]) * hidden_layer_gradients.sum(axis=0, keepdims=True)
+    b1 = b1 - learning_rate * (1 / data.shape[1]) * hidden_layer_gradients.sum(axis=0)
 
     # Calculate the loss using the specified metric
     loss = loss_metrics(output_layer_output, targets,metric="MSE",status="forward")

test/test_knn.py

+import numpy as np
+import knn 
+
+# Test de la fonction distance_matrix :
+def test_distance_matrix():
+    X = np.array([[1, 2], [3, 4]])
+    Y = np.array([[2, 2], [1, 1]])
+    dists = knn.distance_matrix(X, Y)
+    assert dists.shape == (2, 2)
+    assert np.allclose(dists, np.array([[2, 5], [10, 1]]))
+    print("Test for distance_matrix passed.")
+    
+
+# Test de la fonction knn_predict :
+def test_knn_predict():
+    dists = np.array([[2, 5], [10, 1]])
+    labels_train = np.array([0, 1])
+    k = 1
+    y_pred = knn.knn_predict(dists, labels_train, k)
+    assert y_pred.shape == (2,)
+    assert np.array_equal(y_pred, np.array([0, 1]))
+    print("Test for knn_predict passed.")
+
+# Test de la fonction evaluate_knn :
+def test_evaluate_knn_accuracy():
+    data_train = np.array([[1, 2], [3, 4], [5, 6], [1, 1], [2, 2]])
+    labels_train = np.array([0, 1, 2, 0, 1])
+    data_test = np.array([[2, 2], [1, 1], [3, 3]])
+    labels_test = np.array([1, 0, 1])
+    k = 2
+    accuracy = knn.evaluate_knn(data_train, labels_train, data_test, labels_test, k)
+    assert 0 <= accuracy <= 1
+    print("Test for evaluate_knn accuracy passed.")

test/test_mlp.py

+import numpy as np
+
+# Importez les  fonctions 
+from mlp import initialization, train_mlp, calculate_accuracy
+
+def test_mlp_training():
+    # Paramètres du test
+    num_samples = 200
+    num_features = 10
+    num_classes = 3
+    num_hidden_units = 5
+    learning_rate = 0.1
+    num_epochs = 10
+
+    # Générez des données factices pour le test
+    X_train = np.random.randn(num_samples, num_features)
+    y_train = np.random.randint(0, num_classes, num_samples)
+    X_test = np.random.randn(num_samples, num_features)
+    y_test = np.random.randint(0, num_classes, num_samples)
+
+    # Initialisez les poids et les biais
+    W1, b1, W2, b2 = initialization(num_features, num_hidden_units, num_classes)
+
+    # Entraînez le modèle
+    train_accuracies, test_accuracy = train_mlp(W1, b1, W2, b2, X_train, y_train, learning_rate, num_epochs)
+
+    # Vérifiez si l'accuracy est un nombre entre 0 et 1
+    assert 0 <= test_accuracy <= 1
+
+    # Vérifiez si la longueur de la liste des accuracies d'entraînement correspond au nombre d'époques
+    assert len(train_accuracies) == num_epochs
+
+    # Vérifiez si les accuracies d'entraînement sont des nombres entre 0 et 1
+    for accuracy in train_accuracies:
+        assert 0 <= accuracy <= 1
+
+    print("Tous les tests ont réussi avec succès.")
+
+

test/test_read_cifar.py

+import numpy as np
+import read_cifar as rc
+from read_cifar import read_cifar_batch
+from read_cifar import read_cifar
+def test_read_cifar_batch():
+   # Test read_cifar_batch function
+   batch_path = "data\data_batch_1"
+   data, labels = read_cifar_batch(batch_path)
+
+   # Check that data has the right shape and type
+   assert data.shape == (10000, 3072)
+   assert data.dtype == np.float32
+
+   # Check that labels has the right shape and type
+   assert labels.shape == (10000,)
+   assert labels.dtype == np.int64
+   print("All tests passed successfully.")
+   
+
+
+def  test_read_cifar():
+    # Test read_cifar function
+    data, labels = read_cifar('data')
+
+    # Check that data has the right shape and type
+    assert data.shape == (60000, 3072)
+    assert data.dtype == np.float32
+
+    # Check that labels has the right shape and type
+    assert labels.shape == (60000,)
+    assert labels.dtype == np.int64
+    print("All tests passed successfully.")
+
+def test_split_dataset():
+    data = np.random.randn(150, 4)
+    labels = np.random.randn(150)
+    split = 0.8
+    data_train, labels_train, data_test, labels_test = rc.split_dataset(data, labels, split)
+
+    total_size = data_train.shape[0] + data_test.shape[0]
+
+    assert total_size == len(data)
+    assert len(labels_train) == len(data_train)
+    assert len(labels_test) == len(data_test)
+    
+    print("All tests passed successfully.")