# Image classification

## k-nearest neighbors



## Artificial Neural Network

1.

 We have $`\sigma(x) = \frac{1}{1+e^{-x}}`$

ie $`\sigma(x)' = \frac{e^{-x}}{(1+e^{-x})^2}`$

ie $`\sigma(x)' = \frac{1}{1+e^{-x}} * \frac{1+e^{-x}-1}{1+e^{-x}}`$

ie $`\sigma(x)' = \frac{1}{1+e^{-x}} * (1-\frac{1}{1+e^{-x}})`$

ie $`\sigma(x)' = \sigma(x) (1-\sigma(x))`$

2. 

Calculating the partial derivative we have :

$`\frac{\partial C}{\partial A^{(2)}}=\frac{2}{N_{out}} * (A^{(2)}-Y)`$

3. 

We have :

 $`\frac{\partial C}{\partial Z^{(2)}} = \frac{\partial C}{\partial A^{(2)}} * \frac{\partial A^{(2)}}{\partial Z^{(2)}} `$

   but also :

   $`A^{(2)} = \sigma(Z^{(2)}) `$

   So this gives : 

   $`\frac{\partial A^{(2)}}{\partial Z^{(2)}} = \sigma'(Z^{(2)}) = \sigma(Z^{(2)}) (1-\sigma(Z^{(2)})) = A^{(2)} (1-A^{(2)}) `$

We then have : 

$`\frac{\partial C}{\partial Z^{(2)}} = \frac{\partial C}{\partial A^{(2)}} * A^{(2)} (1-A^{(2)}) `$

4. With the same method we have :

$`\frac{\partial C}{\partial W^{(2)}} = \frac{\partial C}{\partial Z^{(2)}} \frac{\partial Z^{(2)}}{\partial W^{(2)}} `$

  and also : $`Z^{(2)} = W^{(2)} A^{(1)} + B^{(2)}`$ 

  So this gives :

  $`\frac{\partial C}{\partial W^{(2)}} = A^{(1)}.T matmul (\frac{\partial C}{\partial Z^{(2)}})  `$

5. With the same reasoning, 

$`\frac{\partial C}{\partial B^{(2)}} = \frac{\partial C}{\partial Z^{(2)}} \frac{\partial Z^{(2)}}{\partial B^{(2)}} `$

and also :

$`\frac{\partial C}{\partial B^{(2)}} = \frac{\partial C}{\partial Z^{(2)}} I_{N} `$

$`\frac{\partial C}{\partial B^{(2)}} = \frac{\partial C}{\partial Z^{(2)}} `$

6. We have :

$`\frac{\partial C}{\partial A^{(1)}} = \frac{\partial C}{\partial Z^{(2)}} \frac{\partial Z^{(2)}}{\partial A^{(1)}} `$

ie :

$`\frac{\partial C}{\partial A^{(1)}} = \frac{\partial C}{\partial Z^{(2)}} mmul (W^{(2)}.T)`$

7. 

$`\frac{\partial C}{\partial Z^{(1)}} = \frac{\partial C}{\partial A^{(1)}} \frac{\partial A^{(1)}}{\partial Z^{(1)}} `$

ie :

$`\frac{\partial C}{\partial Z^{(1)}} = \frac{\partial C}{\partial A^{(1)}} A^{(1)} (1-A^{(1)}) `$
$`\frac{\partial C}{\partial Z^{(1)}} = \frac{\partial C}{\partial Z^{(2)}} mmul(W^{(2)}.T A^{(1)} (1-A^{(1)})) `$

8. 

$`\frac{\partial C}{\partial W^{(1)}} = \frac{\partial C}{\partial Z^{(1)}} \frac{\partial Z^{(1)}}{\partial W^{(1)}} `$

ie :

$`\frac{\partial C}{\partial W^{(1)}} =  A^{(0)} mmul (\frac{\partial C}{\partial Z^{(1)}})`$

9. 

$`\frac{\partial C}{\partial B^{(1)}} = \frac{\partial C}{\partial Z^{(1)}} \frac{\partial Z^{(1)}}{\partial B^{(1)}} `$

ie : 

$`\frac{\partial C}{\partial B^{(1)}} = \frac{\partial C}{\partial Z^{(1)}}`$
  


  






## Authors and acknowledgment
Show your appreciation to those who have contributed to the project.

## License
For open source projects, say how it is licensed.

## Project status
If you have run out of energy or time for your project, put a note at the top of the README saying that development has slowed down or stopped completely. Someone may choose to fork your project or volunteer to step in as a maintainer or owner, allowing your project to keep going. You can also make an explicit request for maintainers.