# Image classification ## k-nearest neighbors ## Artificial Neural Network 1. We have $`\sigma(x) = \frac{1}{1+e^{-x}}`$ ie $`\sigma(x)' = \frac{e^{-x}}{(1+e^{-x})^2}`$ ie $`\sigma(x)' = \frac{1}{1+e^{-x}} * \frac{1+e^{-x}-1}{1+e^{-x}}`$ ie $`\sigma(x)' = \frac{1}{1+e^{-x}} * (1-\frac{1}{1+e^{-x}})`$ ie $`\sigma(x)' = \sigma(x) (1-\sigma(x))`$ 2. Calculating the partial derivative we have : $`\frac{\partial C}{\partial A^{(2)}}=\frac{2}{N_{out}} * (A^{(2)}-Y)`$ 3. We have : $`\frac{\partial C}{\partial Z^{(2)}} = \frac{\partial C}{\partial A^{(2)}} * \frac{\partial A^{(2)}}{\partial Z^{(2)}} `$ but also : $`A^{(2)} = \sigma(Z^{(2)}) `$ So this gives : $`\frac{\partial A^{(2)}}{\partial Z^{(2)}} = \sigma'(Z^{(2)}) = \sigma(Z^{(2)}) (1-\sigma(Z^{(2)})) = A^{(2)} (1-A^{(2)}) `$ We then have : $`\frac{\partial C}{\partial Z^{(2)}} = \frac{\partial C}{\partial A^{(2)}} * A^{(2)} (1-A^{(2)}) `$ 4. With the same method we have : $`\frac{\partial C}{\partial W^{(2)}} = \frac{\partial C}{\partial Z^{(2)}} \frac{\partial Z^{(2)}}{\partial W^{(2)}} `$ and also : $`Z^{(2)} = W^{(2)} A^{(1)} + B^{(2)}`$ So this gives : $`\frac{\partial C}{\partial W^{(2)}} = A^{(1)}.T matmul (\frac{\partial C}{\partial Z^{(2)}}) `$ 5. With the same reasoning, $`\frac{\partial C}{\partial B^{(2)}} = \frac{\partial C}{\partial Z^{(2)}} \frac{\partial Z^{(2)}}{\partial B^{(2)}} `$ and also : $`\frac{\partial C}{\partial B^{(2)}} = \frac{\partial C}{\partial Z^{(2)}} I_{N} `$ $`\frac{\partial C}{\partial B^{(2)}} = \frac{\partial C}{\partial Z^{(2)}} `$ 6. We have : $`\frac{\partial C}{\partial A^{(1)}} = \frac{\partial C}{\partial Z^{(2)}} \frac{\partial Z^{(2)}}{\partial A^{(1)}} `$ ie : $`\frac{\partial C}{\partial A^{(1)}} = \frac{\partial C}{\partial Z^{(2)}} mmul (W^{(2)}.T)`$ 7. $`\frac{\partial C}{\partial Z^{(1)}} = \frac{\partial C}{\partial A^{(1)}} \frac{\partial A^{(1)}}{\partial Z^{(1)}} `$ ie : $`\frac{\partial C}{\partial Z^{(1)}} = \frac{\partial C}{\partial A^{(1)}} A^{(1)} (1-A^{(1)}) `$ $`\frac{\partial C}{\partial Z^{(1)}} = \frac{\partial C}{\partial Z^{(2)}} mmul(W^{(2)}.T A^{(1)} (1-A^{(1)})) `$ 8. $`\frac{\partial C}{\partial W^{(1)}} = \frac{\partial C}{\partial Z^{(1)}} \frac{\partial Z^{(1)}}{\partial W^{(1)}} `$ ie : $`\frac{\partial C}{\partial W^{(1)}} = A^{(0)} mmul (\frac{\partial C}{\partial Z^{(1)}})`$ 9. $`\frac{\partial C}{\partial B^{(1)}} = \frac{\partial C}{\partial Z^{(1)}} \frac{\partial Z^{(1)}}{\partial B^{(1)}} `$ ie : $`\frac{\partial C}{\partial B^{(1)}} = \frac{\partial C}{\partial Z^{(1)}}`$ ## Authors and acknowledgment Show your appreciation to those who have contributed to the project. ## License For open source projects, say how it is licensed. ## Project status If you have run out of energy or time for your project, put a note at the top of the README saying that development has slowed down or stopped completely. Someone may choose to fork your project or volunteer to step in as a maintainer or owner, allowing your project to keep going. You can also make an explicit request for maintainers.