Skip to content
Snippets Groups Projects
Select Git revision
  • main default protected
1 result

README.md

Blame
    • README.md 3.09 KiB

    Image classification

    k-nearest neighbors

    Artificial Neural Network

    We have \sigma(x) = \frac{1}{1+e^{-x}}

    ie \sigma(x)' = \frac{e^{-x}}{(1+e^{-x})^2}

    ie \sigma(x)' = \frac{1}{1+e^{-x}} * \frac{1+e^{-x}-1}{1+e^{-x}}

    ie \sigma(x)' = \frac{1}{1+e^{-x}} * (1-\frac{1}{1+e^{-x}})

    ie \sigma(x)' = \sigma(x) (1-\sigma(x))

    Calculating the partial derivative we have :

    \frac{\partial C}{\partial A^{(2)}}=\frac{2}{N_{out}} * (A^{(2)}-Y)

    We have :

    \frac{\partial C}{\partial Z^{(2)}} = \frac{\partial C}{\partial A^{(2)}} * \frac{\partial A^{(2)}}{\partial Z^{(2)}}

    but also :

    A^{(2)} = \sigma(Z^{(2)})

    So this gives :

    \frac{\partial A^{(2)}}{\partial Z^{(2)}} = \sigma'(Z^{(2)}) = \sigma(Z^{(2)}) (1-\sigma(Z^{(2)})) = A^{(2)} (1-A^{(2)})

    We then have :

    \frac{\partial C}{\partial Z^{(2)}} = \frac{\partial C}{\partial A^{(2)}} * A^{(2)} (1-A^{(2)})

    1. With the same method we have :

    \frac{\partial C}{\partial W^{(2)}} = \frac{\partial C}{\partial Z^{(2)}} \frac{\partial Z^{(2)}}{\partial W^{(2)}}

    and also : Z^{(2)} = W^{(2)} A^{(1)} + B^{(2)}

    So this gives :

    \frac{\partial C}{\partial W^{(2)}} = A^{(1)}.T matmul (\frac{\partial C}{\partial Z^{(2)}})

    1. With the same reasoning,

    \frac{\partial C}{\partial B^{(2)}} = \frac{\partial C}{\partial Z^{(2)}} \frac{\partial Z^{(2)}}{\partial B^{(2)}}

    and also :

    \frac{\partial C}{\partial B^{(2)}} = \frac{\partial C}{\partial Z^{(2)}} I_{N}

    \frac{\partial C}{\partial B^{(2)}} = \frac{\partial C}{\partial Z^{(2)}}

    1. We have :

    \frac{\partial C}{\partial A^{(1)}} = \frac{\partial C}{\partial Z^{(2)}} \frac{\partial Z^{(2)}}{\partial A^{(1)}}

    ie :

    \frac{\partial C}{\partial A^{(1)}} = \frac{\partial C}{\partial Z^{(2)}} mmul (W^{(2)}.T)

    \frac{\partial C}{\partial Z^{(1)}} = \frac{\partial C}{\partial A^{(1)}} \frac{\partial A^{(1)}}{\partial Z^{(1)}}

    ie :

    \frac{\partial C}{\partial Z^{(1)}} = \frac{\partial C}{\partial A^{(1)}} A^{(1)} (1-A^{(1)}) \frac{\partial C}{\partial Z^{(1)}} = \frac{\partial C}{\partial Z^{(2)}} mmul(W^{(2)}.T A^{(1)} (1-A^{(1)}))

    \frac{\partial C}{\partial W^{(1)}} = \frac{\partial C}{\partial Z^{(1)}} \frac{\partial Z^{(1)}}{\partial W^{(1)}}

    ie :

    \frac{\partial C}{\partial W^{(1)}} = A^{(0)} mmul (\frac{\partial C}{\partial Z^{(1)}})

    \frac{\partial C}{\partial B^{(1)}} = \frac{\partial C}{\partial Z^{(1)}} \frac{\partial Z^{(1)}}{\partial B^{(1)}}

    ie :

    \frac{\partial C}{\partial B^{(1)}} = \frac{\partial C}{\partial Z^{(1)}}

    Authors and acknowledgment

    Show your appreciation to those who have contributed to the project.

    License

    For open source projects, say how it is licensed.

    Project status

    If you have run out of energy or time for your project, put a note at the top of the README saying that development has slowed down or stopped completely. Someone may choose to fork your project or volunteer to step in as a maintainer or owner, allowing your project to keep going. You can also make an explicit request for maintainers.