Skip to content
Snippets Groups Projects
Select Git revision
  • main default protected
1 result

image-classification

  • Clone with SSH
  • Clone with HTTPS
    • user avatar
    Update README.md
    BaptisteBrd authored
    a414165b
    History
    user avatar a414165b
    History
    Name Last commit Last update
    data
    results
    .DS_Store
    README.md
    knn.py
    mlp.py
    read_cifar.py
    README.md

    Image classification

    The codes are available in the different .py files. This readme.md file will only present the mathematical part of the Artificial Neural Network.

    Artificial Neural Network

    We have \sigma(x) = \frac{1}{1+e^{-x}}

    ie \sigma(x)' = \frac{e^{-x}}{(1+e^{-x})^2}

    ie \sigma(x)' = \frac{1}{1+e^{-x}} * \frac{1+e^{-x}-1}{1+e^{-x}}

    ie \sigma(x)' = \frac{1}{1+e^{-x}} * (1-\frac{1}{1+e^{-x}})

    ie \sigma(x)' = \sigma(x) (1-\sigma(x))

    Calculating the partial derivative we have :

    \frac{\partial C}{\partial A^{(2)}}=\frac{2}{N_{out}} * (A^{(2)}-Y)

    We have :

    \frac{\partial C}{\partial Z^{(2)}} = \frac{\partial C}{\partial A^{(2)}} * \frac{\partial A^{(2)}}{\partial Z^{(2)}}

    but also :

    A^{(2)} = \sigma(Z^{(2)})

    So this gives :

    \frac{\partial A^{(2)}}{\partial Z^{(2)}} = \sigma'(Z^{(2)}) = \sigma(Z^{(2)}) (1-\sigma(Z^{(2)})) = A^{(2)} (1-A^{(2)})

    We then have :

    \frac{\partial C}{\partial Z^{(2)}} = \frac{\partial C}{\partial A^{(2)}} * A^{(2)} (1-A^{(2)})

    1. With the same method we have :

    \frac{\partial C}{\partial W^{(2)}} = \frac{\partial C}{\partial Z^{(2)}} \frac{\partial Z^{(2)}}{\partial W^{(2)}}

    and also : Z^{(2)} = W^{(2)} A^{(1)} + B^{(2)}

    So this gives :

    \frac{\partial C}{\partial W^{(2)}} = A^{(1)}.T matmul (\frac{\partial C}{\partial Z^{(2)}})

    1. With the same reasoning,

    \frac{\partial C}{\partial B^{(2)}} = \frac{\partial C}{\partial Z^{(2)}} \frac{\partial Z^{(2)}}{\partial B^{(2)}}

    and also :

    \frac{\partial C}{\partial B^{(2)}} = \frac{\partial C}{\partial Z^{(2)}} I_{N}

    \frac{\partial C}{\partial B^{(2)}} = \frac{\partial C}{\partial Z^{(2)}}

    1. We have :

    \frac{\partial C}{\partial A^{(1)}} = \frac{\partial C}{\partial Z^{(2)}} \frac{\partial Z^{(2)}}{\partial A^{(1)}}

    ie :

    \frac{\partial C}{\partial A^{(1)}} = \frac{\partial C}{\partial Z^{(2)}} mmul (W^{(2)}.T)

    \frac{\partial C}{\partial Z^{(1)}} = \frac{\partial C}{\partial A^{(1)}} \frac{\partial A^{(1)}}{\partial Z^{(1)}}

    ie :

    \frac{\partial C}{\partial Z^{(1)}} = \frac{\partial C}{\partial A^{(1)}} A^{(1)} (1-A^{(1)}) \frac{\partial C}{\partial Z^{(1)}} = \frac{\partial C}{\partial Z^{(2)}} mmul(W^{(2)}.T A^{(1)} (1-A^{(1)}))

    \frac{\partial C}{\partial W^{(1)}} = \frac{\partial C}{\partial Z^{(1)}} \frac{\partial Z^{(1)}}{\partial W^{(1)}}

    ie :

    \frac{\partial C}{\partial W^{(1)}} = A^{(0)} mmul (\frac{\partial C}{\partial Z^{(1)}})

    \frac{\partial C}{\partial B^{(1)}} = \frac{\partial C}{\partial Z^{(1)}} \frac{\partial Z^{(1)}}{\partial B^{(1)}}

    ie :

    \frac{\partial C}{\partial B^{(1)}} = \frac{\partial C}{\partial Z^{(1)}}

    Authors and acknowledgment

    Show your appreciation to those who have contributed to the project.

    License

    For open source projects, say how it is licensed.

    Project status

    If you have run out of energy or time for your project, put a note at the top of the README saying that development has slowed down or stopped completely. Someone may choose to fork your project or volunteer to step in as a maintainer or owner, allowing your project to keep going. You can also make an explicit request for maintainers.