Activation Functions
A neuron that only fires 0 or 1 is too blunt for learning by small steps. Real networks pass the total through a smooth activation function. Sigmoid squeezes any number into the range 0 to 1 (handy as a probability). ReLU returns 0 for negative numbers and the number itself otherwise, and is the default in most modern networks.
Examplepython
Quick check
What range of values can sigmoid output?
Your turn: write the code
Print the ReLU of [-3, 2, -0.5, 7].
Finish the quick check (and any code exercise) to complete this lesson, or mark it done yourself.
Tip: use the left and right arrow keys to move between lessons.