Given Summed Input:

x =

Instead of threshold, and fire/not fire,

we could have *continuous* output y
according to the *sigmoid* function:

Note
e
and its
properties.

As x goes to minus infinity,
y goes to 0 (tends not to fire).

As x goes to infinity,
y goes to 1 (tends to fire):

At x=0, y=1/2

We can make this more and more threshold-like, or step-like, by increasing the weights on the links, and so increasing the summed input:

For any *non-zero* w, no matter how close to 0,
ς(wx)
will eventually be
asymptotic
to the lines y=0 and y=1.

Is this linear? Let's change the scale:

This is exactly same function.

So it's not actually linear, but note that
**within the range -6 to 6**
we can *approximate * a linear function with slope.

If x *will * always be within that range
then for all practical purposes we have
linear output with slope.

Or try this:

Is this linear? Let's change the scale:

This is exactly same function.

So we can always get, within that range, an

e.g. *Given * x will be from -30 to 30:

Approximation of any linear function so long as y stays in [0,1]

And centred on zero. To centre other than zero see below.

We can also, by changing the *sign* of the weights,
make large positive actual input
lead to large negative summed input
and hence no fire,
and large negative actual input lead to fire.

This is of course
a threshold-like function
still centred on *zero*.
To centre it on any threshold we use:

y = ς(x-t)

where t is the threshold for this node.
This threshold value is something that is *learnt*,
along with the weights.

The "threshold" is now
the *centre point of the curve*,
rather than an all-or-nothing value.

Can be linear constant y=c, c between 0 and 1. We already saw y=1/2. Can we have other y=c?

By setting a=0,
y=ς(b) constant for all x

By varying b,
we can have constant output y=c
for any c between 0 and 1.

d/dx (fg) = f (dg/dx) + g (df/dx)

Quotient Rule:

d/dx (f/g) = ( g (df/dx) - f (dg/dx) ) / g^{2}

The slope is greatest where? And least where?

To prove this, take the

d/dy ( y (1-y) )

= y (-1) + (1-y) 1

= -y + 1 -y

= 1 - 2y

= 0 for y = 1/2

This is a maximum. There is no minimum.

y = ς(ax+b)a positive or negative, fraction or multiple

b positive or negative

y = ς(z) where z = ax+b

dy/dx = dy/dz dz/dx

= y(1-y) a

if a positive, all slopes are positive,
steepest slope (highest positive slope) is at y = 1/2

if a negative, all slopes are negative,
steepest slope (lowest negative slope) is at y = 1/2

i.e. Slope is different value, but still steepest at y = 1/2