The artificial neuron: should the robot stop?

Two sensor numbers go in, one "how sure am I that I must stop?" comes out. You'll build the neuron from a simple weighted vote, find out why it needs a squash, and then let it learn its own weights with the same trick you used in Lesson 3.

The rest is the full lesson: the problem in detail, the idea built from nothing, a demo to play with, then the homework and a challenge away from the screen.

1. The problem

The robot's front sensor gives two numbers every moment:

• x1 = distance to the thing ahead, in metres;
• x2 = closing speed: how fast that distance is shrinking, in metres per second.

We want one answer: the probability that the robot must stop. In the simulated world, the true rule is "stop if distance − closing speed < 1", which means: if in one second you'd be closer than 1 metre, brake. But the robot doesn't know this rule. And, as you saw in Lesson 2, its sensor is noisy, so near the line the readings are genuinely confusing.

The obvious approach is to hand-write rules: if distance < 1: stop. Try it on a few examples:

distance  speed   true answer   "distance < 1" says
  0.5      0.0       stop           stop   ✓
  4.0      0.2       go             go     ✓
  1.5      1.0       stop           go     ✗  (it'll be 0.5 m away in one second!)
  1.8      1.9       stop           go     ✗

One clue is not enough. We need to combine both clues, and we want the computer to work out how.

2. First principles

Step 1: a weighted vote

Give each input a weight, multiply, add, and add a bias. Call the total z:

z = w1 * distance + w2 * speed + bias

Rule: if z is above 0, stop. Let's choose weights by thinking. Far away should push toward "go", so w1 is negative. Closing fast should push toward "stop", so w2 is positive. Try w1 = −1, w2 = +1, bias = 1:

(0.5, 0.0):  z = −1×0.5 + 1×0.0 + 1 =  0.5   → above 0 → stop ✓
(4.0, 0.2):  z = −1×4.0 + 1×0.2 + 1 = −2.8   → below 0 → go   ✓
(1.5, 1.0):  z = −1×1.5 + 1×1.0 + 1 =  0.5   → above 0 → stop ✓
(1.8, 1.9):  z = −1×1.8 + 1×1.9 + 1 =  1.1   → above 0 → stop ✓

All four right. Look closely: −distance + speed + 1 > 0 is the same as distance − speed < 1. We've rebuilt the true rule as a weighted vote. A neuron with the right weights is the rule.

Step 2: why squash?

z can be any number: −2.8, 0.5, 47. That's awkward. Is 47 "very sure" or just "sure"? And "above 0 → stop" is a hard switch: change a weight a tiny bit and the answer either flips completely or doesn't change at all. You can't learn by nudging something that never moves smoothly. (In Lesson 3 you nudged downhill on a smooth error curve. A switch has no slope to follow.)

So we squash z through the sigmoid:

sigmoid(z) = 1 / (1 + e^(−z))          e ≈ 2.718

   z     sigmoid(z)
  −4       0.018      almost certainly "go"
  −2       0.119
  −1       0.269
   0       0.5        exactly unsure
   1       0.731
   2       0.881
   4       0.982      almost certainly "stop"

Big negative → close to 0. Big positive → close to 1. Zero → 0.5. It's smooth, so a tiny change to a weight makes a tiny change to the answer, and we can learn. And the output is a probability, which Lesson 5 will turn into a careful stop decision.

Computer gotcha: for z = −1000, e^(−z) = e^1000, which is too big for Python: math.exp(1000) raises OverflowError. The fix is the same maths written differently: when z < 0, use e^z / (1 + e^z). Multiply top and bottom of the original by e^z to see they're equal.

So a neuron is just:

probability of stop = sigmoid(w1 * x1 + w2 * x2 + bias)

Step 3: learning by nudging

We guessed the weights. Real robots have dozens of inputs, so we can't guess. We want the weights to come from examples. Start with every weight at 0 and look at one example at a time:

1. Ask the neuron: p = neuron(weights, bias, x).
2. Compare with the truth: error = p − label. (Label is 1 for stop, 0 for go.)
3. Nudge each weight: w_i = w_i − lr × error × x_i, and the bias: bias = bias − lr × error.

Why error × input? If the neuron said 0.9 but the answer was 0 (go), the error is +0.9, so we pull weights down. If an input was big, it was more to blame, so its weight moves more. If an input was 0, it didn't contribute, so its weight doesn't move at all. That's fair blame.

This is Lesson 3 again. There your line's gradient was dw = average of 2 × (predict − y) × x. Same shape: error times input. The differences: we update after every example instead of averaging all of them first, and the "2" is gone because for probabilities we measure error in a slightly different way that makes the maths cancel neatly. It is still gradient descent: walk downhill on the error.

Worked example, by hand

The first two training examples the Python file generates are (7.81, 1.93) label 0, and (6.52, 1.28) label 0. Learning rate lr = 0.05. Start: w = [0, 0], bias = 0.

Example 1: x = (7.81, 1.93), label 0
  z = 0×7.81 + 0×1.93 + 0 = 0       p = sigmoid(0) = 0.5
  error = 0.5 − 0 = 0.5
  w1 = 0 − 0.05 × 0.5 × 7.81 = −0.195
  w2 = 0 − 0.05 × 0.5 × 1.93 = −0.048
  bias = 0 − 0.05 × 0.5      = −0.025

Example 2: x = (6.52, 1.28), label 0
  z = −0.195×6.52 + −0.048×1.28 − 0.025 = −1.36   p = sigmoid(−1.36) = 0.204
  error = 0.204 − 0 = 0.204
  w1 = −0.195 − 0.05 × 0.204 × 6.52 = −0.262
  w2 = −0.048 − 0.05 × 0.204 × 1.28 = −0.061
  bias = −0.025 − 0.05 × 0.204       = −0.035

Notice the second error is smaller (0.204 instead of 0.5): the neuron already learned "far away means go".

What did it learn?

The neuron is exactly 50/50 when z = 0. With the learned weights:

−4.31 × distance + 3.53 × speed + 5.13 = 0
→ distance = 1.19 + 0.82 × speed       (learned boundary)
  distance = 1.00 + 1.00 × speed       (true rule)

Close, and it found this from nothing but labelled examples. It also leans toward stopping a bit early (1.19 m instead of 1.00 m). The bigger weights also make it more confident: the hand-made weights give 0.62 for (0.5, 0.0); the learned ones give 0.95.

3. Play with it

Things to try: set all weights to 0 (everything is 0.5, the line disappears). Press "Train 1 epoch" a few times and watch the line swing into place. Then make w2 negative: what happens to fast-approaching obstacles?

4. Homework

Open project/homework/lesson04_neuron.py. make_obstacle_data is written for you. Write these four functions, in order:

4.1 sigmoid(z)

Return 1 / (1 + e^(−z)), without crashing for huge negative z. Example: sigmoid(0) is 0.5; sigmoid(2) is about 0.881; sigmoid(-1000) is 0.0 (not a crash).

4.2 neuron(weights, bias, x)

Weighted sum plus bias, then sigmoid. weights and x are the same length (2 here, but don't hard-code 2). Example: neuron([-1, 1], 1, (1.5, 1.0)) → z = 0.5 → about 0.622.

4.3 train_neuron(data, lr=0.05, epochs=200)

Start at weights = [0.0, 0.0], bias = 0.0. For each epoch, for each (x, label) in data: compute p, the error, and nudge every weight and the bias as in Step 3. Return (weights, bias). Check your first two updates against the worked example above.

4.4 should_stop(weights, bias, x, threshold=0.5)

Return True if the stop-probability is at least the threshold. Example: with all-zero weights the probability is exactly 0.5, so should_stop([0, 0], 0, (1, 1), 0.5) is True, but with threshold 0.6 it's False.

Run it

cd homework
python3 lesson04_neuron.py

What you should see (this is the real output of the reference solution):

weights [-4.31, 3.53] bias 5.13
(4.0, 0.2) stop prob 0.0
(1.5, 1.0) stop prob 0.9
(0.5, 0.0) stop prob 0.95

Read it: far and slow → 0.0 (go). 1.5 m away but closing at 1 m/s → 0.9 (stop!). Half a metre away, even standing still → 0.95 (stop).

Check it

cd ..
python3 check.py 4

Each ✓ is a passed test. A ✗ tells you what went wrong, for example:

  ✗ sigmoid: sigmoid(-1000) crashed or went negative: handle big negative z
  ✗ should_stop: should_stop compares the probability with the threshold (>=)

A yellow … means that function still raises NotImplementedError: you haven't written it yet. When you see 4/4 passed 🎉 lesson complete!, you're done.

Stretch

• Print the weights after epochs 1, 5, 20 and 200. Does the boundary distance = −(w2 × speed + bias) / w1 move toward 1 + speed?
• Try lr=0.5 and lr=0.001. Which one gets to good weights faster? Does either misbehave?
• Add a third input, the robot's own speed, set to random noise. Does training learn to give it a weight near 0? (It should: useless clues get ignored.)

5. Real-world challenge: be a neuron for a week

Train a neuron on your own decisions, by hand, with paper.

1. Pick a yes/no decision you make every day with two number clues. Good one: "do I take a coat?" with x1 = the chance of rain from a weather app (as 0 to 1, so 40% → 0.4) and x2 = the temperature in tens of degrees (14°C → 1.4).
2. Collect 10 examples. For 10 days (or 10 hours from an hourly forecast, or ask your family about the last 10 days), write x1, x2 and your honest answer: 1 = took a coat, 0 = didn't.
3. Train by hand. Start at w1 = w2 = bias = 0, lr = 0.5. For each row: compute z, look up sigmoid(z) from the table in Step 2 (round z to the nearest one listed, or use a calculator), compute the error, nudge. Do two passes.
4. Record: your final weights, and how many of your 10 decisions the neuron now gets right (probability ≥ 0.5 means "coat").
5. Explain the weights. Is the rain weight positive? Is the temperature weight negative? Which clue matters more to you, according to your own neuron?

What this teaches: a neuron's weights are a summary of a person's (or a robot's) habits, and you can read them. You'll also feel why computers do this part: two passes over 10 rows by hand is already a lot of arithmetic. The robot did 200 passes over 450 rows.

6. Your Robo-Helper so far

Robo-Helper can move (Lesson 1), sense distance (Lesson 2) and knows its braking distance (Lesson 3). Now it has an obstacle detector: train_neuron learns the weights, should_stop turns a sensor reading into a decision.

• Lesson 5 asks "is it any good?": it tests this neuron on examples it never saw, and safest_threshold picks a threshold lower than 0.5 so it almost never misses a needed stop.
• Lesson 11 (the capstone) calls train_neuron when Robo-Helper starts up, and in drive_to calls should_stop(..., (0.5, speed), threshold) whenever something unexpected appears one square ahead. If it says stop, the robot brakes and plans a new route.

7. Common mistakes

• Crash on big negative numbers. math.exp(1000) overflows. The checker says: "sigmoid(-1000) crashed or went negative: handle big negative z". Use the two-branch formula.
• Using > instead of >= in should_stop. At exactly 0.5 the answer must be True.
• Forgetting the bias update, or updating the bias with error × x. The bias acts like a weight on an input that is always 1, so its nudge is just lr × error.
• Getting the sign backwards: error = label − p with w −= ... makes the weights run uphill. Then "an obstacle 0.5 m away should give stop-probability > 0.8" fails. Use p − label and subtract.
• Resetting weights inside the epoch loop. Set them to zero once, before both loops.

8. Check yourself

9. Go deeper (optional)

• Day 28 · CS231n: stack many neurons into layers and you get a neural network that can see images.
• TensorTonic: practice problems that drill the same maths (sums, sigmoids, gradients) one step further.
• Day 08 · AlexNet: the 2012 network, made of millions of these neurons, that started the deep-learning boom.
• Free robotics courses if you want to see how real robots combine sensors and learned detectors.