The inductive bias, or learning bias, of a (machine) learning algorithm is the bias in the form of assumptions prior-knowledge.

This bias can effect the predicted outcome of the algorithm.

In general, what is a bias?

A bias is a leaning towards a certain viewpoint being right.


The term bias comes from the Greek philosopher Bias of Priene, one of the seven sages (c. 566 BC)

• He was renowned for his wisdom, his flawless judicial judgment and his eloquence.

Here are some of his quotations.

• It is better to decide a difference between enemies than friends, for one of our friends will certainly become an enemy and one of our enemies a friend.
• Most men are bad. (his motto, inscribed on Temple of Apollo at Delphi)

Can anyone be truly unbiased?

Any person who claims to be unbiased is biased towards being unbiased, and is therefore not telling the truth.

The appropriate question is, "What is the best bias with which to be biased.".

Can anyone really be tolerant of all viewpoints?


If anyone claims to be tolerant of all viewpoints, then are there viewpoints that will not be tolerated? Anyone who claims to be tolerant of all viewpoints is actually intolerant of any viewpoint that disagrees with their viewpoint. Therefore, anyone who claims to be tolerant of all viewpoints is actually intolerant and not telling the truth. The appropriate question is, "What are the things of which to be tolerant and what are the things of which to be intolerant.".

How you view something can depend on many factors.

• Can two people look at the same data and see different things?
• Can one person look at the same data and see different things?
• Can one person look at the same data and see what they want to see?

What do you see in this wire frame?


Are you looking from above the box, or from below the box.

The same data (i.e., the image) can be interpreted as the following.

• Looking from above the box.
• Looking from below the box.

Which is it? It depends on how you view the data, or, perhaps, on what you want to see.

By filling in some context, can you see the two views here more clearly?


On the left is viewing from above. On the right is viewing from below.

Is one view better, or more correct, than the other view?

If so, then are you showing a bias?

Everyone is biased, so the question comes down to "what is the best bias with which to be biased"?

Note that sometimes, filling in additional context, such as above, can help resolve ambiguities.

A bias is a belief that one particular viewpoint is right and that the opposing viewpoint is wrong.

Are you biased?

Everyone is biased, but some people do not admit or recognize it.

Have you ever been asked the question "You have to be tolerant of other viewpoints."?

What if you reply, "My viewpoint is that your viewpoint is wrong. Will you tolerate my viewpoint?".

Of course, this viewpoint will not be tolerated by the person asking you to be tolerant. Logically, insisting that someone to be tolerant shows that the person asking is very biased and intolerant.

So, the question is not whether a viewpoint is biased, but whether that bias is the best bias with which to be biased. And that bias is, to a large extent, dependent on the "world view" used.

The bias and tolerance issue is related to any of the following (paradox) sentences.

• This sentence is false.
• Is "no" the answer to this question? Answer either "yes" or "no".

In each case, a logical contradiction arises.

Thus, most arguments about tolerance are not logical (or have no logical solution, as in the above paradoxes).

One conclusion. Anyone who insists that you be tolerant of their view but will not tolerate your view and who claims to be unbiased is not reasoning logically. So, trying to reason logically with someone who is not using logic may not be very productive.

Occam's razor is a principle that the simplest explanation is often the best explanation.

To avoid an inductive bias, data is often (randomly) divided into training data (where the result is known) and test data (where the result is to be predicted without bias).

Over-fitting the data is an inductive bias where the answers have, essentially, been memorized. Any data different than what is "memorized" will not be recognized or predicted well.

Students over-fit when they memorize answers but cannot solve problems if anything changes.

Musicians who "memorize" a piece of music may sound great playing that one piece of music.

But give them any other piece of music and it becomes apparent that they are what is sometimes called a "one trick pony".

12 questions omitted (login required)