Speaking of neural networks... You all heard that the machine is only as smart
as the smarts that man puts into it. Well, this is not true. A huge part of
modern AI is dedicated to machine learning, when AI systems can learn and become
MUCH smarter than all programmers put together. But... they learn differently
and their “smarts” might be different from human. For example,the system charged
with keeping water in NYC clean might start adding poison to the drinking water
after concluding that the problem of dirty water is in human activity. It means
that a bunch of other thoughts and moral judgments should be included in the
systems that can potentially harm people.
Another anecdotal case of different ways that
AI learns was the failure in DoD project, when the system had to take millions
of photos made by American spy satellites and find the tanks on those photos. In
the final analysis the system was ably to pinpoint tanks on many photos but NOT
SEEING the obvious tanks on others. After a long study of human and computer
psychologists of the cause of that failure they found out that the system was
taught to understand that there is a tank by using photos done during sunny
days. In some forms of AI it is possible to check how exactly the learning goes,
what is the logic, etc. But in some forms of AI (like neural networks) nobody
knows how the system learns and thinks. You just have to TRUST it. So it turned
out that in the case of tank recognition system devised its own way (somewhat
inhumane) in deciding if there is a tank there – by the presence of its shadow! Then if on a
gloomy day there was no shadow – there was no tank.