Comment on AI

Speaking of neural networks... You all heard that the machine is only as smart as the smarts that man puts into it. Well, this is not true. A huge part of modern AI is dedicated to machine learning, when AI systems can learn and become MUCH smarter than all programmers put together. But... they learn differently and their “smarts” might be different from human. For example,the system charged with keeping water in NYC clean might start adding poison to the drinking water after concluding that the problem of dirty water is in human activity. It means that a bunch of other thoughts and moral judgments should be included in the systems that can potentially harm people.

Another anecdotal case of different ways that AI learns was the failure in DoD project, when the system had to take millions of photos made by American spy satellites and find the tanks on those photos. In the final analysis the system was ably to pinpoint tanks on many photos but NOT SEEING the obvious tanks on others. After a long study of human and computer psychologists of the cause of that failure they found out that the system was taught to understand that there is a tank by using photos done during sunny days. In some forms of AI it is possible to check how exactly the learning goes, what is the logic, etc. But in some forms of AI (like neural networks) nobody knows how the system learns and thinks. You just have to TRUST it. So it turned out that in the case of tank recognition system devised its own way (somewhat inhumane) in deciding if there is a tank there – by the presence of its shadow! Then if on a gloomy day there was no shadow – there was no tank.