I remember vividly my first encounter with computer programming. It was 1975 and it was a simple space-conquest game the kids played at school. Perhaps an Altair, perhaps running Altair BASIC. I wanted to add some rules so I opened the source and typed them in natural language. Of course the compiler barfed, and the lesson I drew from that experience was extremely to the point: "I thought computers were supposed to be smart. This is the stupidest thing I've ever seen."
It was a naive opinion but entirely right.
Now for twenty years I've programmed highly advanced "artificial intelligence" driving the complex, rich, imaginary world TriadCity. It's still the stupidest thing I've ever seen. It's brilliant and nuanced and entirely fake.
Humanity need no fear of computer "intelligence". The proper fear is computer stupidity, paired with the hubristic self-congratulation of programmers and architects and managers and defense officials and generals who genuinely imagine themselves capable of forecasting every conceivable scenario and use case and edge case and intersection. Where the end of the species comes not at the hand of Skynet, but of Dunning-Kruger.
