Hmm… maybe we’re talking different aspects of AI here, possibly different audience too. I was talking general audience, to give them a general impression of the situation we’re looking at. To this end, I’ve been successful with analogies of this sort. To the extent that I have been explaining the underlying technical concepts, I certainly haven’t done so via analogy to the nuclear sector. Off the top of my head, I’ve said things like “like Skynet except not all done up for the movies”, stressing that Skynet is heavily dramatized and not realistic, and “computers take things very literally” which anyone who’s used a computer has some crude sense for. What would you recommend saying?

It may logically hold, but if you use it on the grounds that it’s understandable to non-scientists, you’re going to get people ignoring you (at best) or denouncing you as cultists (at worst). I can’t think of any technical concepts off the top of my head that can be truly understood solely through analogy.

It’s not what we want for nuclear either, so the analogy holds again. My favorite example of nuclear paranoia is “MRI” vs “NMR”. It doesn’t get much more clear-cut than that.

If we’re concerned that people will draw the wrong conclusions when we talk about AI, we should probably just not talk about it. But if we’re going to be talking anyways, then it is good to do so effectively. If you can propose a better strategy than these nuclear analogies, I’m all ears.

Meanwhile, this sounds like a good moment for teaching people rationality/bias/etc.

….

“AGI researchers want to do for the thought process what mechanical engineers have done for the locomotion process.”

I think this is a good expression, although to me it leads right down the “Why the future doesn’t need us” path. But perhaps this is a good thing, if that topic is well worth discussing. The bright side of it includes self-deleting CEV’s or thinking for fun (much like we now jog for fun).

A lot of the charlatans really can help you to ‘get rick quick’- with some ridiculously small probability like 1 in 500, so they can boast about it in testimonials. In general, any very large payoff will have a corresponding very large risk, but humans aren’t good at evaluating the average expected payoff from those parameters (just look at the lottery) and so we fall for it.

“these particular analogies work well.”

The danger of using analogies is that people will think of the analogy as an idea for itself rather than a metaphor for another idea. Or they’ll misinterpret the analogy as being a metaphor for something else entirely. A lot of people are quite frankly scared of anything nuclear, and they have succeeded in getting the government to oppose many new nuclear reactors, processing facilities and research programs regardless of merit, which is not what we want for AI.

When I meet skepticism of AGI’s downside (existential risk), I bring up the early days of nuclear weapons development. (See this for example.) The point is we don’t know and therefore caution must be exercised. Kuipers got that part right, although I’m told he wasn’t too savvy of the tech details when he wrote that.

Regarding the AGI upside (get-rich-quick), the same uncertainty still applies. Maybe compare it to controlled fusion, which wouldn’t solve the same set of problems as FAI (possibly a subset of problems if FAI gets us controlled, energy-productive fusion) but it would still solve many of our problems (possibly all of our energy problems for an astronomically long time). And like controlled, energy-productive fusion, we still don’t know how to build FAI. (See Fusion power for more on that.)

I find that drawing analogies like these helps people get a sense for what we’re looking at here. Since pretty much everyone that I talk to, including non-scientists, are familiar with nuclear weapons and at least loosely familiar with controlled fusion, these particular analogies work well. The analogies can also be starting points for contrasts (such as in protecting nuclear secrets vs AI secrets) in addition to comparisons.