Great post Gabe, I always like reading something that crystallises thoughts I've had in the back of my mind but haven't found the words for. The real question is how to harden vibes into something more concrete and testable, so looking forward to your next post...!
(this comment started by discussing using expected value to assess risk, then went off on a broader subject)
"speculative" is a worrying phrase to see in relation to extinction risk, i agree.
in general this would be prevented by expected-value-like-thinking, like, a *chance* of something REALLY bad is still bad. and the nerds shouting "pascal's mugging!" in my imagination should really shut up :p
a concrete example:
- a 0.1% chance of near-extinction (so 8b dead, without the added badness of losing all future human value) is worse than certainty of a million deaths. because ```8b*0.001=8m > 1m```.
but that just feels so cold and distant of a way of thinking, i don't know how to make that more communicable.
people (me too, maybe not as strongly as most, ) have visceral reactions when you use simple equations where "human lives" is one of the terms, but i really don't have a better way of saying "look dude, even a tiny chance of extinction is extremely bad and arguing about whether it's 99 or 60 or 0.01 is really stupid"
(some ideas/frameworks are simply surrounded by nasty connotations and there is just no good way of dealing with this. being immune to evil connotations is dangerous and you might accidentally turn into a nazi, but then, it's stupid to reject ideas just because someone can *accuse* you of being evil.)
(also im totally okay with worrying about a politician if he starts talking about race/gender/culture differences and EV and calculating human lives xD one's time on a big public platform (and in a position of power) is limited, and the way someone spends that definitely reflects on their deeper beliefs and intentions, much moreso than during loose internet discussions)
about lecunn (referring to your Bengio-Lecunn disagreement example):
i believe lecunn's tweets shouldn't be representative of what ai researchers think (not even as the fringe). they are so ridiculous that at best, it reflects what one can get away with saying and not be ridiculed and shamed off the internet, which reflects what the general public believes are non-insane takes.
(like if a geologist says earth is flat you will tag it as one man's insanity, not as "this is how difficult the science of geology is", and him being considered a serious geologist would become evidence for general societal ignorance)
^ (caveat, i only read his tweets and then dismissed him, maybe i should watch a video of him or read his longer writings, especially since ive been really confused about him for a while)
Great post Gabe, I always like reading something that crystallises thoughts I've had in the back of my mind but haven't found the words for. The real question is how to harden vibes into something more concrete and testable, so looking forward to your next post...!
(this comment started by discussing using expected value to assess risk, then went off on a broader subject)
"speculative" is a worrying phrase to see in relation to extinction risk, i agree.
in general this would be prevented by expected-value-like-thinking, like, a *chance* of something REALLY bad is still bad. and the nerds shouting "pascal's mugging!" in my imagination should really shut up :p
a concrete example:
- a 0.1% chance of near-extinction (so 8b dead, without the added badness of losing all future human value) is worse than certainty of a million deaths. because ```8b*0.001=8m > 1m```.
but that just feels so cold and distant of a way of thinking, i don't know how to make that more communicable.
people (me too, maybe not as strongly as most, ) have visceral reactions when you use simple equations where "human lives" is one of the terms, but i really don't have a better way of saying "look dude, even a tiny chance of extinction is extremely bad and arguing about whether it's 99 or 60 or 0.01 is really stupid"
(some ideas/frameworks are simply surrounded by nasty connotations and there is just no good way of dealing with this. being immune to evil connotations is dangerous and you might accidentally turn into a nazi, but then, it's stupid to reject ideas just because someone can *accuse* you of being evil.)
(also im totally okay with worrying about a politician if he starts talking about race/gender/culture differences and EV and calculating human lives xD one's time on a big public platform (and in a position of power) is limited, and the way someone spends that definitely reflects on their deeper beliefs and intentions, much moreso than during loose internet discussions)
ah damn, substack strips away empty spaces.. but im sorry for not taking taking this into account :/
i had the second-to-last paragraph at "1 tab" indentation, and the last one at "2 tabs".
about lecunn (referring to your Bengio-Lecunn disagreement example):
i believe lecunn's tweets shouldn't be representative of what ai researchers think (not even as the fringe). they are so ridiculous that at best, it reflects what one can get away with saying and not be ridiculed and shamed off the internet, which reflects what the general public believes are non-insane takes.
(like if a geologist says earth is flat you will tag it as one man's insanity, not as "this is how difficult the science of geology is", and him being considered a serious geologist would become evidence for general societal ignorance)
^ (caveat, i only read his tweets and then dismissed him, maybe i should watch a video of him or read his longer writings, especially since ive been really confused about him for a while)