Thank you, very interesting, looking forward to see how you build further topics on this. Also, "death by a thousand cuts" felt very close to my experience.. Would be cool if you share some day what you personally do to avoid that. Is that possible at all, or is it our inevitable end?
I do think it is possible for someone who makes it their focus to avoid that. Otherwise, one would need to be lucky enough to stumble on it, or so unlucky as to die young.
Halfway through this article I went, "oh wait, ANY arbitrary thing you spend a lot of time with becomes part of your identity and you get all these opinions and stances that don't matter at all, I wonder what I'm doing in life that is as irrelevant and wasteful as what the wine and politics guys are doing"
I've started a category in my notes, titled "aesthetics", after reading this. (And I wrote things I value, and things I pride myself on)
------------------
About Ontological Nudges:
I already like to think in terms of symbolism and metaphor and use them when trying to figure stuff out and get the right perspective.
But Ontological Nudges feels deeper than that. I'll try to see which general concepts I use to perceive the world.
What are your thoughts about cog sec at societal level? Are we cooked? Seems like We, human species, develop some antibodies to the more obvious things: people recognize ragebait, terrible attempts at red herrings from politicians, AI slop. But at the same time it seems like it's a product of those being still in their infancy.
If anything, I've seen way too many times people getting overconfident because they can detect obvious cases of cognitive hazards and then assume they are immune to them.
And it seems technology is about to enable things much worse.
We are extremely underprepared, but I don't even know where you would start to make people prepared?
Have you seen any positive examples of progress here? And do you feel like there's a way forward with this?
Also I realize this might require a whole other post to write out, so even some pointers would be super cool!
You touch on many things that I have partial drafts on.
> What are your thoughts about cog sec at societal level?
We do not have reliable mechanisms to share important knowledge at a societal level. Consider that most people do not know about https://safe.ai/work/statement-on-ai-risk
Let alone for advanced concepts like "cog sec".
> Are we cooked?
No!
On one hand, I believe it is possible to build such mechanisms. [1]
On the other hand, I don't believe it is even _needed_. It is enough for live players who act on the interests of others to internalise good cog sec principles.
> If anything, I've seen way too many times people getting overconfident because they can detect obvious cases of cognitive hazards and then assume they are immune to them.
> And it seems technology is about to enable things much worse.
I have a attempted to write a few articles about this. Titles currently in my draft folder "AI Poisoning", "AI Personification & InstructGPT", "I was wrong, AI Poisoning is much worse than I expected."
I am still trying to write a good one on the topic. Until I succeed, I'll write easier ones and work up my way there.
---
[1]:
Imagine a Social Media primitive that is about signalling that one is familiar with a written piece. Upon this, we'd build a profile of all the pieces one is familiar with, and show what is general knowledge within various peer groups.
This would let us systematically transform general knowledge (what everyone knows) into common knowledge (what the group as a whole expects everyone to know).
But the problem is that if someone who doesn't know about "Continuous Learning" reads "[X] learnt [Y] continuously", they don't notice that they are missing on something.
Whereas "[X] was fine-tuned by [Y]" reads strange and like you don't know what it means.
Thank you, very interesting, looking forward to see how you build further topics on this. Also, "death by a thousand cuts" felt very close to my experience.. Would be cool if you share some day what you personally do to avoid that. Is that possible at all, or is it our inevitable end?
Good question.
I do think it is possible for someone who makes it their focus to avoid that. Otherwise, one would need to be lucky enough to stumble on it, or so unlucky as to die young.
The section on ontological nudges reminds me of "man with a hammer syndrome": https://www.lesswrong.com/posts/WHP3tKPXppBF2S8e8/man-with-a-hammer-syndrome
Halfway through this article I went, "oh wait, ANY arbitrary thing you spend a lot of time with becomes part of your identity and you get all these opinions and stances that don't matter at all, I wonder what I'm doing in life that is as irrelevant and wasteful as what the wine and politics guys are doing"
I've started a category in my notes, titled "aesthetics", after reading this. (And I wrote things I value, and things I pride myself on)
------------------
About Ontological Nudges:
I already like to think in terms of symbolism and metaphor and use them when trying to figure stuff out and get the right perspective.
But Ontological Nudges feels deeper than that. I'll try to see which general concepts I use to perceive the world.
That's great to read. That's what I aimed for when writing it.
Cool post, helps to *name* so many things!
What are your thoughts about cog sec at societal level? Are we cooked? Seems like We, human species, develop some antibodies to the more obvious things: people recognize ragebait, terrible attempts at red herrings from politicians, AI slop. But at the same time it seems like it's a product of those being still in their infancy.
If anything, I've seen way too many times people getting overconfident because they can detect obvious cases of cognitive hazards and then assume they are immune to them.
And it seems technology is about to enable things much worse.
We are extremely underprepared, but I don't even know where you would start to make people prepared?
Have you seen any positive examples of progress here? And do you feel like there's a way forward with this?
Also I realize this might require a whole other post to write out, so even some pointers would be super cool!
Hahaha!
You touch on many things that I have partial drafts on.
> What are your thoughts about cog sec at societal level?
We do not have reliable mechanisms to share important knowledge at a societal level. Consider that most people do not know about https://safe.ai/work/statement-on-ai-risk
Let alone for advanced concepts like "cog sec".
> Are we cooked?
No!
On one hand, I believe it is possible to build such mechanisms. [1]
On the other hand, I don't believe it is even _needed_. It is enough for live players who act on the interests of others to internalise good cog sec principles.
> If anything, I've seen way too many times people getting overconfident because they can detect obvious cases of cognitive hazards and then assume they are immune to them.
You may be interested in https://www.lesswrong.com/posts/M5s6WgScRfmeWsLD4/dialogue-is-there-a-natural-abstraction-of-good
> And it seems technology is about to enable things much worse.
I have a attempted to write a few articles about this. Titles currently in my draft folder "AI Poisoning", "AI Personification & InstructGPT", "I was wrong, AI Poisoning is much worse than I expected."
I am still trying to write a good one on the topic. Until I succeed, I'll write easier ones and work up my way there.
---
[1]:
Imagine a Social Media primitive that is about signalling that one is familiar with a written piece. Upon this, we'd build a profile of all the pieces one is familiar with, and show what is general knowledge within various peer groups.
This would let us systematically transform general knowledge (what everyone knows) into common knowledge (what the group as a whole expects everyone to know).
"Continuous Learning" seems like a better name than for this than "Fine tuning" in some ways
Yup.
But the problem is that if someone who doesn't know about "Continuous Learning" reads "[X] learnt [Y] continuously", they don't notice that they are missing on something.
Whereas "[X] was fine-tuned by [Y]" reads strange and like you don't know what it means.