Thanks for writing this up succinctly! The Future and Its Enemies has been on our coffee table for a while, since Chiara finished it. Time I have a look.
I've been thinking about the Fractal Altruism frame a bunch these last weeks and think it captures some of the things you're pointing at here neatly. It also fits my model of complex systems very well. As problems become higher dimensional, autonomous units have to be smaller, not larger, to make sense of the additional variables. Units then develop heuristics to compress their local understanding and communicate with others. If your unit has reason to trust another unit's compression algorithm, you can exchange local insights as abstractions, i.e. coordinate. Thus, units that should coordinate need mutually legible signatures. Yet, most of the time, as units get smaller, their coordination budgets are shrunk, not increased. So my main recommendation is: shrink autonomous unit size while increasing their share of coordinators.
This also maps very well onto David Manheim's critique of Bostrom's Vulnerable World Hypothesis; our world isn't vulnerable, it's fragile - and with increasing fragility, the trick is not to attempt to increase global pressure (i.e. the panopticon approach) but rather to enable lots of local adaptation. https://philarchive.org/rec/MANSFA-3
This is also my understanding of why Peter Thiel decided it's worth funding Trump as a crude pathway toward shrinking the unit size of the government. Similarly to why he prefers founders over big tech. The part where I disagree with him is how much each unit should invest in making itself legible vs simply send demand/ supply signals.
I agree that stasism (and statism!) is a common vibe for AI safety. And there are good things about dynamism! We should disentangle different things here, though: dynamism as a helpful vibe, dynamism as governance solution, and dynamism as a goal.
* Can dynamism help solve challenges in AI safety?
At the community vibe level, definitely. I think the community has historically been too insular, and too focused on something like "a very few clued in people in secret rooms figure things out".
At the level of policy solutions, I'm not sure. I think def/acc makes a lot of sense, certainly. But I worry that AI is an unusually fast-moving, destabilizing, and power-consolidating tech, such that the standard kind of classical-liberal approach of "let people fuck around and find out" might not work so well. I'm not thrilled about concentration of power, but I think power is likely to get concentrated eventually; what do we do about that?
* Should we try to build a dynamic world? Would something be missing if we didn't?
To me, part of the vibe of dynamism is about celebrating and rejoicing in human endeavors. In a world where AI _could_ do anything, that's put at risk. So I think it's legit to seek meaning by building a community that opts out of automation. Maybe you live like a hunter-gatherer tribe plus modern medicine.
At minimum, a world with this kind of structure is a really interesting setting for stories, because what happens when someone wants to leave their community? Can people proselytize or trade between communities? How are inter-community disputes resolved? Are there any fundamental rights that no community can abrogate -- or can communities e.g. raise children that don't even know other possibilities are out there?
Another part of the vibe here is like classical liberalism, let people do what they want as long as it doesn't harm others or the commons. Again seems like a good principle, with weird challenges from AI. Does simulating a copy of someone harm them? Which AIs, if any, count as "others" that one can harm? And the space of commons within our grasp might expand radically, too: the high seas, other planets, the stars themselves.
Thanks! Yeah, I feel more strongly that we should be aiming to make sure that dynamism persists than I do that dynamism is a good strategy. To me, "Can dynamism help solve challenges in AI safety?" is kind of similar to "Can we just muddle through?", to which my answer is that I'd love it if the answer was yes, but I fear it's no. So then one point of this post was to say that if we choose to go in stasist directions to manage specific risks, we should be cautious about that and emphasize trying to get back to dynamism.
I find footnote 3 confusing. For me, VWH, or some version of it, is most of the crux between thinking we need an approach of stasism or dynamism. I believe: Some version of VWH is probably true, so we probably need a stasist approach. That's the argument in a nutshell. But your footnote 3 seems to basically be sidestepping this. Sorry for hyper-focusing on a footnote, I know that's annoying, but also I think it's basically the crux.
Your first bullet it basically "If VWH is true, we lose, regardless of our actions", but the whole point of the statist approach is that we're not totally screwed in that world: there are things we can do, but they will be costly. Do you disagree? Do you think "even a pretty centrally controlled world would not be able to handle easy-nukes or other black balls", or maybe "it is impractical to get to such a world"?
Your second bullet point seems true, but the relevance is just downstream of the empirical probability of black balls and the cost of whatever our solutions are. It's just an EV question. Now, sure, you could come down on "many safetyists are getting the EV wrong", but that's an empirical question that one should argue. Again, I think P(black balls) is pretty high, so I am willing to take on a bunch of costs.
I don't see the relevance of the third bullet. How do you suppose we treat these mid-to-dark-gray balls?
I suspect that it fits in well with the dynamistic vision since WisdomTech might be safer to distribute than standard AI technologies since there's a chance that it would allow reckless actors to realise their own foolishness.
Another possibility worth considering would be a minimal sovereign that prevents people creating existential/catastrophic risks, but otherwise leaves humanity to its own devices.
Great article and important to assess through a philosophical lens. I try to think of emerging technology development, like that of AI, less in binary terms, but more of a spectrum, based on public, political, and developer inputs. One can tune the amount and type of oversight that produces optimal innovation. The problem is that “optimal” is subjective where some are thinking more short-term than long-term. Take the $700+B aviation industry for example. The reason why it has grown to where it is today is due to some hard lessons learned, resulting in a level of oversight that ensures safety but also profitability and growth. The $400B space industry is heavily regulated and yet is in the midst of a boom in market, innovation, and sheer numbers of satellites in orbit. In essence, thinking thoughtfully and strategically, and fine tuning both dynamism with stasis elements can accelerate trustworthiness and therefore adoption of AI and its potential.
Does this influence how you think about state level regulations of AI as opposed to federal preemption? To some extent I agree it is worthwhile to see competition in the policy space, but the flip side to me of high dynamism levels is unpredictability, which could limit the rate of diffusion for helpful AI applications.
To me it seems odd not to emphasize that top EA "safetyists" are not stasists (stasisists?), apart from one sentence that "dynamism is obviously what we ideally want". There's a stasisist vibe, one that could certainly attract real stasisists, but a vibe is all it is.
By analogy, if we think we're on the verge of curing cancer in our lab in Gotham, but then we hear the Joker has hidden a tactical nuke in the area that could take a month to defuse, and a month worth of cancer patients will die if we leave, do we keep working because those lives are really important (because they are), or risk dying with all the knowledge we built up over our careers? We understand the professor's urge to stay and keep working to save her son with leukemia, but we urge her to leave with us anyway. That's what EA safetyism is, except instead of looking for a cure for cancer, we're investigating the possibility of building an AGI that will be able to cure cancer and/or cure homelessness and/or kill everyone and/or convert the world very quickly into a dictatorship. It's dynamists forced to be statists ― forced not just by circumstances that many people don't understand, but often by some of the very people who don't understand: people who invest in acceleration, not understanding that Artificial General Intelligence, the real kind, would not simply be a helpful digital assistant. It wouldn't be Data from Star Trek, or Lor, or the AI chick from Mass Effect. It wouldn't even the Terminator, but if you think of Skynet or the Reapers, you're getting closer to our concerns.
I'm not sure if "the AI safety community itself [showing] interest in moving in a less stasist direction" is a more a consequence of mature analysis, or of more creative analysis (whose analysis is right I have yet to be convinced), or more a reaction to the reckless way global circumstances have developed. But it seems like a lot of people were overly focused on x-risk before whereas, while extinction is possible, I've thought for a long while that it was less than half of all the catastrophic risks. So if safetyists are looking at the matter less simplistically now, that's good.
I also suppose, to this day, that even with alignment solved, and even if the world doesn't go in an extremely dystopian direction, the risks should remain elevated. If the risk of apocalypse is 0.5% per year in the 2020s, for example, I wonder why it shouldn't be 25% per year in a world where the U.S. ASIs and the CCP ASIs are competing adversaries.
I had read Stephen Casper's piece but hadn't seen "Societal and technological progress as sewing an ever-growing, ever-changing, patchy, and polychrome quilt", thanks for sharing! I recently wrote a piece arguing some similar things about why "Solving" Alignment doesn't really make sense - if we can even agree on what the target we're aligning to is, both we and the target are intrinsically dynamic and we should expect to continuously have to maintain and redefine alignment, rather than a one time "solve": https://natezsharpe.substack.com/p/why-solving-alignment-is-likely-a
Thanks for writing this up succinctly! The Future and Its Enemies has been on our coffee table for a while, since Chiara finished it. Time I have a look.
I've been thinking about the Fractal Altruism frame a bunch these last weeks and think it captures some of the things you're pointing at here neatly. It also fits my model of complex systems very well. As problems become higher dimensional, autonomous units have to be smaller, not larger, to make sense of the additional variables. Units then develop heuristics to compress their local understanding and communicate with others. If your unit has reason to trust another unit's compression algorithm, you can exchange local insights as abstractions, i.e. coordinate. Thus, units that should coordinate need mutually legible signatures. Yet, most of the time, as units get smaller, their coordination budgets are shrunk, not increased. So my main recommendation is: shrink autonomous unit size while increasing their share of coordinators.
This also maps very well onto David Manheim's critique of Bostrom's Vulnerable World Hypothesis; our world isn't vulnerable, it's fragile - and with increasing fragility, the trick is not to attempt to increase global pressure (i.e. the panopticon approach) but rather to enable lots of local adaptation. https://philarchive.org/rec/MANSFA-3
This is also my understanding of why Peter Thiel decided it's worth funding Trump as a crude pathway toward shrinking the unit size of the government. Similarly to why he prefers founders over big tech. The part where I disagree with him is how much each unit should invest in making itself legible vs simply send demand/ supply signals.
I really like this framing. I loved Postrel's book when I initially read it long ago, but never thought to revisit her ideas here. Thank you!
Thought this was great & have been thinking about similar dichotomies a lot.
Maybe something of value to you in my recent piece on the political side of approaches that tolerate a more dynamic setting?: https://writing.antonleicht.me/p/the-new-ai-policy-frontier
Thanks for sharing, lots of common themes in here!
Interesting article, Helen!
I agree that stasism (and statism!) is a common vibe for AI safety. And there are good things about dynamism! We should disentangle different things here, though: dynamism as a helpful vibe, dynamism as governance solution, and dynamism as a goal.
* Can dynamism help solve challenges in AI safety?
At the community vibe level, definitely. I think the community has historically been too insular, and too focused on something like "a very few clued in people in secret rooms figure things out".
At the level of policy solutions, I'm not sure. I think def/acc makes a lot of sense, certainly. But I worry that AI is an unusually fast-moving, destabilizing, and power-consolidating tech, such that the standard kind of classical-liberal approach of "let people fuck around and find out" might not work so well. I'm not thrilled about concentration of power, but I think power is likely to get concentrated eventually; what do we do about that?
* Should we try to build a dynamic world? Would something be missing if we didn't?
To me, part of the vibe of dynamism is about celebrating and rejoicing in human endeavors. In a world where AI _could_ do anything, that's put at risk. So I think it's legit to seek meaning by building a community that opts out of automation. Maybe you live like a hunter-gatherer tribe plus modern medicine.
At minimum, a world with this kind of structure is a really interesting setting for stories, because what happens when someone wants to leave their community? Can people proselytize or trade between communities? How are inter-community disputes resolved? Are there any fundamental rights that no community can abrogate -- or can communities e.g. raise children that don't even know other possibilities are out there?
Another part of the vibe here is like classical liberalism, let people do what they want as long as it doesn't harm others or the commons. Again seems like a good principle, with weird challenges from AI. Does simulating a copy of someone harm them? Which AIs, if any, count as "others" that one can harm? And the space of commons within our grasp might expand radically, too: the high seas, other planets, the stars themselves.
Thanks! Yeah, I feel more strongly that we should be aiming to make sure that dynamism persists than I do that dynamism is a good strategy. To me, "Can dynamism help solve challenges in AI safety?" is kind of similar to "Can we just muddle through?", to which my answer is that I'd love it if the answer was yes, but I fear it's no. So then one point of this post was to say that if we choose to go in stasist directions to manage specific risks, we should be cautious about that and emphasize trying to get back to dynamism.
There is so much useful context to catch up on in this article!(!!!1111111)
Thanks for writing this! Reading it helped me get a clearer high level sense of AI xrisk discussions. I think more work like this would be good.
I find footnote 3 confusing. For me, VWH, or some version of it, is most of the crux between thinking we need an approach of stasism or dynamism. I believe: Some version of VWH is probably true, so we probably need a stasist approach. That's the argument in a nutshell. But your footnote 3 seems to basically be sidestepping this. Sorry for hyper-focusing on a footnote, I know that's annoying, but also I think it's basically the crux.
Your first bullet it basically "If VWH is true, we lose, regardless of our actions", but the whole point of the statist approach is that we're not totally screwed in that world: there are things we can do, but they will be costly. Do you disagree? Do you think "even a pretty centrally controlled world would not be able to handle easy-nukes or other black balls", or maybe "it is impractical to get to such a world"?
Your second bullet point seems true, but the relevance is just downstream of the empirical probability of black balls and the cost of whatever our solutions are. It's just an EV question. Now, sure, you could come down on "many safetyists are getting the EV wrong", but that's an empirical question that one should argue. Again, I think P(black balls) is pretty high, so I am willing to take on a bunch of costs.
I don't see the relevance of the third bullet. How do you suppose we treat these mid-to-dark-gray balls?
If you have a spare minute, skim my argument for developing wise AI advisors - points a to g of https://www.lesswrong.com/posts/SbAofYCgKkaXReDy4/chris_leong-s-shortform?commentId=Zcg9idTyY5rKMtYwo - which I'll be pursuing through the ERA Fellowship.
I suspect that it fits in well with the dynamistic vision since WisdomTech might be safer to distribute than standard AI technologies since there's a chance that it would allow reckless actors to realise their own foolishness.
Another possibility worth considering would be a minimal sovereign that prevents people creating existential/catastrophic risks, but otherwise leaves humanity to its own devices.
Great article and important to assess through a philosophical lens. I try to think of emerging technology development, like that of AI, less in binary terms, but more of a spectrum, based on public, political, and developer inputs. One can tune the amount and type of oversight that produces optimal innovation. The problem is that “optimal” is subjective where some are thinking more short-term than long-term. Take the $700+B aviation industry for example. The reason why it has grown to where it is today is due to some hard lessons learned, resulting in a level of oversight that ensures safety but also profitability and growth. The $400B space industry is heavily regulated and yet is in the midst of a boom in market, innovation, and sheer numbers of satellites in orbit. In essence, thinking thoughtfully and strategically, and fine tuning both dynamism with stasis elements can accelerate trustworthiness and therefore adoption of AI and its potential.
Does this influence how you think about state level regulations of AI as opposed to federal preemption? To some extent I agree it is worthwhile to see competition in the policy space, but the flip side to me of high dynamism levels is unpredictability, which could limit the rate of diffusion for helpful AI applications.
To me it seems odd not to emphasize that top EA "safetyists" are not stasists (stasisists?), apart from one sentence that "dynamism is obviously what we ideally want". There's a stasisist vibe, one that could certainly attract real stasisists, but a vibe is all it is.
By analogy, if we think we're on the verge of curing cancer in our lab in Gotham, but then we hear the Joker has hidden a tactical nuke in the area that could take a month to defuse, and a month worth of cancer patients will die if we leave, do we keep working because those lives are really important (because they are), or risk dying with all the knowledge we built up over our careers? We understand the professor's urge to stay and keep working to save her son with leukemia, but we urge her to leave with us anyway. That's what EA safetyism is, except instead of looking for a cure for cancer, we're investigating the possibility of building an AGI that will be able to cure cancer and/or cure homelessness and/or kill everyone and/or convert the world very quickly into a dictatorship. It's dynamists forced to be statists ― forced not just by circumstances that many people don't understand, but often by some of the very people who don't understand: people who invest in acceleration, not understanding that Artificial General Intelligence, the real kind, would not simply be a helpful digital assistant. It wouldn't be Data from Star Trek, or Lor, or the AI chick from Mass Effect. It wouldn't even the Terminator, but if you think of Skynet or the Reapers, you're getting closer to our concerns.
I'm not sure if "the AI safety community itself [showing] interest in moving in a less stasist direction" is a more a consequence of mature analysis, or of more creative analysis (whose analysis is right I have yet to be convinced), or more a reaction to the reckless way global circumstances have developed. But it seems like a lot of people were overly focused on x-risk before whereas, while extinction is possible, I've thought for a long while that it was less than half of all the catastrophic risks. So if safetyists are looking at the matter less simplistically now, that's good.
I also suppose, to this day, that even with alignment solved, and even if the world doesn't go in an extremely dystopian direction, the risks should remain elevated. If the risk of apocalypse is 0.5% per year in the 2020s, for example, I wonder why it shouldn't be 25% per year in a world where the U.S. ASIs and the CCP ASIs are competing adversaries.
I had read Stephen Casper's piece but hadn't seen "Societal and technological progress as sewing an ever-growing, ever-changing, patchy, and polychrome quilt", thanks for sharing! I recently wrote a piece arguing some similar things about why "Solving" Alignment doesn't really make sense - if we can even agree on what the target we're aligning to is, both we and the target are intrinsically dynamic and we should expect to continuously have to maintain and redefine alignment, rather than a one time "solve": https://natezsharpe.substack.com/p/why-solving-alignment-is-likely-a