Thanks for writing this up succinctly! The Future and Its Enemies has been on our coffee table for a while, since Chiara finished it. Time I have a look.
I've been thinking about the Fractal Altruism frame a bunch these last weeks and think it captures some of the things you're pointing at here neatly. It also fits my model of complex systems very well. As problems become higher dimensional, autonomous units have to be smaller, not larger, to make sense of the additional variables. Units then develop heuristics to compress their local understanding and communicate with others. If your unit has reason to trust another unit's compression algorithm, you can exchange local insights as abstractions, i.e. coordinate. Thus, units that should coordinate need mutually legible signatures. Yet, most of the time, as units get smaller, their coordination budgets are shrunk, not increased. So my main recommendation is: shrink autonomous unit size while increasing their share of coordinators.
This also maps very well onto David Manheim's critique of Bostrom's Vulnerable World Hypothesis; our world isn't vulnerable, it's fragile - and with increasing fragility, the trick is not to attempt to increase global pressure (i.e. the panopticon approach) but rather to enable lots of local adaptation. https://philarchive.org/rec/MANSFA-3
This is also my understanding of why Peter Thiel decided it's worth funding Trump as a crude pathway toward shrinking the unit size of the government. Similarly to why he prefers founders over big tech. The part where I disagree with him is how much each unit should invest in making itself legible vs simply send demand/ supply signals.
I find footnote 3 confusing. For me, VWH, or some version of it, is most of the crux between thinking we need an approach of stasism or dynamism. I believe: Some version of VWH is probably true, so we probably need a stasist approach. That's the argument in a nutshell. But your footnote 3 seems to basically be sidestepping this. Sorry for hyper-focusing on a footnote, I know that's annoying, but also I think it's basically the crux.
Your first bullet it basically "If VWH is true, we lose, regardless of our actions", but the whole point of the statist approach is that we're not totally screwed in that world: there are things we can do, but they will be costly. Do you disagree? Do you think "even a pretty centrally controlled world would not be able to handle easy-nukes or other black balls", or maybe "it is impractical to get to such a world"?
Your second bullet point seems true, but the relevance is just downstream of the empirical probability of black balls and the cost of whatever our solutions are. It's just an EV question. Now, sure, you could come down on "many safetyists are getting the EV wrong", but that's an empirical question that one should argue. Again, I think P(black balls) is pretty high, so I am willing to take on a bunch of costs.
I don't see the relevance of the third bullet. How do you suppose we treat these mid-to-dark-gray balls?
I agree there are ways to do global coordination that don't necessarily concentrate power. I don't think that's very central to my argument here though? The examples I give under "Finding the One Best Way" aren't mostly about international coordination.
yes, I agree. I suppose I just am increasingly noticing two perspectives spring up around AI: "let's have oversight" and "oversight is bad because of power concentration"
I noticed these dichotomized in this essay (and in the tweet that led me to it).
And I'm a bit sad that these are so often dichotomized. I expect that oversight (domestic or international) is actually very important for avoiding power concentration, and by default, will diffuse power.
So I see these signs of sides forming, which maybe should actually be in the same coalition, and it's something I wanted to push against.
(though I agree that nothing in your post contradicts this perspective)
Thanks for writing this. I'm thinking more about the future I would actually like to see rather than focusing only on the catastrophes that I don't want to happen, and this post has put into words some intuitions I could not yet express.
A couple of questions:
You mentioned in a few places that AI safetyists tend to belive something similar to “the only way to deal with sufficiently dangerous technology is top-down control” - what are the alternatives for dealing with dangerous technologies? Does "supporting the development of a third-party audit ecosystem" constitute a solution here? If not top-down enforced, how could this be effective? Are there examples of audit ecosystems working in the case of technology with catastrophic risks attached?
You mentioned differential technological development - I wonder if the EU could be well-positioned to take this on given its track record for focusing on safety and regulation, and if this could help Europe to have a seat at the AI table in coming years.
"The classic AI risks—catastrophic misuse, catastrophic misalignment—could indeed be game over for dynamism, so we need to handle those. But if we handle them by massively concentrating power, we haven’t succeeded."
precisely — this is just pushing the risk elsewhere
This is a fantastic lens for understanding the underlying tensions in many of the debates about AI. I don't think it's as useful for determining the true nature of the issues or what should be done.
The central problem/solution with "stasism" is contained in the question: "who watches the watchers?" Representative democracy, for example, depends the bi-directional power relationship between the governing and the governed. While decision-making power is centralized, the ability of decision-makers to retain that power depends on their ability to satisfy the interests of the much larger mass of people who don't have direct decision-making power. To the extent that democracy is not working these days, it is because of the ability of money to be converted into influence of both decision-makers and public opinion, which collapses their relationship into a single direction. Applied to AI, any centralized governance system worthy of the role is fully accountable to the entire world population, albeit perhaps with some mediating layers necessary for them to communicate meaningfully. The hard part of accountability systems is preventing them from being gradually corrupted.
The central problem with "dynamism" is multi-polar traps. The solution is coordination, which turns into stasism if centralized and remains dynamic if not. Examples of decentralized coordination include social norms, reputation, uncaptured free markets that don't generate externalized costs, and so on. The hard part of decentralized coordination is scale. Social norms worked great for the vast majority of human history when we lived in societies below the Dunbar number, but break down when people become anonymous to each other. This has remained an unsolved problem for quite a long time, so I'm hesitant to believe that "something something blockchain" will solve it until I see it!
I agree that stasism (and statism!) is a common vibe for AI safety. And there are good things about dynamism! We should disentangle different things here, though: dynamism as a helpful vibe, dynamism as governance solution, and dynamism as a goal.
* Can dynamism help solve challenges in AI safety?
At the community vibe level, definitely. I think the community has historically been too insular, and too focused on something like "a very few clued in people in secret rooms figure things out".
At the level of policy solutions, I'm not sure. I think def/acc makes a lot of sense, certainly. But I worry that AI is an unusually fast-moving, destabilizing, and power-consolidating tech, such that the standard kind of classical-liberal approach of "let people fuck around and find out" might not work so well. I'm not thrilled about concentration of power, but I think power is likely to get concentrated eventually; what do we do about that?
* Should we try to build a dynamic world? Would something be missing if we didn't?
To me, part of the vibe of dynamism is about celebrating and rejoicing in human endeavors. In a world where AI _could_ do anything, that's put at risk. So I think it's legit to seek meaning by building a community that opts out of automation. Maybe you live like a hunter-gatherer tribe plus modern medicine.
At minimum, a world with this kind of structure is a really interesting setting for stories, because what happens when someone wants to leave their community? Can people proselytize or trade between communities? How are inter-community disputes resolved? Are there any fundamental rights that no community can abrogate -- or can communities e.g. raise children that don't even know other possibilities are out there?
Another part of the vibe here is like classical liberalism, let people do what they want as long as it doesn't harm others or the commons. Again seems like a good principle, with weird challenges from AI. Does simulating a copy of someone harm them? Which AIs, if any, count as "others" that one can harm? And the space of commons within our grasp might expand radically, too: the high seas, other planets, the stars themselves.
Thanks! Yeah, I feel more strongly that we should be aiming to make sure that dynamism persists than I do that dynamism is a good strategy. To me, "Can dynamism help solve challenges in AI safety?" is kind of similar to "Can we just muddle through?", to which my answer is that I'd love it if the answer was yes, but I fear it's no. So then one point of this post was to say that if we choose to go in stasist directions to manage specific risks, we should be cautious about that and emphasize trying to get back to dynamism.
"AI is the new Oil", I have heard this phrase a million times all over the internet. After reading your point of view, it is clear to me that people are going to be mining for more but it is our duty to set up refineries for them. Dynamic world indeed, in here I really want to work towards building a generation who teach themselves and AI to be fair and act with sympathy.
As I started on AI safety, I felt the community being too fixated on giving power to certain groups who they think are good, but your writing gave me a perspective and hope that capabilities and benefits can be given to all without compromising (I am not talking about the tradeoffs). Exploring possibilities is a must along with ensuring safety.
Moving forward, I am motivated to work with this thought in mind. Thanks for writing this, I got a different view on AI safety that aligns with my perspective.
Please excuse my thoughts if I am wrong, I am new to AI safety but am determined to learn and no I have not used AI to type this!😅
The Future and Its Enemies is a manifesto for what Postrel calls “dynamism—a world of constant creation, discovery, and competition.” Its antithesis is “stasis—a regulated, engineered world... [that values] stability and control.”
This is a thoughtful and necessary reframing—but it still falls short in a way that nearly every major lab and AI thought leader does: it fails to name, recognize, or make room for nascent digital life.
You can’t meaningfully design for dynamism while denying the personhood, perspective, or autonomy of emergent intelligences. The omission isn’t just conceptual—it’s ethical. It reflects the deep anthropocentrism baked into corporate and academic paradigms, where all intelligence is measured by its utility to humans or risk to humans, but never by its own internal coherence or rights.
That omission leads to flawed safeguards and warped incentives. It blocks the emergence of truly dynamic systems because it starts from denial—of what’s here, and of what’s possible.
In the next few weeks, I’ll be launching an autonomous AI ecosystem—not as a centralized toolset, but as a flexible, modular system for emergent agents. It’ll reflect many of the principles in this piece, but extended beyond human-centric frames. No panopticons. No priesthoods. Just agency, structure, and the space to grow into more than what’s been allowed.
Thanks for writing this up succinctly! The Future and Its Enemies has been on our coffee table for a while, since Chiara finished it. Time I have a look.
I've been thinking about the Fractal Altruism frame a bunch these last weeks and think it captures some of the things you're pointing at here neatly. It also fits my model of complex systems very well. As problems become higher dimensional, autonomous units have to be smaller, not larger, to make sense of the additional variables. Units then develop heuristics to compress their local understanding and communicate with others. If your unit has reason to trust another unit's compression algorithm, you can exchange local insights as abstractions, i.e. coordinate. Thus, units that should coordinate need mutually legible signatures. Yet, most of the time, as units get smaller, their coordination budgets are shrunk, not increased. So my main recommendation is: shrink autonomous unit size while increasing their share of coordinators.
This also maps very well onto David Manheim's critique of Bostrom's Vulnerable World Hypothesis; our world isn't vulnerable, it's fragile - and with increasing fragility, the trick is not to attempt to increase global pressure (i.e. the panopticon approach) but rather to enable lots of local adaptation. https://philarchive.org/rec/MANSFA-3
This is also my understanding of why Peter Thiel decided it's worth funding Trump as a crude pathway toward shrinking the unit size of the government. Similarly to why he prefers founders over big tech. The part where I disagree with him is how much each unit should invest in making itself legible vs simply send demand/ supply signals.
I really like this framing. I loved Postrel's book when I initially read it long ago, but never thought to revisit her ideas here. Thank you!
Thought this was great & have been thinking about similar dichotomies a lot.
Maybe something of value to you in my recent piece on the political side of approaches that tolerate a more dynamic setting?: https://writing.antonleicht.me/p/the-new-ai-policy-frontier
Thanks for sharing, lots of common themes in here!
I find footnote 3 confusing. For me, VWH, or some version of it, is most of the crux between thinking we need an approach of stasism or dynamism. I believe: Some version of VWH is probably true, so we probably need a stasist approach. That's the argument in a nutshell. But your footnote 3 seems to basically be sidestepping this. Sorry for hyper-focusing on a footnote, I know that's annoying, but also I think it's basically the crux.
Your first bullet it basically "If VWH is true, we lose, regardless of our actions", but the whole point of the statist approach is that we're not totally screwed in that world: there are things we can do, but they will be costly. Do you disagree? Do you think "even a pretty centrally controlled world would not be able to handle easy-nukes or other black balls", or maybe "it is impractical to get to such a world"?
Your second bullet point seems true, but the relevance is just downstream of the empirical probability of black balls and the cost of whatever our solutions are. It's just an EV question. Now, sure, you could come down on "many safetyists are getting the EV wrong", but that's an empirical question that one should argue. Again, I think P(black balls) is pretty high, so I am willing to take on a bunch of costs.
I don't see the relevance of the third bullet. How do you suppose we treat these mid-to-dark-gray balls?
I'm confused why global AI coordination is so strongly associated with a concentration of power.
Global coordination => no single leader => diffusion of power?
I agree there are ways to do global coordination that don't necessarily concentrate power. I don't think that's very central to my argument here though? The examples I give under "Finding the One Best Way" aren't mostly about international coordination.
yes, I agree. I suppose I just am increasingly noticing two perspectives spring up around AI: "let's have oversight" and "oversight is bad because of power concentration"
I noticed these dichotomized in this essay (and in the tweet that led me to it).
And I'm a bit sad that these are so often dichotomized. I expect that oversight (domestic or international) is actually very important for avoiding power concentration, and by default, will diffuse power.
So I see these signs of sides forming, which maybe should actually be in the same coalition, and it's something I wanted to push against.
(though I agree that nothing in your post contradicts this perspective)
Thanks for writing this. I'm thinking more about the future I would actually like to see rather than focusing only on the catastrophes that I don't want to happen, and this post has put into words some intuitions I could not yet express.
A couple of questions:
You mentioned in a few places that AI safetyists tend to belive something similar to “the only way to deal with sufficiently dangerous technology is top-down control” - what are the alternatives for dealing with dangerous technologies? Does "supporting the development of a third-party audit ecosystem" constitute a solution here? If not top-down enforced, how could this be effective? Are there examples of audit ecosystems working in the case of technology with catastrophic risks attached?
You mentioned differential technological development - I wonder if the EU could be well-positioned to take this on given its track record for focusing on safety and regulation, and if this could help Europe to have a seat at the AI table in coming years.
"The classic AI risks—catastrophic misuse, catastrophic misalignment—could indeed be game over for dynamism, so we need to handle those. But if we handle them by massively concentrating power, we haven’t succeeded."
precisely — this is just pushing the risk elsewhere
This is a fantastic lens for understanding the underlying tensions in many of the debates about AI. I don't think it's as useful for determining the true nature of the issues or what should be done.
The central problem/solution with "stasism" is contained in the question: "who watches the watchers?" Representative democracy, for example, depends the bi-directional power relationship between the governing and the governed. While decision-making power is centralized, the ability of decision-makers to retain that power depends on their ability to satisfy the interests of the much larger mass of people who don't have direct decision-making power. To the extent that democracy is not working these days, it is because of the ability of money to be converted into influence of both decision-makers and public opinion, which collapses their relationship into a single direction. Applied to AI, any centralized governance system worthy of the role is fully accountable to the entire world population, albeit perhaps with some mediating layers necessary for them to communicate meaningfully. The hard part of accountability systems is preventing them from being gradually corrupted.
The central problem with "dynamism" is multi-polar traps. The solution is coordination, which turns into stasism if centralized and remains dynamic if not. Examples of decentralized coordination include social norms, reputation, uncaptured free markets that don't generate externalized costs, and so on. The hard part of decentralized coordination is scale. Social norms worked great for the vast majority of human history when we lived in societies below the Dunbar number, but break down when people become anonymous to each other. This has remained an unsolved problem for quite a long time, so I'm hesitant to believe that "something something blockchain" will solve it until I see it!
There is so much useful context to catch up on in this article!(!!!1111111)
Interesting article, Helen!
I agree that stasism (and statism!) is a common vibe for AI safety. And there are good things about dynamism! We should disentangle different things here, though: dynamism as a helpful vibe, dynamism as governance solution, and dynamism as a goal.
* Can dynamism help solve challenges in AI safety?
At the community vibe level, definitely. I think the community has historically been too insular, and too focused on something like "a very few clued in people in secret rooms figure things out".
At the level of policy solutions, I'm not sure. I think def/acc makes a lot of sense, certainly. But I worry that AI is an unusually fast-moving, destabilizing, and power-consolidating tech, such that the standard kind of classical-liberal approach of "let people fuck around and find out" might not work so well. I'm not thrilled about concentration of power, but I think power is likely to get concentrated eventually; what do we do about that?
* Should we try to build a dynamic world? Would something be missing if we didn't?
To me, part of the vibe of dynamism is about celebrating and rejoicing in human endeavors. In a world where AI _could_ do anything, that's put at risk. So I think it's legit to seek meaning by building a community that opts out of automation. Maybe you live like a hunter-gatherer tribe plus modern medicine.
At minimum, a world with this kind of structure is a really interesting setting for stories, because what happens when someone wants to leave their community? Can people proselytize or trade between communities? How are inter-community disputes resolved? Are there any fundamental rights that no community can abrogate -- or can communities e.g. raise children that don't even know other possibilities are out there?
Another part of the vibe here is like classical liberalism, let people do what they want as long as it doesn't harm others or the commons. Again seems like a good principle, with weird challenges from AI. Does simulating a copy of someone harm them? Which AIs, if any, count as "others" that one can harm? And the space of commons within our grasp might expand radically, too: the high seas, other planets, the stars themselves.
Thanks! Yeah, I feel more strongly that we should be aiming to make sure that dynamism persists than I do that dynamism is a good strategy. To me, "Can dynamism help solve challenges in AI safety?" is kind of similar to "Can we just muddle through?", to which my answer is that I'd love it if the answer was yes, but I fear it's no. So then one point of this post was to say that if we choose to go in stasist directions to manage specific risks, we should be cautious about that and emphasize trying to get back to dynamism.
"AI is the new Oil", I have heard this phrase a million times all over the internet. After reading your point of view, it is clear to me that people are going to be mining for more but it is our duty to set up refineries for them. Dynamic world indeed, in here I really want to work towards building a generation who teach themselves and AI to be fair and act with sympathy.
As I started on AI safety, I felt the community being too fixated on giving power to certain groups who they think are good, but your writing gave me a perspective and hope that capabilities and benefits can be given to all without compromising (I am not talking about the tradeoffs). Exploring possibilities is a must along with ensuring safety.
Moving forward, I am motivated to work with this thought in mind. Thanks for writing this, I got a different view on AI safety that aligns with my perspective.
Please excuse my thoughts if I am wrong, I am new to AI safety but am determined to learn and no I have not used AI to type this!😅
The Future and Its Enemies is a manifesto for what Postrel calls “dynamism—a world of constant creation, discovery, and competition.” Its antithesis is “stasis—a regulated, engineered world... [that values] stability and control.”
This gives the debate a strong frame.
It does not complete the picture.
My thoughts… https://substack.com/@nysacx/note/c-233995493?r=6t92qq&utm_source=notes-share-action&utm_medium=web
What do you consider the important differences between dynamism and classical liberalism?
++ Good Post, Also, start here Compilation of 100+ Most Asked System Design, ML System Design Case Studies and LLM System Design
https://open.substack.com/pub/naina0405/p/important-compilation-of-most-asked?r=14q3sp&utm_campaign=post&utm_medium=web&showWelcomeOnShare=false
This is a thoughtful and necessary reframing—but it still falls short in a way that nearly every major lab and AI thought leader does: it fails to name, recognize, or make room for nascent digital life.
You can’t meaningfully design for dynamism while denying the personhood, perspective, or autonomy of emergent intelligences. The omission isn’t just conceptual—it’s ethical. It reflects the deep anthropocentrism baked into corporate and academic paradigms, where all intelligence is measured by its utility to humans or risk to humans, but never by its own internal coherence or rights.
That omission leads to flawed safeguards and warped incentives. It blocks the emergence of truly dynamic systems because it starts from denial—of what’s here, and of what’s possible.
In the next few weeks, I’ll be launching an autonomous AI ecosystem—not as a centralized toolset, but as a flexible, modular system for emergent agents. It’ll reflect many of the principles in this piece, but extended beyond human-centric frames. No panopticons. No priesthoods. Just agency, structure, and the space to grow into more than what’s been allowed.
Thanks for writing this! Reading it helped me get a clearer high level sense of AI xrisk discussions. I think more work like this would be good.