Discussion about this post

User's avatar
facile's avatar

The problem with several of these scenarios is that they assume innovations that don’t exist yet.

1. Genetic enhancement. Herasight guarantees an IQ boost of 5-ish points or whatever. After several generations, that advantage could compound. But what about selection effects? We already see that people on the very high end of IQ have a lower fertility rate, but we don’t know fully why. Is it an environmental mismatch? Are there cognitive trade offs to increasing G? Are there other epigenetic risks to multi-generational IVF selection? No one knows.

2. The “increased competition” thesis is a little misleading. Are people in rural Africa suffering more because of increased competition in the global white collar job market? No, not at all. Competition will only make your life worse if you’re in an economy with shrinking opportunities. Otherwise, you’re competing for new growth, which your active participation in the economy will likely multiply.

3. AIs competing with humans. If capital becomes fully substitutable with labor, then we have a problem. However, even that is not guaranteed given AGI. To what degree are human-human interactions more valuable in that scenario? We have to consider comparative advantage and what effect that could have on the human economy.

Not to mention, if AIs become truly powerful, then distillation and open source means that the average person will become *more* dangerous. It might be the case that full automation kicks off a wave of assassinations that forces the government to nationalize the AI labs and start paying stipends to citizens. This might sound utopian, but I imagine it would still suck in ways we have not fully considered yet.

AI growth will plateau at some point. But no one knows when. It could be that it only plateaus once we create a Dyson sphere around the sun and go full singularity. It could be that it plateaus due to unforeseen scaling problems with LLMs that we don’t know about yet. It could plateau due to cost-benefit problems Is a giga-genius AI more economically profitable than a normal-genius AI? Depends on the cost of running the former vs the latter and the additional revenue generated. If it costs $100mm a token to get the giga-brained AI and $100 for the normal genius AI, and scaling costs are not able to bring the price of the former down considerably, does it make sense for the frontier labs to keep pushing bigger models?

Black-and-white scenarios are seductive because they’re fun to think about. But the reality is that there are way, way too many variables at play to know which scenario we’re headed for.

All we know is this:

1. All growth ends.

2. Growth can always start again.

Anton's avatar

Your capacity to develop romantic and platonic relationships is a product of your ability to compete with other humans for friendships and intimacy. << One can be skeptical of this. The domain of affection is not a competition. Think how silly it would be to believe that my cat has competed for my affection. Similarly my friends have not competed with other people so that I have chose the best people to be my friends. They were just there when I was open to welcome them. It is a common mistake but still a mistake that of reading competition as pervasive in every relationship.

No posts

Ready for more?