Worldbuilding Hopeful Futures with AI – Free course now live

Hi everyone,

Foresight Institute just launched a new free, self-paced course on Udemy:
👉 Worldbuilding Hopeful Futures with AI

The goal: to empower more people — not just technologists or policymakers — to actively think about and shape AI’s long-term trajectory. We use storytelling and worldbuilding to crowdsource diverse, ambitious, and grounded visions of the future, and explore practical pathways to get there.

The course brings together insights from a wide range of thinkers across science, governance, and philosophy, including:

Helen Toner – Director of Strategy at CSET and former OpenAI board member

Anthony Aguirre – Co-founder of the Future of Life Institute

Hannah Ritchie – Lead Researcher at Our World in Data

Ada Palmer – Historian and sci-fi author

Glen Weyl – Founder of RadicalxChange, Microsoft Research

Robert Lempert – Senior Scientist at RAND, expert in scenario planning

Each module includes a short video, reading recommendations, and an interactive assignment to build your own future scenario.

We’d love for you to check it out, share it with others, and let us know what you think:
👉 https://​​www.udemy.com/​​course/​​worldbuilding-hopeful-futures-with-ai/​​

Happy to answer any questions or discuss the project further!

My first reaction was that this seems to be assuming a zero discount rate on the future. I haven’t had a chance to really dig into it though

Really enjoyed this book, it inspired me to start Roots of Progress! https://​​blog.rootsofprogress.org/​​the-idea-of-progress

I would say both immigration and crime are relevant to progress!

Our primitive monkey brains are good at over-estimating very unlikely risks.^[2]

I think this is presupposing the question isn’t it.

If a risk is indeed very unlikely, then we will tend to overestimate it. (If the probability is 0 it’s impossible to underestimate)

But for risks that are actually quite likely, then we are more likely to underestimate them.

And of course, bias estimates cut both ways. “Our primitive monkey brains are good at ignoring and underestimating abstract and hard to understand risks”.

This is a linkpost for https://​​amistrongeryet.substack.com/​​p/​​alphaproof-and-openai-o1

The latest advances in AI reasoning come from OpenAI’s o1 and Google’s AlphaProof. In this post, I explore how these new models work, and what that tells us about the path to AGI.

Interestingly, unlike GPT-2 → GPT-3 → GPT-4, neither of these models rely on increased scale to drive capabilities. Instead, both systems rely on training data that shows, not just the solution to a problem, but the path to that solution. This opens a new frontier for progress in AI capabilities: how to create that sort of data?

In this post, I review what is known about how AlphaProof and o1 work, discuss the connection between their training data and their capabilities, and identify some problems that remain to be solved in order for capabilities to continue to progress along this path.

Ok. Firstly I do think your “Embodied information” is real. I just think it’s pretty small. You need the molecular structure for 4 base pairs of DNA, and for 30 ish protiens. And this wikipedia page. https://​​en.wikipedia.org/​​wiki/​​DNA_and_RNA_codon_tables

That seems to be in the kilobytes. It’s a rather small amount of information compared to DNA.

Epigenetics is about extra tags that get added. So theoretically the amount of information could be nearly as much as in the DNA. For example, methyization can happen on A and C, so that’s 1 bit per base pair, in theory.

Also, the structure of DNA hasn’t changed much since early micro-organisms existed. Neither has a lot of the other embodied information.

Therefore the information doesn’t contain optimization over intelligence, because all life forms with a brain had the same DNA.

Humans are better than LLM’s at highly abstact tasks like quantum physics or haskel programming.

You can’t argue that this is a result of billions of years of evolution. Sea sponges weren’t running crude haskel programs a billion years ago.

Therefore, whatever data the human brain has, it is highly general information about intelligence.

Suppose we put the full human genome, plus a lot of data about DNA and protein structure, into the LLM training data. In theory, the LLM has all the data that evolution worked so hard to produce. In practice, LLM’s aren’t smart enough to come up with fundamental insights about the nature of intelligence from the raw human genome.

So there is some piece of data, with a length between a few bits and several megabytes, that is implicitly encoded in the human genome, and that describes an algorithm for higher intelligence in general.

If it’s a collection of millions of unintelligible interacting “hacks” tuned to statistical properties of the environment, then maybe not.

Well those “hacks” would have to generalize well. Modern humans operate WAY out of distribution and work on very different problems.

Would interacting hacks that were optimized to hunt mammoths also happen to work in solving abstract maths problems?

So how would this work. There would need to be a set of complicated hacks that work on all sorts of problems, including abstract maths. Abstract maths has limitless training data in theory. And if the hacks apply to all sorts of problems, then data on all sorts of problems is useful in finding the hacks.

If the hacks contain a million bits of information, and help answer a million true/​false questions, then they are in principle findable with sufficient compute.

Also, bear in mind that evolution is INCREADIBLY data inefficient. Yes there are a huge number of ancestors. But evolution only finds out how many children got produced. A human can look at a graph and realize that a 1% increase in parameter X causes a 1% improvement in performance. Evolution randomly makes some individual with 1% more X, and they get killed by a tiger. Bad luck.

And again. Most of the billions of years there were no brains at all. The gap between humans and monkeyish creatures is a few Million years.

AIXI is a theoretical model of an ideal intelligence, it’s a few lines of maths.

I’m not saying it’s totally impossible that there is some weird form of evolution data wall. But mostly this looks like a fairly straightforward insight, possessable, and not possessed by us. I think it’s pretty clear that the human algorithm makes at least a modest amount of sense and isn’t too hard to find with trial and error on the same training dataset. (When the dataset is large, and the amount of outer optimization is fairly modest, the risk of overfitting in the outer stage is small)

https://​​www.lesswrong.com/​​posts/​​ZiRKzx3yv7NyA5rjF/​​the-robots-ai-and-unemployment-anti-faq

Once AI does get that level of intelligence, jobs should be the least of our concerns. Utopia or extinction, our future is up to the AI.

> It also seems vanishingly unlikely that the pressures on middle class jobs, artists, and writers will decrease even if we rolled back the last 5 years of progress in AI—but we wouldn’t have the accompanying productivity gains which could be used to pay for UBI or other programs.

When plenty of people are saying that AGI is likely to cause human extinction, and the worst scenario you can come up with is middle class jobs, your side is the safe one.

I think your notion of “environmental progress” itself is skewing things.

When humans were hunter gatherers, we didn’t have much ability to modify our surroundings.

Currently, we are bemoaning global warming, but if the earth was cooling instead, we would bemoan that too.

Environmentalism seems to only look at part of the effects.

No one boasts about how high the biodiversity is at zoos. No one is talking about cities being a great habitat for pigeons as an environmental success story.

The whole idea around the environmentalist movement is the naturalistic fallacy turned up to 11. Any change made by humans automatically becomes a problem.

It’s goal seems to be “make the earth resemble what it would look like had humans never existed”.

(Name one way humans made an improvement to some aspect of the environment compared to what it was a million years ago)

A goal that kind of gets harder by default as humanities ability to modify the earth increases.

One system I think would be good is issue based voting.

So for example, there would be several candidates for the position of “health minister”, and everyone gets to vote on that.

And independently people get to vote for the next minister for education.

Other interesting add ons include voting for an abstract organization, not a person. One person that decides everything is an option on the balot, but so are various organizations, with various decision procedures. You can vote for the team that listens to prediction markets, or even some sub-democracy system. (Because the organizations can use arbitrary mechanisms, including more votes, teams of people, whatever they like)

Approval voting is good.

An interesting option is to run a 1-of-many election.

So you can cast a ballot in the health-election or in the education election or in the energy election or …, depending on which issue you feel most strongly about. (But you can’t vote on both issues at one time.) This has a nice property that the fewer people care about a topic, the further your vote goes if you decide to vote on that topic.

Solving global warming

Most of the current attempts that interact with everyday people are random greenwashing trying to get people to sort recycling or use paper straws. Yes solar panel tech is advancing, but that’s kind of in the background to most peoples day to day lives.

And all this goal is promising is that things won’t get worse via climate change. It isn’t actually a vision of positive change.

A future with ultra cheap energy, electric short range aviation in common use etc.

Building true artificial intelligence (AGI, or artificial general intelligence)

Half the experts are warning that this is a poisoned chalice. Can we not unite towards this goal until/​unless we come to a conclusion that the risk of human extinction from AGI takeover is low.

Also, if we do succeed in AGI alignment, the line from AGI to good things is very abstract.

What specific nice thing will the AGI do? (The actual answer is also likely to be a bizarre world full of uploaded minds or something. Real utopian futures aren’t obliged to make sense to the average person within a 5 minute explanation.)

Colonizing Mars

Feels like a useless vanity project. (See the moon landings. Lots of PR, not much practical benefit.)

How about something like curing aging? Even the war on cancer was a reasonable vision of a positive improvement.

So, even if Sam Altman would declare tomorrow that he has built a digital God and that he will roll it out for free to everyone, this would not immediately lead to full automation.

Superintelligent AI (whether friendly or not) may not feel obliged to follow human laws and customs that slow down regular automation.

The antibodies argument always made the most sense to me. But note that this is an argument for some breast milk, not an all-breast-milk diet—that is, it’s not an argument against formula, just an argument against an all-formula diet. I mention this because when we were in the hospital with our kid, they were pushing against formula very hard.

Also, it’s not an argument for literal feeding at the breast, as opposed to pumping and then bottle-feeding with the breast milk, which is easier for some people.

Emily Oster covers breastfeeding in Chapter 4 of Cribsheet, more extensively than in the 538 article you linked to. IIRC, she notes that there is evidence of benefit for the mother in terms of reduced breast cancer risk (no idea why that would be, though).

(But in general, I agree that Oster is too quick to say “it doesn’t matter” about things that we don’t have rigorous evidence for, rather than trying to make an informed decision about the best course of action based on what data and theories we do have. Other than that minor criticism, though, I am a big fan of her work.)

Here are my initial suggestions, keeping in mind that chaos theory is a subset of dynamical systems theory. For some modern applications it has been subsumed as an element of applied complexity science.

1. The Predictors: How a band of maverick physicists used chaos theory to trade their way to a fortune on Wall Street, Thomas A. Bass, Henry Holt and Company, New York, 1999.

2. Increasing Returns and Path Dependence in the Economy, W. Brian Arthur, The University of Michigan Press, 1994

3. Complexity and the Economy, W. Brian Arthur, Oxford University Press, 2015

4. Complexity Economics, Dialogues of the Applied Complexity Network, W. Brian Arthur, Eric D. Beinhocker, Allison Stanger, Editors, SFI Press, 2020

5. Making Sense of Chaos: A Better Economics for a Better World, J. Doyne Farmer, Penguin Random House UK, 2024

6. Chaos and Nonlinear Dynamics: An Introduction for Scientists and Engineers, 2nd Edition, Robert C. Hilborn, Oxford University Press, 2000. Take a look at Chapter 11 and Chapter 12.

7. Nonlinear Spatio-Temporal Dynamics and Chaos in Semiconductors, Eckehard Scholl, Cambridge University Press, 2001

8. Chaos in Circuits and Systems, Guanrong Chen and Tetsushi Ueta, Editors, World Scientific Press, 2002. Recommended Chapters: 5, 6,10, 12,13, 17, 22, 23, 24, 26, 27.

Many of the engineering applications can be found in application dependent modern circuit designs. Spread spectrum communications and advanced defense electronic applications make use of tools and methods from items 6, 7, and 8.

I hope these are helpful.

Elizabeth and Alex—This is an interesting article. More broadly, there is a significant community of scientists/​engineers/​mathematicians that have worked in dynamical systems, nonlinear dynamics, chaos theory, and applications of chaotic dynamics. A reasonable number of technical monographs and textbooks have been written on these areas since the early to mid 1980’s. Applications exist in electrical engineering, mechanical engineering, fluid dynamics, biology, ecology, economics, networks, medical science, applied mathematics,…. I am not sure whether you want to do a follow up article, or devote time to further investigation, but I am happy to provide pointers to relevant literature if you like.

—Evgeny Sedukhin—“Symphony of the sixth blast furnace” (1979)

This is a huge undertaking, and I admire and respect both your industry and your courage in it.

Two questions/​comments:

Is there a page I can bookmark that will have links to everything that has been published up to that date? I haven’t found it if one exists.

I love the core value of human life first made explicit in what’s been published to date, but I’m struggling with the concept of agency. I feel like the word agency gets thrown around a lot these days, and I’m afraid that may have muddied the waters for me at least in understanding the ultimate point.

But is agency even necessary? What if it was just techno-humanism without this additional concept of agency? That’s what I find myself wondering at this stage. I found some earlier work by the author here: https://​​blog.rootsofprogress.org/​​progress-humanism-agency and I’m guessing the same or similar use of “agency” is intended. I’ll keep reading to see how this plays out, but at least for now I’m not seeing the case for agency clearly — meaning both that I don’t quite understand what agency means and that whatever it means it’s not clear how it’s a necessary moral imperative to human progress.

This may be a semantic argument in the sense that if what is meant by agency boils down to something like attention to pragmatic fitting of means to ends, then this argument for agency would be much clearer. And I may be idiosyncratic in how distracted I get worrying that “agency” in the sense the author is using the term here requires an appeal to “free will,” which I believe both ambiguous and unnecessary.

I’ll keep reading to see what it plays out.

For comparison, Baltimore City public schools have a budget of $1.7 billion and about 78,000 students, or about $21,800 per student (although it’s reported that they only spend $16,370 per student; I’m not sure where the discrepancy comes from). Also note that, in Baltimore City’s budget, they report spending about ¹⁄₃ of their budget on support services, as opposed to instruction.

most of the time if you look close there is an * on official $/​child stats, which excludes buildings, ‘long term’ costs(bond repayments), and cross-government payments(school/​district/​state resource sharing and budget games).

It is convenient when requesting increased funding from voters to be able to use a 25-35% lower number and pretend that everyone implicitly knows you are excluding all facilities and ‘fixed’ costs ofc :/​

recommend skimming your local district’s annual superintendent report sometime. Balance sheet totals/​student are a fun one in appreciating property markets ;)

Will be quite curious to see if this conference is sufficient to revive this forum.

Crossposted from Effective Altruism Forum: https://​​forum.effectivealtruism.org/​​posts/​​vjQ5BhKnDyY35dXXf/​​chomsky-vs-pax-democratica

Hi,
My name is Sean and I’m just a fan of Roots of Progress. Just wanted to introduce myself. I’m a developer, founder, and I’m am trying to get into writing about agriculture and economics. I don’t see a lot of intro posts so I might get in trouble for posting this. But hi anyway!

Relevance of Management Science to Progress Studies

Two questions for anyone here with an interest in them:

1. How do people in the progress studies movement view the science of management?

2. Who else in the group (or the movement) might also interested in this subject, and what kinds of issues or questions seem to be paramount?

By “management” I’m referring to the process of administering and controlling the affairs of the organization, irrespective of its nature, type, and size.

Michael Webb

In my experience, the existing educational model often fails to cultivate curiosity and genuine hunger for learning. It teaches us WHAT to learn but falls short on imparting the more significant aspect of HOW to learn. The knowledge available to us now is such a precious gift but is seldom presented as such. This article was a great read! Thank you for penning it.

Another brilliant resource that covers the same topic is this talk from the late Sir Ken Robinson.

Interesting article! It made me all the more hopeful for the future.

Tony, I could take over operations of the group so that at least the progress movement has some meta presence. I agree with you general conclusion though...it’s not a place for nuanced discussion. Message me on Facebook when you get a moment.

“If everyone agreed to become vegetarian, leaving little or nothing for livestock, the present 1.4 billion hectares of arable land (3.5 billion acres) would support about 10 billion people”—EO Wilson

[though limited fishing + hunting of deer/​pigeons/​waterfowl could be ok]

Great post. It’s nice to hear good news, and more evidence that humanity doesn’t actually run out of resources.

Agreed. The US is a massive country, and we barely use any of the space we have.

I like the idea, and the spirit of trying new ideas and forms of government.

That being said, Wikipedia isn’t without its own issues. The editorial hierarchy, like every bureaucracy, becomes rigid, brittle, and ossified over time. The predominant viewpoint becomes entrenched. And so on.

I very much believe that our representative democracy has grave flaws, and that a wiki-based form of direct democracy answers some of them. But how would you keep the system from being games? At some point, someone has to have the power to approve the edits, and the politicking to be that person becomes fiercer the higher the stakes involved.

Chaotic Progress

A book review in essay format I wrote to help nuance what I see as the unrealistic rhetoric on both sides of the political spectrum right now.

https://​​www.symmetrybroken.com/​​chaotic-progress/​​

Relevant—https://​​www.nytimes.com/​​interactive/​​2024/​​03/​​13/​​climate/​​electric-power-climate-change.html

Important to mention the HHMI funding model, which puts “trust” in PIs/​evaluates them as PIs rather than as research proposals.

Also

“The MacArthur grant is a template for what this looks like on a personal level, with a shift in focus from creativity to integrity, and a bump in compensation – $625,000 is a lot of money, but that money is designed to be seed money for an activity rather than financial security. Those getting a MacArthur grant still face the specter of future financial needs. One needs an order of magnitude more than that over a lifetime to be secure while not compromising one’s interactions with society. ”

What have been the longest scientific experiments? What would be worthwhile 25-year experiments?

I want someone to write an actual post on this but would also be happy to hear some initial, short answers.

I remember reading something on this or hearing these points made in a podcast but can’t recall the source!

There should be some kind of official recognition + prize for people providing public goods on the Internet. There exist prizes for free software and open-source projects, but this does not cover even remotely the amount of intangible value people can deliver on the Internet.
Examples include: https://​​avherald.com, but also a lot of open-source projects. and maybe people like patio11, gwern, Lilian Weng, or Bartosz Ciechanowski. Some YouTubers would also likely qualify, but I’m not very familiar with the medium.

Theory of change: just increase the rate of reminders to people that if they are highly competent and passionate about something not directly marketable, the Internet has made it so that they can easily increase the amount of beauty in the world by making their passion a public project.

First of all, thank you for this post.
This, at least, it was useful to sit and think a bit about the matter.
As I wear the clothes of the Devil’s Advocate, I’d like to do a concurrent post to state why the arguments you put in front were not acceptable (when not actually impossible):

More geniuses

We can state without any doubt that the Earth population incredibly increased in the last decades. All I can experience, especially in the last decades, is the poor quality of the humans (at least, judging from their posts in the social networks). This means:
- more recurring grammatical/​orthographic errors (people who don’t know even their home country’s language);
- people speaking about topics where they’ve got not minimal experience (hence, spreading out ignorance rather than knowledge);
- people speaking not to add something to the debate but just on a “clickbait” basis.

And I mentioned only the too evident issues. There are many other issues, some related to the above.
To cut a long story short: if you can’t provide people a proper education anywhere (use of the technologies included), you are not increasing the chance of getting geniuses but the chance to widen the ignorance pool where the humanity is going to feed on.

More progress

• Total investment. I wish it was so. This is (at least) a naive conception of humanity. I’m going to give you good news and bad news: good news are that the money for such investments already exist; the bad news are that the owners of their money don’t want to spend a single penny for them. So, if you want to get more investment, you have to create politically and economically the reasons for they would find profitable (or more profitable) to do so;

• Specialization. Here we go to a previous issue: if you can’t provide people a proper education, you are not creating many more specialized workers but only more people who always have a worse knowledge of the basics of their job (and of the other jobs too, which means always more people wanting to have their say to “actually teach papa to screw”);

• Larger markets I’m going to give you the same bad news and good news I gave you before. These markets would actually exist already now, but we’re using them in the wrong manner. Gandhi used to say: “Earth has enough resources to satisfy anybody’s need but not to satisfy anybody’s GREED. If only we could understand (and make it understandable to the right people) the difference between need and greed your hopes would be somehow possible;

• More ambitious projects
  Stop.
  If you studied Ancient History properly you must know that the way Ancient Egypt took large masses of people to work to these more ambitious projects was force them to do that in SLAVERY. We should have put behind us that word from the Enlightenment Age (which—it is useful to remember—also made a significant contribution to the creation of the United States of America).
  Are you REALLY sure that you want to turn back the clock of Time to THAT? (not to mention that, in spite of that, still today in the 21st Century we have large portions of Earth where we keep on accepting, socially or politically, slavery).

“In fact, these factors may represent not only opportunities but requirements for progress. There is evidence that simply to maintain a constant rate of exponential economic growth requires exponentially growing investment in R&D. This investment is partly financial capital, but also partly human capital—that is, we need an exponentially growing base of researchers.”
-
You keep on considering the issue in terms of QUANTITY, forgetting totally the QUALITY, which is what, in your perspective, really makes the difference.

“But in the long run, an exponentially growing base of researchers is impossible without an exponentially growing population. In fact, in some models of economic growth, the long-run growth rate in per-capita GDP is directly proportional to the growth rate of the population.”
-
What kind of models of economic growth are we talking about?
With numbers at hand, in the current models of economic growth that are applied globally from those self-named “civilized countries”, the only growing thing is poverty. In favor of the concentration of wealth in fewer and fewer hands. Is this your conception of “growth” and “civilization”?
All the above is also weakening Earth security.
Because the poorer are people, the lesser are valuable their lives.
And the lesser are valuable their lives, the easier would be they are going to devote them to criminal projects in exchange of some quid, whatever they may be.
Just because they’ve got actually nothing to lose (neither their own life itself).

To cut a long story short, in the current models of economic growth the only thing growing is—with poverty—the manpower of criminal organizations (terroristic ones included, that you state you wanna fight in words but in the end you are feeding in practice).


More options

• Better matching for aesthetics, style, and taste.

• Better matching to careers.
  
  It’s the same old story: humanity will have more of these if they will be trained to.
  Otherwise, the only things growing—together with poverty and the manpower of criminal organizations—are Fascisms.
  There is an increasing list of countries where, foreign people are not only felt as an advantage, but rather a danger (“they come here to feed criminal activities”, “they come here to steal our jobs”, “they come here to steal our women”, etc.).
  The reason is, from my point of view, quite simple. Let’s say we have a house with three rooms: this house would be comfortable to—let’s say—no more than 3-4 people (one room per person). If you wanna host some more people, you will have to increase the quality of services provided by the house, to compensate that there will be a bit lesser.
  Because one of the fault of your scheme is that you completely ignore that the Earth resources are limited. You want more people but you didn’t tell us how to feed them, how to water them, how to educate them, in a world that’s already at stake to do that with the current number.
  So, going on with my metaphor, we’ve got currently the three room house (that—it has to be clear—won’t have any more room, at least in the short term) with about 6 people in it, where 1 person has his/​her own room and the other 5 are fighting among them to compete for the remaining 2 rooms. Let’s say that 2 of the 5 people fighting will find an agreement to get another room together in the end (hopefully, in an optimistic perspective). There are still 3 people remaining and fighting to compete for the only room left. And let’s also say that, if in a house there are three people fighting among themselves, cohabitation is not the maximum of happiness even for the other three who are trying to live in peace.
  And would you want to get in some more people in such a toxic environment?
  What exciting perspectives of life could you provide them?
  

• Better matching to other people. Even on this side, it’s the same old story.
  You won’t solve the problem if you increase the number of chances without increasing the quality of chances.
  I can offer my personal single heterosexual man, if someone can find it useful.
  What I experienced on my own skin is that our consumerist vision of society has led us to also consider interpersonal relationships as “products”. This means that we no longer want to build and spend on improving a relationship with anyone when, given the huge availability of choices, we can fuck off our partner and take another one with the same nonchalance we have in changing the brand of our washing machine soap.
  And when you’ve got another partner the game starts again.
  Women (I talk about them because I’m interested in women, but I guess I can tell the same thing about men) spend their lives collecting dozens of toxic relationships where they always pour on the new partners all the toxicity they have had from the previous ones, charging the righteous for the sinner. And in the end, the only thing left to do is escaping from that hell.
  The result is that, in the end, neither of them both will believe in a healthy relationship anymore, but they will always think the other as an automatic dispenser of money, favors and sex (if you manage to get to sex). And when they can’t accept the other one has gone away and has the right to have a less toxic relationship, the story often ends with a murder or a suicide, when not with them both.
  I think this is not the right way to achieve an increasing Earth population.
  I think the time has come to provide our children with a “sentimental education” before a sexual one. Something that will teach them to respect other people and to treat people as people and things as things, and not people as things and things as people. But it’s very hard to teach to someone something you haven’t learnt for yourself yet. So, before teaching to them, we should start to learn this for us in advance!
  To cut a long story short again, we have to invest more in the quality of what we’ve got instead of more quantity!

• More niche communities.

• More niche markets.
  You will have to build up in advance a world with a social, cultural and economic system that would encourage this. Until then, your project is simply not possible, when not naive or incoherent.
  Because in the current one, you are doing exactly the opposite.

Deeper patterns

• The existence of non-rival goods.

• Economies of scale.

• Network effects and Metcalfe’s law. Value in a network is generated not by nodes but by connections, and the more nodes there are total, the more connections are possible per node. Metcalfe’s law quantifies this: the number of possible connections in a network is proportional to the square of the number of nodes.
  
  Metcalfe’s law talks about the number of nodes. You can get all the nodes you want, but if the quality is poor, they’re going to last the twinkling of an eye.
  And, in the end, the number won’t be exactly what you expected!

A dynamic world

I assume that when Ehrlich and Goodall advocate for much smaller populations, they aren’t literally calling for genocide or hoping for a global catastrophe (although Ehrlich is happy with coercive fertility control programs, and other anti-humanists have expressed hope for “the right virus to come along”).
-
Nice one to talk about these good ideas.
So, if maybe they still have not taken them into consideration (I can’t believe that, but you’ll see...), now they’ll surely do!

A world with a large and growing population is a dynamic world that can create and sustain progress.
-
I remain with my opinion that, before thinking of a larger population on Earth, we should work on improving the quality of life of those who are still living here, in advance.
And there are still loads of things to do just to make it become acceptable.
I’d love to change my mind if you’ve got more consistent argumentations than those you already provided in that sense.

Thanks anyway for raising the topic, which is everything but not relevant.
Best regards,
A.F.

America’s First Dual-Use Technology

This is a linkpost for https://​​kinetic.reviews/​​p/​​americas-first-dual-use-technology

With disintegrating 737s in the news, a lot of people are wondering how things got so bad. I’m here to rewind the clock 100+ years to reassure everyone that the aviation industry has always been beset by challenged business models, bad incentives, and shoddy aircraft.

From the first takeoff in 1903 through the late 1920s, “dual-use” aircraft were merely a promise – there was no commercial aviation market. There was, in fact, barely any U.S. market, and it was the inherent appeal of aviation that attracted both entrepreneurs and capital.¹ As nascent markets for passenger transport and airmail transport emerged, so too did the first dual-use application for bombers, accelerating innovation in the early 1930s. Fighters did not lend themselves as nicely to commercial applications, and absent a procurement strategy and meaningful expenditures from either the Army or the Navy, U.S. fighters at the time of Pearl Harbor were inferior to their German and Japanese counterparts.

The mythos surrounding the Wright Brothers, Charles Lindberg, and successful mass production during WWII belies the reality of a pre-WWII aviation industry plagued by poor policies and dysfunctional relations between the government and manufacturers. The Army and Navy remained tepid – often hostile – buyers right up until the outbreak of WWII, and in partnership with Congress, they managed to inflict maximum abuse on manufacturers. During the interwar years, industry subsidized the military for both the development and production of aircraft, losing lots of money along the way. In return, manufacturers were deprived of their intellectual property rights and accused of graft and wartime profiteering. Acquisition law was so punitive towards aircraft manufacturers as late as 1939 that firms were convinced it was a conspiracy led by the automotive industry. It wasn’t.

Early on, there were some Cassandras: “In 1919 Secretary of War Baker called for a long term procurement program for military aircraft and warned Congress that “it cannot be expected that industry will long engage in an unremunerative line,” but he underestimated the aircraft manufacturers for whom the industry’s appeal defied rational calculation (The Politics of Aircraft, pg 45).”² Indeed, the optimism-cum-masochism of early aviation entrepreneurs (e.g., Douglas, Grumman, Martin, etc.) enabled bad policy to persist.

The military-industrial complex as we know it did not exist prior to WWII, yet this period serves as a reflecting pool for the best and worst attributes of our modern defense industry.

Pre-WWI and WWI

During the first decade of the twentieth century, the U.S. led the world in heavier-than-air aircraft. We ceded this early lead to Europe in part because of patent disputes between the Wrights and Curtiss and in part because of late entry into WWI. However, even before the outbreak of war, Europe showed a much greater interest in the military utility of aviation. From 1909-1911, the U.S. Army owned one military aircraft. In comparison, France owned over 250 planes by 1912. By the time the U.S. entered the war, its domestic aircraft industry was so far behind that of its peers that the U.S. almost exclusively used foreign combat aircraft of French or British design, with the American Expeditionary Force flying the French SPAD. (The U.S. Combat Aircraft Industry, 1909-2000).

The first true test of U.S. industry-government relations was an abject failure. In 1917, Congress passed a Hail Mary, $640 million “Aero Bill,” which at the time was the largest congressional appropriation ever.³ Industry was meant to fulfill the government’s vision of mass production of aircraft, but aircraft of the era were truly “crafts” made of wood and fabric and not ready for the assembly line. This problem was exacerbated by the decision to have industry manufacture foreign designs, as there were no cutting-edge American combat designs ready for mass production.

Building a foreign fighter and retooling a factory around it proved too challenging. The U.S. licensed the SPAD design from France and contracted Curtiss Aeroplane to manufacture it, but the firm was wholly unequipped to produce the thousands of fighters expected and nearly went bankrupt trying to do so. Its early units of SPADs were immediately declared “worse than useless,” and the Army Signal Corp told Curtiss to produce the British Bristol instead. But when Curtiss failed to produce a light enough version of the Bristol, it was back to the SPAD – this time, a new variant (The Politics of Aircraft, pgs 33-36).

Glenn L. Martin refused to commit to unreasonable production expectations, explaining it was only possible for his firm to manufacture three planes a day. He was blacklisted from the war effort until summer 1918 when the Army had him build prototypes of his excellent MB-1 bomber.

At the end of the war, the government did not receive anywhere near the value of aircraft for the money spent. Rather than consider that high technology aircraft were incompatible with mass production under the conditions, President Wilson, Congress, the media, and much of the War Department concluded the failed effort was a big business conspiracy to raid the wartime coffers of the government. It was the last time the industry operated largely unregulated.

The Interwar Years: The Myth of the Fungible Engineer

After WWI, the government decided it would no longer license foreign designs, but it did not learn any lessons about the difficulty of having one firm produce a different firm’s plane. In the interwar years, the root cause of industry’s struggles was the government’s decision to separate design contracts from production contracts.⁴ Ostensibly, this was to ensure there was competition at every phase of the acquisition, ensuring the best deal for Uncle Sam. In reality, the attempts to cleave R&D from production were disastrous for all parties. Here is how the process worked:

Firms would respond to government specifications with their paper designs and the estimated cost to build the prototype. The government would select two to four firms to build a prototype and have a fly-off. The government then acquired the design rights to the winning firm’s prototype. Firms consistently lost money on prototyping in the hope of making a profit on the production contract – provided they could win it, that is.

The government then held a separate competition to determine which firm(s) would produce the winner’s prototype. Production competitions were almost solely based on price. This meant the firm who had the winning prototype was at a huge disadvantage because it needed to amortize its design costs, so its production bid inevitably came in higher. The firm that won the contract for production received just the winner’s prototype and had to reverse engineer the blueprints; although as we will see, having the blueprints would have been of little use.

The government treated aircraft as standardized goods to be manufactured, but aircraft were not widgets. This decision to hold pure-price manufacturing competitions resulted in manufacturing failures that make Boeing’s recent plane window blowout look positively quaint (emphasis mine):

1) Martin sold his aforementioned excellent MB-1 prototype bomber to the Army at a loss. “In 1919 [he] was underbid by three contractors on the bomber’s production. Martin was given twenty to build anyway but lost money on the reduced volume, suspended the bomber’s development, and in disgust, declined to deal with the army until 1931. L.W.F. Engineering built fifty, Aeromarine built twenty-five, and Curtiss built fifty. The bombers in the field proved to be completely different airplanes of widely varying quality (The Politics of Aircraft, pg 59).”

2) “In 1923 Curtiss lost $182,000 on a $175,000 development contract for the successful navy Curtiss Scout …In the competition to build 40 Scouts, Curtiss submitted a bid of $32,000 per plane, but Glenn Martin won the contract at $23,000 apiece. Martin complained that the plane came with no blueprints but admitted that they would have been useless in his shop anyway. His staff drew up new blueprints and in the process produced an entirely new plane inferior in performance to the Curtiss design (The Politics of Aircraft, pg 62).”

3) In 1919, the Ordnance Engineering Company developed Orenco-D, “the best pursuit in that day,” but Curtiss won production for 50 of the planes. “Ordnance Engineering liquidated; and the fifty planes built by Curtiss had to be destroyed as unsafe (The Politics of Aircraft, pg 59).”

4) “The demise of Thomas-Morse, the designer of a reasonably successful indigenous fighter design (based on the French SPAD), can be directly traced to the award of the production contract in 1921 to Boeing, which had little ability to design an advanced fighter but which underbid Morse on the production contract.”

And so the myth of the fungible engineer was born.

The myth holds that any worker is the same as any worker, and any production line is the same as any other production line. It’s a fundamentally un-American concept and reveals a captured, collectivist mindset. The belief that the sum total of innovation could be captured in a prototype or blueprint rather than in a network of humans doing exceptional things was demonstrably false. It didn’t matter that the government owned the most innovative aircraft designs – absent the designers, the prototypes were unlikely to be faithfully manufactured at scale.

It’s hard to overstate how damaging the myth of the fungible engineer is in a dynamic industry undergoing rapid technological change. With aircraft, a prototype was almost immediately rendered obsolete upon selection. Had the firm with the winning design been the one to manufacture it, it would have been able to incorporate new innovations during the manufacturing process. But the firm most able to manufacture the winning prototype was the least likely to win. The implications for innovation are clear: Firms were incentivized to keep a bare-bones design staff to minimize R&D expense, since there was no money to be made in designing the most innovative airplane.

To quote Freedom’s Forge author Arthur Herman, “it’s through making things that we learn what can be made better, which is why the most productive companies also tend to be the most innovative.” In what appears to be an oxymoron, aircraft manufacturers were deprived of the opportunity to manufacture their aircraft. And because aviation is a relatively low volume industry, missing out on the limited military production orders of the interwar years had a high opportunity cost. As

Brian Potter writes on production learning curves:

If you’ve produced 1,000,000 of something, whether you make another 500 or 5,000 will make almost no difference in learning curve terms. But if you’ve only made 50 of something, making another 500 makes a huge difference in the level of cost reduction that can be achieved. Thus, if you only plan to sell a few hundred of something, a relatively small number of sales will have a large impact on how efficiently you’re producing and how profitable you are.

Separately competing design from production was a short-sighted acquisition strategy, although the taxpayer temporarily got a great deal. The government was paying less than the full cost for a prototype, and then it paid less than the full cost for production. Industry was subsidizing the government! But it was unsustainable for industry to operate unprofitably, and more importantly, it resulted in the warfighter getting a bad product. Note how the health of industry and national security share an intimate relationship.

Not everyone had blinders on. The legendary U.S. Navy Admiral Moffett, then director of the Bureau of Aeronautics and credited with introducing the aircraft carrier, believed “the distinction between design and production was meaningless and an obstacle to procurement” and “procurement laws dishonored the government.” Although price competition on manufacturing orders was the default, there were some loopholes that allowed for “negotiated contracts,” where a contracting officer (CO) could award the manufacturing contract to the firm with the winning prototype without a full competition. Moffett pushed for the use of negotiated contracts whenever possible, but extreme risk aversion from COs prevented them from being used with any regularity (The Politics of Aircraft, pg 86).

This piece is supposed to be focused on the origin story of the myth of the fungible engineer, but I can’t help but do a quick diversion on the origin story of the CYA, risk-averse, CO. Many will read the above manufacturing failures and ask how it’s possible that the COs – faced with such incontrovertible proof – would still pursue price competition over negotiated contracts. Were they simply useful idiots?

A closer examination of incentive structure reveals they were just acting rationally. Congressional scrutiny, allegations of wartime fraud, charges of favoritism or collusion, and an extra-long contract review process loomed large for the CO who went with the negotiated contract. It was much easier to pick the firm that could do it the cheapest – actually receiving the plane purchased was of secondary concern.

Things haven’t changed much, and we continue to encourage COs to adhere to process at the expense of outcome. As Pete Modigliani and Matt MacGregor write in their recent summary of the DoD Inspector General’s Audit of Cost-Plus-Award-Fee Contracts:

To conclude, in reviewing $32B of contracts, COs regularly did not follow policies, but still effectively managed the contracts for most of them, with a 0.015% of improper payments not fully justified in the contract files. As a result, DoD IG recommends increased oversight and controls. And people wonder why many COs are risk averse.

The Fungible Engineer is Alive and [Un]Well

I wish we could say we learned from this chapter in history, but the myth of the fungible engineer is the central tenet in acquisition today. We see it manifested in two ways. First, the government largely acquires software with a labor-based, butts-in-seats model that does not account for individual exceptionalism. From Trae Stephens still-relevant 2016 piece “Innovation Deficit: Why DC is Losing Silicon Valley”

The average Request for Proposal (RFP) involving software development services requires estimates on count of full-time equivalents (FTEs) who will be engaged in the effort, a “basis of estimate” (BOE) modeling FTEs, their labor categories, and the FTE blend deployed against RFP requirements and delivery dates (e.g. requirement “x” needs 2.5 engineers for 6 months to complete). This process massively disadvantages efficient and more talented teams. Because the talent gap between average and excellent is so large, it would generally be better to have one Lebron-level coder than to have 100 average ones.

Second, the government continues to equate ownership of source code, diagrams, and prototypes with innovation – once again, not realizing it is the networked people surrounding these artifacts that breathe life into them. While it is no longer an option for Boeing to bid on the production of Lockheed’s fighter, the pre-WWII mindset around the value of owning atoms has extended to owning bits. A recent report from the United States Geospatial Intelligence Foundation (USGIF) enumerates in meticulous detail different solicitations from the National Reconnaissance Office (NRO) effectively boxing out commercial software by requiring aggressive ownership of IP – and these examples are just from a single government agency.⁵

Running and maintaining a software system is much closer to producing an aircraft at scale than it is to designing and delivering a prototype for a fly-off. Despite its proliferation of software factories, the government still has not internalized this point. For a thought experiment: if the government acquired the code base of OpenAI and turned it over to [insert favorite Systems Integrator], do you think said Systems Integrator would continue on OpenAI’s trajectory of building a next generation AI company? Relatedly, as any entrepreneur raising venture capital money for his/​her startup will tell you, leading your pitch with a list of patents is not a winning strategy.

One final point. Right after WWII, there was a race by the Allies to seize as much knowledge from the Germans as possible. America’s “acquisition” of one Werner von Braun via Operation Paperclip was by far the most successful of these technology transfers. But America and the UK also pursued an expensive strategy of microfilming and translating millions of documents, which was not successful. France, who didn’t have the resources or political capital to pursue this strategy, instead embedded trainees into German research centers, maintaining the intangible value of a scientist’s network. This people-centric strategy was very successful for the French at a fraction of the cost.⁶

America has a deep bench of Founding Fathers and entrepreneurs we celebrate. China only has Mao, maybe Xi. They had to disappear Jack Ma because the CCP does not accommodate outliers. Our acquisition system should reject the myth of the fungible engineer and instead reflect the time-honored American tradition of elevating the individual over the collective.

I’m only comfortable with more humans if we make serious efforts to increase the percent of vegans/​vegetarians, or reduce the amount of meat needed by human, given how wasteful meat is to land use.

[limited hunting of deer/​pigeons/​waterfowl could be ok, but people would still need to eat way less meat than they do now]

maybe, just maybe, cultured meat will come just enough time to save us. But the probability range of this is still too high for comfort.

I very strongly believe any pro-fertility incentives must be coupled with incentives to decrease meat consumption/​increase vegetarianism (or increase % of power from nuclear, given that excess solar energy will also destroy habitat).

EO Wilson once advocated that 50% of the world is set aside for wilderness. Maybe that’s a bit too ambitious, but I think 35% would be good enough to preserve enough biodiversity. Taiwan/​Hong Kong/​Japan are all densely populated and have roughly that % of wilderness/​forest (they also rely a lot on seafood, which does not destroy wilderness , but is still inherently a limited resource given overfishing). People get huge health/​happiness benefits from nature exposure too, esp when the nature is close to where they live

“If everyone agreed to become vegetarian, leaving little or nothing for livestock, the present 1.4 billion hectares of arable land (3.5 billion acres) would support about 10 billion people”—EO Wilson

https://​​www.pnas.org/​​doi/​​10.1073/​​pnas.2204892120 is one of the most depressing findings ever.

How do you optimize your life for serendipity?

Ken Stanley has a new social network and I asked the question here: https://​​app.heymaven.com/​​discover/​​25623

But progress studies forum should have more people who can answer this (esp b/​c serendipity and progress are both close allies)

That requires a long process and takes a lot of research time

I wrote my rebuttal to this article here.

This is a brilliant article. My father used to work at IBM ARC back in the 1990s, and you’re describing how things worked there, during a period in which numerous Nobels were earned working within a corporation.

The jack-of-all-trades approach to being a PI was also part of what drove me personally out of a Ph.D. program and into industry. I didn’t want to be a solo entrepreneur constantly writing grants for peanuts. The most attractive jobs to me back then (early 2000s) seemed to be the government lab jobs that had no teaching responsibilities, and relatively less pressure to raise grant funding constantly.

We need to encourage more divisions of labor in the practice of scientific research. This is a great diagnosis of some of the fundamental problems of our current system. Kudos, Jason!

I don’t fault you for not mentioning it because it’s definitely more speculative. But as a voiceover I would add to this pitch for block funding the observation that in many cases, getting a large enough group together seems to lead to a critical mass, whereby new ideas start to fission off of each other. I think we’re seeing something like that now at the Flatiron institute, and probably because they’re doing many of things you’ve prescribed.

Great discussion and examples. I often struggle to explain why bad business fails without regulators but is propped up by them far too often. This was a helpful article on the topic and your ideas are valuable. Keep writing!

>Throughout history, fearmongering has been used to justify a lot of extreme measures.

And throughout history, people have dismissed real risks and been caught with their pants down. What, in 2018 or Feb 2020 would appear to be pretty extreme measures at pandemic prevention would make total sense from our point of view.

Countries can and do spend a huge pile of money to defend themselves from various things. Including huge militaries to defend themselves from invasion etc.

All sorts of technologies come with various safety measures.

>For a more concrete example, one could draw the possibility that video games might cause the players to emulate behaviors, even though you have to be insane to believe the video games are real, to then start advocating for bans of violent video games. However, one could go a step further and say that building games could also make people believe that it’s easy to build things, leading to people building unsafe houses, and what about farming games, or movies, or books?

If you are unable to distinguish the arguments for AI risk from this kind of rubbish, that suggests either you are unable to evaluate argument plausibility, or you are reading a bunch of strawman arguments for AI risk.

>The community wants you to believe in a very pessimistic version of the world where all the alignment ideas don’t work, and AI may suddenly be dangerous at any time even when their behaviors look good and they’re constantly reward for their good behaviors?

I do not know of any specific existing alignment protocol that I am convinced will work.

And again, if the reward button is pressed every time the AI does nice things, there is no selection pressure one way or the other between an AI that wants nice things, and one that wants to press the reward button. The way these “rewards” in ML work is similar to selection pressure in evolution. And humans were selected on to enjoy sex so they produced more babies, and then invented contraception. And this problem has been observed in toy AI problems too.

This isn’t to say that there is no solution. Just that we haven’t yet found a solution.

>The AI alignment difficulty lies somewhere on a spectrum, yet they insist to base the policy on the idea that AI alignment lies somewhere in a narrow band of spectrum that somehow the pessimistic ideas are true, yet we can somehow align the AI anyway, instead of just accepting that humanity’s second best alternative to survival is to build something that will survive and thrive, even if we won’t?

We know alignment isn’t super easy, because we haven’t succeeded yet. We don’t really know how hard it is.

Maybe it’s hopelessly hard. But if your giving up on humanity before you spend 10% of GDP on the problem, your doing something very wrong.

Think of a world where aliens invaded, and the government kind of took a few pot shots at them with a machine gun, and then gave up. After all, the aliens will survive and thrive even if we don’t. And mass mobilization, shifting to a wartime economy… those are extreme measures.

Well

>Invader countries have to defend their conquests and hackers need to have strong information security.

One place where offense went way ahead of defense is with nukes.

However nukes are sufficiently hard to make that only a few big powers have them. Hence balance of power MAD.

If destruction is easy enough, someone will do it.

In the war example, as weapon lethality went up, the fighters moved further apart. So long as both sides have similar weapons and tactics, there exists some range at which you aren’t so close as to be instakilled, nor are you so far as to have no hope of attacking. This balance doesn’t apply to civilian casualties.

Research oriented government agencies already sometimes give grants to private organizations (corporations, typically) for largish chunks of work—climate tech startups get nontrivial amounts of project funding from organizations like NRECA for example. Do these grants work better than grants to PIs? That seems worth finding out. Similar, though less research oriented, is the phenomenon of “Beltway bandit” companies whose revenue comes mostly from defense contracts—I suspect that for better or worse, the incentive structure faced by grant seeking labs in your proposed model would come to resemble these.

Also posted in EA Forum: https://​​forum.effectivealtruism.org/​​posts/​​jJap6KhzFe3mgh32M/​​electric-vehicles-and-renewable-electricity

The thing is, we have many options that aren’t just accelerating or decelerating the whole thing. Like we can choose gain of function research and cutting edge AI capabilities, and accelerate everything except that.

Science is lots of different pieces, differential technological development.

“25% probability that the domain experts are right x 50% chance that it’s not too late for science to affect the onset of the
time of perils x 50% chance that science cannot accelerate us to safety = 6.25%”

This smells of the “multistage fallacy”

You think of something. List a long list of “nesscessary steps”. Estimate middling probabilities for each step. And multiply them together for a small end stage probability.

The problem is, often some of the steps, or all of them, turn out to not be that necessary. And often, if a step had actually happened, it would do so in a way that gave you strong new information about the likelihood of other steps.

Ie if a new device needs 100 new components to be invented, and you naively assume the probability is ⁵⁰⁄₅₀ for each component. But then a massive load of R&D money gets sent towards making the device, and all 100 components are made.

In this particular case, you are assuming a 25% chance the domain experts are right about the level of X-risk. In the remaining 75%, apparently X risk is negligable. There is no possibility for “actually it’s way way worse than the domain experts predicted”.

“x 50% chance that it’s not too late for science to affect the onset of the
time of perils x 50% chance that science cannot accelerate us to safety ”

If the form of the peril is a step. Say a single moment when the first ASI is turned on, then “accelerate to safety” is meaningless. You can’t make the process less risky by rushing through the risky period faster. You can’t make Russian roulette safer by playing it real fast, thus only being at risk for a short time.

Marvelous article! Correlates with my readings in Pinker, Rosling, and the authors (sorry, I don’t have their names, but they have done their homework) of the book “Superabundance.”

Bravo!

I don’t think I have a single criticism...paradigm changing writing. And I have indeed have had a paradigm change over the last few decades, from “Club of Rome “Limits to Growth” and Paul Erlich to graduate to the Abundance mentality of Pinker, Diamandes, and Hans Rosling.

This channel is doing important work in my opinion, as even most progressives and intellectuals are leery of growth as a virtue.

Rudi Hoffman

Port Orange, FL, multiverse.

It isn’t clear that the offense-defense balance directly affects the number of deaths in a conflict in the way that you claim. For example, machine guns nests benefitted the defenders significantly, but could quite easily have resulted in there being more deaths in warfare, due to the use of tactics that hadn’t yet accounted for them.

If you had told people in the 1970s that in 2020 terrorist groups and lone psychopaths could access more computing power than IBM had ever produced at the time from their pocket, what would they have predicted about the offense defense balance of cybersecurity?

I don’t know why you’d think that compute would be the limiting factor here. Absent AI, there are limited ways in which to deploy more compute.

Interesting, well-researched article, IMO. The one component that may make the article more relevant has to do with digital Intellectual Property, which may or may not be the same as patentable ideas. While it seems obvious to me that IP is still property, my nephew, a self-avowed hacker, once said to me of his unabashedly stealing IP “The failure of these business models is not my problem.” Which I thought at the time and still think is a moral/​ethical lapse of good judgement. Is stealing work product like or exactly like patent infringement? “Informations wants to be free!” is a rallying cry, but lacks nuance. Anyway, great article, thank you!

Brilliant and clever summaries of really exciting projects, mostly in the conceptual phases, but still adjacent to current capabilities given appropriate attention and funding!

I wonder if another avenue might be to permit partial retirement via allowing people to draw a prorated share of their Social Security to subsidize remaining with a current employer but perhaps in a part-time role. Currently, there are very strict income limits for those who draw SS before full retirement age (67 years old for my generation). With the growing dearth of young people and labor-force participation stuck in the doldrums, we need to find ways to keep the older yet very productive people—in the first half of their 60s in particular—engaged. It would be good for society, the economy and for individual well-being.

Lovely, fact-based, paradigm shifting article. Bravo! 40 years ago, in college and grad school, my heroes were the “Club of Rome” and others calling for “simple living, high-thinking”. I realize now just how short-sighted, blazingly wrong I was. I wish Paul Erlich and most articles in “Free Inquiry” would have a similar epiphany!Rudi Hoffman, Port Orange, FL

This idea makes a lot of sense, after you’ve decoupled health from age to a much greater degree than today. However, at that point I think the better idea would be “There is no longer a compelling reason for retirement or long gaps between jobs to be a matter of public expense at all, unless you’re doing it for some set of purposes the public cares about.” I think we need to normalize the idea of years-long gaps on resumes, among other things. But I don’t feel compelled to share the expense of different people’s career paths and life choices.

The reason we originally set the retirement age approximately equal to the average lifespan was to ease the burden on people planning for their own futures, so that no one had to suffer for living longer than they’d expected. The fact that we now live many years in relatively poor health near the end of our lives is a very different situation. and we’ve adapted the policy tool we had to try to cover it. The idea of retirement as a long vacation or reward for having worked 40 years is an anomaly. In the near term, we shouldn’t force people to work when they’re not able to, regardless of age, but we also maybe shouldn’t be paying living costs just because someone hits a certain age. Protests and strikes aside, I think a better solution would be to eliminate age-based pensions, and greatly expand disability-related and poverty-related social security programs to the point that everyone who outlives their ability to support themselves is still covered. If you like, phase it in gradually over the next 30 years so that anyone not near the beginning of their careers doesn’t have to worry about it. I realize this would still be a political firestorm, of course, but as noted, this is a scenario where every available option is considered unacceptable to large swaths of the population.

The takes here are suspiciously similar to Vitalik’s techno-optimism d/​acc post https://​​vitalik.eth.limo/​​general/​​2023/​​11/​​27/​​techno_optimism.html#compatible Wondering if there are any thoughts on this?

Wonderful summary of the history and analysis of the economic advantages and disadvantages to a patent system. I’m a patent lawyer who has been worrying about the question of whether patents promote progress for a couple of decades. https://​​www.symmetrybroken.com/​​whats-wrong-with-the-patent-system/​​

Lately, I’ve been partial to the model of progress articulated by North, Wallis, and Weingast (NWW) in Violence and Social Orders. https://​​www.symmetrybroken.com/​​what-the-patent-system-can-learn-from-violence-and-the-social-order/​​

Their observation is that a combination of free entry into political and economic competition has enabled adaptive efficiency, and hence promoted progress, in what they call open access societies.

Patents and corporations both began as royal prérogatives. The Venetian Patent Act is a historical anachronism, representing one of the first (if not the first) open access orders in human history. But it was circumscribed in geography (but not necessarily temporally!). NWW might say that Venetian patents opened access to Venetian markets for foreigners and non-elites who had improvements to the state of the art.

But more generally in view of NWW, I believe the question to ask is whether a given patent is necessary to formation of a corporation or not. Patents as a system of insurance (think third party debt collection for investors) is less appealing, although probably not as socially costly as critics of NPEs would have it. And note that some would distinguish between NPEs and PAEs (patent assertion entites) because universities, government labs, and others never intend to practice what they patent but also don’t rely upon litigation and licensing as their primary source of revenue.

“As Simon Rifkind, the Co-Chairman of the 1967 United States President’s Commission on the Patent put it, ‘the patent system is more essential to getting together the risk capital which it required to exploit and to develop and to apply the contributions of the genius inventor than to provide a stimulus for the actual mental contribution’”

Nothing has changed in over 50 years in this regard. The best reason to file a patent is because without one nobody reasonable is going to feel comfortable investing. But what fields remain in which this is true except for those in which regulatory approval is required before revenue?

“The question, then, is whether patents are associated (1) with more inventing activity overall, and (2) with more productive uses of the inventions once they are invented. In economic terms, we are more looking for evidence that patents facilitate capital formation and technology diffusion, both of which are well-studied and associated with economic growth.”

The evidence for multiple inventions is devastating to the case that patents are required for (1).

NWW have the right framework for analyzing (2), which is directly relevant to the first part of the economic terms you identify. The second part (diffusion) is more subtle, but best understood (I believe) through comparisons to alternative institutions of open source licensing and trade secret torts. Patents look favorable relative to trade secrets in many cases, but not to open source licensing in most.

The overarching theme seems to be what do we need the get people to work together toward bringing some technical dream to market after it has been demonstrated as possible by scientists? In the case of drugs or medical therapy, there’s no question that exclusive rights are required to attract capital. In the case of software, it’s less clear, and the dead weight costs of the system look more dubious.

Thanks again for the wonderful summary of a complex and nuanced question.

I love the work you’re doing. I believe there are dysfunctions in the way curiosity-driven academic research gets funded that have and will continue to have major implications for technological progress and economic development. I’m also sympathetic to the sketch of possible reforms. Increasing competition and closing feedback loops on the performance of funding decisions would likely produce tangible benefits.

What I’m wondering about more these days is how things work at the micro level. In my experience, there is a very clear and observable difference between the kinds of researchers who are driven by curiosity and the kinds of researchers who are driven by a desire to see their work have a tangible impact on the world during their lifetime. VC funding has done plenty for the latter. How can we encourage and help the former?

There is probably a baseline level of curiosity driven research that will happen no matter what we do. There always has been. Many of the people who do curiosity driven research do it despite having to make huge personal sacrifices. Mostly what we need to do for them is keep other people out of their way.

But people are people, and very few can live as hermits forever. I believe that in many cases, the curiosity driven research accelerates as the curiosity-driven researchers find a community of kindred spirits in which they can share their ideas freely. When fear of being scooped and losing funding is replaced by a infinite, positive-sum game of seeing who can come up with the coolest new theories or results, the work tends to accelerate and multiply in a combinatorial way.

Where are those physical environments today? Bell Labs had one. IBM ARC had one. I feel like the Flatiron Institute has one. Where else?

In the end, funding is only part of the answer here.

I’d add that Polanyi worried about the question of the integrity of the fabric of scientific knowledge. That is, he asked what was the ultimate source of a geologist’s or a layperson’s confidence that the state of the art knowledge in say — x-ray crystallography — had integrity. His concern was that, in science there was nothing analogous to the function of arbitrating prices in markets and between markets to ‘keep people honest’ in their valuations.

His answer was what he called his ’theory of overlapping neighbourhoods”. It was “held by a multitude of individuals” connected in a fiduciary network, “each of whom endorses the other’s opinion at second hand, by relying on the consensual chains which link him to all the others through a sequence of overlapping neighborhoods”. Polanyi would have appreciated businessman Charlie Munger’s claim that “the highest form a civilization can reach is a seamless web of deserved trust”.

We can apply an analogous process of relying more on those with a good record of shorter term prediction in making longer term predictions.

Section 6 of this report discusses the way in which one might ‘bootstrap’ longer-term vision by using shorter term forecasting performance as one criterion for determining longer-term forecasting prowess — together with deliberation on determining shorter run expectations that can be used to test the progress of longer term projects.

Though I don’t think he’s cited (his contribution is in noting the need for such a thing rather than identifying it — because that is pretty obvious), this builds on Polanyi’s overlapping neighbourhoods idea.

Thanks for the post Jason. Most or all the mechanisms proposed tend to assume that people are self-interested rather than a mix of self-and-other interest. They play to Hobbesian not Aristotelian stereotypes of the way we are. The scientist turned philosopher Michael Polanyi would argue that many of these functions have powerful ethical dimensions and I’d argue that the structures we’ve put in place, in appealing to self-interest tend to crowd out more ethical motivations. Bureaucracies breed careerism. So I’d want to introduce more randomisation and other kinds of mechanisms to minimise the extent to which the accountability mechanisms we set up don’t just produce more accountability theatre. There are also ways of selecting for merit that don’t involve people performing for their superiors in an organisation. Thus the republic of Venice used the mechanism of the Brevia by which a sub-group of the 2,000+ strong population with political power were chosen by lottery. They then convened behind closed doors and then had a secret ballot. The idea was to insulate the merit selection process from favours to power. It seems to have worked a charm helping Venice to be the only city state in Italy that got through the 500 years from the beginning of the 13th century without any successful coups or civil wars.

It’s an interesting idea.

I generally disagree with some of the premise, but I do think it’s interesting. Taking advantage of one’s health and youth while one has it seems tempting.

On the other hand, worker productivity presumably goes down pretty fast once the worker in question is in their 60s-70s, just because they’re getting older, so there may not be anyone who wants to employ them.

You’ve also got the issue that social security benefits do somewhat depend on how much someone earned in their life (I believe, I’m no expert) - so what would be the benefit someone takes in their 30th year?

Helen Toner was apparently willing to let OpenAI be destroyed because of a general feeling that the organization was moving too fast or commercializing too much.

I suspect that the board will look better over time as more information comes out.

Here’s some quotes from the Time article where Sam was named CEO of the Year:

But four people who have worked with Altman over the years also say he could be slippery—and at times, misleading and deceptive. Two people familiar with the board’s proceedings say that Altman is skilled at manipulating people, and that he had repeatedly received feedback that he was sometimes dishonest in order to make people feel he agreed with them when he did not. These people saw this pattern as part of a broader attempt to consolidate power. “In a lot of ways, Sam is a really nice guy; he’s not an evil genius. It would be easier to tell this story if he was a terrible person,” says one of them. “He cares about the mission, he cares about other people, he cares about humanity. But there’s also a clear pattern, if you look at his behavior, of really seeking power in an extreme way.”

… Some worried that iterative deployment would accelerate a dangerous AI arms race, and that commercial concerns were clouding OpenAI’s safety priorities. Several people close to the company thought OpenAI was drifting away from its original mission. “We had multiple board conversations about it, and huge numbers of internal conversations,” Altman says. But the decision was made. In 2021, seven staffers who disagreed quit to start a rival lab called Anthropic, led by Dario Amodei, OpenAI’s top safety researcher.

… For some time—little by little, at different rates—the three independent directors and Sutskever were becoming concerned about Altman’s behavior. Altman had a tendency to play different people off one another in order to get his desired outcome, say two people familiar with the board’s discussions. Both also say Altman tried to ensure information flowed through him. “He has a way of keeping the picture somewhat fragmented,” one says, making it hard to know where others stood.

… Altman told one board member that another believed Toner ought to be removed immediately, which was not true, according to two people familiar with the discussions.

In other words, it appears that Sam started the fight and not them. Is it really that crazy for the board to attempt to remove a CEO who was attempted to undermine the board’s oversight over him?

They were definitely outclassed in terms of their political ability, but I don’t think they were incompetent. It’s more that when you go up against a much more skilled actor, they end up making you look incompetent.

The thing about e/​acc is it’s a mix of the reasonable and the insane doom cult.

The reasonable parts talk about AI curing diseases ect, and ask to speed it up.

Given some chance of AI curing diseases, and some chance of AI caused extinction, it’s a tradeoff.

Now where the optimal point of the tradeoff lands on depend on whether we just care about existing humans, or all potential future humans. And also on how big we think the risk of AI extinction is.

If we care about all future humans, and think ai is really dangerous, we get a “proceed with extreme caution” position. A position that accepts the building of ASI eventually, but is quite keen to delay it 1000 years if that buys any more safety.

On the other end, some people think the risks are small, and mostly care about themselves/​current humans. They are more e/​acc.

But there are also various “AI will be our worthy successors”, “AI will replace humans, and that’s great” type e/​acc who are ok with the end of humanity.

I don’t see any specific criticism of effective altruism other than “I don’t like the vibes”.

And the criticism from “acrimonious corporate politics”.

“Helen Toner was apparently willing to let OpenAI be destroyed because of a general feeling that the organization was moving too fast or commercializing too much.”

Between the two of them, a philosophy that aims to prevent catastrophic risk in the future seems to be creating its own catastrophes in the present.

Shutting down a company and some acrimonious board room discussion is hardly “catastrophic”. And it can be the right move, if you think the danger exceeds the value the company is creating.

Ie if a company makes nuclear power plants that are melt downs just waiting to happen, or kids toys full of lead or something, shutting that company down is a good move.

A potential area of overlap between effective altruism and Roots of Progress is the non-profit New Harvest, which funds research into making meat, eggs, and milk without animals.

Great point. I wish we had more ideas about how to improve this. So many places we might try to fix this: philanthropists might redirect funding. We might try to provide career paths for these institutions’ employees that spanned the space of current problems and not just the one problem they work on.

There is an argument to be made that e/​acc is the Jungian shadow to EA.

There is a fundamental difference in principles between the two movements in that EA gradually and then suddenly fell into a paternalistic disregard (if not disdain) for the negative feedback that the market provides—e.g., Helen Toner’s belief that the dissolution of OpenAI was an acceptable alternative to resolving differences with the CEO. But with this exception, most of the principles espoused by EA (scientific mindset, openness to falsifying evidence, integrity, and teamwork) are shared by e/​acc.

But EA started with philosophical principles and became a mass movement. e/​acc more or less has begun as a mass movement, and is only gradually and haltingly identifying its principles.

Both EA and e/​acc reflexively repress the valid differences they have in their approach to promoting progress. While e/​acc is now on the ascendant and EA on the ropes, until e/​acc or EA can integrate their shadow, both will fall short of their potential in activating human energy in service of progress.

What would a fully integrated vision of progress look like? It would acknowledge the valid view of e/​acc that markets generally provide the best mechanism for gathering and processing information about the needs of dispersed groups of individuals while at the same time acknowledging and grappling with the reality that there are some important needs that cannot be met by markets (either because the preconditions for market formation have not been or cannot be met).

But I would be very careful posting this sort of essay online right now. You are either for or against at the moment. Anybody trying to nuance things is likely to be sidelined.

In theory, if they could be made to work, self-driving cars would be one of the best technologies ever. In practice, the technology seems stuck in a rut. Although exact statistics are hard to come by, the number of human interventions seems to remain high.

There is a very high burden of proof for self-driving car companies like Cruise and Waymo; they need to convincingly demonstrate, using robust statistical evidence, that their vehicles are indeed significantly safer than human drivers in the same locales. Cruise, Waymo, et al. have certainly had plenty of time to produce such evidence, but they have yet to do so.

As bullish as I once was on self-driving cars, I think it is reasonable for people to be worried about the potential danger posed by these prototypes driving around their streets. If self-driving car companies can’t prove their worries are misplaced, then, well, banning such testing on public roads doesn’t strike me as unreasonable. At the very least, there seems to be little excuse for taking safety drivers out of prototype cars.

My view on self-driving car bans is influenced by my view that fundamental research breakthroughs are needed to make wide-scale commercialization of self-driving cars a reality. I don’t think the bottleneck is more public road testing. Deep learning researchers need to figure things out like self-supervised video prediction. Until then, self-driving cars will continue to spin their wheels.

I think Ezra Klein has a lucid take on the “manifesto”. Ezra observes that it’s a covert anti-wokeness rant:

It’s a mistake to read his manifesto as about technology. It’s about how we were once brave and strong and we have become soft and weak.

In Ezra’s New York Times column on Andreessen’s rant, he writes:

It’s a coalition obsessed with where we went wrong: the weakness, the political correctness, the liberalism, the trigger warnings, the smug elites. It’s a coalition that believes we were once hard and have become soft; worse, we have come to lionize softness and punish hardness.

I would describe myself as a techno-optimist, but I find Andreessen’s rant distasteful and alienating. I think allowing Andreessen to define what constitutes techno-optimism would do significant damage to the techno-optimist cause.

Hi Jason,

This is great, I would love to read more about how you believe Progress Studies could become a philosophy on par with Effective Altruism. I think an advantage EA has is its roots in John Stuart Mill and some of his contemporaries. Personally, I’ve found it harder to pinpoint which philosophers were early proponents of Progress Studies—my sense is that the idea of building, whatever the trials and tribulations, is fundamentally a Stoic idea. Indeed, I think Ayn Rand’s ideas, particularly on the importance of individualism, are important if one would like to create an epistemic history of Progress Studies.

Thanks for sharing this draft.

Style suggestion. You could put the penultimate paragraph before the preceding one and delete the final paragraph. That will decrease the preachiness factor at the end and the repetition of ideas in the last and third to last paragraph. Plus going straight from we need serious people to the paragraph about those people is what your structure is asking for.

I’m working on an essay on patents and progress. Does anyone want to give it a read and give me some feedback?

Since you’ve highlighted the women in these paintings, it’s worth noting that one of the first scientific studies advocating the use of masks in surgery was written by Alice Hamilton, a Chicago physician, in 1905. Quoting my article here (it may be paywalled):

In 1905, Chicago physician Alice Hamilton publishes an article in the Journal of the American Medical Association, reporting on experiments measuring the amount of streptococci bacteria expelled when scarlet fever patients cough or cry. She also measures the strep bacteria from healthy doctors and nurses when they talk or cough, leading her to recommend masks during surgery.

“I was told by a student in a large medical college in Chicago,” she writes, “that he had often noticed at the clinics of a certain surgeon that, when the light was from a certain direction, he could see, from his seat in the amphitheater, a continuous spray of saliva coming from the mouth of the surgeon while he discoursed to the class and conducted his operation. Obviously, protection of the mouth, of some sort as to catch and impression the droplets of sputum, should be a routine precaution for surgeons and for surgical nurses during operations.”

For something that is long term, but only effects the property of 1 person, like the field example, the market prices it in and it’s not an externality.

No one is significantly incentivized to stop climate change, because no one person bears a significant fraction of the damage caused.

Politicians are far from perfect, but at least they have any incentive to tackle these big problems at all.

I wonder if there is also a psychological factor at work. Something along the lines of: a major world crisis, especially one that causes a lot of deaths, makes you think about the ephemeral nature of life and what’s really important, and you either decide to have kids or to stop putting it off. Imagine someone reading about deaths among the elderly and thinking, “I really want Mom and Dad to meet their grandkids—better get on it.”

Curious if Claudia Goldin’s work is relevant here?

The Shirky Principle: “Institutions will try to preserve the problem to which they are the solution.”

Eli, this is great. And useful!

I went to the EAA event in Oshkosh last year to see the experimental community first-hand. I was struck by who was there. It was amateurs, sure, but also a lot of pro pilots who had a side interest in experimental aircraft. It’s not just another type of person. The designation gives everyone a place to try new things. Also, it was full of EVTOLs. When we eventually get flying cars, I suspect we’ll point at the experimental designation as an enabling part of the process.

Nice piece, thanks. I hadn’t noticed HC’s latest video, but am also a fan so look forward to watching the full thing.

An initial thought, though: there seem at least a couple of places where one can grant his facts yet argue his interpretation of these as “good” is backwards.

For instance, one can argue that lower preindustrial working hours reflect chronic underemployment and unemployment, one of that society’s chief problems, and that industrialisation alleviated this problem (and even that this was one of the great initial benefits of industrialisation).

And HC’s following point ought to be seen as evidence against the superiority of preindustrial life:

Notice how the workers were constantly eating. That was one of the perks of being a day laborer. Food was a worker’s biggest expense, and so part of their compensation was that their employer would take care of the food for that day. It would be like if part of your compensation was that your boss paid your rent. It relieved a massive financial burden.

Essentially he’s just pointed out that living standards were so low that the majority of budgets had to be spent just on food.

Those many free days the workers had came as a courtesy of Catholicism. Look up the amount of regulation and holidays they had. This was, in fact, one of the reasons why Protestants were so comparatively successful. It was also part of the reason why they were resented in Catholic-majority countries; they got richer and were doctrinally snobbish about it. But I guess attacking Protestantism in favor of medical Caholocism would probably be the wrong sort of spicy for this YouTuber.

Regarding the structure of the workday. Modern-day physical labor still looks like this in many cases. Having worked over the summer in Landscaping and Carpentry this schedule is still present. And OP mentioned the case of office Jobs already.

What a helpful and comprehensive guide to the economic study of human capital and productivity. Thank you for writing this! I like the idea of a model as a good way of testing various hypotheses. Models are undefeated for inferring causality. As a matter of prediction or even for generating hypotheses, I’m less optimistic about models.

Pulling on the institutional thread does seem like the right move, as opposed to ideas. They do at least roughly map to the slow pre-enlightenment, fast modernity and slowing post-modern pattern of progress. Still, privileging institutions over individuals as the most important factor doesn’t make much intuitive sense to me, since each new idea only happens once. It’s possible that institutions are getting in the way of idea execution (regulation, etc.), but missing key ingredient seems to be coordination more than anything.

The “one-time-boom” version of the ideas are getting harder to find argument is one you hear often that does seem to map to the general pattern of progress and is Cowen’s favored variant. I’m partial to the idea that in the wartime hurry to push innovation we reordered our institutions to achieve great short-run gains at the expense of a kludgy emergent system that was working fairly well. Over-optimization of talent sorting and centrally directed R&D may extract progress from promising individuals at the same time that it destroys the system for creating promising individuals. Somewhere between a “Seeing Like a State” argument and a more individual and metaphysical explanation.

I like the idea, especially for the experimentation in governance it would foster.

One possible issue is that the democrat/​republican divide often tends to be urban/​rural, rather than by state.

Ideally, this sort of change would be accompanied by a redrawing of state lines, enabling both a) more than 50 states and b) better alignments of geography/​population to statehood.

I see an issue with all of this, and it is the bottleneck that is storage/​infrastructure

You cannot expect all of the solar energy storage to be serviced by batteries, because those degrade if they hold energy for too long

The solution to this is intercontinental HVDC lines, and a bit of hydrogen too

The hydrogen for storage is very easy, even tho it’s a tricky material, we can manage it for unexpected solar dips

However intercontinental HVDC, which are VERY needed once we start to use more than half of our energy needs from solar, require INTERNATIONAL cooperation, and a lot of commitment from many countries

I don’t know if the international geopolitical scene is prepared for such a project, and I think this will delay solar adoption in your timeline after 2030ish

This is a great piece! Thanks very much for posting.

Maybe I’m missing something, but this post doesn’t seem to actually back up the (strong!) claim in its title. Seems more like you’re defending “Something Is Getting Harder To Find But It May Or May Not Be Ideas”.

Agree on these points in general—and believe this is one of the major reasons for optimism around AI. AI models seem particularly good at navigating high dimensional landscapes if we structure them appropriately. My theory is this will allows us to hugely increase #2, as we now have a better method for searching the solution space.

Comments in no particular order:

Marc Andreessen continues to sound just like himself. I think this is good for the piece, it feels very genuine. In the main I agree.

Markets is the biggest section. This feels telling and also kind of wasteful. It also had the clunkiest bits which were, but of course, the ones about economics. By contrast the Technology section felt a bit thin, but I could easily forgive a certain amount of c’mon, you know why you are here in the effort.

What is this for, really? I can tell who it is for, because it doesn’t seem like it would register much with people who didn’t already mostly agree. But what problem is the manifesto solving? Guessing by some of the keywords included in the bad ideas list, this feels like maybe trying to further crystallize e/​acc into a broader concern?

On the flip side, this bit here under The Enemy makes it seem more like talking book:

Our present society has been subjected to a mass demoralization campaign for six decades – against technology and against life – under varying names like “existential risk”, “sustainability”, “ESG”, “Sustainable Development Goals”, “social responsibility”, “stakeholder capitalism”, “Precautionary Principle”, “trust and safety”, “tech ethics”, “risk management”, “de-growth”, “the limits of growth”.

Purely as a matter of style, I thought the “we believe” and “we had a problem” chunks were great, that’s what I want out of a manifesto. I would jettison all the quotes and argumentation, moving names and sources to footnotes or something; I mostly found it distracting, like he was so used to the argumentation side of things he had trouble letting it go (Not that I blame him, I would have the same issue were I to write a manifesto). I thought some of them were compacted too much, like compressing all the progress in agriculture into the green revolution, which sort of deprived it of emotional impact (not least because of term confusion, since green revolution shows up in advertising campaigns and slogans constantly meaning something entirely different).

All that said, I liked it and I wish more public figures like Marc would do things like this.

I disagree with (5), that the long-term historical pattern is acceleration (and more specifically, I don’t think that the first three charts in your linked piece are sufficient to demonstrate this).

At the frontier, real GDP per person growth has remained remarkably constant for the past ~200 years.

The growth rate for the world might show acceleration, but I understand this as a compositional effect, as more countries leave the zero/​low growth regime and experience rapid catch-up growth. But in the long-run each country’s growth rate will converge with the roughly constant rate at the frontier. Eventually we’ll run out of countries joining the modern growth regime, and I’d then expect world real GDP per capita growth to slow. This paper from Robert Lucas describes the dynamics I’m talking about—see Figure 3 in particular (world growth accelerates and then slows and converges to the rate at the frontier).

And due to the near universal pattern of the Demographic Transition, I’d also expect population growth to trend towards zero in the long-run. So I wouldn’t expect acceleration in the growth rate of total GDP either.

(FYI, I’m repeating my reasons for being unconvinced of David Roodman’s piece on accelerating growth, which are in also in this Twitter thread).

Hope this is helpful!

Interesting and thought-provoking, great read!

I read recently on some substack that a lot of the gains in cheaper Solar power are actually a result of the industry being highly subsidized, much more than other forms of energy—presumably indicating that it’s an unsustainable form of growth we can’t expect to continue were it to be pitted against e.g Nuclear in a free(er) market(?).

Does this change your conclusions in any way? What am I missing? Also, how do you view the future of nuclear (fission) energy in light of the potential of solar?

I see the appeal and I like the aliveness, but I dislike the lack of nuance and disagree on the specifics.

I think that this is broadly correct. One of the biggest problems of health care in America is that the feedback mechanisms that control cost (such as honest and public prices) are completely broken, and fixing them would likely go a large way towards solving the problem.

I enjoyed this post, especially the concept of an error budget which was new to me.

As an implementation detail, these meta-regulations seem to fit into concept of House Rules, a version of which probably exists in most legislative/​regulatory bodies. However, AFAIK these are usually set by the body itself, so perhaps the meta-regulation would need to be written into the constitution or charter of the body.

I’ve heard people criticize this for lacking nuance, not engaging with critics, and not citing sources. I feel this misunderstands the genre. It’s a manifesto. It’s not supposed to be nuanced or appeal to critics; it’s supposed to be even a little divisive, drawing a line in the sand, recruiting those who are already sympathetic and ignoring or even repelling those who are not. It’s not supposed to argue for its claims, it’s supposed to stake out some beliefs and declare them.

If you just don’t like manifestos of any stripe, then fine; but it never makes sense to criticize a piece for not being in a different genre.

Marc is a modern-day Thomas Jefferson. I agree with every word he wrote and find it inspirational and a great encapsulation of how I feel about tech and what it can do for society. The movement needs a rallying cry and a big tent to go with it. I intentionally mention Jefferson, as someone whose actions often fell short of his inspirational vision and words. Marc is not without criticism in this regard, but who is? This is not the time for criticizing our friends, and as such, I won’t nitpick the things I would alter in such a manifesto. I hope it can help formalize this ethos into a practical movement for society, especially for those of us in the US who feel politically homeless.

Community self-regulation among lobster fishermen.

https://​​www.pnas.org/​​doi/​​abs/​​10.1073/​​pnas.0702241104

Some negative externalities and other problems of progress that may or may not be considered solved:

• how technology can make war more deadly

• how it can assist oppressive/​authoritarian regimes

• technological unemployment

• unclean/​unsafe food, water supplies

• resource depletion

• pollution of all kinds

• carcinogens

Positive externalities are like, almost all of human wealth?

Yale economist William Nordhaus estimates that innovators capture a mere 2.2% of the total “surplus” from innovation. … If it is anywhere close to being an accurate estimate, the implication is that “society” pays a paltry $2.20 for every $100 worth of welfare it enjoys from innovating activities.

Quoted by Tyler Cowen here

My list now includes:

Sulfur Dioxide

Clean Air Act

Montreal Protocol

Lead Paint

Lead in Gasoline

Public Radio

Mandatory 8th grade education

Secondhand Smoke

Public Transit Subsidy

Automobile Emissions Standards

Fishing Quota System Australia

Bees and Orchards

Cattle Ranchers and Crop Farmers

—

Climate Change

Space Debris

Traffic Congestion

Violent Crime

Assuming the Data is Correct: legal Marijuana

To add on to what John Buridan already correctly shared about major changes to the tax code in the early 20th century you might want to check out the Federalist Paper No. 30-36: https://​​guides.loc.gov/​​federalist-papers

Hamilton and the Federalists were arguing for taxation and a central bank to deal with the Revolutionary War debts that the U.S. had racked up (foreign loans and veteran backpay and pensions). Since direct taxes were declared illegal in the constitution, most of the money raised by the Federal Government was in the form of tariffs: https://​​www.ttb.gov/​​public-information/​​special-feature

Abraham Lincoln instituted the first national income tax to raise funds desperately needed to prepare the Union Army for the Civil War, and to administer a more complicated system of taxation, he also created the IRS. There were numerous repeals and court battles until in 1909 the constitutionality was settled by the 16th amendment. https://​​www.amazon.com/​​Ways-Means-Lincoln-Cabinet-Financing/​​dp/​​0735223572

To answer your question, the big driver of federal taxation for the past 80 years has been social security (and medicare to a lesser extent), instituted in the 1930s(medicare in the 1960s). If you sum up state and local revenues (mostly property and sales) they are greater than federal, when you take out insurance programs ($ you get back) and intergovernmental transfers ($ state/​local get back) from federal taxes. Here’s a good breakdown of the revenue sources: https://​​www.taxpolicycenter.org/​​briefing-book/​​what-breakdown-revenues-among-federal-state-and-local-governments

To answer why there’s so much federal taxation relative to state and local, even if there’s greater spending by state and local, you have to examine the expansion of federal government programs from the New Deal through WWII, to the Great Society, the Cold War, the Green Movement, and 9/​11.

• Dept. of HHS was created in the early 1950s along with medicare/​medicaid.

• NASA in the 1950s to carve out space programs from DoD.

• NSF in the 1950s to unify and manage federal R&D.

• HUD in the 1960s to support urban renewal.

• Dept. of Transportation in the late 1960s to regulate interstate travel (mostly air traffic and highways).

• EPA in the early 1970s to regulate clean air and water.

• US Consumer Product Safety in the 1970s for consumer protection.

• Dept. of Energy to carve out nuclear weapons from DoD and manage U.S nuclear power in the 1970s.

• Dept. of Education in the early 1980s to measure performance and redistribute funds to ameliorate inequities.

• VA in the late 1980s to carve out veteran health from the DoD.

• Dept. of Homeland Security to standardize and unify national security.

As you can see, many of these agencies and departments deal with externalities states may impose on their neighbors, collective action problems that encourage bad actors to defect and of course, plenty of pork projects to spread federal money around the country.

I agree with your conclusion, Maxwell, and this piece was a joy to read. Jason’s comment also seems correct to me in that subdomains very clearly exhibit the phenomenon of ideas getting harder locally. Still, the fallacy of composition tells us to be wary of summing up these subdomains. Diversification across subdomains may the answer to how the innovation frontier can continue to expand despite ideas getting locally more challenging.

I’m curious to hear what you think is the scarce resource. After trying my hand at starting a company and working in venture capital, I’ve come to appreciate that often the idea is quite important. The old hobbyhorse of execution vs idea feels like a false dichotomy though. The best companies do not spring forth from entrepreneurs’ heads like Athena, fully dressed for battle, but they are also not A/​B tested into existence. Similarly, science seems to move forward through a combination of dogged empirical work and theoretical insight.

Here are a few areas I’d like to read more about: courage, ignorance/​fools and subversion/​tricksters. I always think of the strange case of Medicine Nobel Dr. Barry Marshall, who debunked long established medical beliefs about stress being the cause of gastric ulcers by performing risky self experimentation that involved infecting his gut with bacteria: https://​​asm.org/​​Podcasts/​​MTM/​​Episodes/​​The-Self-Experimentation-of-Barry-Marshall-MTM-144

I am pretty convinced that β>0, even though I also think there is some contribution to slowdown from other factors. Some reasons:

• Bloom et al observe β>0 not only in the economy at large but in various subdomains, such as transistors and Moore’s Law. If you want to argue that this is all caused by society’s resistance to innovation (which is real), then you have to argue that society also resists innovation in integrated circuits (which… doesn’t seem like a thing).

• There are simple intuitions for β>0 based on low-hanging fruit and burden of knowledge (both of which you mentioned).

• Scott Alexander gives some intuition for why we should expect β>0 here; if it weren’t then progress would long ago have accelerated to insane levels. In fact, I haven’t checked the math, but I think if β=0 you end up with a hyperbolic curve where GDP goes infinite in finite time; see this other Alexander post (although the paper that post is based on is kind of silly and not written by an expert in the field; Kremer 1993 is a better reference for this idea and is worth reading).

• We arguably see similar patterns not only in technology and the economy but also in science and the arts. Holden Karnofsky makes an argument for this here.

That said, I am also pretty convinced that there is a contribution from institutions and other factors. (To paraphrase something Eli Dourado said to me, TFP growth is slow in Venezuela, and no one thinks that is because ideas are harder to find in Venezuela.)

Thank you for this thought-provoking post! Maybe ‘WTF happened in 1971’ should be renamed to ‘WTF happened in 1973’? Energy scarcity being at the root of the long term trends in productivity and prosperity seems like a more plausible explanation than the Nixon shock.

Long term abundance is a great future to look forward to. What has me more concerned is the inertia of the climate system (committed warming, feedbacks) and the turbulence it’s going to cause in the next decades. It seems like a race to deploy enough solar to allow people to adapt to extreme weather (air conditioners for wet bulb heat waves, fresh water for droughts etc.).

“~sixth … industrial revolution”—please Casey, you’re killing me

Excellent analysis, good to see optimism about the future (the pessimist in me is saying but what about AI☹️).

I like this perspective in the reality of atoms, but I’d like to offer a cofactor that also compounds with this.

Electrical power is never freed from thermodynamic constraints. In simple terms, it would be obtuse to maximize the transmission of electrical power, because that inherently maximizes waste heat & associated destructive risk. Fundamentally, our environment favors transmission of signal upon local consumption of power, usually digital bits above a minimum SNR. This is not a single paradigm, because our economy has been growing considerably in the scope of financially valuable bits, typically with a growing capacity for risk instruments.

I’d like to extend this a little further. So average joules per capita might scale up tremendously, maybe it’s shunted to larger FLOPs (operating bits) per capita, and coincidentally there’s more accurately implied risk per capita. I’m willing to assert that there’s a happy providence where radical energy abundance is complemented by a radical, elastic demand for accelerating production of novel signals. In other words, we’re not just sticking with a static manufacturing base that can brute force upon energy abundance, perhaps it is also capable of diversifying & optimizing for energy-efficient exploration of undeveloped frontiers. Moreover, because the footprint for novel concepts like autonomous agency fall under the threshold of consumer electronics, there may even be an imminent paradigm where monetary velocity upon energy abundance leads to much more extensive conversion of joules to financial surplus, therefore much more extensive capacity for debt. Which is critical when communities need the capex for resources like freshwater-producing facilities.

As to the Kardashev challenge, I think we need to consider that solar technology has been optimized for wattage/​mass (though I’m sure this oversimplifying some nuance). Since we can only really grow outwards, beyond land-bound real estate, I’d contend that radical energy abundance relies in part on optimized upmass. Not only is this necessary, but there has been a decades-old, thoroughly-planned dream to maximally collect the necessary commodities in microgravity and continue scaling up en situ. We might only attain 1.0 on the Kardashev Scale by endeavoring to construct the precursors to attaining 2.0. This bridge isn’t going to be crossed in our lifetimes, but it’s fascinating to me, at least, that there may be an acceleration stack:

more joules → more FlOPs → more financial agency → more serviceable debt → more terrestrial capex → more demand for extraterrestrial ISRU → more indication that we can thrive & moderate environmental downsides

this definitely doesn’t touch upon the nuances of manufacturing, societal displacement, and perhaps other black swans. yet I think this could be a by-the-numbers providence that further vindicates a sense of optimism, and perhaps a more focused purpose for the human condition beyond Malthusianism, beyond noisy zero-sum subcultural bloodsport.

then again, this is just my overly optimistic take.

What about land cost for solar? At what point does that become a significant part of energy cost? Solar is less diffuse than wind but more diffuse than any fuel-based energy technology.

If it’s not significant now, surely at some point on our way to becoming a Kardashev Type 1 civilization it becomes a problem?

Does anyone have a good essay about federalism—particularly the history of the US and how we have divided power between the federal governments and the states?

“Land value taxes are generally favored by economists as they do not cause economic inefficiency, and reduce inequality.^[2] A land value tax is a progressive tax, in that the tax burden falls on land owners, because land ownership is correlated with wealth and income.^[3][4] The land value tax has been referred to as “the perfect tax” and the economic efficiency of a land value tax has been accepted since the eighteenth century.^[1][5][6] Economists since Adam Smith and David Ricardo have advocated this tax because it does not hurt economic activity, and encourages development without subsidies.”

https://​​en.wikipedia.org/​​wiki/​​Land_value_tax

Despite this rather glowing summary, and support by economists from Milton Friedman to Paul Krugman, land taxes are rare. The Economist explains:

“The bigger barrier is political. LVTs would impose concentrated costs on landowners [a politically powerful constituency], who face a new tax bill and reduced sale price. The benefit, by contrast, is spread equally over today’s population and future generations. The problem is unlikely to be overcome. Economists will continue to advocate LVTs, and politicians will continue to ignore them.”

https://​​www.economist.com/​​the-economist-explains/​​2014/​​11/​​10/​​why-land-value-taxes-are-so-popular-yet-so-rare

Some jurisdictions have managed to implement land value taxes. Denmark and Estonia have a form of the tax, and a handful of municipalities in Pennsylvania (Allentown, Harrisburg, Altoona from 2009-2018ish and Pittsburgh from 1911-2000) have experimented with it or its cousin split-rate taxation. Split-rate taxation taxes both land and structures like a traditional property tax, though it taxes land at a higher rate (often 5:1) than structures.

I’m still researching various income streams for governments and I’m wondering: will income tax be the best way to tax in the future? (I’m thinking about remote work and how we might be taxed in one location but living in another one). Or are there governments who are pulling tax revenue from other unique places? Are there any good books or essays about unique tax structures that have worked well/​make sense?

Hi Evan,

Very interesting piece and I think San Francisco isn’t alone in the need for some form of greater regional governance. My mind immediately jumps to all the regional planning bodies throughout the U.S., like CMAP (Chicago Metropolitan Agency for Planning) in Illinois, which covers an area vastly greater than the City of Chicago but whose powers are purely advisory. From what I understand, they put out excellent regional plans full of great TOD and YIMBY proposals. Yet it’s ultimately the Aldermen of Chicago who exert a kind of feudal authority over every little land-use decision in their patches of turf. With a playing field like this, it’s no wonder NIMBYs dominate (or why Aldermen keep going to jail, for that matter). Consolidating land-use authority in a regional body certainly would tilt the balance of power towards pro-housing forces.

But the transit issue is where I think your case is strongest and most generalizable. I live in DC, where our regional metro system WMATA is chronically under-funded and terrified of making long-term investments needed to ensure the decent and reliable service a metro area of our size deserves. WMATA has to go hat-in-hand to Maryland and Virginia soliciting voluntary contributions for a system that reaches very deeply into both states. The share of operating costs recovered by fares is pretty low, but I don’t think it’s totally out of the ordinary for U.S. transit systems. DC residents end up bearing a much greater cost for what is a truly regional system. But we simply can’t compel Maryland and Virginia to pay their fare share. It sounds like BART has the same issue? We could use some regional bodies with actual taxing authority whose jurisdictions actually align with the reach of our infrastructure needs. Maybe “save our metro” is a useful political rallying cry for this.

Are there any examples of cities around the world you’ve seen that do this particularly well? Greater London, essentially coterminous with the Tube, comes to mind. But I’d love to do some research on further case studies, perhaps for one of my ROP BBI pieces.

Wow, Evan—what an amazing essay this is! As a former SF resident I had not mental picture of 101 municipalities but of course am very aware of the challenges. The New York analogy is great and super helpful.

The solution makes total sense—and yet it’s also clear that inertia and self-interest of bureaucrats and local politicians will work against it. What do you think will it take to get an advocate (or a group of advocates) like Green in NYC to get this going? (I guess getting your essay as much readership as possible with SF people is a good first step!)

US tax policy changed during the WWI era. It was all quite local until 1917. https://​​taxfoundation.org/​​data/​​all/​​federal/​​historical-income-tax-rates-brackets/​​ Also, you’ll notice that even at the state level, income tax was rare. Income tax has historically been ill-regarded. Until 1905 most of the state of New York was funded through alcohol taxes: https://​​www.jstor.org/​​stable/​​1882558 They then passed a mortgage tax and a stock tax to counter the rising costs of a growing population.

Every government level has the power to tax in the US, and they all find very different ways to do it.

Who is your target market? For an adult to take nearly a year out of their life is a big ask. For whom do you expect it will be worth it? Staff/​campaigners at existing non-profits? Existing bloggers/​youtubers etc on related subjects? Anyone else? How large is the potential pool of students and how will you reach them and convince them that the course is worth the time and money?

I realise this is not a commercial project, but I think you would benefit from some start-up like thinking: who is this product for, what is the market, how do you reach customers, what is a minimum viable product from which to iterate? Apologies if you have already done that, but it wasn’t communicated in your post.

I don’t think there would be broad agreement within the progress community about the Singer argument, or more generally about utilitarianism.

Personally, I am neither a utilitarian nor an altruist, and I don’t agree with the drowning child argument as I understand it.

I think how much to spend on yourself vs. charity or other causes that you believe in is a personal decision, based on what is meaningful and important to you.

Hmm, I honestly don’t know whether progress studies can be applied to any random job or company. I think of it more about applying at a society-wide level. Of course, it might inspire some people to take jobs at more ambitious /​ cutting-edge companies (or start such companies!) But that also doesn’t mean there’s anything wrong with companies that aren’t cutting-edge—it takes all kinds of companies to make a functioning economy.

If anything, maybe progress studies can help remind you all of the moral value of economic growth. To the extent you all do you job well, and create economic value, and produce an honest profit—you are contributing to the well-being of the world. That makes it worth taking pride in a job well done. Trite but true.

Tribal-based SEZs could be the perfect place for people to sell normally Rx-based drugs OTC.

(Canada is way more liberal wrt respecting tribal sovereignty than America—the courts in Canada consistently rule in favor of tribal reservations and the RMCP in British Columbia has encountered public outrage when it tried to do enforcement before, which has nudged it towards non-enforcement of some actions where tribal actions had a conflict with the province). Canada also allows tribes to “buy up more land”, which is way easier in Canada than the states. Canadians in general are more left-wing than Americans and are more likely to believe that tribes really should have their own sovereignty outside of federal jurisdiction (and it’s interesting that the Canadian police just don’t enforce certain laws—which is why there are now so many psychedelic mushroom stores in BC....)

Look up Tetla economy from one of the Cowichan Tribes.. (https://​​members.viatec.ca/​​news/​​Details/​​the-launch-of-an-indigenous-cryptocurrency-is-near-142419)

https://​​champions.substack.com/​​p/​​the-coast-salish-blockchain-stock

For example, there is someone who is working on setting up an financial exchange on a BC tribal land.

https://​​laws-lois.justice.gc.ca/​​eng/​​acts/​​i-5/​​FullText.html

====

https://​​www.newyorker.com/​​magazine/​​2024/​​08/​​12/​​how-tribal-nations-are-reclaiming-oklahoma [tesla also is managing to sell cars in a NY reservation to circumvent restrictive NYC laws about selling cars]

On reservation land, there is already massive distrust of the “process of modern medicine” (including doctors), which prevents many people from getting the care they need. Diabetes rates (and “addiction rates”) on reservations are ultra-high (one Oglala reservation featured by Peter Santenello [he showcases the differences between Indian reservations on his YouTube videos] had dialysis centers). If diabetes medication [eg empagliflozin, sotagliflozin, metformin, rapamycin, semaglutide]+statins+[potentially revolutionary treatments to addiction, which could include some native-based psychedelic plants] could be made OTC (OTC means trusting the customer, it means being pro-choice), it could do a lot to prevent health problems from turning into health disasters, especially on tribal lands. And since there is also substantial demand for these medications OTC among the ordinary population, it could serve as a huge source of taxable income, kind of like the casinos.

[native reservations, esp in South America, seem to have some ability to give people psychedelic plant therapies—there could be a way to rework experimental “right to try” bio-therapies into the legends/​prophecies of some native american tribe]

It’s already possible to get many of these Rx-based drugs OTC in Latin America (and perhaps from India/​Turkey through the grey market) [Latin America has its own unique issues with high diabetes rates], but then one has to deal with inconvenient flights [or travel costs]/​customs/​etc, which massively reduces their convenience.

Similarly, they can be places where people base experimental medical treatments like exosomes or stem cell treatments (where it’s a massive hassle to offer them in the US due to regulation, and where it’s possible to get them in Latin America). Eg https://​​garmclinic.com/​​ (which is in Honduras, and thus tough to get to for most Americans). Any form of non-mainstream medication innovation (that the FDA holds back!) would have a strong argument for being more-than-welcome on tribal land

I’ve visited Roatan before, and the amount of land needed to set up these jurisdictions is quite small. If Minicircle could be offered on tribal land rather than Latin America. Of course, one would have to do due justice to issues that Native Americans face, but bypassing the untrustworthy “white man/​FDA” medical system is, I think, enough of a good reason.

[OTC removes many barriers that indigeneous/​poor populations face when getting the medication they need, especially preventatively.]

Upon looking further: https://​​www.fda.gov/​​regulatory-information/​​search-fda-guidance-documents/​​cpg-sec-100350-fda-jurisdiction-indian-reservations might be an obstacle.. (it might not be impossible to change this in the name of generally increasing tribal soverignty...)

https://​​www.foodsafetynews.com/​​2013/​​09/​​not-ready-fsma-and-fdas-proposed-safety-rules-disregard-sovereignty-and-federal-case-law/​​

LINKDUMP

https://​​latecomermag.com/​​article/​​what-happened-to-molecular-manufacturing/​​

https://​​www.leversforprogress.com/​​

forum.quantifiedself.com

SOME RANDOM LINKS:

https://​​logancollinsblog.com/​​2023/​​08/​​31/​​logans-catalog-of-useful-resources-for-creating-the-future/​​

https://​​curius.app/​​alex-k-chen

https://​​nanoscale.blogspot.com/​​

https://​​blog.spec.tech/​​p/​​speculative-technologies-2023-year

SOME interesting people:

orthogonal to the popular ML/​DL/​AI people [or orthogonal to “scaling is all you need” people]

https://​​hillelwayne.com/​​post/​​ [he’s also in the jkoppel/​​predrag circuit]

[also I listed some on LW]

https://​​github.com/​​VictorTaelin

https://​​gist.github.com/​​VictorTaelin/​​9061306220929f04e7e6980f23ade615

https://​​higherorderco.com/​​

more in the PC of popular ML/​DL/​AI/​”scaling is all you need” people

https://​​www.yitay.net/​​blog/​​2022-best-nlp-papers

“My theoretical challenge is: Progress Studies ought to be able to explain why two communities can view progress differently, and do so in its own terms; that is, in a theory that adequately defines what progress is...”

These are interesting questions, but, in my opinion, on the periphery of what “Progress Studies” is. Progress Studies generally pre-supposes that industrial, scientific, and economic development are good things. Most discussion seems to focus on better understanding how and why these things come about. At least, that’s what attracted me to this forum—somewhere where people nerd out over why the steam engine was developed where and when it was, how we can accelerate the pace of technological development, and how we can achieve more of these “good” things in the future by studying the past.

To determine whether or not this should be considered progress is, in my view, more likely to be satisfyingly answered by the field of Philosophy. There are centuries of writing that will delve far deeper into this, and have far more satisfying answers, than what’s generally discussed here—and the general supposition that, say, the Apollo mission or mRNA vaccines were capital-P Progress.

“I have reason to believe there is contestation here. From my highly informal sample of well-educated people in the Bay Area, for some, space colonies seem to represent progress; for others, an inhabitable and less technological earth seem to represent progress. These two, highly local cultural communities have different views of what progress is. My theoretical challenge is: Progress Studies ought to be able to explain why two communities can view progress differently, and do so in its own terms; that is, in a theory that adequately defines what progress is.”

I think this is actually a good example of the point I’m trying to make. The Philosopher would study both arguments and try to understand what each would say about the Good, and make a determination about what this says about the human condition or what progress should be. Your median Progress Studies enthusiast likely starts off pretty firmly on team Space Colony—and, if they’re being honest, is mostly interested in this question as a means for understanding how we could encourage more of team degrowth to switch sides.

To be clear, I don’t mean this in a negative sense toward either Progress Studies or Philosophy. Both questions are interesting and worthy of discussion. Seeking knowledge and understanding is an inherent good! But it’s also nice to have a space to talk about steam engines or designing effective industrial policy, without getting into the Philosophical weeds each time.

Update: I’m already planning to give brief remarks at a few events coming up very soon:

• Thurs, Aug 24: Recur Club founders meetup in Indiranagar. Register/​apply here

• Sun, Aug 27: LessWrong /​ Astral Codex Ten meetup at Matteo Coffea

If you’re in/​near Bangalore, hope to see you there!

This post is fantastic. Also, the last 8 months have been an eternity for LLMs. Have you been tinkering with any of the new ones since this experiment, and if so how’d they do?

Thanks for sharing the “the unreasonable effectiveness of insurance” books and pieces about fire safety. This is also a theme that fascinates me, and while I was working in reinsurance modeling for cybersecurity I became fascinated with the history of steam boilers. This technological innovation is a great case study for progress that creates problems that requires more progress to solve.

The Hartford Steam Boiler Inspection and Insurance Company (now a part of MunichRe) was created to address this problem and the blog post below describes why the sinking of the Sultana steam ship in 1865 was a critical turning point: https://​​blog.hsb.com/​​2015/​​03/​​30/​​sultana/​​

I believe this precedent setting case predates the UL by about 30 years and well before the commercialization of electricity (besides nascent telegraph systems). Prior to the HSB Inspection and Insurance Company, insurance was limited to covering mostly natural causes of marine/​storm (ancient) and property/​fire (Stuart Era) losses. There was risk of piracy, so some of the losses were man-made, but prior to the Industrial Revolution technological risk was solved by lack of progress.

Experimental results from getting a 3yo interested in technology:

1. I bought her a little broom, which she liked, and then showed her that our robot vacuum could do the same work automatically, After a year she’s still not totally comfortable with the sound, and gets scared when it’s going towards her. Best decision we made was getting a model that did /​not/​ have a mobile app, instead using a remote control (I believe it was this model: https://​​us.eufy.com/​​products/​​t2108124?ref=navimenu_2_2_4_1_img). She has learned to walk over to the shelf, find the button which makes it return home, and carefully watch it until it docks. I stress that, even when it feels scary to her, she’s always in control because she can find the remote.

2. Assembled a LEGO-like robot dog after she got excited seeing one on YouTube, but again it was too loud for her, and she gets very skittish when a device is moving towards her.

3. She enjoys asking questions to Bing Chat. I hold up my phone and turn on Siri dication, which she’s learned to recognize. She asks her question and I use the keyboard to clean up the text, then submit. I read her the output. She mostly asks under-specified questions about plot points in her favorite books, but doesn’t mind that the answers are basically a re-hash of a given character’s Wikipedia page. She’s never gotten interested in image generation, contra my expectations, mostly DALL-E can’t do specific characters (I haven’t tried the others).

I don’t know exactly how seriously to take it—but I know Michael Dearing, whose site that is, and he is 100% in favor of capitalism, so… at least partially serious?

It seems a bit ahistorical to suggest that “Rome” could have industrialized by “skipping the Dark Ages” for a couple of reasons.

The list of things the Romans didn’t have is almost entirely composed of “Dark Age” inventions.

And, although the city of Rome fell, the imperial capital had already moved to Constantinople. The Roman empire continued to develop new technology for many centuries. (Although for peculiar historical reasons we start calling the Roman empire “Byzantium” at that point.)

Consider reading “Cathedral, Forge, and Waterwheel” by Frances Gies and Joseph Gies for a more realistic view of progress in the period after the Roman empire moved its capital to Constantinople.

What a great article! I think it’s so important to have specific historical examples of how technology impacted jobs, vs. speculating in a vacuum. Your emphasis on culture and dynamism is especially on point:

”For coal miners, for example, this would mean job training programs^[23] that widen the aperture of cultural pride from providing coal to providing energy, even in the energy forms that will win the future. It also means reusing physical infrastructure when possible, such as the recent Berkshire Hathaway effort to convert a West Virginia coal plant into a nuclear power plant.^[”

Focusing on the upside or “widening the aperture” of work people consider will matter a lot, as will just being open to change (vs. using regulation to stop it before we even fully understand it).

This also reminded me of a study that analyzed the impact of AI on taxi drivers. The key finding was that AI helped less experienced/​less skilled drivers be more productive—rather than replacing drivers wholesale: ”. We find that AI improves drivers’ productivity by shortening the cruising time, and such gain is accrued only to low-skilled drivers, narrowing the productivity gap between high- and low-skilled drivers by 14%. The result indicates that AI’s impact on human labor is more nuanced and complex than a job displacement story, which was the primary focus of existing studies.”

https://​​docs.iza.org/​​dp15677.pdf

Thanks for sharing this article here.

I realize I’m a little late to ask this, but have you thought about how to license the resulting works? There is significant long-term value in putting them under a free license (such as CC BY-SA 4) as it enables further proliferation and reuse by others who want to provide information about those inventions; but the contract with the artist might need to specify that it allows for that kind of reuse.

So hyped for this! If anyone has reading suggestions, please let us know!

Can we enhance medical devices to the point where an MRI is as easy and cheap to use and own as a thermometer?

Very fun piece. It might be interesting to take a longer view. I’m under the impression that pre 10k BC, with small populations and limited “researchers,” technologies were occasionally lost. Famously, the ability to make fire and to fish might have been lost in Tasmania. So is it a (the?) core fact of civilization that we are able to accumulate technology while losing essentially none of it.

Focus will be on the actual arguments in section on optimization pressure, since that seems to be the true objection here—previous sections seem to be rhetoric and background, mostly accepting the theoretical basis for the discussion.

I take it this essay presumes that the pure version of the argument is true—if you were so foolish as to tell a sufficiently capable AGI ‘calculate as many digits of Pi as possible’ with no mitigations in place, and it has the option to take over the world to do the calculation faster, it’s going to do that.

However I interpret you as saying in practice, that wouldn’t happen, because practical considerations and countermeasures? Is that right?

I take the quoted sections here to be the core arguments:

It’s true that you can’t get the coffee if you’re dead. But that doesn’t imply that any coffee-fetching plan must include personal security measures, or that you have to take over the world just to make an apple pie. What would push an innocuous goal into dangerous power-seeking?

The only way I can see this happening is if extreme optimization pressure is applied. And indeed, this is the kind of example that is often given in arguments for instrumental convergence.

...

But why would a system face extreme pressure like this? There’s no need for a paperclip-maker to verify its paperclips over and over, or for a button-pressing robot to improve its probability of pressing the button from five nines to six nines.

More to the point, there is no economic incentive for humans to build such systems. In fact, given the opportunity cost of building fortresses or using the mass-energy of one more star (!), this plan would have spectacularly bad ROI. The AI systems that humans will have economic incentives to build are those that understand concepts such as ROI. (Even the canonical paperclip factory would, in any realistic scenario, be seeking to make a profit off of paperclips, and would not want to flood the market with them.)

The implication here is that there are reasons not to do power seeking or too much verification—it’s dangerous, it’s expensive and it’s complicated. To overcome the optimization pressures acting against doing that, you’d need to exert even more powerful pressure to do it, which wouldn’t be present if you had a truly bounded goal that already had e.g. p~0.99 of happening if you didn’t do that. Because the risk of disruption, or the cost in resources, exceeds the gain from power seeking.

Let’s consider the verification question first. If you give me affordances, and then reward me based purely on a certain outcome, we agree I’ll use those affordances as best I can even if the gains are minimal. A common version of this is someone going over their SAT answers for the sixth time, because the stakes are so high, so might as well use all the time given to you. There are always students who will use every second you give them, they’d fall asleep at their desk if you let them then wake up and keep trying.

The question is, why in practice wouldn’t you stop at a reasonable point given the cost? That ‘reasonable’ is based on the affordances given, and what terms you effectively built into the reward function. Sure, if you put in a cost term, at some point it stops verifying, but you have to put in the cost term, or it will keep verifying. If you didn’t say exactly 32 paperclips or make it deliver you exactly the 32, it will make 32,000 paperclips instead because that is a good way to ensure you made 32 good ones, etc.

Thus your defense is to start with a bounded goal ‘make 32 paperclips’ or ‘fetch me the coffee.’ Then you put in penalty terms—asymmetrical ones I hope! - for things like costs and impacts. That could work.

You still have to worry that there will be a ‘way around’ those restrictions. For example, if there’s a way to make money that can then be spent, or otherwise gain power or capabilities in a net profitable way, and this is allowed without penalty, suddenly there’s a reason to go maximalist again, and why not? It’s certainly what I would do. Or if sufficient power lets it change the rules around its reward, of course. Or if there’s a way to specification game that you didn’t anticipate. Again, what I would look to do.

It is not trivial to specify exactly what you want here, but yes it is possible to prevent IC this way in a given case. The problem is, as the affordances and capabilities of the system increase, the attractiveness of these alternative strategies and its ability to find them increases, and your attempts to block them become more likely to fail—not that it’s impossible in theory to solve the issue in any given case.

The other problem is that if some people solve this problem, while others do not, some systems will seek power and others will not seek power, which does not solve our collective problem at all. The systems that don’t seek power quickly become irrelevant. And this is a strong argument, from the perspective of such a system and for its owner, for seeking power. If you intend to kill me to ensure you can fetch your boss’ coffee, then I cannot sit on my hands and be a humble assistant, or I will fail.

With fully maximalist goals you are in much deeper trouble, and often people give AIs maximalist goals—the most clicks or engagement, the most profits or paperclips, and so on. Then what do you actually want to happen?

Often the best way to do something really will be to seek power, or humans do choose this on reflection.

(E.g. IRL: Oil companies overthrow governments, people fight world wars in order to ensure their freedom on their farm or to implement their favorite distribution of resources, people engage in grand conspiracies or globe-spanning decades-long epic quests to win someone’s heart, wreck entire industries in order to protect a handful of jobs, work every day their entire lives to earn more money without ever having a plan to spend it, etc)

Most people spend most of their time pursuing instrumental goals—power, money, knowledge, skills, influence and so on. If you tell a system to ‘make the most money’ as many people will, what happens? It’s not that easy to put in sufficient correction terms, and when you do, you really do hurt the capabilities of the system to achieve the goals specified.

(Happy to do a call, I deleted like 3 attempts on this, and higher bandwidth /​ feedback likely helps here)

Great article, i’ve been trying lower the technical bar to entry in creating and testing BCI pipelines with LSL enabled devices by creating my new python backage PyBCI (https://​​github.com/​​LMBooth/​​pybci). It’d be great to get feedback or input from like minded people looking to create similar technologies.

I would call it metascience, and I would include Convergent Research and Speculative Technologies. See also this Twitter thread.

There is no history that I know of, it’s almost too new for that. But here’s an article: “Inside the multibillion-dollar, Silicon Valley-backed effort to reimagine how the world funds (and conducts) science”

See Jack Devanney’s blog, e.g.:

• Market Based Regulation of Nuclear Power

• Implementing Underwriter Certification of Nuclear Power

• Cleaving the Gordian Knot
  
  
  

I don’t think most of these “next einstein” arguments prove what you think they do.

If you want to increase the chances of string theory breakthroughs, you want to find the sort of people that have a high chance of understanding string theory, and push them even further. If any genetic component is relatively modest, then it becomes mostly pick someone, and throw lots of resources at educating them. If genetics or randomness control a lot, but are easily observed, then it’s looking out for young maths prodigies and helping them.

Ensuring widely spread education is more about the people making lots of small ideas rather than the lone geniuses. It’s getting people from being peasant farmers to codemonkey programmers.

This feels like a failure at actually engaging with the views you are purporting to criticize. If someone believes AGI will likely kill every living being, exactly what benefits should they consider to ensure that the evaluation is balanced? That our last couple years will be marginally more comfortable? How should the solutionist approach look?

Also, as Marc Andreessen points out in his piece, AI can also increase safety (this point seems unaddressed in your essay): https://​​pmarca.substack.com/​​p/​​why-ai-will-save-the-world

I don’t understand the point you’re trying to make. Is it “safety is good”? That seems pretty obvious?

I think the problem is that some people think the state should regulate/​interfere with how safe something can or should be. Related: https://​​worksinprogress.co/​​issue/​​anti-growth-safetyism

Thanks Jason for the article, I think it’s a useful lens for looking at the past.

The idea of having the right economic conditions for progress and invention reminded me of what Bill Gates calls the “Green Premiums” for the changes we need to make to move away from higher emissions (I imagine others have made the point before). Gates’s argument is that we need to create the right environment to “prime” the world for investing and creating these more sustainable technologies, and it seems like a proactive way that one may try to recreate the an Industrial Revolution Britain. (Whether it would work or not is a different matter!)

In the modern era, we can trace the history of elite universities from Germany to the United Kingdom, and up to today in the American northeast and Bay Area.

What do you mean by elite universities in Germany? In the UK there’s a sharp difference between an elite university like Oxford and Cambridge and the rest that doesn’t exist in Germany.

As the saying goes, talent is equally distributed, but opportunity is not.

Why just assert that this is true without providing any arguments about why you think it’s true?

The demand side of innovation has been highlighted many times in the past. As Liebenstein put it “The knowledge may have been there already and a change in circumstances induced the change in technique”. More generally the Boserup hypothesis sets out how agricultural innovation is driven by increasing population densities. Adam Smith made the same point in his discussion of agriculture in the American colonies. That these ideas not widely known, or just forgotten, reflects how much more attractive people find the silver bullet, hero-inventor story of innovation. And how much more useful those ideas are for politicians.

Not sure if this is quite what you are looking for, but I’ve been keeping a list of progress-related museums that I have visited or want to visit, large or small, including:

• Antique Gas & Steam Engine Museum in Vista, CA

• Charles River Museum of Industry and Innovation in Waltham, MA

• Henry Ford Museum in Dearborn, MI

• Davistown Museum in Maine

• Museum of Craft & Design in San Francisco

• History of Science museum in Harvard (one room)

• Jenner Museum, Gloucestershire; also a statue of Edward Jenner in Kensington Gardens?

• Bibliotheque de la Faculte de Medecine in Paris, which houses The Jubilee of Louis Pasteur, by Jean-André Rixens

• Fleming’s original Petri dish in the British Museum

• Institute of Making, part of University College London

Wages are both about productivity and about how much value workers capture of their work. I asked GPT4 for the factors of why London grew in the 16th century and one of the reasons it comes up with is:

Dissolution of the Monasteries: When Henry VIII dissolved the monasteries in the 1530s, vast amounts of land and wealth were redistributed, often ending up in the hands of the mercantile class in London. This led to further growth and development in the city.

Monks at monasteries were a class of people that were very poorly paid. GPT4 describes the economic effects further as:

Economic Shift: The dissolution of the monasteries led to significant economic changes. The rural economy, in particular, was affected, as monasteries had been major landowners and employers. After dissolution, many former monastic lands were converted into profitable sheep pastures, contributing to the growth of the wool trade, but also leading to rural displacement and unrest, a phenomenon known as the enclosure movement. Furthermore, the dissolution ended the monasteries’ roles in providing social services such as healthcare and education, which had to be taken over by other institutions or simply vanished, leading to increased poverty and suffering among the poor.

Please let me know how to delete my account and all comments and posts related to it. Much appreciated

One other factor, which I would class under demand factors: at each stage of the industrial revolution, you need to have enough demand for the primitive early version of your new technology, so that people will use it, iterate on it, and develop it.

You write “early Newcomen engines consumed ~45 pounds of coal per horsepower-hour; the most efficient engines of the late 1800s used less than one pound.” But to get to those efficient engines, it had to be worth someone’s while to use the very inefficient early Newcomen’s engines, so that there was a version 1 which could be iterated into the 1lb/​hp-hour version.

Bret Devereaux has an argument that this was specific to Britain, whose geology happens to contain lots of coal mines which were prone to flooding. (I recommend reading the whole thing.) So the very first use of a steam engine was to pump water out of coal mines. Because the use case was literally at the coal mine, there were zero transport costs for the coal (a major consideration in an era before railroads and steamships). The very inefficient Newcomen engines still raised more coal than they burned, which would otherwise have been inaccessible, so there was an economic use for them. The UK by this point had already cleared most of its forests, so there was demand for coal (not necessarily true in earlier periods). And the higher wages in England, which you discuss, meant it was cheaper to pump the water with steam engine than with muscle power even where muscle power was technically practical. His argument is effectively that without all of these factors, the very inefficient early steam engines would not have been economic, so no one would have iterated on them.

I don’t think this is the only factor—you/​Allen have gone through a lot of others, which also make sense. But I think it’s worth considering that part of the answer to ‘why did the Industrial Revolution start in England?’ is geological accident.

typo: products or launched or other --> launches?

Nice post! Seems like a really important but unsexy thing to understand /​ improve regulation. It’s a shame to be limited not by technical problems but by misaligned orgs. It’s like stunted growth due to malnourishment

So true, Tony! Finding ways to creatively bring the study of human progress into schools is important, for so many reasons. It will be fun to work through the specifics on how we can make this happen—and to think specifically about how this teaching can inspire kids.

I would guess Alexander Hamilton (or other founding fathers). I assume you added the words AI in exchange for Britian.

I would wager Hayek, Huxley, or Orwell

My first instinct is Frank Herbert. But that seems too obvious, and your note that you brought it up to date makes me think it might be earlier.

I have been following this for years, but the project does not progress very much (there are more machines, but the community has not grown). The village set would be both useful for civilization RE-construction and for civilization (First) construction in poor countries. In my first post in the EA Forum I tried to produce some interest either in this project or some susbtitute, because no other open source project could be more useful.

Related: The Long Now Foundation’s Manual for Civilization

“What books would you want to restart civilization from scratch?”

The Long Now Foundation has been involved in and inspired by projects centered on that question since launching in 01996. (See, for example, The Rosetta Project, Westinghouse Time Capsules, The Human Document Project, The Survivor Library, The Toaster Project, The Crypt of Civilization, and the Voyager Record.) For years, Executive Director Alexander Rose has been in discussions on how to create a record of humanity and technology for our descendants. In 02014, Long Now began building it.

The Manual For Civilization is working toward a living, crowd-curated library of 3,500 books put forward by the Long Now community and on display at The Interval.

See also Lewis Dartnell’s book The Knowledge.

On my opinion there are several adaptation-related factors which act differently in different times and circumstances. The newest most important now seems to be fear based on ignorance and, well, weak congintive abilities (50 years ago it was called stupidity...). COVID had demonstrated how quickly the mob requires and demands to stop, to shut down, to close and to lock everything and everywhere. Rational thinking was not applied. Mob is not 100% of population but it’s formed by it’s share large enough to push the states to act in certain direction. Discussion was not possible, the mob had to study the whole texbooks first. Social media, press news cycles (this is not an exlcusive list!) amplify fears, forecasts of soon imminent extinction and organise and agitate the mob. How the people will react to really fast changes like, say, automated finances system replacing accountants and whole finance departments everywhere? Film “Interstellar” gives a good picture how it may happen (“We were never in space and we definitely do not talk with children aboit things that sinful”). Such factor sure always existed but now it’s qualitatively stronger. At times and situations it may amd probably will reverse, well, at least local progresses in certain fields if not the whole way how we do things.

Yes. I recommend this article which has some helpful data pulled from the World Bank and Our World in Data, and explains the trends.

https://​​www.ft.com/​​content/​​967e1d77-8d3c-4256-9339-6ea7025cd5d3

But to summarize, as economic development advances, more people can spend more time inventing cleaner and better ways to do things, and industrial and economic development makes clean energy cheaper to deploy.

I thought I’d post this here as the program content is partially inspired by Progress Studies (the unofficial working title even was ‘Progress Fellowship’ for a while!). Hope it’s helpful!

I bet GPT-4 could already do a lot of this work, perhaps with some fine-tuning and/​or careful prompt engineering.

The problem with automating compliance documents is not just the time/​effort to prepare them. It’s also the time spent waiting to get a response, and in some cases, “user fees” paid to the government to review them. If everyone started using GPT to do compliance, I suspect that the various agencies would just start to build up an ever-growing backlog of un-reviewed applications, until they’re all like immigration and they have decade-long wait times.

The entire argument rests on current nuclear arsenals being powerful enough to kill all people right now, but the author does not cite a single link supporting that key assertion. And for a good reason: even pessimistic scientific estimates say around 40% of people would survive.

If regulation, money, and public opinion were no constraint, how many IQ points could a “CRISPR-baby” gain with modern technology?

How much of an issue is the “causal tagging problem” (i.e. knowing that the genes you’re editing actually increase intelligence)?

How much of an issue are off target mutations?

How many IQ points could be gained with just embryo selection?

what are some examples of beating regulations when it hinders innovation?

Hi. Maybe ten years ago, thanks to your role in maintaining #hplusroadmap, you seemed to me a leader of a new generation in transhumanism. Since then, while technology has kept advancing, and various futurist dreams have remained alive, I feel that “transhumanism” as a movement or subculture or identity, lost some of its momentum and its avantgarde character. One way to put it, is that in the 1990s there was a transhumanist futurism centered on Eric Drexler and nanotechnology, but in the 2010s it was eclipsed by the rationalist futurism centered on Eliezer Yudkowsky and artificial intelligence. Of course there have been many many other movements, personalities, and technologies of interest; but I think one could tell a plausible story about the radical techno-futurism of recent decades, in which a cultural transition from transhumanism to rationalism, is at the heart of the narrative.

Meanwhile, your own bio says that you got heavily into crypto, something I never got into, but which went from a “cypherpunk” fantasy, to a headline phenomenon, and ultimately to a financial ecosystem with a nominal worth of something like a trillion US dollars, and which, pardon my bluntness, seems to be riddled with at least as much hype, scamming, and corruption, as anything in mainstream finance. I would be interested to know how the recent history of transhumanism and futurism looks, from the perspective of someone who took your path, into the world of startups and legal commerce.

What are the most cogent fears about genetic engineering, given the state of the art and what’s technically feasible?

Conversely, what are some prominent fears that are unlikely to ever be realised?

How much is innovation in genetic engineering held back by regulation?

Why do you think we don’t have more people starting ambitious genetic engineering projects?

What are the best near-term/​foreseeable applications of genetic engineering? What is the low-hanging fruit here that we can see and define and should go after first?

Related factors: it appears that harms due to technological change are much smaller than benefits due to technological change, and also much smaller than harms that we already suffer on an ongoing basis (like deaths due to disease).

Slightly tangentially: I have so many questions about that Irish population graph, like:

-- That is a huge population growth rate between 1800 and 1840 before the famine. What was going on there? I graphed it vs a few other European countries and while many of them were growing steadily during that period, nobody else had that high a percentage of growth (54% growth in 40 years if I’m doing the math right) except the UK, which has a crazy quick and apparently unrelated knee upward in the graph in 1810-1820. Was that just a time when both Ireland and the now-UK lands escaped the Malthus trap sooner than everyone else?

-- Then after the famine deaths and emigration wave, there’s a long steady decline that seems to be levelling off around 1920-- but then drops again, quite quickly, so that the population declines another 25% or so between 1920 and 1930. I can’t help thinking it must be related somehow to the Irish Free State gaining independence in 1922; but if OWiD had suddenly stopped counting the Northern Ireland population as part of Ireland at that date, I’d expect an even more abrupt decline. Was it emigration of the Protestant minority in the south to the UK? Economic hardship associated with independence? Something else?

Thanks for the great article :-)

I am commenting as someone who has spent a lot of time thinking about AI alignment, and considers themselves convinced that there is a medium probability (~65%) of doom. I hope this is not intrusive on this forum!

I hadn’t considered the crux to be epistemic, which is an interesting and important point.

I would be interested in an attempt to quantify how slowly humanity should be moving with this: Is the best level comparable to the one with genetic engineering, or nuclear weapon proliferation? Should we pause until our interpretability techniques are good enough so that we can extract algorithms from AlphaFold2?

I am also interested in possible evidence that would convince you of the orthodox (“Bostrom-Yudkowsky”) view: what proofs/​experiments would one need to observe to become convinced of that (or similar) models? I have found especially the POWER-seeking theorems and the resulting experiments enlightening.

Again, thank you for writing the article.

What could a superintelligence really do? The prophets’ answer seems to be “pretty much anything.” Any sci-fi scenario you can imagine, like “diamondoid bacteria that infect all humans, then simultaneously release botulinum toxin.” In this view, as intelligence increases without limit, it approaches omnipotence. But this is not at all obvious to me.

The idea of creating ASI as an omnipotent being, far superior and all-knowing, strikes me as a pseudo-religious argument wrapped in technical and rational language that makes it palatable to atheists. It’s a bit like how the wildest predictions from longevity/​curing aging feel a bit like heaven for people who don’t believe in god.

I get how to get from ANI to AGI and then to ASI. It makes sense. But at the same time, something about it doesn’t. Perhaps this is why this position (AGI as a harbinger for extinction) lacks mainstream appeal.

If some rogue AI were to plot against us, would it actually succeed on the first try? Even genius humans generally don’t succeed on the first try of everything they do. The prophets think that AI can deduce its way to victory—the same way they think they can deduce their way to predicting such outcomes.

Is this rationalists anthropomorphizing AI to behave/​think like they thing, perhaps?

Great article! I think you expressed The Argument well and similarly to how I see it expressed by those who believe it.

I’m always surprised by how many tools are available to evaluate the argument…and that its fans rarely use any of them. It’s great to see you use some of these tools to critique it!

By way of comment: at the same time, your article leaves the argument looking more plausible (to me) than it probably is, just because your critiques don’t include as many angles as it might from progress studies (especially the scientific method and the history of technology). My attempted survey of the possible angles, some but not all of which you tackle:

Most catastrophic risks have a lot of evidence to tell us how much we should worry about them (the history of infectious disease outbreaks, nuclear accidents and near-accidents, etc). The argument never comes with any evidence. Worse yet, it’s rarely presented as a hypothesis to be falsified, but instead as speculation. This is especially surprising because their main catastrophic scenario is an accident, and accidents are one of the most common and well-studied kinds of risk (auto accidents, the Tacoma Narrows Bridge, airplane accidents, policies for canceling ferries in dangerously bad weather, nuclear power plant accidents, accidents involving Covid in a Wuhan laboratory vs. Wuhan seafood market, etc). Accidents are studied by all sorts of people including actuaries, government technocrats, and popular authors. Successful predictions of catastrophe (or anything) are almost always based on evidence.

More generally, the argument is usually presented without any scholarship or context outside of speculative philosophy. But there is lots of scholarship to know (beyond the above) from the histories of technology, human well-being, and predictions of apocalypse, and probably many other domains.

A cost-benefit analysis would be needed if the argument were to be made credible. Lifespans are about 35 years shorter in poor countries than they are for Japanese and Swiss women, and about 15 years longer for the richest US females than the poorest US males, so it’s a good estimate that 25+ years of life are lost by the average person due to risks that can be attacked by anti-poverty, public-health, and economic growth measures alone. Peter Attia is probably right that exercise, sleep, and food account for another 10 years. As you say, the argument glibly assumes that AI will solve pretty much any problem it needs to solve to kill us all. We have no reason to believe that, but those who do surely should also believe that the AI will solve any problem it needs to to gain that 35+ years of life for the average person among the 8 billion of us. At this rate, even Scott’s estimated 2% risk of an AI apocalypse looks like a bargain. The context provided by cost-benefit analysis also reminds us of where we ourselves should focus our attention. And of course the likely upside of AI doesn’t just depend on a glib assumption of AI capabilities — AI is a general purpose technology, so progress studies tells us something about what upside to expect.

Finally, the argument is rarely presented with a plausible mechanism.

What do you make of Eli Dourado’s take that AI probably isn’t a transformative/​revolutionary tech because of regulations and the lack of innovation in physical meat-space etc? (“We wanted flying cars, and all we got was 140 characters”-kind of thing)

ones that require human creativity, empathy, and collaboration.

Curious why you think these can’t be replaced by AI creativity, empathy, and collaboration.

Why are there so many medical studies using sloppy research methods, and how big a problem do you think this is?

I noticed this when trying to figure out how common Long Covid is—most of the studies being reported in the media, at least early on, did not have a control group. On the basis of these studies, the media was saying that Long Covid affects 30, 50, or even 60% of people who get Covid.

Many of the studies also use methods which suffer from responder bias like surveying online support groups. Studies which track cohorts over time and have a good control group find more modest figures like 10-15% of patients experiencing greater than expected symptoms at 3 months. However nearly all of these are retrospective studies which as I understand it are not as good as prospective studies. More recently a study came out which does what should be done all along—it compares outcomes of Covid patients with patients who got symptomatic non-covid upper respiratory infection. They found more symptoms in the control group than the Covid group at 3 months. This calls into question whether Long Covid is actually an actual phenomena in its own right or just another iteration of post-viral illness /​ post-viral chronic fatigue syndrome (see Vinay Prasad’s video).

I wonder, if low quality studies can be so misleading, is it worth doing them at all? It seems to me we should be pooling resources to do more high quality studies rather than many low quality ones.

How do you think the role of physicians would be affected by AI?

how far we are from bio engineering to treat diseases/​conditions?

Are we winning the war on cancer? Is it reasonably fast/​steady progress, or has something gone wrong?

What has gone wrong in the fight against Alzheimer’s? Did a “cabal” prevent funding for anything other than the amyloid plaque hypothesis?

What do you think is the cause of Eroom’s Law? Why has it (fortunately) stalled in the last decade? Do we have any hope of reversing it?

To see the other perspective, try replacing “consumption” with food and “consumerism” with obesity. We only have 1 earth (for the foreseeable future), and rampant consumerism leads to a very inefficient conversion from its resources to value.

Also, you can still be anti-consumerism while agreeing that the global south would ideally see higher consumption. Reducing obesity doesn’t mean we shouldn’t feed the starving.

I would agree with you about growing costs for equipment in trendy “big science” (dark mater, hot fusion, gravity waves, accelerators), and I see this trend in military domain, like in IT in old-fashioned companies, like in nuclear industry… It is aggravated by a growing increase of regulation.
It seems that some domains push providers to improves performances to the point nobody can buy the product, but it is really perfect. I’ve heard that for nuclear reactors (they can resist to anything, but nobody can afford them), for tanks (they are smart, agile, powerful, robust), for military drones (they can work in civil air zone, do any mission, transport much, but cost like a helicopter)...

Meanwhile, in IT I’ve seen the trend to RAID disks, to Cloud, to SaaS, while Ukraine war showed the efficiency of simpler drones, not so overengineered canons, old tanks and old planes, tinkered by motivated staffs...

Here I’ve caught an article about the cross-pollination of AI and new experimental methods, reducing costs by 10x.

https://​​www.wsj.com/​​articles/​​biologists-say-deep-learning-is-revolutionizing-pace-of-innovation-eeb79c1b

It makes me think about the exponential learning curve drawn by piling S-curves… As if the end of an S curves goes to unaffordable perfect technology, and that a revolution make you start again the exponential phase of a new S-curve...

See how quickly African labs have used Crispr-CAS technology for their own needs, to fight emerging diseases or climate change in their agriculture. It was done for much cheaper than for transgenic GMOs.

Have you considered printing off a few sheets of paper, getting some glue, and just adding a few signs yourself? ;-)

Some tech, like seatbelts, are almost pure good. Some techs, like nukes are almost pure bad. Some, like cars, we might want to wait until we develop seatbelts and traffic lights for before we use widely. It depends on the technology.

Thanks for posting and considering. I agree. It would be great if more people researched this.

A simple way to start would be to study the open science hardware momentum. A few anecdotes from that scene over the past decade:

OpenROV (us) — made ROV prices >10X cheaper Open qPCR (Chai Bio) — made qPCR multiple X cheaper OpenTrons — made liquid handling multiple X cheaper Cubesats (not really OSH, but similar idea) — made satellites 10x cheaper

They are all orders of magnitude more affordable, and many have completely rearranged who uses the tool, sometimes opening up big new markets. The lack of demand pull assumption should be tested.

-David

Consumers, in the “consumerism” worldview exist only to receive goods. It’s a primarily self-centered orientation to the world, and that’s why people sneer the word with such a moralizing tone.

Imagine the opposite of consumerism is producerism. Producing time-saving conveniences, building stuff, retaining walls and, heck, even trivial trinkets. Producing is a “nice thing to do.” An active life working and valuing the things you wished you valued while helping others in the small, tedious ways that the economy rewards a person for.

But a “consumerist” is a distracted, binge-watching, GrubHub couch potato perhaps sporting a part-time BS job. People are afraid of living in a “distraction/​hedonistic/​morally corrupt/​selfish society”. And part of the reason this objection to society comes up so much is that (probably mistakenly) they think the following:

1. many jobs are BS 2a) much of what people purchase is not actually valuable, only perceived as such. 2b) much of what we buy, our best selves would not.

2. much of what society nudges us towards (in music, in carbon emissions, in social values) is not in our own or our collective best interests.

What is the problem of missing novelty in drug development, and what can be done to fix it?

Which chapter of the book was the most interesting to write?

What is the impact of social media on medical discovery?

You asked for steelmanning. I speak as someone who’s much on board with much of what you say above.

For me, the key reason why I’m worried about consumerism is that it can easily become an addiction and it tempts me to give the unimportant things in life too much attention.

Imagine if someone asked why cocaine or alcohol have a bad image. Obviously, cocaine and alcohol make people happy—very much so. But they’re also addictive. What was supposed to be a tool for achieving happiness turns into a master. And then it destroys my ability to focus my full attention on the things that truly make us happy: relationships, God, achievements, nice meals, generosity, art, etc

Great points. In (good) science, scope matching is one of the most important concerns. I’ve always wondered why it doesn’t have a (widely used) name.

Scope matching failures really do come up constantly in modern criticisms of new technologies, whether it’s social media or AI. Probably happened centuries ago too

I think part of what the critics of “consumerism” are motivated by is the increased specialization of production which ensued from the Commercial and Industrial Revolutions. Instead of molding their own bowls and carving their own spoons of wood and fashioning their own stools to sit on, people bought much better and cheaper stuff from Wedgewood, Chipendale and other manufacturers. People had consumed before, but the production was invisible as it occurred in in cottages.

I think another part of what critics have in mind is that a good bit of consumption is in the form of positional goods—keeping up with the Jonses (or showing you got ahead of them). Now I think the competition for social status is a human universal and not likely to be changed by mewling moralists. I think competition by striving to get the baubles—the Ferraris, the Guccis, etc. - is far less damaging than many of the alternatives. People will compete for social status; it is simply a question of what form the competition will take. The great triumph of modernity is in replacing the quest for glory and honor with the quest for stuff.

Great post. I would add that when we talk about lifting people out of poverty we’re literally talking about increasing their consumption. Consumption is also a synonym for the economic part of well-being.

I’d venture to speculate that the main reason we aren’t better at reducing poverty, increasing middle-class well-being, making life better for families, fighting disease, and other important goals, is because we don’t pay enough attention to increasing median consumption.

For anyone who’s interested: I’ll be teaching the next cohorts of The Foundations of New York soon! It’s an accelerated introduction into NYC government/​law that also touches on dependencies at the state and federal levels. Class begins in mid-April and goes through May.

https://​​maximumnewyork.substack.com/​​p/​​the-foundations-of-new-york-applications

How do you think that framing a discussion about the effects of future technologies as potentially leading to scenarios of existential hope as opposed to existential risk can be helpful? Or is positing a dichotomy between existential hope and risk sort of missing a key point?

It looks like the next major technological wave will be AI. How might this change Foresight’s plans or focus areas? Would you focus more on AI? Or, can AI help us with nanotech, longevity, etc. (and how exactly)?

What areas of research or technology are most underrated by the broader research world, and why?

What are some new projects you are working on but haven’t published yet?

What are some projects you want to see but wish someone else or another org was working on?

How does Foresight Institute allocate its time/​effort across projects? Do you have a way of thinking about how much attention to spend on different speculative areas?

Do you see most of the value/​impact/​benefit that the Foresight Institute produces as coming from a few key outputs, or from a larger list of projects each of which only produces a fraction of your total value? If the first, what are your key outputs?

What does Eric Drexler think about the Foresight Institute? If I recall correctly, he was one of the founders?

What are some example decisions that the Foresight Institute’s work has helped influence?

Thanks for this, Lev. Some things I’d like to understand deeper if you were to write about this more:

• What are the remaining challenges of BCI? (vs. what has been solved already?)

• What are the most promising approaches? How do they differ and what are their pros/​cons?

• Do we need scientific discoveries to make this work, or is this science known and it’s mostly engineering?

• You talk here mostly about motor output; what about sensory input?

• What is the legal framework around this? Does existing regulation even handle this? Will that be a problem?

Most of the EA longtermist arguments are about future people existing at all. If there’s an extinction event, there will be no future people with complex values.

Usually “optimistic news on science” means writing newspaper article that take a scientific study and misrepresent it as being more important than it really is to entertain people who like reading about science. A big part of human progress is about fixing problems and roadblocks to it.

One big undercovered progress story might be the new CTIS in the EU that became mandatory for new clinical studies at the end of last month. Clinical trials are important to advance science.

I’d love to read good reporting on CTIS that actually goes into what’s good and bad with the new system and not just reporting that’s optimistic about it.

Another interesting story that might benefit from good journalism is that the National Electric Code in the US is written by the National Fire Protection Association and thus for the interests of firefighters. It seems that they added a requirement for single module shutdown to solar cells with results in solar installation being more expensive while there’s no clear evidence that it produces a significant safety gain.

In both cases, the subject matter of how regulations relate to progress is very complex, but good reporting about those could do a lot more for progress than cheering on individual things.

Elon musk is very good at making himself the center of as many conversations about technology as possible.

He should not be taken as a source of information of any reliability.

Living on mars with tech not too far beyond current tech is like living in antarctica today. It’s possible, but it isn’t clear why you would want to. A few researchers on a base, not much else.

Think ISS but with red dust out the windows.

At some point, which might be soon or not so soon, tech is advanced enough that it becomes easy to get to mars. But at that point, traditional biological humans on mars might be stupid, compared to say self replicating robots containing computers running uploaded human minds in the asteroid belt.

A mars base is cool scifi. But it might turn into the largest white elephant in history. It doesn’t solve any obvious practical purpose in increasing human wellbeing or industrial capability.

Sure, at some point you are disassembling all the planets to build a dyson sphere. But before that, a mars landing doesn’t actually need to mean any real progress.

I don’t think the “aside from the internet, nothing much”. Firstly comuter and internet tech have been fairly revolutionary across substantial chunks of industry and our daily lives. This is the “a smartphone is only 1 device so doesn’t count as much progress” thinking. Without looking at the great pile of abacuses and slide rules and globes and calculators and alarm clocks and puzzle toys and landline phones and cameras and cassette tapes and … that it replaced and improved on.

Secondly, there are loads of random techs that were invented recently, solarPV, LED’s. Mrna Vaccines. Electric (self driving?) cars.

And finally, a substantial part of progress is the loads of tiny changes that make things cheaper and better. If you don’t include things like 3d-printers and drones that haven’t really gotten good yet, then of course you will see less inventions recently. The first fridges were expensive and not that good either.

If longtermists existed back when blacks were widely regarded as morally inferior to whites, would the moral calculus of the longtermists have included the prosperity of future blacks or not? It seems like it couldn’t possibly have included that. More generally, longtermism can’t take into account progress in moral knowledge, nor what future generations will choose to value. Longtermists impose their values onto future generations.

It is true that we can’t predict future moral knowledge. However.

1. An intervention by someone from that time period that helps modern whites and doesn’t harm modern blacks would still be seen as better than doing nothing from the point of view of most people. (excluding the woke fringe) Most random interventions selected to help future white people are unlikely to cause significant net harm to blacks.

2. If their intervention is ensuring that we are wealthy and knowledgeable, and hence more able to do whatever it is we value, then that intervention would take into account progress and moral knowledge.

3. In reality, you have to choose to do something. When making decisions that effect future generations, either you impose your current values, or you try to give them as much flexible power to allow moral knowledge, or you basically pretend they don’t exist.

This is an intresting new combination of standard mistakes.

Another issue is that if altruistic morality is taken to its logical conclusion, then everyone would be trying to solve everyone else’s problems. How could that possibly be more effective than everyone trying to solve their own problems?

Altruistic morality in the total utilitarian sense would recognize that solving everyones problems is equally valuable, including our own. In the current world, practically no humans are going to put themselves lower than everyone else, and most of the best opportunities for aultruism are helping others. But in the hypothetical utopia land, people would solve their own problems, there being no more pressing problems to solve.

If we are here to help others, what on Earth are the others here for?

Well imagine the ideal end goal, if we develop some magic tech. Everyone living in some sort of utopia. At this point, most of the aultruists say that there is no one in the world who really needs helping, and just enjoy the utopia. But until then, they help.

What we actually need to be is selfish, not altruistic. We need to make as rapid progress as possible so that the people of the future themselves will be at a starting point where they can make even more rapid progress.

A aulturist argument for selfishness. You are arguing that selfishness is good because it benefits future people.

If you were actually selfish, you would be arguing that selfishness is good because it makes you happy, and screw those future people, who cares about them.

I also don’t know where you got the idea that selfish=max progress.

Suppose I am a genius fusion researcher. (I’m not) I build fusion reactors so future people will have abundant clean energy. If I was selfish, I would play video games all day.

Altruism is subordinating one’s own preferences to those of others. It’s a zero-sum game. It’s not win-win.

In the ideal utilitarian hypothetical utopia, who exactly is loosing. If hypothetically everyone had the exact same goal, the well being of humanity as a whole, valuing their own well being at exactly the same level as everyone elses, that would be a 0 difference game, the exact opposite of a 0 sum game.

Some thoughts on Meaning & Modern Job Satisfaction

Jason recently shared a thread on the tension between the objective criteria that make work meaningful increasing while the subjective experience of perceived meaning of work seems to be decreasing. As with most things related to progress, much of this likely stems from a combination of rising expectations and the current emotional climate of pessimism. However, with the help of several conversations, I believe that I’ve identified two elements that may help further explain the gap between objective and subjective experiences.

I was talking about this with a friend — Ashley — who is an upper middle manager at Nike. She’s worked at Nike for 10+ years, with numerous promotions and “career success” by most standards, she enjoys her work, loves the people she works with and has a fairly high degree of autonomy. She’s also an athlete, mostly a runner, who engages in the running community and does Nike sponsored events every year.

Based on all of this — the mastery, autonomy, recognition, human connection and the intersection of her work with her personal life — Ashley should experience a high degree of meaning in her work, but she shared that she experiences almost no meaning. That said, she has no plans to leave and her job has lots of emotional upside including being supportive of her family life.

In digging into why, it largely came down to two things (that she did not enunciate exactly, but I summarize as):

• Feeling 100% replaceable — Ashley explained how many people she’s seen come & go over the years and how professionally, its meant very little to Nike. They may be missed personally and there may be some short term pain from a transition, but that, in her words, ‘the whole point of the corporation is so that no individual matters. We are all replaceable — and that’s a feature.’
  
  I can imagine a past where, even a low meaning job by today’s standard, would not have felt so replaceable. Growing up in a (very) small town, I can tell you with confidence that when the pizza place closed, no one starved, but it was MISSED in a way that even the most popular pizza place in a city never could be.
  
  

• Genuine uncertainty of causing harm vs benefit — While Ashley can repeat the marketing premises (and, yes, Nike has an entire team whose sole purpose is to market internally, to employees), she is genuinely uncertain of whether Nike produces a net benefit on the world. She conceptually embraces the ideals of Nike, but does not trust that Nike acts in a manner that expresses those ideals consistently nor that it is even possible for Nike, within a capitalistic system, to act as a net-positive for society.
  
  Anecdotally, I had a very different conversation with a friend who works at OpenAI that lead to a similar conclusion (s/​he wishes to remain anonymous) . While he does not feel replaceable, he is very concerned about how his job has shifted to become significantly less meaningful and more challenging to be fully engaged with as his ethical concerns about the company and general concern about the future have increased in the last 8 months.
  
  To beat a horse dead with anecdotes, my father, who mines garnet, finds enduring and genuine meaning from unlocking resources from their raw state into one that is usable. He feels little uncertainty about the net benefit of his work. Meanwhile many people that I meet in my day-to-day (highly educated /​ not ever going to be miners) are honestly appalled by the idea of mining, let alone that the mine is within the boundaries of a protected wilderness and generally view his work as detrimental, rather than beneficial (and thus not meaningful).
  

While both of these are highly subjective criteria, so is an individual’s assessment of meaning. Framing matters. I think that people, and especially younger generations, are weighed down by their genuinely uncertain about how to positively impact the world — and a huge chunk of that is what progress studies is looking to address! It’s also why, in my opinion, the clarity and confidence of the EA worldview was able to spread so rapidly.

--

Here’s some research that dances around supporting the ideas, although I wasn’t able to find anything that nailed it in a cursory search:

• You’re probably well aware of studies that indicate decreasing trust, which in turn leads to a dearth of confidence in the actions that will lead to their desired result. EG: the well supported idea that there is declining trust in institutions, scientists, and how greenwashing has /​ is significantly diminishing trust in corporate ethics (as well as scandals like ENRON, near disasters like the 2008 banking crisis, etc)

• Here is an HBR thought piece about potential relationships between talent & corporations that is relevant Plus a Quora thread on “Are employees really replaceable?” that I think highlights normal people’s thoughts on it (clue — YES!)

--

As one further, bonus anecdote, Ryan Holiday just wrote a piece about completely changing his marketing strategy to align with building a more meaningful life.

This may be a faux pass of the community rules about disagreement, but I don’t think you’ve created anything interesting here. Your “new movement” comes across a lot more like a cult than a concrete way to find meaning.

You describe a crisis of modernity in very vague terms in this piece. You point to things that are bad, gesture that our emotions might not be positive, and suggest that we need to develop a new way of living and finding meaning. To find meaning, you recommend watching a video you made.

This is generally how your video comes across too. You describe the “void” which is a shorthand term for anxieties, concerns, and nervousness about the future, and then say it’s solved by “the light in the void” which is further explained over the next 20 minutes to be, more or less, the meaning of life. You provide a few exercises about how to find this meaning for yourself, but the exercises tend to focus more on making the viewer feel a certain way rather than clear exercises that the viewer can build upon or share with others. It is a vibes-based way of thinking, not a structured one. Furthermore, it is not clear why a new movement is needed for people to find this meaning, rather than, say, to read a self-help book. (For a book that actually helps you to find meaning in your work/​life, I recommend “What Color is Your Parachute” personally. It has been in print for 50 years)

To learn more about your movement/​idea, you provide a link to your website. This website has links to two youtube videos (including the one discussed earlier) and another which is an hour and thirty minutes. I couldn’t/​didn’t get through that one. The website touts the variety of people in the movement (Successful hedge fund managers! Interesting and cool anarchists!), but is light on how the system actually works. The only other thing on the website are two donation buttons, one for a $200 donation and and one for a $2,000 donation to the organization that you’ve started.

I am not going to blatantly assume bad faith here, but if you do not intend for this to be a cult, then you need to make your pitch significantly more clear and cut to the chase of what you’re trying to say.

I wrote about Class 1 /​ Class 2 in the context of blockchain for my blog today and wanted to share my updated thoughts after spending a few days thinking.

I think fundamentally, Class 2 problems is just a rephrasing of tragedy of the commons issues. Rephrasing is useful because it gives us a new perspective to approach an issue.

In the piece, I suggest that we can predict Class 2 problems by thinking about the specific features of the technology, eg blockchain, which motivate entrepreneurs to solve the Class 1 problems, and thinking about how those features could be bad when overdone (classic market failure ideas of over supply)

Rather than coming up with a checklist of things to look out for, which we might never complete, I think using the lens of ‘what persuades entrepreneurs to solve the Class 1 problems, and how could this be bad’ gives a useful way to approach Class 2 safety topics. It also lets us make the argument that ‘these Class 2 problems are only here because the technology was so good that we fixed all the Class 1 problems, so let’s face them head on, rather than banning the technology (or similar)’

My blog post here

We will be hosted on Saturday March 25 at 2:00 pm in a room of the New York Public Library on 5th Ave and 42nd.

In a previous post I commented that before we deal with spaceships, we need a complete circular and almost self replicating human economy:

https://​​progressforum.org/​​posts/​​62XwqubkcAKNG6Wqf/​​space-colonization-and-the-closed-material-economy

This is the hardest problem

When it comes to the philosophy of space technology, the effects of it on earth shouldn’t be undercounted. Cheap satellites have a lot of implications for privacy when every spot on earth can be surveilled ²⁴⁄₇.

Orbital bombardment is a powerful weapon with a strength comparable to nuclear weapons and in cases like attacking underground bunkers even more powerful. At the same time, the existing fears of radiation don’t exist for orbital bombardment. It’s important to think well about how to handle the implications of powerful technology in orbit.

SpaceX tanker ships might be powerful enough to do serious damage and yet they don’t have the safety against cyber attacks that military weapons usually had in the past. What happens if a SpaceX tanker crashes in the White House, removing it completely from the map and nobody really knows why it happened?

Gavel-kind succession laws, whereby all children got an equal share of their parents’ estates, rather than it all going to the eldest. English primogeniture, by contrast, apparently left a lot of gentlemen’s younger sons having to become apprenticed to merchants.

It would be interesting to understand more about the effects of those laws. Being able to give all the wealth to a single son makes it easier to invest all the wealth in a single commercial enterprise because that enterprise doesn’t have to be broken up.

″

Just a sense of why I wrote this article, it will be part of a collection of articles about technology pertaining to the brain and consciousness (all forms of BCI and virtual-reality), and technology mimicking the brain and consciousness (AI and computational models about consciousness). But really all of this is in terms of progress and how all this is possible.

Hello, I’m Kassandra Dick, a freelance writer from Canada. Progress is the only viable option for humanity and I’m all about surviving and thriving. I love the work and writings of Peter H. Diamandis and Steven Kotler, as well as Yaron Brook, Ayn Rand, Alex Epstein and David Eagleman. You can find my work and my poetry here: https://​​kassandra-dick.medium.com/​​

I’m Lev, at least that’s what I go by online. Most of my interests that I will be writing about have to do with AI, consciousness, brain technology, and space travel. I care about these things in terms of progress, and I want to promote the philosophical grounding that progress requires.

Impressive application Tony, I admire your commitment to human progress and look forward to seeing what you will achieve!

Sounds interesting, thanks for announcing this here!

I hope we get more attempts to bridge economics research and the broader public, like this and New Things Under the Sun.

Congrats, Tony!

Hello everyone, I am Isaac, a CompE and math double major at UMBC. I am interested in working on medical devices and am interested in starting a for-profit startup. This startup would hopefully work on both direct impact and allow money for donations (see here).

I am also considering a PhD or Masters in Biomedical engineering, computer engineering, or electrical engineering.

Thanks Thomas! There is a community section of this forum where you can create local groups and meetups /​ other events. Please post there and let me know, I’ll help spread the word.

What EU regulations that aren’t really justified in terms of providing safety are most responsible for holding back innovation?

Your work seems like a spiritual successor to Simon’s Ultimate Resource, sounding some of the same themes for a new generation. What are the biggest or most interesting updates since Ultimate Resource was published? And/​or what did you find in your research for the book that surprised you?

I have heard it claimed that Julian Simon got a bit lucky in his bet with Paul Ehrlich, and that if a different basket of metals or other commodities had been chosen, he might have lost. Is that true? What do we make of that?

I

I have read in Where Good Ideas Come From by Steven Johnson that innovation can be modelled by a graph where create a vertex on the graph with edges on the used technologies that are combined. In The Structure of Scientific Revolutions, Thomas Kuhn develops paradigms where they are built upon with normal science and new paradigms are created by disproving an assumption and building a new paradigm. The Innovators Dilemma by Clayton Christenson has a similar idea where revolutionary technologies start out as lower quality and build up over time while competitors invest in the best short term innovation (iterative technology).

What sort of mathematical /​logical model do you use for modeling innovation?

I want one, if any remain! DM sent.

Love seeing the experimentation in research models, as well as nanotech projects specifically! I was very excited to see the announcement, and will be looking forward to seeing what you come up with. I’m late to the party, but a couple of questions if you get to them:

I’m guessing the work most of your projects will be doing will (if successful) generate some patent output. Is that correct, and if so do you have a concrete plan for those?

You mentioned experimenting with new ways of managing research. Do you have ideas for management experiments you’re excited to try, or is that primarily in attempting a private ARPA-style project (which is certainly a huge project on its own!)?

P.S. -- Just realized this morning that you’re the same Ben Reinhardt I went to high school with for two trimesters (before I got mono and fell off the face of the earth). Small world—wish I’d gotten to know you better then! Great to see you working on such cool stuff!

Thanks, I appreciate the pushback. Let me push back in turn:

• Water in streams can’t be counted on to be clean, which is why we have water filtration plants. Also it often isn’t found where we want it, and so we have to transport it to our homes, farms, etc.

• Berries have to be collected, and wild sources quickly run out. We also have to identify which berries are safe to eat. And in general all plants have evolved through human selection to be more productive, tastier, etc.

• Air… is pretty usable as is, true.

I suppose there might be a very small number of resources we could consider almost fully natural. Air perhaps. Gravity? But we generally don’t think of these things as “resources” at all.

What are some research areas you think are important but wish someone else would work on?

Besides institutions for science, what other types of organizations do you think the world needs?

When you succeed tremendously with Speculative Technologies what are the most exciting changes a normal person just living their lives will see?

How will we experience the world differently in 2051? What will our homes look like? Our daily experience of work and going places? Our leisure activities? … or anything else that will be enabled by the new materials and technologies that are beyond the cutting edge now.

Or, put differently, why should people far outside the progress and tech communities be excited about this work?

I’m also very curious on what we, as a progress community, can contribute to the telling of more good, inspiring stories. If you have thoughts, please comment or get in touch: heike@therootsofprogress.org.

When I read this I felt a need to shout “YES!!” to all of this. In the past couple of weeks, as I’ve talked to people in the broader progress studies community I heard a lot of personal stories, and many of them included either influential stories (often Science Fiction books, but sometimes also historical tales of human ingenuity and invention or movies) or experiences in spaces that made possibilities real (like Disney Epcot or even museums with inspirational themes).

I’d love to hear from people in this community how these types of experiences played into their journey of being interested in scientific, industrial, technological, and human progress.

Will Speculative Technologies offer internships this summer? If so, will they be limited to purely STEM students or is there space for generalist interns within your exciting organisation?

(Found out about SpecTech from Matt Clifford of ARIA/​Entrepreneur First)

Regardless, best of luck.

Do you think there is anyone you left out in this analysis?

What is the roadmap, as far as we can see it, to the kind of nanotech envisioned in J. Storrs Hall’s books (Where Is My Flying Car? and earlier Nanofuture)? What are the big unsolved problems? What are the most promising approaches or near-term goals?

Is the new industrial policy for semiconductor manufacturing going to result in new materials and manufacturing innovations, or is it mainly about bringing existing know-how to US from Asia? Will it spur any research, or result mainly in execution of known industry methods?

Will the industrial policy jobs be of interest to the types of people you would like to bring into private ARPA, or will it only create non-R&D jobs?

How would you allocate $1-10m personally towards maximising progress? Would a FRO or PARPA be the right choice over say angel investing in startups or funding smaller research projects?

which types of new materials /​ manufacturing technologies will have the most economic impact ? what are you most excited about being built with these new materials /​ technologies ?

Is Speculative Technologies part of the upstream process of creating alternatives to plastics? (alternatives that don’t have the “persistent pollutant” or microplastic/​nanoplastic decomposition problems)

What startup opportunities do you think exist now in the manufacturing sector in the United States?

How much pressure is currently being applied to Congress to break some of the bottlenecks on energy abundance? How much more is needed?

What’s your theory of management going in to Speculative Technologies?

What advances in material science would have to occur for it to be as exciting to investors and average people as software? i.e. is the world of bits going to remain the dominant arena of novel creations for the next century?

I resonate with this. The issue is that culture is particular, but the type of progress the progress studies is generally committed to is civilizational (tech and institutions) not cultural (art and meaning). A community dedicated to progress would instantly become significantly more narrow if it committed to some particular vision of what is valuable and meaningful. While I am fairly committed to a particular vision of how to integrate civilization and culture to create a meaningful life, I wouldn’t want the Progress Forum to commit to a particular view of, say, family values or the status of rituals in society.

Your video is still a broad tent view of meaning. Yet, I’d like to hear how one can actually engage in the project you describe without becoming partisan some particular view of what makes a meaningful life.

A while ago there was a kickstarter project called Origami Bag. They basically have a material that has for most purposes a smooth purpose but at the same time behaves like velcro.

While the product itself is not that interesting the material itself seems interesting. Naively, I would expect that it would find applications in which velcro is currently used like shoes because it seems much nicer.

To me, there’s an open question of why a new material like this didn’t find adoption. The fact that it didn’t seems also to have applications that might be hard for other new materials to find adoption. Do you have an idea here?

How likely is it that commercial products will be manufactured in the next one or two decades in space, making use of the zero-G environment?

Excellent description of how stories play a critical role. I’m interested in whether the same sorts of stories could be updated and played again, or whether it has just become harder to share these kinds of stories. In the UK, in 1951, there was the Festival of Britain, which was similar to other events of the time: showing how the future could be great. It was at the newly built Southbank Centre. Such events require lots of public sector funding and, particularly to hold frequently, bi-partisan commitment. It seems like this is a prerequisite for national-level storytelling? Today, we have global ExPos, but they are every five years and garner little attention (Dubai 2020, Japan 2025). I would be curious to know what people think it takes to tell these stories. Good piece; really enjoyed reading & have shared with my team (who are planning an Expo just like this!)

Is additive manufacturing, or 3D printing using different materials /​ composites, a viable alternative to manufacturing processes today? Or will the cost curve & material integrity never be as good when compared to high volume standardized manufacturing?

Industry 4.0 is a popular marketing slogan for, broadly, the future of manufacturing. What’s your take on the harder science /​ tech underpinning this positioning? Anything you’re particularly excited about?

What is the outlook for graphene? I saw that there is this new process called flash graphene, but don’t know what to make of it.

What are your 3 favorite things that are coming soon in materials?

There are obvious avenues for progress in our society: more income security, less working hours, ideally some degree of population de-concentration. Addionally, relatively poor countries still have a large room for economic improvement, and political and military risks are un un-acceptably large.

On the other hand, your criticism of the “rockets and flying cars” progress ideal looks spot on to me.

Hi Jason, Matt is certainly one the substackers I sometimes most regret not having a sub to!

Thanks Sam, this is a great list.

A few more, although you may have intended some of them to be grouped together with one of the other entries.

• Can we cure mental health illnesses like depression, anxiety disorders, and PTSD?

• Can we create an actually good pain medication that cures chronic and/​or acute pain with few or no side effects?

• Can we build monitoring and imaging technology that allows us to cheaply track every process occurring in our body, enhancing early detection of disease, improving diagnosis accuracy, and informing treatments?