AMA: Matt Clancy, Open Philanthropy
Hi everyone, I’m Matt Clancy (website, twitter), an economist specializing in science and innovation. I’m currently a Research Fellow at Open Philanthropy, where I work on metascience issues. In concrete terms, I split my time between making grants, doing research, and writing New Things Under the Sun. The last thing is a living literature review on what academia knows about innovation that I’ve been writing professionally part time since late 2020. Prior to that I worked at the Institute for Progress as senior innovation economist, and before that I was an assistant teaching professor of economics at Iowa State University.
Ask me anything! I will be answering questions from Monday, February 13 until Wednesday, February 15. Use the comments below to add questions, and upvote any questions you’d like to see me answer.
- The Roots of Progress 2023 in review by (LessWrong; 31 Dec 2023 18:16 UTC; 22 points)
- Matt Clancy AMA on the Progress Forum by (LessWrong; 12 Feb 2023 20:23 UTC; 17 points)
- Progress links and tweets, 2023-02-15 by (LessWrong; 16 Feb 2023 0:04 UTC; 10 points)
- Jason’s links and tweets, 2023-02-15 by (16 Feb 2023 0:04 UTC; 4 points)
In the economics of innovation, what are the big questions where there now is a relatively settled consensus? And what are the big open questions that the field is currently debating?
We have some big picture stuff pretty nailed down. To start, long-run improvements in material living standards have long been understood to arise primarily from improvements to technology, which emerge from R&D. It’s also pretty settled that much of the value of R&D spills over to people who are not the performer of R&D, and this means that in a laissez-faire system, R&D will be supplied at less than desirable levels. This problem is especially acute for fundamental science, which is primarily aimed at understanding how the world works. Most economists think the government is going to have to play a role in supporting, for example, fundamental scientific research.
Given that a laissez-faire system is going to undersupply R&D, what’s the best set of policy interventions? Here, there’s a lot more debate and uncertainty. For example, in this 2019 paper by Bloom, Van Reenen, and Williams, they try to describe some of the main policy tools available to influence innovation. In a table at the end of the paper, they evaluate the quality and conclusiveness of the evidence for different policies.
Settled issues, with high quality evidence:
Skilled migration is good
R&D tax credits work
Stuff we’re less sure about, either because evidence conflicts a bit, or we lack very good evidence:
Direct R&D grants
Patent boxes
Impact of universities on STEM supply
Stuff that we’re very unsure about:
Impact of providing incentives to universities
Effects of intellectual property rights
Mission oriented policies (like DARPA)
If you were given a million dollar grant, which question in innovation policy would you want to answer and what might that dataset look like?
I will cheat and list two ideas, plus one irrelevant one.
First, let’s start with some large-scale descriptive statistics of practices!
How do different labs organize and manage their staff?
How do different editors manage their journal and peer review process?
How do different grant reviewers pick grants?
The goal is to see if there are obvious best practices; what kinds of stuff is correlated with good outcomes? This would be some kind of large-scale survey.
Second, I’ve always wanted to know more about the political economy of R&D. How do governments decide how much to spend on R&D? I’m not sure the best way to study this though.
Third, my dream research project, after we have sorted out the more important stuff, is to use the decadal Sight and Sound Greatest Movies of All Time poll to study how people’s perception of artistic greatness changes over time. Do people change their minds? Or is it all about new (younger) critics embracing newer films?
What are the primary characteristics you think an effective “culture of progress” would have?
Do you see any concrete, meaningful steps being taken to get to such a culture? Any opportunities in that direction that you see that you wish were being worked toward right now but aren’t?
Among other things it’s important that people think that of themselves as having the agency to solve big problems; that innovation and entrepreneurship—basically just pioneering new paths—is one of the things that it’s normal to do with your life. I have written a bit about that here and here.
You can try to do that at scale by trying to impact pop culture, and I don’t doubt that works to some degree. But I think the challenge is you are trying to convince someone that people like you can innovate too; and there is an inherent distance between what’s in pop culture and yourself for most people, simply because most people are not the subject of movies, TV shows, profiles, etc. More effective if it’s your brother or classmate or coworker who models the innovative career path, since it’s a small leap to think “if they can do it, why not me?”
What do you think about the question of “ideas getting harder to find”? What do you think of the discussion of this topic in the progress community—is there something people are misunderstanding or getting wrong about the issue?
I think three different things might be true:
Scientific and technological progress is getting harder
But the pace of progress is steady because we try harder
Nostalgia bias, the esoteric nature of modern advances, and general human crankiness makes people falsely believe the rate of progress is slowing more than it actually is
I’m pretty confident about #1 and #3, less sure about #2. We certainly are trying harder, but maybe not enough to keep the overall rate of progress steady. I go back and forth on this, but at least think the case for a slowing rate of progress is not nearly as strong as the case for #1.
Why is #1 true? I think it’s a combination of things. To a large degree, this just seems to be an inevitable feature of advancing knowledge; the burden of knowledge gets heavy, and maybe we also pluck some of the low hanging fruit. But a non-negligible part of the slowdown is probably due to the way we organize and conduct R&D. Improving that would have really high benefits, even if we’re probably not going to go back to the era when a handful of gentleman scientists could make giant advances.
What evidence-based advice would you give to a corporate CEO to increase innovation in her company?
Are there specific management practices, elements of corporate (not national) culture, or management behaviors that promote innovation? Can we apply insights from research on funding organizations (DARPA, NIH) to for-profit organizations? Are there robust principles (e.g., tolerate short-term failure, reward long-term success) that should apply in a corporate setting?
What are your favorite journal articles, working papers, or books on corporate innovation? Who is doing good research on this topic?
Probably the most concrete topic I’ve written the most about is the geographic distribution of an R&D workforce. I think one of my main takeaways is to not assume that everyone working together in the same building is as good as it gets, in terms of innovation, though it depends on the industry. Reasons to consider a more distributed workforce:
You can hire people who are a better fit for a specific role if you are set up for a remote workforce
A distributed workforce is exposed to a greater diversity of information environments
Modern communication technology makes it feasible to collaborate effectively at a distance, so that in person communication is no longer so crucial
That said, a distributed workforce offers its own challenges. You have to be more intentional about facilitating random meetings among different parts of the org for one, or you risk excessive siloing. And I think occasional in-person meetups are also important.
If you think nothing matters as much as getting the right people, then this is all the more important!
Some of my favorite papers on corporate innovation is the works of Ashish Arora and his colleagues, who are actively working on this stuff.
What has the experience of New Things Under the Sun becoming so popular been like for you? For instance, has it changed how you write or what you decide to write about? And what do you think is hard about writing NTUS that might not be widely appreciated?
It’s been great that people I respect like the project, but the most important thing about it becoming popular is that I now get paid to do it. People sometimes say if you get paid to do something you previously did for fun, it kind of ruins it for you because it becomes work. That hasn’t been my experience, probably because (1) I don’t stress about view counts or anything anymore, and (2) I work on it a few hours each M-Th, but it’s not my entire professional life (much less my whole identity).
The main thing that has changed as the project has become bigger is that it’s more ambitious than it was at the outset. Mostly you can see that in the length of the articles, but also a bit in the work I put into the site infrastructure. I still write what I think is interesting, though other stuff I’m working on at the Institute for Progress and now at Open Philanthropy has some influence on what I think is interesting.
Probably the hardest part is knowing how much cool stuff is out there that I don’t have time to get to!
What role do government and politics play in supporting scientific innovation (coming from academia)?
There are good/interesting arguments on both sides of “governments have nothing to do with progress in science vs look at all the useful things we’ve got from the NASA space program”, etc. Where do you fall?
The biggest single thing is the government simply pays for a very large chunk of science! In 2019, the federal government paid for 41% of US scientific research.[1] That was more than the private sector (33%), university system (13%), or the philanthropic sector (10%). It’s true that if the US government stepped back these other sectors would probably step forward to shoulder some of the burden, but I doubt they would cover the majority of the short-fall.
More broadly, I think science and innovation is pretty hard to predict. That means we ideally want an innovation ecosystem that explores and tries lots of different approaches, even if some of those approaches don’t seem to hold the highest promise at there outset. One way you can get that is to have a private sector that is open to new entrants and startups who want to try different stuff than the incumbents. But those new firms are still going to be ultimately chasing the same signal as everyone else, namely profit, which might limit the extent to which this ecosystem explores the potential space of technological possibility.
So I think it’s useful to have some innovating organizations who are pursuing goals entirely untethered from market success. The government is one viable solution to this problem of getting a bit more exploration into the innovation ecosystem. I haven’t seen a study that tries to rigorously quantify the value of, e.g., NASA or DARPA, in terms of their spillovers to the rest of the economy, but I would be surprised if they don’t come out looking like good investments.
From table RD-3 of the NSF science and engineering report, taking basic research as a proxy for science.
Anton Howes has been writing about an innovation mentality as critical in brining about the Industrial Revolution. Joel Mokyr discussed the importance of culture—the beliefs, values and preferences that influence behavior—as critical.
How do you think about our culture in the US today as it relates to innovation? Where are we strong, and where are there opportunities for improvement? I’m especially interested in how this may be changing: can we see differences in cultural attitudes across US generations? E.g., are younger people more innovation oriented now than they were decades ago—or less? And especially if less—what do you think is the best way of fostering an innovation culture especially among young people? (Or, put differently, what are the key obstacles you see?)
Where are we strong?
Across the sweep of history, the contemporary USA has got to be in the top 5% for it’s cultural support for innovation. But I think that’s mostly because the default state through human history has been so bad, rather than that we are so good![1] But it could be a lot worse! Elon Musk was person of the year in 2021!
It’s true that a lot of people are down on innovation, but I think to some degree that has to be an inevitable part of the kind of free society you want where lots of different perspectives (itself important for innovation!) are welcome and collide. A world with universal acclaim for innovation and progress would itself be kind of stifling. But that’s not to say we have the balance right already.
Where are there opportunities for improvement?
In my experience, different regions of the USA differ a lot in terms of what you ambitious people think they should do with their energies, and indeed how ambitious they should be. In some places, the ambitious thing to do might be to go to an Ivy League school, and then to be funneled into finance or medicine or something. In others, it’s to found a startup. In my own home state, neither of these was particularly emphasized. I think it would be great if people had a bit more exposure to what ambition means in different places, to broaden their own views about what a good life means. Not everyone would opt to be an innovator, but I think at present a lot of people who should probably are not because they simply don’t consider it much.
Joe Henrich’s book The Secret of our Success has some really interesting examples of traditional societies where if you tried to be innovative and more efficient than your peers, you would end up subtly killing yourself. In a world where humanity’s causal knowledge about how the world works is weak, innovation probably is dangerous to the individual, and so society’s rationally encouraged doing things the traditional way.
I hadn’t thought about the regional aspects, but it makes total sense, and it reminds me of this post by Paul Graham that talks about cities and ambitions: http://www.paulgraham.com/cities.html
What learnings can we draw from your work on innovation on bringing together thinkers in progress studies? When do you think we should think about in-person gatherings and events, and when is it fine to work at a distance/via Zoom/here in the forum? As we think about events and bringing people together to learn and exchange ideas, any insights from your research on how to best structure those types of events?
In this context, Eric Gilliam had an interest post on his substack on conferences and Feynman’s take on their declining usefulness. Curious to hear if you have any thoughts based on your research as well on organizing gatherings that lead to more innovation.
Eric’s post: https://freaktakes.substack.com/p/feynman-on-journal-reviews-conferences
My basic view is that modern communication technology makes it pretty easy to communicate and exchange ideas with people, as long as you already have a relationship with those people. It’s less well suited for forming relationships because it’s not so good at helping you discover people outside what you think you’re interested in (because we tend to go to websites catering to our interests), and because it’s a bit harder to build deep trust without face to face meetings.
In-person events based around groups that have common interests but don’t already know each other can be really useful for forging deeper relationships, and the internet means those relationships can remain productive at a distance. They can also be useful for meeting new people, if these meetings are packed with people you don’t already know and the meetings force you to interact with new people (for example, by making you queue for a buffet, find a table with an opening, sit next to someone on a bus to a second location, circulate around and look at posters, etc.).
Maybe big conferences aren’t that good for this kind of thing, as Feynman and Eric speculate? At a big conference, you’re more likely to know more people and so you might end up just hanging out with them and not meeting new people. And perhaps, knowing you unlikely to run into any given person a second time, it just doesn’t feel as important to introduce yourself and get to know them? Or maybe it just takes two or three minor encounters to really build a relationship for many people.
For more on academic conferences, I wrote something here.
Thanks, Matt, for the thoughtful response. My key take-aways are that (1) in person events are helpful to get new relationships going because of trust & discovery, and (2) we should keep these meeting small (or create smaller sessions in larger meetings) and (3) purposefully get people together who otherwise might not talk, and ideally have them not just be passive but work on something together in those sessions. Some good initial thoughts in any case; thank you!
Any initiative that presents evidence to lobby for change is confronted with the question of how exactly that evidence may feed into the political or bureaucratic process that can bring about said change. To what extent, do you think, should the Institute of Progress concern itself with the study of this evidence-to-policy pipeline? How interesting is this topic to you in general, and which other players are well positioned to contribute to research on the topic?
Yeah, this is a hugely important topic that is tough to appreciate from outside. I’m fond of this quote by Duncan Watts:
”For 20 years I thought my job was, as a basic scientist, publish papers and throw them over the wall for someone else to apply. I now realise that there’s no one on the other side of the wall. Just a huge pile of papers that we’ve all thrown over.”
The conventional wisdom is that, at least in the USA, you need a think tank apparatus that specializes in digesting academic literature and packaging it for policy-making. That’s one of the roles that the Institute for Progress plays on the metascience front. Also, this general topic is something I hope to look into more as a New Things Under the Sun post.
Excited that you’ll do some work on this, Matt—looking forward to reading it.
What does the economics of innovation have to say about a late adopter/slow adopter industry like Construction, where productivity has been flat at best for four decades, and not just in the USA? If Construction is somewhere around 6% of GNP, you’d think the issue would get more attention.
A really good research agenda needs to have two features: it should be an important problem, and you need an angle of attack on it. In the case of the economics of innovation, I think that biases the field towards studying sectors where there is readily available data, whether those are patents, granular data on productivity, approved drugs, or whatever. A lot of low-growth sectors—education, construction, government—are maybe areas where high quality data isn’t as readily available. Or in some cases, innovation-relevant data might actually exist, but a critical mass of scholars don’t understand it, which makes it hard to study and get good feedback on your work.
That said, there is some work on this. Personally, I’m excited to look into this handbook that examines the pace of entrepreneurship and innovation across a bunch of different sectors. Also check out Brian Potter on the construction industry specifically!
thx—I’m aware of Brian’s work, will check out the handbook.
How should one create institutions which drive innovation activities and create technologies improving welfare ? Is Venture Capital model of investment a good model to support innovation?
Different kinds of institutions are good at different kinds of things, and you ideally want a portfolio of different institutions.
Academic research is good at promoting non-commercial exploratory research that hews close enough to the community of experts to take advantage of critical feedback and allow for people to build on top of each other’s ideas (classic article here)
Venture capital is good at innovation that is very valuable if it works, very uncertain if it will work, but where it is possible to learn relatively easily if an idea will work. (classic article here)
Incumbent firms are probably pretty good at incremental improvement of their technologies, provided they face a bit of competition (citation needed!)
Large corporate science labs are good at projects that require a lot of resources, a longer time horizon, and a focus on integrating disparate results into a coherent package (classic article here)
There are probably other institutional arrangements better suited to different kinds of innovation too; most of these institutional arrangements are not that old, as institutions go! I think, for example, that there are probably ways to improve the organization of academic research. And the private sector has to some extent retreated from basic science, and we may need new institutions to fulfill the kind of role they used to do.
Actually, this essay on the topic is great and actionable: support Ben’s Speculative Technologies project!
What are the implications of your work for competition policy and big tech regulation?
Afraid the implications are limited for the time being. Competition policy and innovation is a big and active research area, and one that I haven’t dug into yet. It’s too big a topic for one article.
What areas of economics do you think the economics of science and innovation might want to draw upon more? More theory? More behavioral economics?
I don’t think we necessarily need more economic theory related to innovation, unless it’s informed by new data and facts (which it is sometimes!). I just feel like we had that in spades for decades.
The biggest blindspot for economists of innovation that I worry about is that, actually, incentives just don’t matter as much as we think. Engineering innovation may be much more a problem of finding the right people (not just right skills, but have a creative personality open to novel solutions), bringing the right people together, and so on. I’m not sure the right field to learn from on these topics though; still it’s something I think a lot about!