Transcript: How Well Does Patent Screening Work?
Tim Phillips talks to Mark Schankerman
Mark Schankerman
Is the patent office doing a reasonable job? I think the answer must be yes.
Tim Phillips
How does the patent system ensure that the right innovations are protected? My guest today has analyzed how examiners grant and deny protection, so how well are they doing? Today on VoxTalks Economics, ensuring property rights create innovation. Welcome to VoxTalks Economics. I’m Tim Phillips. Gaining a patent can be a long and contested process that’s important because we want to protect only genuine innovations, and when there are those innovations, well, they should get the protection they deserve. Mark Schankerman of the London School of Economics and the CEPR has used artificial intelligence to evaluate how well this system is working in practice. I’ve come to his office, I’m with him now. Mark, welcome back to Vox Talks Economics.
Mark Schankerman
Thank you very much, Tim. It’s a pleasure.
Tim Phillips
Mark, we’re talking about patent screening today. Screening patent applications isn’t something we genuinely hear a lot about in a conversation. Maybe we don’t think about it very much. What does screening mean?
Mark Schankerman
Applicants, that is, firms — typically firms, could be individuals — have inventions, and these inventions they would like to cover with property rights. Now, why would they like to do that? It’s basically because when they have property rights over their inventions, and these property rights last typically for 20 years, these property rights give them the ability to charge other people if they want to use or build on their invention. Example, to license their invention. If they don’t have property rights, they don’t have any legal capability of extracting these returns, so screening really is about whom you want to give property rights to. In a procedural sense, what happens in the patent office is true in all patent offices, by the way, not just the US one. What happens is that the applicant submits his application, and in that application, there’s a set of so-called claims. And these claims are descriptions of the main elements of their property right that they would like covered, and typically you would have two or three main claims, but some have dozens. The examiner in the patent office, who is assigned to do the screening on this particular patent, the examiner searches prior art, which is anything that’s in the public domain prior to this applicant applying. And the key feature here is that this application, this request for an invention to be covered, it must not be an invention that’s too close to things that have been done before. Because if it is, then the examiner is supposed to say no, you can’t have a property right on it, because it’s already been done by somebody else who is in the public domain. And that’s essentially how screening works. Which means that the screening process is really an interaction between the applicant and the examiner who’s assigned, and it takes place over multiple rounds. It’s not just one interaction. The examiner may say, I don’t like this claim, I want you to modify it because it’s too close to something prior. Then you can modify it, and you can come back to the examiner, or you can just abandon it.
Tim Phillips
And the examiners, therefore, have a lot of influence, a lot of power. Who are they?
Mark Schankerman
That’s a good question. Patent examiners are first of all knowledgeable. They often have PhDs in some kind of STEM, some kind of hard sciences, but they’re experts in the science and the technology of the area in which they’re assigned. So they operate in very specific units, which are fairly detailed. So, somebody in IT might be doing communication devices of a particular kind. Of course, for new areas — for example, in the 1990s software, now AI — having knowledge in that is, of course, more difficult because it’s a new area. So, there’s always concern about how up to date examiners are. But they’re skilled people.
Tim Phillips
But no one is ever right 100% of the time. No matter how knowledgeable they are. We might worry about what would happen if patents were either awarded or refused incorrectly. What’s the economic outcome if that mistake happens?
Mark Schankerman
That’s very important. Really a core question. Let me start by answering the following: Whom do you want to give property rights to? Somebody comes with an invention, why would you give them patent rights? Or why would you not give them patent rights? The key economic objective, as opposed to the political or procedural objective in the patent office, is that you want to give property rights to inventions that would not otherwise be developed if you didn’t give them property rights, and which generate positive social welfare. It’s a really important and often misunderstood point. Because if you give property rights to inventions that might be highly valuable — highly valuable — but they would have been developed even if they didn’t get property rights — why? Because they could extract money in other ways — then you’re giving property rights where they’re just redundant. So, it’s often said by legal scholars and others that you want to give property rights to valuable inventions. That’s simply not correct. I like to think of this as kind of innovation-inducing. What happens if they make a mistake in relation to that economic benchmark? Well, there are two types of mistakes they can make. They can grant a property right on some patent claim that they shouldn’t have. If they do that, first you’re giving a property right to somebody who doesn’t deserve it, in the sense of this standard, and that allows them to raise prices. Because they can extract royalties from licensees, which otherwise they couldn’t do. Then, when you raise prices, that has economic costs. Economists jargon, it’s deadweight loss, but it’s basically the social cost of prices being higher than they otherwise need to be. There’s another cost, though, which is very important, and in the US context, dominant, which is when you give property rights on something that they shouldn’t have gotten them on, they’re willing to protect them. And the way they protect them is through litigation. So, maybe they shouldn’t have gotten this property right, but you gave it to them. So, now they can sue others for infringement, or for other things that can be litigation, which otherwise wouldn’t occur if you didn’t give the property rights, because they shouldn’t have gotten them. That’s the other component. So, deadweight loss or higher prices are necessary, and litigation costs that are unnecessary. We call that type one error. In the US context, it turns out that litigation is the dominant factor, because it’s so costly. That might be different in Europe, where it’s less costly, though still not cheap. The other kind of mistake that an examiner can make is not giving a property right where he should have. That’s called by economists type two error. In the US context, it’s important to understand that if I don’t give you — you’re the applicant — I don’t give you the grant on the claim that you’re asking for, or you can abandon, you can just say I withdraw.
Mark Schankerman
If you withdraw an application, it might be that there were some claims in that application that were valid. So, you can end up with a situation, in a situation, where the applicant does not get property rights on something that he should have done. And the social cost there is reduced incentives for innovation. There’s very little discussion — by business, yes — but very little discussion in the public debate about type two error, because it’s unseen. I can see stuff that I say, well, that should not have been granted a patent. But you don’t see things that should have been. That means that type two error is often underrated, and in this work we’re trying to measure the frequency of these types of errors and also the social costs, and that’s important precisely because it’s hidden.
Tim Phillips
There is anecdotal evidence that mistakes have been made and costly mistakes have been made?
Mark Schankerman
Let me give you two examples. Now the most egregious, I think, that I’ve ever seen was the patent on the swing. There’s a diagram in the patent that is a piece of wood with two ropes on it. Now, I don’t know about you, but I used to play on swings when I was a youngster. That was long before this application, but it was granted. Now, the question is: How common is that? And the answer is, before this work, this research we’re talking about, we didn’t know.
Tim Phillips
We just didn’t know.
Tim Phillips
Everyone would have an idea, no one really knew.
Mark Schankerman
Yes, and everybody would have their anecdote, and I’m not denying the truth of those anecdotes, but the question is: How frequent and how costly? What I’m trying to do in this work, and in the work that I hope follows this, is to provide methodologies for an evidentiary base for policymaking in this area. Because otherwise it’s just anecdotes, or even worse, ideology that drives policy. And so, that’s what I’m trying to do ultimately. Now, the other example I wanted to give you is one that’s more problematic, but makes us think, which is Amazon one click shopping.
Tim Phillips
Right, yes.
Mark Schankerman
Amazon got a patent on it. Clearly highly valuable, hugely valuable. The question is: Would they have done this invention had they not gotten patent rights on it? Some say yes, because it’s not costly to do it. You have an idea and implementing it is very cheap, or relatively cheap, so that they would have done. In which case we shouldn’t have given them, from an economic point of view, patent rights. One example, which is obviously not valuable, the other highly valuable, but it doesn’t necessarily mean that we should have given them property rights.
Tim Phillips
We’re in Europe. In Europe now, there is a unified patent court, isn’t there, since 2023. Does this improve the examination process, do you think? Also, does it raise the stakes? Does it make it more important that this examination process works well in Europe?
Mark Schankerman
Does it improve screening? The answer is no. The Unified Patent Court does not do any screening examinations.
Tim Phillips
Okay.
Mark Schankerman
Its remit is to adjudicate enforcement issues. It probably will help in the enforcement, because it’s going to have a unified enforcement, which means that you don’t have to try to enforce in each country, which is going to lower the cost of enforcement. That fact makes screening even more important, because if we have a type one error — that is, if we grant what we shouldn’t do — and it’s cheaper to enforce it, it’s more likely to be enforced. So I think it’s absolutely correct that the importance of screening is not undermined, it’s reinforced by the establishment of the Unified Patent Court.
Intermission
We last spoke to Mark in 2021. We were talking about how patent pools could speed up the diffusion of new drugs in low-income countries, and how we could do a much better job of licensing these life-saving generic medicines. Follow VoxTalks Economics wherever you get your podcasts, and then you can find the episode called ‘Patent Pools for Generic Drugs’.
Tim Phillips
Ok Mark, let’s talk about your research, because nobody really knows exactly how many errors there are. You are trying to close that knowledge gap in your work, so you set out to model how the patent application system really works. How can you do that rigorously?
Mark Schankerman
It’s a very good question, Tim. The short answer is with great difficulty.
Tim Phillips
Yes.
Mark Schankerman
We’ve been working on this paper, my co-author and I — Will Matcham — for years. Literally for about seven or eight years. The key to doing this in the patent office is to understand what the institution actually looks like, how it actually functions. You have to know the institutional detail, which we spend a lot of time learning about. In the case of US Patent Office, you then have to abstract and to try to focus on the core elements. Then you try to model — as much as you can in any model — the core elements of the actual procedure that’s followed and the actual incentives of the parties involved in that procedure. In our case negotiation. So the actual incentives of the applicant. What are his incentives? Does he have to pay fees? What are his actions that could improve his position? And so on. And for the examiner, what kinds of incentives does he get, or constraints from the patent office? Maybe he’s intrinsically motivated, he just cares about doing the right thing. You have to try to capture all of that. You want to build what economists call a structural model, because what you want to do is to model what actually is going on as best one can. Then estimate such a model using data. And then do counterfactual experiments to ask, given these estimates that we have, what would happen? Assuming those estimates don’t change, which is the idea behind a structural model, but when the regime changes, for example, you change the fees that the applicants face, or you change some constraints on the process — what would happen?
Tim Phillips
So it’s already becoming obvious exactly how difficult this is. Data doesn’t have a easily measurable form. One of the basic things you have to do is to work out if there is an application, how different that is to other patents that have been granted. A sort of a distance between the application and existing patents. How do you measure that distance?
Mark Schankerman
Yeah, that’s the key. It’s a key to this project. It’s not the only key, but it’s one of the keys. The Patent Office, until very recently, does not think explicitly in terms of the distance measure we’re using. However, they do think explicitly about how close is it to prior art, and that’s part of the mandate of the examiner.
Tim Phillips
Yes.
Mark Schankerman
Our approach to measuring distance, and measuring whether you’re too close, is our construction about what, effectively, we believe is going on in the patent office. All the major patent offices — the UK, the USPTO, the EPO — are all actually starting to use algorithms to help examiners do this. So, I do believe that we’re on on the right track in modeling it in this way. There’s an algorithm that we train on other patent documents that learns — if I can put it that way — how to interpret words in a given context. But, the basic idea is that you compare the similarity of the words in the paragraph context in a claim of that’s being submitted to all previous claims, whether they’re in granted patents previously, or patents that weren’t granted.
Tim Phillips
Importantly, yes.
Mark Schankerman
All previous claims. Then these algorithms transform the information into some kind of distance measure. A quantitative measure. So now you have a distance measure to all previous claims. Claims not patents, claims within patents, we’re talking about millions. Then you look for the closest previous claim. Then the question is, if that claim is too close, I should be rejected. Basically, we use this distance measure to compare to a threshold. Imagine that one of the examiners was completely intrinsically motivated, by which I mean they want to do the right thing, no matter what.
Tim Phillips
Yeah.
Mark Schankerman
And the right thing for them is to do what they understand is the patent office’s mandate. For someone like that, they would never knowingly grant a claim which is below the threshold, because they would feel so bad about it.
Tim Phillips
Yes.
Mark Schankerman
They wouldn’t do it. We’re essentially measuring the threshold as the lowest distance that they grant a claim on, for that kind of guy. So that’s the intuition. And so it’s not just an arbitrary thing, it has some foundation. And then the question is: Are these claims too close? And if they are, they should be rejected. And if they’re not rejected, it’s a type one error. And if they’re above this threshold, so they’re not too close, they should be granted. But if they’re abandoned, then you don’t get that. So that’s where the type one and type two errors are measured exactly by this distance measure.
Tim Phillips
So we’re in this wonderful position now, where you have this algorithm that can spit out these errors or non-errors. How big a data set did you test this against?
Mark Schankerman
The data set that we used in this paper covers all of the patent applications, and all the claims in them, for the cohorts of the years 2011, 2012, and 2013. That’s about 1 million. About 300,000 per year, roughly. That covers 20 million claims.
Tim Phillips
Wow.
Mark Schankerman
We have decisions by the examiner on each of those claims at each round of negotiation.
Tim Phillips
Right, because, well, a lot of the time the examiners will go back and say modify.
Mark Schankerman
Exactly.
Tim Phillips
Not now, but modify this, and maybe yes.
Mark Schankerman
Right, and something like two to two and a half rounds, each round’s about a year.
Tim Phillips
Wow.
Mark Schankerman
Some last much longer. So it covers 55 million decisions by the examiner. And the claims that our distance measure compares them to previous goes back to 1976 and that covers about 100 million. The computational requirements of this obviously are huge.
Tim Phillips
Give me some idea of the results that you’re getting now. So, for the first applications that people are making, how many of those applications are are too close?
Mark Schankerman
The answer to that is about 81% of claims with independent claims, the main claims, when they’re applied for, are below the threshold.
Tim Phillips
Wow.
Mark Schankerman
81%. Now, you might say, yeah, I mean, that would be a worry, right? If you gave them all.
Tim Phillips
If you gave all of those.
Mark Schankerman
Exactly, then you’d have a type one error of 81%. Now it turns out, however, that among all granted patents —so this is after it goes through this period of negotiations where the claims are either rejected, or rejected and then narrowed, you know, you’re claiming too much, and then you, you reduce that until it’s acceptable to the examiner — about 13% of granted patents have at least one claim below the threshold. So 13% of granted patents have at least one claim that should not have been granted. However, among all claims — that’s called the extensive margins — but among all claims, only about 6% of all claims granted are below the threshold. Okay, 13% have one or more than one, but only 6% overall. So, when you compare 6% to 81%. 81% came in with claims below, and only about 6% of claims afterwards. That’s part of the answer. The question is, the patent office doing a reasonable job? And I think the answer must be yes.
Tim Phillips
But tell me something about this 81% figure. What’s going on here? Is this ignorance of other patents that have been granted, or are the applicants gaming the system a little bit?
Mark Schankerman
So, the answer to that, Tim, is that I don’t know. We can’t know. We certainly know from talking with legal scholars and with legal practitioners, and also from admittedly reading the blogs, that first of all, law firms get paid a lot for this.
Tim Phillips
Yeah.
Mark Schankerman
Okay, they get paid for something, and part of that is strategic behavior, casting your patent application in such a way, a) that it’s most likely to be granted, and b) that it has as much property right scope as possible.
Tim Phillips
Yeah, you don’t want to leave anything on the table.
Mark Schankerman
Exactly. But then you might say, well, why don’t you just ask for everything, and the answer to that is, when you ask for too much — when you push your closest distance below the threshold — you make it more likely that the examiner is going to say no. Then you can revise it, but that’s going to involve costs and time and all the rest. So, in order to get an expeditious grant, you don’t want to do that too much. And its that trade off between I want more property rights, but I want to reduce the risk of not getting it.
Tim Phillips
Uh huh.
Mark Schankerman
That’s the trade off that’s mediated by each applicant, and so there’s undoubtedly strategic behavior. Now, could there also be ignorance of past claims that they’re unaware of? It’s possible, but I would point out that when you apply for a patent, you have to list the relevant prior art. The key thing is you want to expand your property rights, subject to this constraint, that you can get it by the examiner, and expanding those property rights is what we call in the paper ‘padding’. Whenever you write an article, you always have a few things in there that you know, a scientific article that maybe is not completely original, you know, it’s making it look a little bigger than it should do. Okay, same thing with padding. If you can get it by the examiner, you have more scope on your property right, but you risk rejection. Padding in our model is an outcome which we actually can measure, how much padding is actually going on, and we definitely find that it is going on. So, there is some degree of strategic behavior.
Tim Phillips
Look at it from the other side then Mark. When applications are refused, can we estimate whether there are many of those that should have been granted?
Mark Schankerman
We can do, because remember, you have a distance for claim — closest distance to prior art — and you have a threshold. And you know whether the claim was granted at the end of the day or not granted. If you go through three rounds — and it’s narrowed by our estimates around 35% per round — what you’re ending up getting granted is much less than you originally asked for. But the answer is, we can measure those that were above the threshold that were in fact abandoned. 32%, so about a third of all abandoned applications have at least one valid claim.
Tim Phillips
Wow.
Mark Schankerman
And 16% of all abandoned claims are valid. So, type two error is actually more frequent than type one error.
Tim Phillips
The error that you were saying is not discussed as much.
Mark Schankerman
Exactly, exactly. So, one of the things we hope this will do is to raise awareness of this. Now, the fact that type two error is more common than the type one error doesn’t necessarily mean it’s more serious. Because that’s about the social costs imposed by each, which we also try to measure. But the frequency is quite striking.
Tim Phillips
So Mark, it’s incredibly interesting to finally have some idea about the scale of what might be a problem or might be a system working well, which previously have been discussed anecdotally. So we probably need to think about how we could make the system work better. First of all, this is a huge system. It’s a very complex system, involves a lot of people and a lot of knowledge. Is it a very expensive, difficult system to run?
Mark Schankerman
We estimate that the social cost of screening is about 15.5 /, 15.4 billion US dollars.
Tim Phillips
Right.
Mark Schankerman
So, about 12 billion pounds per year, so per cohort application, that’s on the order of 5% to 6% of all private sector performed R&D. So, is that big or small? It’s not insignificant.
Tim Phillips
No.
Mark Schankerman
This is the social cost of screening, not the social costs of the whole system. But how much does it actually cost to do this? Now, what’s striking about this, and I have to admit that I didn’t expect this. About 25% — because we actually quantify this — about 25% of these social costs of screening come from type one and type two errors. So, where is the other 75% coming from? It’s coming from the prosecution costs — by which I mean the cost of the patent office, actually the examiners paid their salaries and all the rest of it — and most importantly, from the legal fees and litigation costs in the US context. The vast bulk of that is not the patent office cost, it’s just the legal cost of going through this system. Okay, because it costs a lot to do this. In another setting, like Europe, where the legal costs might be somewhat lower — especially in continental Europe — this might change. But that’s the case in the United States. Litigation costs really are a dominant social cost of screening. Now, as far as improving the system. Even though my own take on this is that the patent office is not doing a bad job, even though they’re making now quantifiable errors, and maybe their use of AI algorithms will improve that. Who knows? Nonetheless, in this paper we actually show, because remember, structural models allow you to do counterfactual thought experiments. We look at things like increasing the fees that applicants have to pay.
Tim Phillips
Right, yes.
Mark Schankerman
So that you don’t get just a lot of junk coming through. We look at various other reforms. The one that really matters most is restricting the number of rounds of negotiation that an applicant can go through. In the US patent context, there’s no limit.
Tim Phillips
There’s no limit?
Mark Schankerman
There’s no limit to the number of rounds you can go through as an applicant. It’s just that after two rounds you have to pay what’s called a ‘Request for Continued Examination’. An RCE. So you have to pay fees, but they’re not real high fees.
Tim Phillips
No?
Mark Schankerman
$10,000 or something. You can go on forever, and yet that’s the system. The vast majority are done after six rounds. That’s a long time. The mean number is about two and a half, but there’s no limit. You can keep coming back for another bite at the apple, but you just have to narrow the apple, as it were. This is not the case in other patent offices. In the European patent office, for example, you don’t have the right to do this. In China, you don’t have the right to do this. So, this raises this really important point, which is that saying that you can’t do more than two rounds, or in our case, we actually look at one round, really substantially reduces the social cost of screening. And it makes applicants more aware of not coming in with junk. If you come in with a lot of padding, you may not get to where you need to be to get acceptance after two rounds, and then it’s killed. So, it induces a less strategic and more reasonable behavior on the part of applicants. And it really does reduce the social cost of screening, not least because fewer rounds means fewer legal costs, which is the major component. Now we don’t know whether in other institutions that have other design features like the European Patent Office, whether those same reforms would help there. In order to do that, you really need to study these other institutions. But the overall objective here is to be able to say something quantitatively about what the performance is in these different institutions, how the design features differ, and whether those design features matter.
Tim Phillips
Now, you can say something quantitatively about the performance of these institutions and the examiners in them. Many of your colleagues down this corridor might say, well, let’s pay for performance, let’s incentivize them in that way. Would that work?
Mark Schankerman
That’s a big question, and a very controversial one.
Mark Schankerman
I once proposed an RCT, a randomized control trial, in the patent office, US Patent Office, at quite high levels to test whether incentives of various kinds would work. How should I put this in a polite way? The unions didn’t like the idea. So it was shot down, and this was presented to the then head of the patent office and his colleagues. In any event, the first thing to say is that examiners do have incentives. They have both extrinsic or monetary incentives, quasi-monetary and potentially intrinsic motivation, just doing the right thing.
Tim Phillips
Yeah.
Mark Schankerman
We actually measure the magnitude of their intrinsic motivation. This is important, by the way, because in many public institutions people think that people who work in public institutions have intrinsic motivation, otherwise they make more money going out, in our case, working at private patent offices. That may be true, but we actually can measure. And what we find is that there is a lot of intrinsic motivation among examiners. But they also have credits. They get credits for each decision they make, depending on when they make it, and various other factors, and these credits accumulate. Their bonuses are based on how much you exceed the expected number of chits. So, these credit structures, or extrinsic incentives, if you want — they’re not monetary exactly, but they translate into potential bonuses — give certain kinds of incentives. And in fact, they give incentives to try to do it early, and they give these incentives because the patent office wants it not to take too long. So, there are incentives. Now, the question is, would things be better or worse if we intensified or got rid of these incentives? We do some of that in the counterfactual analysis. Basically, we get rid of the credit incentives. If you get rid of the intrinsic motivation.
Tim Phillips
You’re in trouble.
Mark Schankerman
Yeah, they go pear-shaped. So, intrinsic motivation actually is really important in this institution. When you get rid of the credits or the monetary incentives, they don’t seem to matter. Surprisingly to me, that doesn’t seem to matter. But if you get rid of intrinsic motivation first, and then get rid of credits, getting rid of them makes things worse. To say that in another way, when you have very strong intrinsic motivation, which is what we’re estimating, these extrinsic incentives don’t seem to be powerful. But if you don’t have intrinsic motivation, then these things do seem to matter. But I think that’s as far as I would push that point.
Tim Phillips
So, Mark, we’ve reached this first stage, this base camp, after eight years of hard work. There is huge potential here. What’s next with the work?
Mark Schankerman
I actually have a recent — last year— a European Research Council advanced research grant for five years. Really to pursue some of these issues in this direction. Two major challenges I think going forward that I’m particularly interested in. I would like to see this kind of analysis, whether done by me or others, on other leading patent offices. And the leading ones would be EPO, China, Japan, South Korea, those would be the big ones. And some of them have different features than the USPTO. And doing this kind of analysis, on one or more of those, would allow us potentially to say something about whether these different features are there’s a best practice or maybe not. So this quantitative evaluation of institutional design, that’s really the way I think about it, is the overarching thing. And so that’s the first thing I’m hoping. Now within this ERC grant, my co-author on this paper and collaborator with that — William Matcham at Royal Holloway — we’re going to be looking at the European patent office. The other big thing, which I think is very important, is to understand that our evaluation is about the performance of USPTO, given the patent standards that it is mandated to enforce. And these patent standards come in the first instance from the US Code — which is US legal code on their various provisions there — but most importantly, the evolution of that code through the judicial decisions, including the Supreme Court. They’re doing a pretty good job, it appears, given the standards. But the big remaining question is: What about the standards? Maybe the whole standard that they’re imposing is too strict or too lax. Maybe this threshold that we’re estimating — that they’re using — is too low or too high. And if it is, they might be doing a very good job at implementing the standard that they’re mandated to do, but the standard is wrong. We still have a bad outcome. So, not to pour water on, on the excitement around this paper, but there is this other big challenge. Which is understanding whether the standard itself is set too high or too low, because if we can do that in any setting, then those two pieces together will tell the whole story that we need.
Tim Phillips
It’s a fascinating story, and good luck with the telling of it, and I’m sure we’ll talk about it again. Mark, thank you.
Mark Schankerman
Thank you very much, Tim. Been a pleasure.
Tim Phillips
If you want to read this paper, it is called ‘Screening Property Rights for Innovation’. At the moment, a discussion paper, 18334 at CEPR, authors William Matcham and Mark Shankerman. Mark, you also have a publication for this, don’t you?
Mark Schankerman
Yes, it’s now forthcoming in Econometrica.
Tim Phillips
Congratulations.
Mark Schankerman
Thank you very much indeed.
Outro
VoxTalks Economics is a Talk Normal Production. The assistant producer is Megan Bieber, and our editor is Andrei Zagarion. Next week on VoxTalks Economics, Digital Money and Money Creation.







