@LukasDay I came here to post that, I but from the perspective of betting “NO“. this use case seems about as slam dunk as you could ask for, which makes the efffect size disappointingly small. Here is another angle from the paper:

An agent using the AI tool who had just two months’ tenure at the firm performed as well as an agent with six months’ tenure working without the tool.

It seems to be functioning as a very good auto-complete, not galaxy brain:

agents, who could participate in multiple chats at once, spent about 9 percent less time per chat, handled about 14 percent more chats per hour, and successfully resolved about 1.3 percent more chats overall.

So, maybe if there was instant, seamless, roll out across every low skill knowledge job we would see a bit of a bump, but as we think about the entire complex economy (and the fact that the system is already capacity constrained!) I just don’t see it happening

@LukeHanks If no one has posted any evidence here then I'll see if there's a way to examine it systematically, using the NAICS definitions posted below.

@Nostradamnedus can you please add the NAICS level that you will count as a sector to the description? Maybe "anything at the 2 digit level" with 3/4 digit codes evaluated on a case by case basis?

@JustNo fiscal year Vs calendar year clarity might be good too.

@JustNo to get an idea on impact... Has it been proven under such a test that Amazon for example has had a significant impact on brick and mortar retail?

If we used Amazon in place of AI would this resolve yes under such criteria?

@JustNo That sounds like a good criteria, will add it. Thanks!

@Fivelidz I planned to use calendar year, but maybe these stats are mostly available for financial years.. Need to think about it.

@JoshuaBlake it doesn't have to be "rolled out" across and industry. It can be something like GPT-4 being widely available by students so they pay for less online tutoring and there is an impact on growth in that sector as a result of lower demand.

That would satisfy the yes conditions and be a positive resolution.

I think there could also be widespread economic impact on online advertising for example. Search engines not using AI will likely be negatively impacted as Bing and Google bard take up greater market share?

If bing has record growth against non AI based search engines could that resolve it as a yes?

It had a 15% user increase with the integration announcement.

https://www.similarweb.com/blog/insights/ai-news/bing-chatgpt-ai-chat/

@Fivelidz - if there is an impact, who determines what made the impact?

Eg, if GPT-4 is released and in Q4 of 2023 the online tutoring sector comes out and says they saw a loss in revenue/lack of growth/decreased demand how will it be determined that it was GPT that caused the impact vs other confounders such as a softening economy, rising inflation, fewer people going to college, etc?

Like sure, the tutor companies could come out in their earnings report and say "hey, there was an impact and GPT is to blame". But just recently Walgreens said they were shutting down stores due to shoplifting (and definitely not because they overexpanded), but then later came back and said "Maybe we cried too much last year" in an earnings call. So clearly companies can't be trusted to accurately report what is impacting their business.

https://www.cnn.com/2023/01/06/business/walgreens-shoplifting-retail/index.html

@LukasDay

I think it's fair to think that AI having an economic impact at some point is certain but this market is concerned with proof of it occuring this year?

So what test needs to pass to say that it is unequivocally due to AI?

These following articles are about generative AI replacing a quarter of all creative jobs.

https://www.weforum.org/agenda/2023/05/generative-ai-creative-jobs/

https://www.key4biz.it/wp-content/uploads/2023/03/Global-Economics-Analyst_-The-Potentially-Large-Effects-of-Artificial-Intelligence-on-Economic-Growth-Briggs_Kodnani.pdf

Even if this year we were to see significant reduction in employment in those jobs and the employers state it is due to using AI systems instead and broadly the explanation from employers is AI can do the job more cheaply, would you still deny this?

Would you say instead that these employers could be collectively having an ulterior motive in explaining culling their staff and argue confounding factors?

For a tutor company to admit that they are suffering due to AI is far more devastating than to suggest growth slowing due to virtually any other confounding factor. If across field the online tutoring sector drops; what "proof" of it being directly linked to AI would be needed?

It seems like a correlation Vs causation discussion but ultimately I think it will be found that the variable with the most impact is AI becoming widespread and the economic change can be reasonably attributed to it due to it.

The frustrating thing about this though is as you point out confounding variables like lower college attendance or the economy is hard to gauge for a field like online tutoring as it is such a recent sector.

Right now it seems that if papers were to come out to say a sector was incredibly likely to have been impacted by AI this could devolve into how nothing can be proven as there are confounding variables about everything. People even stating AI being the cause in the field themselves doesn't pass this test so what can?

By that rational I'd say this market can't resolve as NO either as it needs to be proven that that every noticeable sector change this year wasn't due to AI. It's a market that is YES or N/A but definitely can't resolve as NO as it can't be proved there wasn't a noticeable effect. I'm leaning to this as settling YES rather than N/A though as I think it will be unequivocal by the end of the year.

@Fivelidz -

> So what test needs to pass to say that it is unequivocally due to AI?

I mean, the description is a little vague, but here's how I was interpreting the question:

Will there have been a noticeable NAICS sector-wide economic effect from widespread commercial use of AI Technology in fiscal year 2023 (1 October 2022 - 30 September 2023)?

There is ambiguity as to whether this effect needs to be seen in US economic sectors or other national economies are considered. One of my assumptions is that the question is specifically US focused and that market sectors are the 20 broad sectors (Table 2) defined by the North American Industry Classification System (NAICS) [But again, ambiguity because the market description specifically mentions car manufacturing (NAICS 336100) which is technically a subsector Transportation Equipment Manufacturing (NAICS 336000) which is itself a subsector of the Manufacturing (31-33) sector.]

This market would resolve YES if there is a noticeable economic effect (positive or negative) in any NAICS defined sector or subsector that can be attributed to widespread commercial use of AI Technology. Attribution can come from a government source (such as Bureau of Labor Statistics), a reputable economic analyst (such as Goldman Sachs or Bloomberg), or if 90% or more companies in a defined sector file 10-Qs or 10-Ks that attributes the performance of some economic metric (revenue, employment, productivity, etc) to AI Technology.

The market would resolve NO if there is no performance attribution to AI Technology or if the economic effect was only seen in a minority of a defined subsector. (E.g., if Tesla sees a massive increase in productivity from AI Tech, but Ford, Audi, BMW, GM don't then the market resolves NO.)

@Nostradamnedus - thoughts?

@LukasDay Thanks, this looks like a pretty good resolution criteria! We will use NAICS and the US economy.

@Nostradamnedus - glad to help. Just for clarification, are we just considering the 20 main economic sectors (as seen in the other thread) or are we considering subsectors as well?

@LukasDay I think the subsectors should be included as well as the original examples of car manufacturing, etc are on that level.

Agree subsectors should be included

The NAICS Subsectos I'm confident have been impacted in spirit (i'm not sure how the numbers hash out) is: "Internet Publishing and Broadcasting (NAICS 516)"

GPT-4 is unequivocally better than most humans, including most employed humans, at the kind of copywriting and content generation much of internet publishing uses

I'm not sure if Motion Picture and Sound Recording Industries (NAICS 512) is affected that much this year, but it will be within the next few years.

@Fivelidz Online tutoring is a whole sector of the economy? It doesn’t seem that way to me.

@LukasDay I strongly doubt drive through bots will have a a serious impact this year. That said, I'm still team yes.

@JustNo That was a good video. I forgot why I was watching it and just kept watching.

@JustNo I’ve been Rick Rolled! Thanks for the video, it was great!

@JustNo https://wapo.st/42ykPpT

@JacobSkauvold how can you read that and predict no?

@JacobSkauvold also, thanks so much - sorry if that comes off as rude - I'm just genuinely confused.

@JustNo Haven't read it yet. Will make sure to read it later and update accordingly.

@JustNo one book got scooped is very far from a noticeable effect across the entire publishing industry

@JoshuaBlake I didn't ask why the market wasn't resolved, and I'm not arguing that it should be resolved on this basis - there's no good number quantifying the impact of new LLM or any other tech for the year, and no rush to resolve a market about the year in May. More to your point, the gist of the article is that this isn't an isolated phenomenon.

I am baffled that folks are buying no here at these prices and I'm curious at the thought process. From where I'm standing, it seems preposterous to say there's a 37% chance that no artificial intelligence product will have a noticeable, sector wide economic effect on any sector before the end of the year.

Any given exact prediction may be shakey, but you don't think publishing, visual media, online advertising (including search or breaking search into its own sector), software development, education, or a single other sector will see a noticeable impact?

@JustNo - how are you defining "sector"? Cause strictly speaking, there's no such thing as an "online advertising" or "search" sector.

@LukasDay that's a great question I don't have an answer for, but I'd point out that online advertising is given as an example in the descriptor. Is there an official list of sectors somewhere?

@JustNo - North America (Canada, US, Mexico) uses the North American Industry Classification System (NAICS) for breaking down the economy into sectors/subsectors.

https://www.census.gov/naics/

[Which, after reviewing my own source I guess Web Search (519290) is a sector. Online advertising is just rolled in with Advertising Agencies (541810)]

That's how I've been defining "sectors" usually. Although using this sources comes with the ambiguity of what is a sector and what is a subsector.

Like, car manufacturing is a subsector of transportation manufacturing which is a subsector of the Manufacturing sector. Are these considered three different sectors? Or all they rolled into the Manufacturing sector? If we see a noticeable impact from AI Tech in car manufacturing but not transportation manufacturing how does the question resolve? (That's a problem for the market creator tho)

(Related, this is why I'm betting NO. Depending on how economic sectors are defined, even if one or two companies see massive impacts from AI Tech, will it be enough to shift the entire sector as a whole in a noticable way? Depending how how large the sector in question is, unlikely)

@JustNo - Here's the 20 main economic sectors according to BLS

https://www.bls.gov/ces/naics/

@LukasDay NAICS looks pretty good.

@RonWiener Needs to make a statistical impact.

@Nostradamnedus Previous writer stirke had such impacts, bigger question is do you attribute the strike to AI

@RonWiener @Nostradamnedus I don't think it can, as of yet, be attributed to AI. AI is included, but it feels like an after thought as of yet.

AI isn't mentioned in this piece from Vulture in their section on "What are the Demands"
https://www.vulture.com/2023/05/wga-strike-2023.html

Unless it becomes a publicized fact that AI is a hang up, and it substantially increases the duration of the strike I don't think the strike can be attributed to AI.

@JustNo there's a number of articles now that seems to indicate that AI is a hang up. Just Google "writers strike GPT AI"

@AlexanderLeCampbell there should be a change in key statistics, such as employment or revenue, to a degree that is clearly noticeable from normal variation and can be attributed to AI.

@LachlanMunro yeah I think this should satisfy that it's significantly impacted the online education/tutoring sector.

@Fivelidz I think it's reasonable to hold off resolving until a broader and more substantial impact has been documented. A single company doesn't quite constitute a sector. That said, I'll be continuing to buy pretty much as many Yes shares as I can afford - I'd say up to 90% at least - because I think it will be very clear by year-end.

@JustNo might be easier to document the overall impact it has had during 2023 after 2023 then. Might make the resolution date a bit tricky tbh.

@Fivelidz keyword is noticeable I guess.

@Fivelidz The closing date isn't necessarily the resolution date (: but yea, I don't think it's going to be an issue.

@xxx It's affected basically all digital art communities, but as far as I can tell, it hasn't had major quantifiable economic effects on digital art as an industry.

@horse Agreed. For there to be a big impact you'd need to see a big impact on people where the production of art is / was already providing substantial income, and it's unlikely that many people working on in-flight projects are going to adopt new tech stacks or workflows midflight - so even as they start training up or using tools in-house, it's people looking to move from hobbyist or amateur to pro that are going to have challenges getting through the barrier-to-entry of reasonably well paid art.

@PipFoweraker But it has an impact on growth of the sector?

If there were more jobs in the digital art field year by year until the new AI models have come out though existing artists may have a job for a while it's still has affected demand and competition.

A big impact can just be rapid growth stopping/slowing not necessarily receding right?

@YoavTzfati I'm still sticking with NO - I think for it to scale appropriately by the end of the year it will need to be adopted by large orgs, and the discussions I've seen from engineering, security, risk, compliance people etc inside them suggests an adoption curve pushing into 2024 before the sum effects would hit the 'noticeable' threshold. Although I'm thinking if I'm wrong it'll probably be in a field like advertising where the lifecycle of campaigns and deployments are much shorter. Good argument though!

@PipFoweraker It doesn't have to be implemented by a company to have significant impact in a sector.

Just using chat GPT-4 is viable for many to use instead of getting an online tutor, website building or legal advice. Reducing demand in those areas will impact growth in those industries.