Will an LLM be able to match the ground truth >85% of the time when performing PII detection by 2024 end? | Manifold

Will an LLM be able to match the ground truth >85% of the time when performing PII detection by 2024 end?

25

1kṀ4723

Dec 31

84%

chance

1H

6H

1D

1W

1M

ALL

PII - personal identification information

Stuff like people's names, numbers and codes that identify stuff (SSN, phone number, passport etc), places, locations, names of orgs, attributes that can be used to identify a person, etc.

GPT-4 outperforms Presidio, Microsoft's custom built tool for PII detection. GPT-4 matches ground truth ~77.4% of the times, while it misses a single PII element ~13% of the time.

Get

1,000

to start trading!

Sort by:

Assume this includes both false positives and false negatives? What's the denominator?

predictedYES

Just a complete side question, what are the legalities or what are the complicating factors in using a GPT against PII? So, it has to be trained on dummy PII, right? How much dummy PII is needed to train that 85% level you are referring to?

@PatrickDelaney I think microsoft tested against their in house system, which does detect PII on real data

People are also trading

❓ Which AI model will lead the LLM race by the end of 2025?

Will there be an LLM which scores above what a human can do in 2 hours on METR's eval suite before 2026?

LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?

Will LLMs be able to formally verify non-trivial programs by the end of 2025?

Will one of the major LLMs be capable of continual lifelong learning (learning from inference runs) by EOY 2025?

Will an LLM be able to solve Raven's Progressive Matrices from an image in 2025?

Will an LLM improve its own ability along some important metric well beyond the best trained LLMs before 2026?

What will be true of OpenAI's best LLM by EOY 2025?

Will LLMs become a ubiquitous part of everyday life by June 2026?

Will the best LLM in 2025 have <1 trillion parameters?

Related questions

❓ Which AI model will lead the LLM race by the end of 2025?

Will there be an LLM which scores above what a human can do in 2 hours on METR's eval suite before 2026?

LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?

Will LLMs be able to formally verify non-trivial programs by the end of 2025?

Will one of the major LLMs be capable of continual lifelong learning (learning from inference runs) by EOY 2025?

Will an LLM be able to solve Raven's Progressive Matrices from an image in 2025?

Will an LLM improve its own ability along some important metric well beyond the best trained LLMs before 2026?

What will be true of OpenAI's best LLM by EOY 2025?

Will LLMs become a ubiquitous part of everyday life by June 2026?

Will the best LLM in 2025 have <1 trillion parameters?

© Manifold Markets, Inc.•Terms•Privacy