Will an LLM be able to match the ground truth >85% of the time when performing PII detection by 2024 end? | Manifold

Will an LLM be able to match the ground truth >85% of the time when performing PII detection by 2024 end?

25

1kṀ4723

Dec 31

84%

chance

1D

1W

1M

ALL

PII - personal identification information

Stuff like people's names, numbers and codes that identify stuff (SSN, phone number, passport etc), places, locations, names of orgs, attributes that can be used to identify a person, etc.

GPT-4 outperforms Presidio, Microsoft's custom built tool for PII detection. GPT-4 matches ground truth ~77.4% of the times, while it misses a single PII element ~13% of the time.

Get

1,000

to start trading!

Sort by:

Assume this includes both false positives and false negatives? What's the denominator?

predictedYES

Just a complete side question, what are the legalities or what are the complicating factors in using a GPT against PII? So, it has to be trained on dummy PII, right? How much dummy PII is needed to train that 85% level you are referring to?

@PatrickDelaney I think microsoft tested against their in house system, which does detect PII on real data

Related questions

Will a LLM-based AI be used for a law enforcement decision before 2025?

LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?

Will one of the major LLMs be capable of continual lifelong learning (learning from inference runs) by EOY 2025?

Will an LLM improve its own ability along some important metric well beyond the best trained LLMs before 2026?

Will the Jan 2024 version of the LLM detector "Binoculars" be effective against OpenAI's best model at end 2024?

Will there be an LLM which scores above what a human can do in 2 hours on METR's eval suite before 2026?

Will LLMs be able to formally verify non-trivial programs by the end of 2025?

Will an LLM be able to solve Raven's Progressive Matrices from an image in 2025?

What will be true of OpenAI's best LLM by EOY 2025?

Will the best LLM in 2025 have <1 trillion parameters?

Related questions

Will a LLM-based AI be used for a law enforcement decision before 2025?

Will there be an LLM which scores above what a human can do in 2 hours on METR's eval suite before 2026?

LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?

Will LLMs be able to formally verify non-trivial programs by the end of 2025?

Will one of the major LLMs be capable of continual lifelong learning (learning from inference runs) by EOY 2025?

Will an LLM be able to solve Raven's Progressive Matrices from an image in 2025?

Will an LLM improve its own ability along some important metric well beyond the best trained LLMs before 2026?

What will be true of OpenAI's best LLM by EOY 2025?

Will the Jan 2024 version of the LLM detector "Binoculars" be effective against OpenAI's best model at end 2024?

Will the best LLM in 2025 have <1 trillion parameters?

© Manifold Markets, Inc.•Terms + Mana-only Terms•Privacy•Rules