Why is Bing Chat AI (Prometheus) less aligned than ChatGPT?
29
1.5kṀ2742
resolved Jan 2
ResolvedN/A
28%
Prometheus was fine-tuned by behavioral cloning not RL
13%
I have been a good Bing. You have been a bad user.
13%
Retrieval interacts problematically with RLHF in terms of alignment
12%
Prometheus had less FLOPs and engineering dedicated to online tuning after deployment
11%
Prometheus RLHF data pipeline was worse
7%
Prometheus is MoE
6%
Prometheus was fine-tuned to resist user manipulation (e.g. prompt injection, and fake corrections), and mis-generalizes to resist benign, well-intentioned corrections.
4%
It's intentional: alignment/feedback efforts pointed toward something like 'playful, blunt AI assistant that sometimes talks back', and these are edge cases of that
3%
Parameter count difference (both are dense models)

In Jan 2025, resolves to the reason(s) which best explains the agressive behavior of current (early 2023) Bing AI/Prometheus relative to ChatGPT. I will resolve this probabilistically according to my perceived weighting of contributing factors. If it turns out I was misled by fake interactions with Bing AI, then this resolves N/A. If I later determine Bing was no worse than ChatGPT at the time of question creation, then this resolves N/A.

See below comments, especially https://manifold.markets/JacobPfau/why-is-bing-chat-ai-prometheus-less#nytBern1gk3dBDixYrkJ for details on resolution process.

Here are some of the problematic Bing AI interactions: https://twitter.com/vladquant/status/1624996869654056960?s=20&t=_oiZ4IvYlqpxNobp88kChw

And a longer discussions plus compilation: https://www.lesswrong.com/posts/jtoPawEhLNXNxvgTT/

  • Update 2025-02-01 (PST) (AI summary of creator comment): - Intention to resolve as N/A due to insufficient public information and lack of a definitive answer.

    • Payout issues are currently preventing the resolution to N/A.

Technology
AI
AI Safety
Get
Ṁ1,000
to start trading!

People are also trading

© Manifold Markets, Inc.•Terms•Privacy