[High context mech interp question] In https://transformer-circuits.pub/2024/jan-update/index.html Anthropic introduce "ghost grads". This is a fairly complex technique (e.g. it includes treating various scaling factors as constant wrt gradient descent, equivalent to using stop gradients). It also leaves some details ambiguous. I've heard of subtle bugs implementing this technique, some of which don't impact performance! So, is Anthropic's implementation also bugged?

This markets resolves yes if the Anthropic's implementation as of the posting of the circuits update was bugged. If there is a detail they didn't specify, this does not impact market resolution. Resolves yes/no based on updates to the circuits update post and/or my subjective impression on discussions