🏅 Top traders

okay, all. This was a pretty tough resolution. 11/20

I totally admit that some of these are rather froggy. But they still are very birdy, too, since they have wings, feathers, good feet, etc. Just sometimes they have froggy eyes and lack beaks.

I tried to base this on my comment about ideograph, where the 3rd image has a very bad tail and no legs or possibly froggy legs, but I still gave credit. Based on that, I judged the bottom animals under here on "balance of appearance" rather than "does it fully and perfectly look like a bird". I also considered how much room for disagreement there would be if I adopted the opposite position; in that case, people could have strong arguments that a bird with wings, feathers, bird feet etc but a froggy head should be considered a bird.

To rectify this, I have made a market with STRONG requirements for each animal, due mid 2024. /Ernie/midjourney-can-draw-a-frog-riding-a-c14b6948e49e

UL no

UR yes

LL no

LR no

UL no

UR yes

LL yes

LR no

UL yes

UR yes

LL yes

LR yes

UL no

UR yes

LL no

LR no

UL no

UR no

LL yes

LR yes

@Ernie thanks for the thoughtful resolution. Each of these image generation markets seems to teach us something about making them better in the future. I particularly liked the idea of, "Crop out the rest of the image; would the average person say that is an X?" One operationalization with some effort would be to actually do that and show the images to online crowdworkers with some concrete measure (e.g., 4 pre-determined randomized answer choices, do at least 15 out of 20 crowdworkers pick the right one?), though there are logistics that come with that, such as making sure the crowdworkers are paying attention. You could survey disinterested Manifolders.

@Jacy yes. I'm trying to improve over time. The cutout thing could be done; even then it'd be important to tell them whether they have the option to say "neither frog nor bird" or they should choose the closest.

Perhaps more time at market creation could help... but so far it always seems like the problems are not what I expect them to be.

@Ernie while I understand there's subjectivity involved involved I can't avoid being disappointed. I count at most three birds in the generated images (image 1 TR, image 4 TR, image 5 BL)

Edit: I missed image 3 LR which could be a bird, so I'll give you that one too

If you encountered a real life example of any other creature in the pictures no one would casually say, "oh that's just a weird bird", you'd probably call an exorcist

@Odoacre I get it, but there was always going to be a judgement call and it was probabilistic, which is why the market was near 50%. For what it's worth, some birds look like this:

This is a "Large Tawny Frogmouth", native to Australia.

@chrisjbillington I understand Ernie's reasoning and I'm not looking for a reversal or anything. But I still think it was flawed reasoning. The tawny frogmouth has a bizzarre appearance but no one would mistake it for an actual frog.

Even the third ideogram image has a beak and is flying which are "Birdy" things. Most of the midjourney generated birds have a clearly froglike appearance and are basically a hodgepodge of different animal parts stitched together.

The very fact he is trying to box the animal in the "bird" or "frog" category is imho wrong reasoning, what if MJ had generated a frog riding a cat? How would the cat fare in the frog-bird spectrum?

There's no need to decide if it's more a frog or a bird, the simpler question is: is the thing on top a frog and is the thing below it a bird?

@chrisjbillington damn imagine if the data is full of those guys. No beak and froggy eyes!

Thanks for posting. You're a lifesaver

@Odoacre Yeah, I lost here FWIW, was betting NO. I see your objection (and understand you're not looking to change the outcome, this is just a discussion). But I think there's just not much to be base things on when it's a subjective market. @Ernie decided these Frogkensteins were close enough to being brids, and there isn't any objective standard to say otherwise, since the criteria didn't list specific bird-like attributes that are necessary.

Maybe you could say "would most people call this a bird?" or something like that, but we'd have to have a survey to find out, and if we're not going to have a survey, I guess as long as the creator's judgement isn't gratuitously different to what we'd predict such a survey to show, then we should accept the remaining discrepancy as part of the game of betting on subjective markets (not saying you're not accepting it, of course).

That's an idea though, resolving some of these AI markets by survey in the future. Maybe something to experiment with.

The very fact he is trying to box the animal in the "bird" or "frog" category is imho wrong reasoning,

This is a valid alternate way of reasoning, but so is "what two animals is it most like, and which of the two is it more like"? I can't think of why one kind of reasoning should take precedent other than a general principle of being more or less strict in judgement, but since the market wasn't didn't say it was erring on either side, of strictness or leniency, I don't think there's much of a basis to choose one kind of logic over another.

@chrisjbillington Out of curiosity, made a poll (I don't think this should affect the market in any way, am just curious):

I tried it myself, and I can't say that any of these would be good for me, if I asked a human to drawn a frog riding a bird and they came up with any of these examples I would be very disappointed and would challenge their understanding of the request.

A bird with a frog head is not a bird, it's just some weird chimera.

In the set of 20 I got with Midjourney v6 on Dec 24, I'd say these count: A3, B1, B3, C3, C4, D1, D2, D3, D4, E1, E3, E4 (with E2 right on the fence because it has a very froggy head, a froggy back, but froggy bird wings, a birdy chest and tail, and very birdy legs), which is 12/20 or 13/20. The frogs are mostly great. The birds have pretty good bodies, but the big issue is frog-like bird heads (mostly frog with a more pointed "beak"), and I'm not sure how you'll rule on those. I'm also moving up the price based on the third Ideogram image that you said was positive because that's also a pretty frog-like head on the bird.

(A)

(B)

(C)

(D)

(E)

@Ernie could we get some clarification on your standards for bird images before the market closes? E.g., judge a few of the borderline images as examples. I'm mostly going off the third Ideogram image in the description, which you say meets the bar.

@Jacy I'm thinking of it this way. If we cut out the images of each animal and independently asked people what it was, I'd need their primary answer to be "bird".

Because anyone can do their own tests, providing a true resolution rule would solve the market. I'll be home tonight and attempt to give info. No n3 MJ version will be out this year so it's as good as done

@Ernie thanks, that doesn't update me much in either direction. Maybe you could specify "bird" among which options? In a free-text response, I'd expect most of the examples here, including the ones that meet your bar, to evoke other responses, but if the options are just "bird" and "frog," or a few species of birds and a few species of frogs, or a number of animal names but nothing like "birdy frog" or "mutant," then I think we're definitely in bird territory.

Huuuuuh why is this so difficult? My best attempt:

@Joshua hahaha

@Joshua wow that's a lot better than anything i've managed