I'm considering the first IFT "Not Successful", in that it didn't insert Starship into a near-orbital trajectory. Even though personally I think it was a successful data gathering flight. The guiding principle for success is something like "Did this flight prove to customers their payload will go where they want it to go?".
Essentially, will SpaceX make 500 launches that a customer would consider proves satisfactory reliability and safety of the system by Jan 1st 2030.
Clarified criteria:
A "successful launch" means any launch in which:
A Starship–Superheavy full stack lifts off from the surface of Earth under the thrust of its engines, and
Starship reaches an altitude of at least 100km, and
Starship either:
Successfully deploys or delivers a payload to its intended trajectory or destination, or
If a test flight with no payload, reaches its target trajectory or destination - disregarding later parts of the flight pertaining only to vehicle recovery or disposal.
The last point is context-dependent:
If a test flight is attempting to land Starship on e.g the moon or on Mars, then Starship must land in one piece at the target destination to count as a success.
If a test flight is testing Earth orbital capabilities, then Starship must reach its target orbital or near-orbital trajectory, but the launch will still count as a success if it burns up on re-entry.
If a test flight is of point-to-point suborbital hops, then Starship must land in one piece at the target destination for it to count as a success. Point-to-point hops can be identified by their highly elliptical trajectories, with target perigee well inside the Earth.
In all cases it doesn't matter whether the booster survives.
1,000
3.00I updated the description to match the proposed new criteria and unpinned the comment. I'd have left just the tally pinned if I could, but it doesn't make sense to keep it in the description + that comment is super long (and now largely redundant).
I'm borrowing from the original creator's criteria:
I think we have to fit the same criteria to the 1st launch that we do to the 500th. I'm considering the first IFT "Not Successful", in that it didn't insert Starship into a near-orbital trajectory. Even though personally I think it was a successful data gathering flight. The guiding principle for success is something like "Did this flight prove to customers their payload will go where they want it to go?".
and will say that the guiding principle will remain "Did this flight prove to customers their payload will go where they want it to go?". If this means at some point flights are being made that reach 99km by design, the 100km rule could be revised. I trust this will not be the case...
in defence of the 100km rule, from the original description:
I'm considering the first IFT "Not Successful", in that it didn't insert Starship into a near-orbital trajectory.
Thanks @chrisjbillington and @ViniciusMoreira for your considerable effort in making this market better!
If there are any standing objections, let me know - I am not an expert
I read through some (not all) of the comments. I think it is reasonable to adopt the criteria outlined in the pinned comment -- assuming it really does include the special case suborbital flights that meet
The guiding principle for success is something like "Did this flight prove to customers their payload will go where they want it to go?".
from the original criteria
I'll give an opportunity to respond if you like, I'd rather everyone agreed so we can move on, but if anything is insufficient lmk before I change it.
I don't know who is counting though, 500 launches seems horrible to keep track of and if there is no good source that tracks this criteria we may have trouble in 5 years if it's close to that number..
I figured surely somebody is counting, but I had no idea how right that was: https://en.wikipedia.org/wiki/List_of_Falcon_9_and_Falcon_Heavy_launches
Not that wikipedia is a good resolution criteria, but it is thoroughly cited
I would like to propose a more objective and straightforward method for resolving this market to minimize disputes. Using an independent organization, not affiliated with Manifold or SpaceX, such as the FAA, can provide clarity and fairness.
Each time a Starship Super Heavy + Starship launch occurs, it adds 1 to the count of total launches. If the FAA initiates a mishap investigation for any of these launches, that flight will be considered a failure and will be subtracted from the total count.
If, before January 1st, 2030, the count exceeds 500 successful launches, the market resolves to YES. If the count does not reach this number by that date, the market resolves to NO.
I have previously commented on why I believe Chris Billington's clarification method is not ideal. Therefore, I am proposing this alternative method, which I believe is more objective and fair for all parties involved. This approach ensures that success is determined by an impartial and independent body.
Here is the FAA definition of what would be defined as a failure.
The new FAA regulations describe nine events (see below) that would constitute a mishap (14 CFR 401.7). The occurrence of any of these events, singly or in any combination, during the scope of FAA-authorized commercial space activities constitutes a mishap and must be reported to the FAA (14 CFR 450.173(c)).
Serious injury or fatality
Malfunction of a safety-critical system
Failure of a safety organization, safety operations or safety procedures
High risk of causing a serious or fatal injury to any space flight participant, crew, government astronaut, or member of the public
Substantial damage to property not associated with the activity
Unplanned substantial damage to property associated with the activity
Unplanned permanent loss of the vehicle
Impact of hazardous debris outside of defined areas
Failure to complete a launch or reentry as planned
@ViniciusMoreira
> Unplanned permanent loss of the vehicle
This one is specifically allowed to happen in some situations by the prior criteria, so there would need to be a strong consensus to override that one IMO?
It seems like the original intent of the question is really heavily focused on "payload goes where it is supposed to go", to a degree that makes some of the reasons the FAA might consider a launch a failure a little bit less relevant. I wonder if that one would be different enough to constitute a separate question entirely.
Is there a chance they launch some of these vehicles within this timeframe in a manner that somehow does not involve FAA jurisdiction?
---
I don't want it to look like I am stepping on Chris's toes here either. I did respond to your other comment since Chris's original post does say he'll allow other moderators to intervene if there are controversies. Maybe we need to get an additional moderator involved, then.
Regardless, it's much better to figure this stuff out now, than in 3 years or 9 months or whatever. It's good that we are trying to figure out if the current count is 1 or 2 or whatever, and more objective criteria. Maybe they'll easily do 600 in which case it was all a waste of time, or maybe they won't even get to 400. But if it's 'close', then this stuff matters.
@ViniciusMoreira I appreciate the desire to make criteria more verifiable, and if the market were being made from scratch, perhaps counting non-FAA-mishap launches would be a good criterion to base a market around. However, FAA mishaps and successful payload deliveries (or demos of such capability) are simply different things, and this market is about the latter. You can have unsuccessful payload delivery and yet no FAA mishap, and you can have successful payload deliveries (or plausible demos thereof) despite FAA mishaps.
As for your point below about the 100km criterion, it's a space rocket and 100km is the most common definition of space. Any launch that could plausibly have a customer or be a test of something a customer might want will definitely be reaching at least 100km. That very much includes point-to-point flights on Earth, which I deliberately wanted to include with these criteria.
This is not a picky criterion. In recent flights, at the time Starship had reached 100km of altitude, it had only travelled about 130km downrange. After stage separation in the most recent flight, Starship ignited its engines at 72km of altitude, with about 0.75 km/s of vertical speed. Ignoring wind resistance, without igniting its engines, Starship would have continued upwards an additional ~30km over the next ~80 seconds, reaching an altitude just above 100km, before falling back to Earth. So a launch that only reaches 100km is one where Starship doesn't even ignite its engines.
The only launches that are excluded by this criterion are low-altitude tests that are not a trajectory any customer would want, or close to what any customer would want. For example, there were many Starship low-altitude test flights before Starship first flew with SuperHeavy. The 100km criterion is simply to exclude out such low-altitude test flights, which I don't think the original market intended to include, since it defined success as delivery of payload to "intended orbit or landing zone for interplanetary bodies". Previous low-altitude flights are excluded by the original criteria already by virtue of not involving SuperHeavy, but I think it's unlikely the creator would have wanted to include similar flights that do.
I'd be happy to replace the altitude criterion with something else, but you need something to exclude test flights that aren't a trajectory any customer would want. And "no mishap" isn't a good criterion, since SpaceX could do some low-altitude tower catch tests or whatever and have it go off without a hitch, and yet it would not be a demonstration of anything a customer would plausibly want - which is about taking a payload somewhere.
For what it's worth, I don't think we're going to see any successful Starship–SuperHeavy flights reaching less than 100km of altitude ever. If we remove the criterion I think it won't make any difference, so I don't think it's the case that it's a significant change. But on the off-chance there is some low-altitude test flight, it's good to have a way of excluding it without us having to argue about whether a customer would have wanted it. I think this very conservative altitude criterion is a good threshold - it would be a very weird customer who charters a space rocket to go a couple of hundred km without leaving the atmosphere when they could have taken a plane.
@chrisjbillington
it's fine not to use my definition of success, but I still believe your 100km criterion is arbitrary and was not in the initial question. However, if you include Earth-to-Earth the flights as shown in this video https://www.youtube.com/watch?v=zqE-ultsWt0 , regardless of the altitude reached or whether they have passengers or not, that works for me. If not, I want moderators, I took a look at this market.
@Eliza
As I mentioned in a comment to Chris Billington, it's fine with me not to use my definition if the types of flights shown in this video are included: https://www.youtube.com/watch?v=zqE-ultsWt0
Regarding the resolution of this market, I believe it will be very unlikely to be 'close.' It's either going to be 'embarrassing' for the YES side with fewer than 100 flights, or it will be the opposite with over 700+ flights. It all depends on when the slope of the curve starts to go exponential.
@ViniciusMoreira those flights will be included, fear not. They will be greater than 100km in altitude, I promise you. There is no plausible customer-relevant Starship flight that doesn't involve it reaching at least 100km.
The 100km criterion was chosen in order to include such flights (and other technically suborbital flights) whilst excluding low-altitide demos such as the trajectories of SN8,9,10,11, and 15.
I have a concern regarding Chris Billington's recent "clarification". The added requirement that a "successful launch" means reaching the Karman line (100 km altitude) was not part of the original prompt when I invested.
I believe SpaceX's Starship will achieve 500 successful launches by January 1st, 2030, especially with the potential for Earth-to-Earth transport. These flights may not reach 100 km but can still be considered successful by any reasonable definition. Additionally, the Karman line is not universally accepted; for instance, in the U.S., where SpaceX is headquartered, space begins at 80 km according to NASA and the U.S. Air Force.
Chris's new criteria significantly change the original terms and could unfairly exclude many successful flights. I invested because I believe in Starship's potential, and this added requirement alters the original understanding.
I urge the moderators to consider the original criteria and not impose additional requirements that were not initially stated. This ensures fairness and maintains market integrity.
@ViniciusMoreira It seems like a reasonable argument. I'm a No holder but if they use this system to do launches that are intended to NOT reach 100km and they do so successfully, they should probably count.
Thank you for your response, @Eliza .
As a suggestion, for the sake of clarity and fairness, it might be beneficial to define a "successful launch" as one that is straightforward to verify. In my opinion, a launch should be considered successful if it takes off, reaches its mission goals, and, upon completion, does not trigger any investigation by the FAA (Federal Aviation Administration). The FAA is a reliable and independent from SpaceX, making it a trustworthy authority on this matter. Additionally, any significant failure of the Starship anywhere in the world automatically triggers an FAA investigation, which provides an independent measure apart from SpaceX.
Using this definition, we currently have 1 successful flight (2024-06-06 IFT-4), contrary to Chris Billington's count of 2. If an FAA investigation is triggered for IFT-4, the number of successful launches would drop to zero. This approach ensures that success is determined by an impartial and independent body.
@ViniciusMoreira
For those who are curious what the FAA defines as Mishap and automatically trigger an investigation, if you use my definition it would be defined as failure.
The new FAA regulations describe nine events (see below) that would constitute a mishap (14 CFR 401.7). The occurrence of any of these events, singly or in any combination, during the scope of FAA-authorized commercial space activities constitutes a mishap and must be reported to the FAA (14 CFR 450.173(c)).
Serious injury or fatality
Malfunction of a safety-critical system
Failure of a safety organization, safety operations or safety procedures
High risk of causing a serious or fatal injury to any space flight participant, crew, government astronaut, or member of the public
Substantial damage to property not associated with the activity
Unplanned substantial damage to property associated with the activity
Unplanned permanent loss of the vehicle
Impact of hazardous debris outside of defined areas
Failure to complete a launch or reentry as planned
As the creator is absent, I'll be clarifying resolution criteria and keeping track of what launches count for the purposes of this question being resolved by moderators in the future. This is something of a test case for new moderation policy being implemented by Manifold, that will allow moderators a more active role in clarifying questions and tracking what aspects of the criteria are satisfied already, ahead of time.
As I'm obviously trading in this market, if this turns out to be controversial, I will ask other moderators to intervene instead.
On to clarification:
A "successful launch" means any launch in which:
A Starship–Superheavy full stack lifts off from the surface of Earth under the thrust of its engines, and
Starship reaches an altitude of at least 100km, and
Starship either:
Successfully deploys or delivers a payload to its intended trajectory or destination, or
If a test flight with no payload, reaches its target trajectory or destination - disregarding later parts of the flight pertaining only to vehicle recovery or disposal.
The last point is context-dependent:
If a test flight is attempting to land Starship on e.g the moon or on Mars, then Starship must land in one piece at the target destination to count as a success.
If a test flight is testing Earth orbital capabilities, then Starship must reach its target orbital or near-orbital trajectory, but the launch will still count as a success if it burns up on re-entry.
If a test flight is of point-to-point suborbital hops, then Starship must land in one piece at the target destination for it to count as a success. Point-to-point hops can be identified by their highly elliptical trajectories, with target perigee well inside the Earth.
In all cases it doesn't matter whether the booster survives.
Launches so far:
2023-04-20 IFT-1: didn't reach target trajectory ❌
2023-11-18 IFT-2: didn't reach target trajectory ❌
2024-03-14 IFT-3: reached target trajectory ✅
2024-06-06 IFT-4: reached target trajectory ✅
Count of successful launches so far: 2
@mods Can someone with the appropriate permissions please review Chris's criteria here and, if you are satisfied, add it to the market description for the benefit of all future participants? I have reviewed his submission along with the context of the original market description and it seems to be well-aligned. There has been a debate with Vinicius Moreira about some of the points in Chris's criteria, but it looks to me like Chris is very confident in this and it's unlikely to change greatly. (If we have to modify it again later, we can do so.)
Also, this is an interesting market with 100 participants. It might be a good market for some targeted subsidy from the house -- maybe not too much, since it is a long-term market that people might not want to tie up mana in forever, but a little bit more than 1000 would be awesome. Maybe bump it by 5k or 10k?
I think, as a general rule, it should not be considered good practice for mods who are participating in markets to "take them over" from inactive creators and clarify or alter resolution criteria.
It seems substantially 'worse' than creators betting in their own markets, where at least other participants can have more of an idea of what they are getting into.
This isn't in any way a comment on Chris or the specific clarifications in this market, just a general rule of thumb suggestion.
@JoshuaWilkes In addition to you thinking that, the guidelines also say that.
When Chris posted his new suggested criteria, it specifically said in his post that he'd defer to other moderators if it turns out to be controversial, and I took that to mean he would allow any other moderator to adopt his criteria or modify them as they see fit. He's obviously well informed on this issue and has written some detailed criteria, but he's also the largest No holder on the market at this time.
The actual guidelines for moderators to 'take over' markets specifically prohibit moderators with a 'large' position from becoming the new 'owner' of a market, so in this case it seems like, for example, Joshua with his 121 shares of No would be allowed to do this, but Chris with his 100k No shares probably would not.
The steps from the guidelines:
Someone else becomes the owner
The description is updated
The creator has 48 hours to come back to undo this
So far it seems like we're most of the way done with the first step and haven't gotten to the 2nd step yet. (....I'm not really convinced the market needs a single new moderator to be defined as the question owner -- it should probably either just be owned by all the moderators/the site, or else transferred to a new, active user of any kind....)
Out of all the other moderators on the site, there are at least some who have some knowledge about this domain, so we could ask one of them to review Chris's criteria and either adopt them, rewrite them a little, or completely rewrite them. But in any case, someone who is an admin will need to actually edit the description and there is no way for the site to actually show who is the 'new owner', so.....
I like these criteria. I'm running some very similar markets and need to come up with criteria; they're likely to be extremely similar to this and I might adopt them entirely. I could become the owner (I have 110 shares on yes here), but I think I'd like to maintain my ability to trade in it. (I care more about that with my existing shorter term 2027 market, though.)
My related markets:
/EvanDaniel/will-spacexs-starship-have-40-succe
/EvanDaniel/will-starship-have-a-99-success-rat
An interesting edge case we should figure out (pointed out on my success rate question): does the propellant transfer on IFT-3 count as successfully delivering a payload to its intended trajectory? I think on this criteria set it does.
definitely a nitpick, but:
>Essentially, will SpaceX make 500 launches that a customer would consider proves satisfactory reliability and safety of the system by Jan 1st 2030.
seems poorly worded with respect to the actual question; I would say a 1% failure rate is generally accepted in launch systems; if SpaceX made 500 launches and 5 failed to put the load in orbit, it could be argued that this criteria is achieved.
conversely, if there were 1000 launches and 500 blew up on the pad (with the remaining 500 successfully placing the payload in orbit), I feel like this criteria is arguably not met.
"proves satisfactory reliability and safety" feels like a criteria that's off-axis from the actual question being asked here.
@Adam The creator is inactive and 2030 is a while away, but I think if moderators are considering how to resolve this question in the future, they'll just count the number of successful launches.
The detail given by the creator here seems intended to confirm that suborbital flights with no payload count, so they defined "success" as including demos that a customer would see and think "Yeah, if that were my flight, I'd be happy that it could have delivered my payload".
And whilst the customer does indeed care about the success rate, I don't think that should be considered relevant to the question, the definition of "success" should be limited to determining whether individual flights count.