Will the project "Better Debates" receive receive any funding from the Clearer Thinking Regranting program run by ClearerThinking.org?
Remember, betting in this market is not the only way you can have a shot at winning part of the $13,000 in cash prizes! As explained here, you can also win money by sharing information or arguments that change our mind about which projects to fund or how much to fund them. If you have an argument or public information for or against this project, share it as a comment below. If you have private information or information that has the potential to harm anyone, please send it to clearerthinkingregrants@gmail.com instead.
Below, you can find some selected quotes from the public copy of the application. The text beneath each heading was written by the applicant. Alternatively, you can click here to see the entire public portion of their application.
Why the applicant thinks we should fund this project
If you aren’t familiar with Kialo, please watch this 2-minute explainer video before reading on. The intention is to use this as an alternative to the [EA Forum] specifically for authors contributing to key debates with big implications (e.g. randomista vs systemic change)
Collective rationality is the lifeblood of EA. Kialo facilitates it best. Below are some of EA’s biggest issues, and how Kialo can alleviate them.
Important arguments on key debates are scattered about the internet.
Kialo groups all contributions to a given debate into a single easily navigable page.
There is no efficient way of locating counterarguments of interest.
Kialo reveals all counterarguments with a single click.
The attention posts receive depend on factors that have little to do with the convincingness of the arguments contained within.
Kialo ranks points solely by their user-rated convincingness; the best rise to prominence
Bad points in otherwise good posts and good points in otherwise bad posts are given inappropriate levels of attention
Kialo breaks arguments into logically distinct pieces. Weak surroundings cannot drag down strong points, nor can strong surroundings drag up weak points. They rise or fall on their merits alone.
EAs tend to write verbosely and use excessive jargon. It’s thus costly for readers to extract information.
Moderators work with authors to distil their writing to be clear, concise and situated appropriately within the debate’s logical structure
How much funding are they requesting?
$50,000
What would they do with the amount specified?
Recruit and train the project manager
Recruit moderators who are experienced with Kialo to consult
Narrow down our shortlist of theses and select one to use for an MVP
Conduct the beta test, survey users, and thoroughly examine the results
Iterate systematically until we’re happy with it
If all goes to plan, take the debate public
Advertise it on Slack, Discord, the EA forum, and the EA Subreddit
Reach out to high-profile EAs to take part
Use the Ten Conditions of Change Framework to systematically address barriers to increased user engagement
Survey users and conduct a thorough analysis of our outcome measures
If the results are promising, we’ll form a plan for how best to scale and start executing!
Here you can review the entire public portion of the application (which contains a lot more information about the applicant and their project):
https://docs.google.com/document/d/1pTJwmT4b5kJ5_Nhsha_EpXaLhlfPd8VAL67rTjL56PU/edit#
Sep 20, 3:26pm:
Sep 20, 3:51pm:
🏅 Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | Ṁ3,456 | |
| 2 | Ṁ323 | |
| 3 | Ṁ269 | |
| 4 | Ṁ215 | |
| 5 | Ṁ213 |
Not having any payout or top comments selected for this tournament after a month since it ended is very disappointing. Manifold should have the money in their account before these tournaments start and the top comments should have been announced simultaneously to the winners.
If you received $30,000 USD from this regranting program six weeks from now, what would your plan be for the six months following that? Please be really concrete about what you’re trying to get done.
We’d request an additional $20K from other funders e.g. the EA Infrastructure Fund. If successful, we’d execute the project. Otherwise, we’d return the money.
I appreciate the honesty here but at the same time I also find it a little disappointing to read this all or nothing view. 30k can go a long way and I don't immediately see great reasons why there wasn't an attempt to break this down.
The issue for me with this project is that it's so married to the particularly platform (Kialo). This makes it inflexible. If the project was more broadly creating a register of arguments and counterarguments in accessible formats for different readership, I'd be backing it. But as people pointed out on LW, there are other alternative platforms (e.g. debateart) that do the same(ish) and as folks both on LW have mentioned (and in fairness, the app acknowledges) adoption is tricky and so far adoption of Kialo hasn't been popular. If it hasn't worked in 5 years since those LW why would it now?
What would be a more effective proposal in this domain for me would be a pilot exercise like taking some of the longer sequences on LW/EA forum, some of the more technical AI discussions e.g. about timelines and different progress stages, some of the populat discussions around economic growth, RCT evaluations, and trying to distil (opposing) views to a wider readership. For further embedding/ longer pilot, I'm also thinking a lot of the intro fellowships, summer fellowships, rationalist workshops would be places to look. Some of my local EA meetings really delve into pro and cons and I can see having that more organized and easily reachable as a useful thing.
Still going no because this is a Kialo-centered project and if it hasn't been adopted in 5 years I think there's enough UX data to see this not being what EA users would use.
@RinaRazh I don't understand how people can compare Kialo and Debateart: they don't seem even remotely similar (in the referential context of "debate platforms"), although perhaps there's some feature of Debateart I'm just missing?
Kialo has had some adoption—a non-trivial amount compared to the graveyard of totally lifeless, hard-to-use platforms—but regardless I don't see why people should take non-EA adoption as a conversation-ending insight into whether it would be valuable in EA.
I really just do not understand why so many people look down on Kialo. (If you'd like to suggest a better alternative to forum comment threads, feel free to do so!)
So conflicted here. I desperately want this project to succeed, I stumbled across LessWrong in high school, absolutely abhorred the format and the suggested posts, and never returned. I deeply, deeply relate to the problem they're trying to solve.
Unfortunately I think this faces the same issue as the Adversarial Collaboration project, once a collective norm is established it can be exceedingly hard to change it, even to a better alternative. Here the norm is use of a specific forum, and if the EA forum is anything like Reddit or social media platforms, the network effects will be too difficult to overcome. Everyone posts there so that's the best place to get attention so everyone posts there (and so on), people's accounts develop their reputations and prestige, and the inaccessibility keeps randos (like me!) away and keeps "quality" high. I'm not involved in the forum, just guessing based on other platforms. But as much as I'd love to see this succeed, I don't think its a promising proposal, and sadly I can't think of any better alternative :(
Came across this 2017 post sharing Kialo on the Less Wrong forum which got a few comments there, but not too many: https://www.lesswrong.com/posts/g3odvaj8opqCF9egv/kialo-an-online-discussion-platform-that-attempts-to-support
Mostly seems a similarish take to the comments here.
Kialo is a great looking system and one of the easiest debate-mapping platforms to use for crowdsourcing purposes and general audiences. No denying that. However its shiny UX and popularity should not be conflated with usefulness (especially if positing itself as something that can improve the structure of discourse). I have been watching this space for years, and I see a few limitations about Kialo which may not have been considered, which non-experts in this space may miss, as well as inconsistencies with how it is presenting itself in this application. Please see a number of these below:
[Kialo argumentation is superficial] Re. the claim that "Kialo reveals all counterarguments with a single click." Sure, you can unpack trees of arguments and claims in Kialo and they are identified as pro/con, however there is no rigorous logical relationship between the pro/cons, they are just side by side and show a general orientation of being opposed to each other in relationship to the parent. This is one of the more superficial argument-mapping relationships that could be had. This makes Kialo easy for scaling an audience, but not great for actually diving into complex argumentation and truth-seeking.
[Kialo argumentation permits vagueness] Unlike curation projects which standardize language so that complex argumentation can be mapped (meaning that precise adversarial relationships between evidence and claims can be fleshed out as well as differentiating all of the tangential, but non adversarial claims in between), Kialo allows general natural language input which allows for vagueness. Again, this is great for casual use and scaling an audience, but not for rigorous truth seeking. For rigorous truth-seeking, complex, natural language statements should be deconstructed to their premises, so that arguments and evidence can be aggregated for and against each premise. Although in the application Kialo claims it "breaks arguments into logically distinct pieces" - it doesn't really do this in a useful way. It may allow users to break down blog posts into arguments (to input as comments), but it does not break down arguments into pieces. A user may break down their argument into claims, but Kialo does not do that, nor specifically require that, therefore it permits vagueness. And vagueness in discourse mapping generates more argumentation based on misunderstanding or partial dis/agreement than necessary.
[Kialo wasn't built for truth-seeking] If a user were to unpack every claim into premises, and map arguments for and against each premise being supported/refuted by evidence, then this expands a debate around every premise, in every argument, which requires more of a graphical map interface, and not a hierarchical tree-interface. Kialo was simply not designed for truth-seeking, just crowdsourcing vague arguments for general browsing. This has a value for those who may not otherwise encounter diversity of ideas, but I don't see this as being useful to a group like EA. Comprehensive mapping (a structure to accommodate comprehensive inquiry and truth seeking) is not a capability of Kialo, but is a capability in other systems already on the web. Speaking of other systems -
I am surprised that Kialo identified the EA Forum as the project most similar to itself in its application (per the question: "Of all the existing projects you’re aware of in the world, which is the most similar to yours, and why do you think yours represents an improvement (or is worth doing despite the existence of this other related project)?"). This seems like a misleading tactic since there are literally hundreds of debate and deliberation mapping platforms that are more similar in telos and format to Kialo than Kialo is likened to the EA Forum. Either Kialo is not paying attention to colleagues in the space, or it is attempting to appeal to the EA community by likening itself to them.
[Kialo is inherently not designed to improve the rigor of thought in depth, only breadth] Again, Kialo is a great catalogue for browsing a diverse set of high-level ideas, but the node structure that contains each claim/argument in their mapping hierarchy is very superficial. They rely on embedded hyperlinks for evidence, which is a very superficial way of linking knowledge and should not be acceptable as a way of improving discourse. With hyperlinks, there is no way to identify the relevant bit of evidence in the media artifact, and also there is no way to argue against the validity/relevance of the evidence in their architecture. Their system would not, therefore, be a marginal improvement to the blog format for discourse, wherein at least you are not confined to short comments as inputs (like with Kialo). There is hardly a use in partially deconstructing discourse into arguments without fully deconstructing the reasoning in order to identify logical fallacies, biases, errata, etc. Does the EA Forum need a hierarchical 'listical' of its ideas? I don't think so. Even if so, a user would get better visualization of that data in a color-coded outline format.
[They're choosing the wrong advisors] In their application, Kialo identifies a championship debater as one of their advisors. This shows a fundamental misalignment with the use-case they are claiming to support. Debate is both a sport, and a tool. A champion debater is not the right person to advise on a project that's supposedly using debate as a tool for truth-seeking. They come from the world of "debate" as a sport. Instead, Kialo should have epistemologists/scientists/philosophers on board for a truth-seeking project. However, if they did get such advisors on board, I would bet M$ that they would quickly be advised that the structure of their system is too superficial for the task of truth-seeking in any rigorous sense, and they would have to rewrite a lot of their code, redesign their user story pipeline, and update a lot of their priors about the reliability of their users.
In sum - Kialo is great for general public audience. It is a starting place, kind of like Wikipedia. I can't imagine $50k could afford all of the redesign that would have to occur for Kialo to make a marginal improvement to the kinds of discourse and deliberation that the EA community would want to engage in. I.e. Kialo mentioned it would host debates like: "Could AIs have consciousness? Does AIs being conscious make them more deserving of legal protection?" The architecture of Kialo can not accommodate the complexity of that debate in a sincerely meaningfully way (though of course it could just launch the topic tomorrow very easily). Anyway, even although Kialo upranks arguments based on their "user-rated convincingness" (which, depending on some goals, this would actually be a kind of anti-rational design), it does not accommodate the rigorous process of deconstructing those arguments to assess its strength/cogency/soundness/validity in all directions of the premises relationship to counter-arguments and relationships to their argument conclusions. It is not a place for serious discourse, and since their platform is already built - I don't see why they need $50k to start the topic on their platform and then post it around a few platforms to get EA users. It does not seem like an effective use of funds for either the discrete project nor contributing to improving the discourse capabilities of large groups.
Anyway - sorry to say all this. I have respect for Kialo for what it is, but felt I needed to plainly state what it isn't from my POV.
TLDR “Guy who writes three-page bullet points, and 300-word sentences does not like platform aimed at clear/concise arguments”
Over to the more serious commentators to more eloquently examine whether it is material that the Kialo proposal is vastly easier to understand than the arguments made against it 🤔
@Gigacasting Had 7 points to make. Nice hyperbole and ad hominen. Sorry you have a tough time reading.
Just because the claims in Kialo's application are easy to understand, it doesn't mean they're true - and sometimes it requires extensive explanation and nuance to debunk something.
Let's try again:
Debates on Kialo are superficially structured by design.
Contrary to the application, Kialo doesn't "[break] arguments into logically distinct pieces" in a logically meaningful way.
Kialo doesn't have features that improve knowledge infrastructure, it perpetuates bad knowledge-linking standards, and it confines certain inputs unnecessarily.
Kialo would accept the arguments as vague as those ones above ^ on their platform, which has limited use if you actually care about what's true.
Kialo seems to lack self-awareness (it thinks the next most similar platform is EA Forum itself and it wants someone who is an expert on debate-as-a-sport to oversee what would be a truth-seeking project [wrong expertise for the deployment of debate as a tool]).
In sum, this doesn't seem like a good use of money.
@SagaciousAims ...doesn't seen like a good use of money since the platform is already built and they could launch the proposed topics right now if they wanted to.
@SagaciousAims You should be clear this application is NOT from Kialo, that is just the proposed forum and the applicants are open to alternative applications, as you can see here https://manifold.markets/clearthinkbot/will-we-fund-better-debates#q3TNf9Pf5owgB5Py0eds
@SagaciousAims you should probably read the application before tearing into any individual piece of it.
@ “sagacious dude”
I’m sorry you got cut from debate and someone at Kialo told you to learn to distill your arguments—but showing up an anon account with a pretentious name and rambling against one project is just weird.
Learn to write!
https://www.amazon.com/Elements-Style-Fourth-William-Strunk/dp/020530902X
@BTE I think this gets into the distinction between "do I think these applicants could have a project worth money they could find" and "should this specific grant be approved".
This grant application is written to be about Kialo, unmodified, with money used for content generation, and the answers to the questions about reasons for failing to achieve goals, mechanism for positive outcomes, and the pitch for the software under why this should be funded, are based around that.
I think the applicants could get together a grant application that did some interesting stuff in the domain that I'd assign higher odds of being approved, they're definitely very focused on it, but it'd take a lot of change from the one being looked at here.
@BTE Hey thanks for improving my understanding. I read the substance of the project and missed that the drumbeat of "Kialo" "Kialo" "Kialo" was from people wanting to deploy it (I skipped the personal details of the application). Apropos that I made a sly comment about reading to the Gigaguy, hah! I'll eat humble pie. Anyway, what a sweet compliment to Kialo, in that case. However, I am not sure it massively changes my arguments about the inappropriateness of that platform and some of the comments about judgement, though they need to be redirected. My apologies for my errata in that instance!
@Gigacasting lol I made this account because someone told me that feedback was welcome on this contest, and I have a lot of background knowledge on Kialo and debate mapping in general - soo this is the project I am commenting on.
Why you so angy?
So much ad hominem - learn to argue (just throwing it back at you for funsies).
or be like @BTE (thanks again for correcting that error)
Is this top quartile? Clearly not.
Is it better than all the “research/lobbying” projects asking for 10x the funds to reach 1/1000th people? Definitely.
Whether or not this guy has any good arguments—it’s nice to see he’s learning to speak succinctly!
@Gigacasting Oh rly? Why you think so? How you know it true? XD
Re. "Is it better than all the “research/lobbying” projects asking for 10x the funds to reach 1/1000th people? Definitely."
This text exchange between you and I is evidence that user-inputted argumentation and evidence is not really a constructive and substantive exchange, lol (< there's my hyperbole - which is tame, oh-so tame, but also marginally true, at least marginally true).
@SagaciousAims LMAO!! This made my day! I couldn't agree more @Gigacasting should definitely be more like me!!
@BTE I think I must be missing some context, but making someone's day is the least I can do for being a jerk earlier :D
I hope y'all have a good day.
@SagaciousAims Thanks for the detailed feedback! I think there's three categories of counter-arguments, which I'll respond to in turn:
<Debates on Kialo are still suboptimal>
I view this as analogous to getting people off heroin. It doesn't make sense to get them to quit cold turkey, better to get them trying methadone first. I'm dubious whether we could get enough EAs to learn the other more complex, less intuitive, and more labour-intensive platforms without first seeing the benefit that a more structured discourse can bring. I view this as the first stepping stone on across the river. It isn't the ultimate solution, but rather, an important first step on the way to it.
<Users have the freedom to be non-constructive (bad links/vague claims)>
This is why we don't simply want to create a debate on Kialo and let it moderate itself. We want to have trial runs, and really think through what are the rules and norms that optimise for an engaging discussion, and then execute on moderating for them.
<Expert advisor is wrong; need philosopher rather than debater>
I thought this too until I read a little deeper into competitive debate. These people are phenomenal at concisely and engagingly breaking down arguments into their purest essence and understanding their logical structure. I think that they strike a good balance between knowing how to write well, how to persuade, and how to reason well.
I think one thing you're perhaps overlooking is that we're trying to build up a new organisation from scratch very quickly, on a tiny budget, and require people who are both skilled at doing that and sufficiently knowledgeable about EA - technical knowledge in constructive argumentation doesn't suffice.
https://www.linkedin.com/in/hamza-tariq-chaudhry/
https://www.linkedin.com/in/john-salter-b685181ba/
I think, paired with a project manager with a good track record of inspiring collective truth-seeking, we do in fact have the capabilities required to make this project a success.
@JohnSalter Hello John, I appreciate the time you spent to further articulate your ideas and appreciate your interest in this space in general! It's really great that you care about formalizing debate. I have not been convinced by your arguments, however, and here is why:
<Re. overlooking the starting of a new organization on a small budget>
I know of an organization in this debate space that has run on less than $50k per year and they have accomplished more than this project is proposed to - and they had to build their own tools. Not that anyone should have to be that scrappy. I just definitely did not over look that fact, rest assured. Also, it doesn't cost very much to ask a few EA friends to use a debate that you set up on Kialo and immediately see how they like it.
<Debates on Kialo are still suboptimal>
In response to your comments about incrementally improving the space (getting people off heroin), I think this incremental step is wholly unnecessary and will ultimately be a waste of the community's time. I have seen at least one project that: processes the data that people have already generated, in the natural language that they generate them (and standardizes that language), deriving them from the platforms where they're already expressing them. Essentially, crowdsourcing projects are inefficient in some dimensions as compared to curation projects - they just look more efficient superficially. With a team of curators and a few tools - I imagine someone could create a Kialo-like representation of the AI consciousness debate in 20hrs from the EA community. I would bet M$ on that. Add a comment feature + moderation and there you go. Now, will EAs care about this knowledge artifact? If you care about baby steps here, maybe shop around those existing platforms to see if anyone finds the output useful, or if it will just be more complex blogging into the void.
<Users have the freedom to be non-constructive (bad links/vague claims)>
All of the other issues I brought up about the features of Kialo itself still stand (i.e. can't handle complex logical structures, perpetuates bad knowledge linking norms, etc.). No matter what you do it is the inherent architecture of Kialo itself that will perpetuate some of these issues. Vague claims may be amended, but this will be hard to accomplish because you will have to teach EAs a new standard. Totally achievable, but everything else will still not be a serious improvement to structured thought IMO - only superficially so. You could create a machine learning model that standardizes this language - that may be worth a few hundred bucks to train.
<Expert advisor is wrong; need philosopher rather than debater>
Breaking down "arguments into their purest essence" sounds like a rhetorical strategy - which is used in debate sports. Deconstructing arguments into all of their implied premises as base logical units is something that would be more useful if you care about truth-seeking. As you said, they are striking a balance between reasoning well, persuading, and how to write well - it doesn't sound to me like these strategies are optimized for truth-seeking, which is a much more rigorous logical process than simplifying language itself. I would concede that your team knows more about debate sports than I do, but I do not accept that therefore you are correct that this is the best choice of advisement. I have taught a class on the logical deconstruction of language for truth-seeking purposes. Distilling language is a part of it, but there are more steps involved.
Anyway, John - you seem really sincere and I can't help but really appreciate how much you care! I am 100% on board for structuring discourse more formally and making structured discourse more accessible. I am probably a rare ally to you in that regard. I don't think many people seem to understand how much of an imperative it is to have collective, structured discourse in order for human reasoning to evolve (whether wholly computed by AI or not) - even if informally things already operate that way (scientific literature, etc.). Good luck to you! And let me know if you have further refutation to my thoughts above ^
@SagaciousAims Might you be so kind as to link to some of the projects you've referenced? They sound really interesting.
@SagaciousAims Gigabro is our resident troll. We (or at least I) enjoy arguing with him and he is a good sport when he is wrong. I guess he can be a bit aggressive, but I am convinced he is just trying to bait people into betting against him because he is really good at making money on here.
Anyway, love your comments! Keep them coming!
I swear I keep getting front run by john & eli 😭
(like i'm thinking "yeah i should probably bid this down bunch more, let me just double check the doc to make sure i'm not missing something important."
Meanwhile john/eli are just max bidding seconds ahead of me. rip)
@GeorgeVii maybe if i had fewer tabs, it'd load quicker and i'd be swell
@BTE I think the API might be better, but I don't use it myself. I should really throw together a manifold CLI command though if there isn't already one.
@GeorgeVii I have not used a site this slow since high school in the 90s. API presumably doesn't have the latency problems the UI has??
This John guys seems amazing. I can't see him failing at any project.
@JohnSalter Well, Amazing John the Author, lets run you through the gauntlet a bit, shall we?? First, I would love to see your direct responses to the previous naysayers on this thread. Second, I would also like to know how attached you are to the Kialo platform you suggested and whether you are open to alternatives?
I'm the primary author of this application. I will tip anyone who replies to this comment with actionable suggestions to improve the project!
I'm pretty enthusiastic about it and will likely apply elsewhere should we not receive funding this time around!
@JohnSalter Have you put all your mana down on yourself yet or do we need to challenge you??
@JohnSalter I am currently betting with you, but I can think of at least one guy who will take a $1000 challenge bet against you @jbeshir ...
@JohnSalter I like the spirit! Some ideas, I've no knowledge about what Clearer Thinking like but these'd get me to up my probability:
1. Get some initial non-paid buy in from EAs. If there's existing EAs on EAF using it and reviewing it positively, or a bunch of independent EA groups using it for internal discussions regularly and also reviewing it positively, then it becomes much more plausible to me that trying to get wider use will be both successful and positive.
2. Two big classes of debate that I think would be compelling to host:
- "Premises" debates, like "should we donate money overseas or locally". I don't expect these to be debated much by committed EAs, but kept as FAQs they give new arrivals the ability to see that their arguments are made and counted and see the counterarguments against, make new ones if they're not, and some might be persuaded, or at least feel less like "dogma" is going undiscussed.
- "Decisions" debates; a recent example might be "Should EA Global be Open Access", and there's probably a bunch of others possible from the recent critique contest. Because these are around a binary decision, it's less often that the right answer is "here's a better way to think about it" although there's still non-zero of that.
"Should this grant be accepted?" as a debate would be a cute (maybe too-cute, I dunno what people like) way to let the software pitch itself, if it goes to a future round that's not hosted by Manifold, though I don't know if you already suggested this.
3. Getting buy-in from independent EA local chapters, EAF admins, CEA, or any other EAs would both unlock ways to cheaply advocate for wider use, as they're in a position to recommend it to others, and demonstrate that non-selected EAs were finding it compelling, both of which I think would lead to a more compelling application!
4. Least actionable, since it likely requires Kialo to be interested in bespoke work since the software's not open source as far as I can tell(?), but if you can pivot towards iterating on a customised or integrated (requires buy-in from EAF) version like @BTE mentioned based on feedback from an initial paid debate, I expect that could go interesting places.
I'd be up for a challenge bet but I'm out of balance right now and would have to spend a bit going around shaving it off long-term low return markets- I think the 1015 balance may just have been bet anyway though!
@JohnSalter How much usership does Kialo have?
If EAs could use it, why aren't they already? I suggest this is a dev issue rather than content.
@NathanpmYoung One of their subdivisions called Kialo Edu (same thing, but aimed at schools, has 250k+ as of April 2021, having grown 50% since start of year according to their twitter. I'd guess the whole thing has a million or so, but I'm speculating here.
https://twitter.com/KialoEdu/status/1382005591187525633
@JohnSalter Imagine I'm running a Double-crux session with a group. We are writing pen and paper, we are conversing about the cruxes. The session ends. What's the most time-efficient way to save this exchange?
@JohnBuridan store the paper in a drawer? I think you're getting at something that I'm missing, could you please state more explicitly what you mean?
The proposed budget is shoestring and the team is more than qualified to organize debates.
We can hem and haw over the specific strategy and tooling, but the core idea of implementing a community-driven structured debate app within existing the EA community is long overdue. I think $50K to experiment with adding Kialo or a forked version customized for EAF is definitely worth it.
In fact, someone should fund a contest that provides $50K to design new community features for growing/engaging the EA universe.
@BTE Why spend $50k when they could just launch the debate topics they said in their application that they'd want to host...right now?
@SagaciousAims In brief, the attractiveness of this project to EAs is proportional to the number of EAs taking part. It takes a push to get that snowball rolling.
There's a couple of barriers to overcome. If you engage in a debate on Kialo at present, there's a high chance that no EAs will respond. If you try to start a debate, then you'll have to be the one moderating it. Prospective participants have no guarantees that the debates they contribute will be actively moderated a week from now. It just isn't very attractive due to the risk of abandonment / non-engagement. This project solves both of those major hurdles to the user engagement we need to attract further user engagement.
The choice of project manager could make or break this project if funded. There is a shortlist off folks who could lead this project down the right path...
Why not have some goals in each question and get the community to forecast those goals as well. If we fund, will X? If we don't fund, will X?
I think development and pitching of software to try to make debates more effective, even though a lot of a lot of this software doesn't take off, is potentially a high return activity and seems pretty central to the grantor.
However, this particular proposal I'm not as sure on as some of the others. I've no external evidence not in the grant application itself, but this was my take from that:
1. I'm not sure that spending 50k on trying to organise a set piece debate with the software to essentially advertise it to EAs is a good use of a grant.
If the proposal was to adjust or tune the software to meet EAs needs, after gathering a round of feedback or similar, this would make much more sense to me, but it doesn't look clear that Kialo's developers are involved in this pitch (although one of the people involved is an ex-employee).
If it was a broader audience being targeted than just putting forward essentially an alternative to EAF, that would also make sense to me. With just EAs as the target, though, if people are enthusiastic about it, they can presumably linkpost one on the EA forums, and have takeup rise if people find the software genuinely improves their ability to communicate and pleasant to use. If it isn't taken up, throwing 50k at advertising it doesn't seem like a good plan.
2. Their list of proposed debate topics don't seem to me like they're likely to be high expected value, for the debate in itself, being rather value-loaded rather than a place like, say, grants themselves, which benefit from the aggregation of evidence:
Improving the long-run future should be the primary focus of today’s altruists
Could AIs have consciousness?Does AIs being conscious make them more deserving of legal protection?
It’s more important to prevent suffering than to create happiness
(Global priorities debates such as) Biosecurity is a more important long-termist priority than artificial intelligence.
3. In general I don't have high expectations for a model of debate that takes one high level binary question and lists arguments-as-soldiers for and against, rather than trying to arrive at a better model of a domain, without necessarily committing to exactly whether that model makes any particular original statement right or wrong. If you can taboo the word "conscious" in this kind of model, even if it occurred in the top level question, and have the results of that effectively modelled and become part of the collective "answer" if favoured, that would make me more optimistic about this.
There's some usefulness in having essentially an interactive FAQ that has counterarguments to common objections, but I don't think there's huge compounding gain there.
I don't expect it to supplant EAF posts as suggested; the ability to describe a full model, and then for people to take out and make variants on or draw connections to subparts of that model afforded by freeform comments is genuinely useful.
4. It strikes me that a the first set of metrics proposed here for success- number of users, number of comments, etc- are essentially Kialo's commercial metrics. And the second batch- enjoyment, reported better understanding, reported changing mind- are essentially Kialo's consumer satisfction metrics.
My bottom line would be to look at using this for collecting information for/against internal decisions, but not spend 50k on intracommunity advertising.
@jbeshir I think you are misunderstanding what they mean by "advertise". They DO NOT mean "buy ads on reddit, discord, slack, etc." I admit it is a poor choice of words, but nowhere in the full application do they say they are spending a single penny to advertise. All of the money is going to pay several individuals basically bottom dollar ($30ish/hour) to create debates, generate content and promote it in on free forums and social media.
@BTE That's what I understood it as meaning, no confusion there! I'm dubious about a grant to perform paid content generation targeting the EA community in aid of promoting a particular piece of debate software that the applicants think is good, rather than just linkposting to debates using the software that they think will help the community and allowing organic uptake if other people agree- which usually won't be the case, but so it goes.
@NathanpmYoung 's info below that this software has been around for 5 years, is also something I take as a pretty good hint that if it isn't showing signs of organic uptake by now a paid campaign is unlikely to produce lasting uptake. This is pretty typical for debating and idea-mapping software- it's a pretty packed niche which struggles to break out of the niche- so it's not super suprising. I was disappointed that we don't have more use and iteration on Guesstimate, although I think use ticks along still.
I think an example of a grant I'd find more compelling would be one to create a bunch of debates on introductory 101 questions in it that could be linked as references, or similar- I'd still not be sure but it's trying to produce something of immediate value.
@jbeshir Guesstimate is one of the best web apps ever developed!! So useful, but not super (novice) user friendly.
@BTE | I was disappointed that we don't have more use and iteration on Guesstimate
Ozzie & QUIR team are building out Squiggle. Which Future Fund is experimenting with at the moment.
Will be interesting if we can start building out large composable models for cause prioritization etc. & hopefully ppl can start appreciating second&third order effects can drastically change EV, instead of too often being an after-thought
@GeorgeVii I was just reading through the Future Fund grants this afternoon. The simulated outcomes feature in Guesstimate always impresses during presentations, particularly for financial projections. I wish it was easier to plug different models together similar to Yahoo Pipes, probably a familiar reference for those of you who have been using the internet at least since release 2.0.
@jbeshir Before I respond, might you be so kind as to verify that I've correctly summarised your positions
1. If there is no uptake, it will have just been 50k wasted on advertising. That would be terrible, and accordingly grantmakers should hold that against the application. Implicit: the features kialo has at present are not sufficient to meet EA's needs.
2. The debate theses are weak and should be really be changed
3. The nature of a binary debate (yes/no), which are the only ones kialo supports, hinder truth-seeking discussion e.g. deep claims shouldn't have to argue in favour of, or against, the thesis.
4. The proposed metrics seem to be to the benefit of Kialo, rather than actually figuring out whether the platform improves debate within EA.
I really hate how you can't edit comments even one or two seconds after! I meant to include a question mark after my first sentence; this wasn't meant to be a passive-aggressive demand!
5 . Kialo's lack of organic growth since it's launch five years ago implies that paid advertising now won't have much of an effect on it's overall user base now.
@JohnSalter This is a good approach!
For 1, I think that's part of it. I think if you need to pay EAs to participate in it, it's unlikely they will use it while not being paid. It indicates they're not perceiving themselves as getting positive utility from it, and that would mean uptake won't happen and wouldn't necessarily be a good thing to happen.
The other half is that if uptake does happen, then you probably didn't need to pay them to participate in the first place- if it's positive-utility-generating, making a debate and linking it would get participation and produce the kinds of content the grant aims to produce via paid content generation. So the 50k grant isn't needed either way.
Whether something can achieve uptake when offered for free to people is a pretty good test of whether they're getting utility from it. EA is pretty small, so by the time you've gotten that confirmation, it's already going to be pretty visible, and there's better avenues available for further reach depending on what kinds of usage end up popular, e.g. to EA chapter leaders.
I admit some sense of feeling that "this is the kind of thing I'd expect to see, if organic uptake had been attempted, and no one was really interested", but in that case I'd look at why the tool didn't achieve that and what could be improved rather than advertising.
So for 1, it's not so much that it won't achieve uptake as that I think advertising is unlikely to change whether it achieves uptake within EA relative to organic growth. In a larger space like "science" or "poltiics" you might want to do it after seeing uptake of the tool within one part of the space, but within EA once that happens you've probably reached a pretty good fraction of active EAs already. We really only have one major forum.
2 is about right!
3 isn't so much a suspicion this could cause harm as a specific reason to think this tool might not meet needs, my expectation is that if binary yes/no doesn't meet what people want to say the likely outcome is just that people engage less.
For 4, yep! I think it's the least significant of the initial four- I roughly put them in order. It kind of feels like a sales pitch for a piece of software in parts, which pairs poorly with success criteria that kind of boil down to "does the software get sold". I think it might be justifiable by the software being sold probably correlating well with whether it's generating value for people though.
And 5 is about right too, yeah. If it was brand new I would understand lack of popularity as just EAs not having had heard about it yet. It being at least five years old... makes it pretty plausible that it has been encountered by EAs, but @NathanpmYoung 's experience was typical so it didn't get shared much. The .edu growth is an update in the opposite direction, which is good to hear, and growth in a place where the users are more directly choosing to use the software would be better still.
@jbeshir Thank you for taking the time to help me clarify my understanding!
1. Kialo is structurally disadvantaged relative to the forum because of network effects. At present, the EA forum has all of the EAs on it. The attractiveness of a discussion platform to EAs depends on the number of EAs taking part. It will take a push to overcome these network effects, such that Kialo can complete on a level playing field.
On the left is the current situation. On the right is what we hope to bring about if we get funded.
Note: we only want debates specifically to be held on Kialo. We don't see it as a replacement for the forum. We see it as a complementary platform which deals with some discussions better than the forum itself.
@jbeshir 2. We're up for changing the theses, perhaps even having a survey to decide
3. I believe you're mistaken on two fronts
Kialo allows debates with more than two answers. For example, one can have a debate where all of the cause areas are on the top level, and discussion beneath each relates to that specific causes merits and demerits. We should have made this clearer on the application - Apologies!
Deeper levels do NOT have to argue in favour of, or against, the debates thesis. They relate specifically to the claim above. This allows people to object to parts of a model, and suggest better models. For example, if we're discussing the merit of Lemarkian evolution, someone can post a "con" beneath stating that it's inferior to Darwinian evolution because X, Y, Z. That con can link to a description of Darwinian evolution, and beneath that con a discussion can take place that exclusively discusses the merits of darwinian evolution (without needing to be in favor of, or arguing against, Lemarkian evolution.
4. I feel this is a good thing, it makes gathering the information quick and easy. However, we agree it's not measuring the central purpose of the project. Kialo has features that let's you track how beliefs change over time. We'd like to make use of that to better measure whether engagement with our debates causes beliefs to change / become more nuanced.
5. Please refer to answer 1 for to see why I think it hasn't appealed to EAs just yet (chicken and egg problem). I'll add one thing though - Depressingly, the niche for rational discussion software is very small. I believe Kialo has captured a healthy junk, if not most of it, despite being a charity which can't generate revenue from users (and thus fuel it's own growth via aggressive advertising which pays for itself).
@JohnSalter Under 5 you're referring to Kialo as a charity- I think it's not a super key point since getting better use of for profit technology is good too (implications around whether EA customisation is available down the line aside), but can I check if that's correct?
https://www.crunchbase.com/organization/kialo and https://www.ft.com/content/4c19005c-ff5f-11e7-9e12-af73e8db3c71 describe it as a for profit, maybe Crunchbase is out of date?
@jbeshir I had assumed it was because it doesn't charge, has no ads (that I've seen), and doesn't sell data. I did a little digging and can't find any information. I retract my previous comment. I now believe that it is for-profit, although probably acting as a social enterprise in practice.
I used Kialo ~5 years ago and honestly it just wasn't that engaging. As someone who likes understanding debates a lot, that's pretty damning for me. Maybe it's improved since then.
@NathanpmYoung I am not suggesting Kialo is THE solution to the problem they are addressing. Typically software doesn't do the engagement part, that is on the community, and EA is certainly an enthusiastic and engaged community. However, EA is also largely an echo chamber with a strict hierarchy of leaders that should be known (if it isn't already) as the 'EA Mafia'. Getting any of the made men in the community to engage with you on subcomponents of complicated essays, or (gasp) having the audacity to challenge the premises put forth by the bosses of the EA Mafia, is a waste of time unless you have a fetish for being mansplained to. Questioning EA Dogma is met with ridicule leaving no room for debate at all. Kialo clearly isn't going to get much traction in EA as a destination, but the set of features it offers could be used (even here on Manifold hypothetically) in ways that could be very useful.
@NathanpmYoung Interesting! Can you tell us more about why you didn't find it engaging?
Been playing around with the app mentioned in the application and it is really cool. Tools like this would be extremely helpful to folks who are getting their first exposure to the EA community, especially when exploring many of the more esoteric AI Alignment conversations on Less Wrong or AI Alignment Forum. It can be intimidating to newbies to speak up in such a swarm-like ecosystem, so providing as much structured and interactive knowledge as possible would make EA a much more accessible and sticky movement.
.jpg%3Falt%3Dmedia%26token%3Dfb83f18b-ca71-49db-8b32-7b23aa4cf70a&w=64&q=75){kind=small}
.jpg%3Falt%3Dmedia%26token%3Dfb83f18b-ca71-49db-8b32-7b23aa4cf70a&w=96&q=75){kind=small}
