Randomised trials have suggested that L-Theanine meaningfully reduces the anxiety-inducing effects of caffeine, while maintaining its cognitive benefits.
I will be undertaking a blinded trial with L-Theanine using the following protocol:
1. Take 210mg L-Theanine or a placebo (blinded) just before my morning small iced coffee.
2. 90 mins after coffee consumption, I will record my anxiety level as a subjective measurement between 0-10.
3. Repeat for 20 days during which I will engage in sustained periods of focused work.
(Protocol taken from here, which has sources: https://n1.tools/experiments/anxiety/lTheanine)
Resolution criteria: Resolves YES if a naive model (i.e. that doesn't include confounders etc.) suggests at least an 80% probability that consuming L-Theanine reduces anxiety.
In other words: "Compared to placebo, a difference in anxiety between -xx% and 0% is at least 80% likely".
For reference, my previous market on whether L-Tyrosine improves focus used 20 data points and resulted in mean focus (placebo) of 7.4 and mean focus (L-Tyrosine) of 7.0. The model suggested that "Compared to placebo, a difference in focus between 0% and 30% is 19.4% likely". Image from that experiment below.
Extra notes:
- Starting this experiment today.
- I may visit different cafes, but I'll always order a small iced coffee and try my best to consume about the same amount of caffeine each day (without being too strict).
- I won't bet on this market.
- I am sensitive to caffeine and do experience quite pronounced caffeine highs and lows.
- Dosage is 210mg because the brand I use from Amazon provides 210mg capsules (they suggest two capsules per dose).
----------------------------------
Model details (for those interested)
I'll model the posterior distribution of anxiety values with the placebo and with L-Theanine, and then merge those to create a posterior distribution for the % difference in anxiety between the two. I'll be using a Bayesian model for this, which requires a prior to update. That prior will be the mean anxiety for the placebo and L-Theanine from the data itself (which is informative, and means the data will basically define the posterior distribution).
Priors:
Mean L-Theanine ~ Normal(mean_data, std_data)
Mean Placebo ~ Normal(mean_data, std_data)
StdDev L-Theanine ~ HalfNormal(5)
StdDev Placebo ~ HalfNormal(5)
Likelihoods:
Data L-Theanine ~ Normal(Mean L-Theanine, StdDev L-Theanine)
Data Placebo ~ Normal(Mean Placebo, StdDev Placebo)
Deterministic Transformation for Percentage Difference:
Percent Difference = ((Mean L-Theanine - Mean Placebo) / Mean Placebo) * 100
Posterior Inference:
Probability(Percent Difference ≤ 0) ≥ 0.80
1,000🏅 Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | Ṁ30 | |
| 2 | Ṁ23 | |
| 3 | Ṁ13 | |
| 4 | Ṁ13 | |
| 5 | Ṁ12 |
Why are the standard deviations determined according to random variables with constant parameters? (And why 5?) Shouldn't it depend on the actual distribution?
I don't understand how it's possible to get a posterior of 83% with a p-value of only 0.31 although I don't really understand your math.
Also I think you should be using t-distributions, not normal distributions.
@MichaelDickens For this market I wanted to make the resolution criteria clear - but I'm by no means an expert and welcome improvements!
1) The HalfNormal(5) prior says "I don’t expect the day-to-day standard deviation of my 0–10 anxiety scores to exceed five points".
That's intuitive to me and doesn't force a completely uncertain prior and IIRC I took this approach after seeing how analyses like this are typically done - but I can see how using 5 here might be arbitrary.
2) Since there are only 20 data points I don't expect p-value to get anywhere near the typical p<0.05 threshold, but given the weak prior I'd expect the posterior to shift if the effect is large enough (mean anxiety of 5.0 w/o Theanine vs 4.2 w/Theanine seems like a big enough effect).
3) I kept it to a normal distribution for simplicity, for a t-distribution I'd have to give an opinion on how tailed I expect the data to be. I can see the case for adding that prior since I'm only collecting 20 data points, but I wanted to avoid the extra assumption.
@LuisCostigan Having a normal prior is fine, what I should have said is that the data should follow a t-distribution, with 10 degrees of freedom for the theanine group and 8 for the placebo group.
FWIW I re-tested your data with a uniform prior and got pretty much the same posterior probability.
After thinking about this some more, I think the biggest problem with this methodology is that a continuous prior has zero probability mass on the effect size being zero. But in real life, many effects are genuinely zero. (Like, most arbitrary things you could eat don't have any effect on anxiety.) So I don't think the 83% posterior is realistic—much (or most?) of that 83% probability is pretty close to 0, and really should be represented as exactly zero. Adjusting for this, the probability of a positive effect is considerably less than 83%, although I don't know how to formally adjust for this.
The way I would do this analysis, although I don't know if this is the best way, is to calculate the likelihood ratio: P(mean diff = 0.8 | true diff = 0.8) / P(mean diff = 0.8 | true diff = 0). Which in this case equals 1.77. So the experiment is 1.77:1 evidence in favor of the observed result over the null hypothesis. If your prior was 50% (= 1:1 odds) then the posterior is 1.77:1 = 64%.
My method is maybe not ideal because it's looking at the evidence that the mean is exactly 0.8, rather than that the mean is positive* at all. But I am not sure of a good way to do the latter.
I believe the likelihood ratio for "the mean is positive", if calculated correctly, must be less than 1.77:1, because 0.8 is the maximum-likelihood value. Any other positive value has a lower likelihood.
I am probably overthinking it at this point but I did a Bayesian analysis where the prior has 50% of the mass at 0, and 50% of the mass following a normal distribution with mean 0 and SD 5. I am not entirely sure I did the math right, but the experimental evidence appears to argue in favor of a zero effect, with the posterior being: 3% chance of negative effect; 79% chance of zero effect; 18% chance of positive effect.
The intuition behind this result is that the prior distribution expects a large positive effect conditional on there being a positive effect, but in fact there was only a small positive effect.
5 SD seems unrealistically large—it implies that just under 5% of interventions would change someone from zero anxious to maximum anxious or vice versa. So I tried setting the prior SD to equal 1 and got: 10% chance of negative effect; 53% chance of zero effect; 37% chance of positive effect. So even in this case, it's evidence in favor of the effect being zero. Although at least in this case the observation also increased the probability of a positive effect (the prior is 25% negative, 50% zero, 25% positive).
Seems like L-Theanine works to reduce post-coffee jitters!
Mean post-coffee anxiety with L-Theanine: 4.2
Mean post-coffee anxiety without L-Theanine: 5.0
82.6% chance of an anxiety reducing effect (honestly higher than I expected).
(Note: I updated my app to use Absolute Difference on the KDE plot x-axis but for the resolution of this market quickly switched the analysis back to Percent Difference, so ignore the x-axis label)
I bet NO because I thought 83% chance of effect was pretty high. I think there’s more than a 17% chance of no effect, let alone potentially negative effects or different kinds of anxiety. I’ve heard anecdata about L-theanine giving a different sort of anxiety that is still edgy despite being less jittery than caffeine, so increased total anxiety also seems possible to me.