Sometimes experiments do not work out quite the way we expect. The study drugs experiment is one of those cases. It was fraught with problems, from shipping errors to technical problems to all-out hardware failures.
I do not consider the experiment to be a failure by any means. I did manage to collect at least two weeks of solid data to be analyzed, though I only managed to cover one of the three drugs I intended to test. I do plan to come back and finish testing the rest of the drugs at some point, hopefully starting the trial from scratch, but for now I’d like to share what results I was able to gather. I think you’ll agree that the data is quite interesting. You can read the hypothesis, examine the raw data, skip to the compiled data, consider some observations, or just jump to the conclusions… or, most importantly, what it means for you!
Free from Skill Cookbook
An Evidence-Based Approach to Self Improvement, available from Amazon and on the Kindle store.
The Joy of Craft
How to improve focus, practice better, stay motivated, and everything else you could want to know about better studying.
In order to formulate a conclusion at the end of this post, we first need a null hypothesis. In this case, I shall state that:
The use of the drug Huperzine A will produce a 5% (or larger) increase in attention, meditation, or ease (recall) values.
In other words, if any of these 3 values increases by more than 5%, it will be considered as having a significant effect. 5% is generally accepted as a reasonable value for statistical significance, so it seemed like an appropriate value to use here.
You can check out the Google Document which contains all my data here; the graph below shows the raw data over time. One important note is that the “sleep time” is measured in 10th of an hour (ex, 60 = 6 hours). I did this in order to make the 0-100 scale used for the rest of the data be applicable here as well. For information about what meditation and attention mean, there is more information in this post (click to enlarge).
As a point of interest, here are the number of cards (words) in my Anki deck over time:
The following chart is an average of the values for the drug and for the placebo, in order to be able to compare them against each other:
Because it is a bit hard to see the minute differences in the above chart, here is a chart which examines the offsets of each value from the overall average:
I had wondered if the Meditation and Attention values provided by my EEG device would reflect upon my learning. In other words, if the EEG showed that I was paying more attention, did it translate to better results? Out of curiosity I created the following chart, which plots the meditation and attention values (as well as an average of the two) against the ease on the next day. The intention here is to demonstrate the effect of focusing harder while studying on future recall:
I’ll leave you to decide for yourself if you believe this demonstrates a strong correlation between attention/meditation and the ability to learn information. I would like to return to this question at a later point, but I’ll leave you with this link to the white papers written by NeuroSky about what meditation/attention values actually mean in terms of brain activity.
So: the big question! Was the change in meditation / attention / ease from the use of Huperzine A enough to disprove our null hypothesis?
The way to read this chart is:
- There was a 4.5% decrease in attention with Huperzine A as compared to the placebo
- There was a 11.7% increase in meditation with Huperzine A as compared to the placebo
- There was a 3.3% decrease in ease with Huperzine A as compared to the placebo
As you can see, the meditation and ease values fell well within the 5% range set out by out null hypothesis. In fact, I actually experienced a decline in ease and attention! We could say that the null hypothesis was disproved for these two properties. However it seems to me that the giant spike in meditation is suspect, making me believe that there is insufficient data to draw a full conclusion.
In short, I was unable to demonstrate a significant change with huperzine A in 2/3 meaningful categories, but still feel that more data over a longer period is warranted.
What It Means for You!
There is lot of data in this post. I hope that you found it useful… but what does it mean for you?
Well, as I have already stated before, I am generally opposed to taking drugs unless it is necessary. Even if the null hypothesis is not ultimately disproved… would a 5% increase in your ability to remember things really be worth it? Could that same benefit not simply be achieved by studying a little harder? Keep in mind, most nootropic drugs are primarily meant for long-term brain health – the application of them for purposes like studying a foreign language is a result of companies using Alzheimer’s’ research to create a product marketed at students. My objective in this experiment was to determine if this was anything more than marketing hype, and I believe I have demonstrated that the drug does not show a significant change (at least for my purposes of learning a language, when it comes to Huperzine A). The companies who make other nootropic drugs will no doubt claim that it is their specific formula that makes a difference, but the fact is that most of the research they cite/use is based upon one or two key ingredients (and this almost always includes Huperzine A). But, again, this trial could have been much longer in order to be more complete.
What do you think? Is this even a worthwhile question? Should I be testing other drugs, or shrug it off and say that it is not worth it? I already have a few more experiments in progress that seem much more promising (more details coming soon!), but I do not necessarily want to give this one up. Let me know your thoughts below!