Planets & Me

How we are testing it

The front page tells you what we are doing and why. This one is the working out.

It is longer, it is more technical, and nothing on it is written to persuade you. Every number here was counted out of the project’s own files when this page was built, and the ones that count against us are set in the same size as the ones that do not.

What is actually new here

We wrote down what would change our minds, before we looked.

Every astrology service says its readings are accurate. None of them can show you the test. We think that is the whole problem, so we did the opposite thing first: a pre-registered specification, frozen and timestamped, that states the hypotheses, the statistical tests, and — in its own section — what result we would accept as a refutation. We committed to publishing the outcome either way, including a null one, and we expect a null one.

The published literature on astrological validity is almost entirely negative. Carlson in Nature in 1985 was null. Dean and Kelly's 2,100 "time twins" in 2003 were null, and that one is worse for us than the sentence makes it sound — we set out why below, because it is the most adverse thing in the literature for the study we care most about. The Gauquelin Mars effect did not replicate. We treat the prior probability of a large predictive effect as low, and we designed the studies so their main deliverables survive a null result: an executable, source-cited formalisation of the rules, a measurement of how often the tradition contradicts itself, and a validated chart engine, which we have committed to publishing as open source when the results are published.

Three times a figure of ours turned out to be measuring us

This is not a hypothetical risk and we are not going to be coy about it. Three times a number we reported turned out to be measuring our own encoding rather than the texts. The worst of them arrived looking like a confirmed hypothesis at p = 0.0004; the effect vanished when we removed pairs that our own chapter structure had made mutually exclusive, and on a later pass its sign flipped while the p-value stayed impressive. That hypothesis is now suspended as uncomputable on our corpus, and it is suspended in public.

Each time, the fix was the same and it is the habit the product is built on: where a quantity is our choice rather than the tradition's, it is declared in the data — not in a footnote — and the tests refuse to let the figure be quoted without the declaration travelling with it.

What the toggle on a sample report opens

Every figure that produced the reading.

The front of a report is the reading. Behind the toggle is what made it: the six Shadbala components for each planet, every rule that matched with its source and its standing, the combinations that were never tested with the reason code for each one, the four readings of the two open textual questions, and the provenance of every parameter in the scores.

That includes the thirteen strength scores, each ranked against a hundred thousand synthetic charts — so a score is not a number on its own but a position in a distribution we generated and can show you. Both figures are counted out of the repository below, like every other number on this page.

Open a sample report and turn it on. The birth is invented; the workings are real.

Counted from the repository at build time

What has been built

Not one number on this page was typed in by hand. Each was counted out of the project's own files when the page was built, and each says underneath it how it was counted.

One set of figures further down is not a count and says so where it appears: the Dean and Kelly measurement is a simulation we ran. It carries the command that reproduces it, so you can check it the way you would check a count — by running it yourself.

    And the figures that do us no favours

    These are the same size as the ones above, in the same list format, for a reason. A page that shows you the flattering counts and buries the rest has told you nothing you can use.

      The strongest null in the field is a null on our own instrument

      Dean and Kelly drew their time twins from every birth in Great Britain in a single week — 3 to 9 March 1958. One week is near-perfect control for everything that separates one cohort from another. It also removes most of what a chart is made of, and we measured how much.

      • Six of the nine grahas take exactly one sign across that entire cohort. The Sun, Venus, Jupiter, Saturn, Rahu and Ketu are constants in it. Their dignity, and every combination resting on them, is untestable on that sample at any size and by any method.
      • The dasa lord at birth keeps 96.3% of its variance, because the Vimshottari sequence starts from the Moon’s nakshatra, and that still takes nine values inside a week. The ascendant keeps all of its variance.

      So the most-cited null in the field is a null on the feature our dasa timeline is built from. Not a general null about astrology that we can hold at arm’s length — a null on our own instrument. It is the single most adverse fact in the prior literature for what we are about to do, and we would rather put it here than wait for somebody else to find it.

      One step in that is an inference, and we would rather mark it than let it pass as a fact. Dean and Kelly tested whether time twins resemble each other across a hundred-odd outcome variables; they did not test whether a named feature predicts anything. What distinguishes a time twin from another child born that week is the ascendant and the Moon, and the dasa lord travels with the Moon — so the contrast they measured is carried by those features. We think the chain holds. It is still a chain, and you should see where the join is.

      The same measurement bounds much less about the slow grahas, which are the constants in that week. That is true, and it is the kind of true thing that makes a convenient excuse, so we are not going to lean on it: it does not soften the paragraph above.

      Measured, not counted. Unlike every figure above, this one comes from a simulation rather than a count of our files. Reproduce it from a checkout of the project — after pip install -e . and scripts/fetch_ephemeris.sh, because the ephemeris data is licensed by Astrodienst and is not ours to ship — with python3 -I sim/cohort_confound.py -n 6000 --perms 3 --cohort-window 1958-03-03:1958-03-09 --window-place 51.5,-0.13 — the full table and the reasoning are in docs/study-4-design.md, section 2.6.

      On the register of the material

      The dasa-phala chapters are not wellness literature. They speak of disease, loss, imprisonment and death alongside gain and honour, and where we show you what they say we show it in the schema's own words rather than in softer ones. We have not decided how plainly to put the adverse material in the product, and we would rather say that than pretend the question does not exist.