magnolia psychometrics is looking for organizations to pilot test Lunaria, our first commercial model for higher accuracy and more precise decision-making in assessment development, thanks to explainable features.
Lunaria predicts how strongly people’s answers to two survey items will correlate, from the item wording alone. It needs no response data. From the same text it predicts correlations between scales and their reliability (Cronbach’s α).
Use cases
Lunaria answers your empirical questions without collecting data. Its predictions miss the observed correlations on average by about as much as a survey of 71 participants misses them through sampling error alone. Instead of surveying 71 people, you get an answer instantly, at a fraction of the cost. Before acting on an important finding, we still recommend confirming it with empirical data.
The same predictions support instrument development. Finding items that fit poorly with their scale or duplicate another item, building short forms that keep a target reliability, and spotting scales that measure the same construct under different names.
Validation
We compared Lunaria with several models on a survey held-out during training, and report the previously leading one here, SurveyBot3000.[1] Across 30,008 item pairs, Lunaria’s predictions correlate at r = .79 with the observed correlations, compared with r = .57 for SurveyBot3000 (Figure 1).
Explainable features
Unlike earlier AI models that predict correlations between survey items, Lunaria not only estimates them more accurately but also explains why two items or scales correlate. It decomposes each prediction into feature categories: construct alignment, general similarity, response statistics, ratings of the wording, content categories and linguistic features. Figure 2 shows a selection of these features for two items about voting.
Both items concern voting, from opposite sides of the political spectrum. Item A describes a lasting habit, item B a single act. Accordingly, Lunaria rates A as a stable trait and B as a passing state, and B as more concrete. Lunaria attempts to tell apart two questions that one correlation blends together. Construct alignment says whether two items address the same construct, and from which end. The predicted correlation says how strongly their answers will relate. Two items can be clearly aligned and still correlate only modestly.
Pilot testing
Lunaria v0 is in beta. We are looking for organizations that develop or maintain assessments and want to test it on their own items. Pilot partners get early access with generous rate limits for a pilot phase of about three months, and help shape the model with their feedback.
Lunaria’s predictions of empirical correlations and reliabilities are well tested and robust. Some of the explainable features are still experimental, and feedback from the pilot phase will help refine them.
Registration is open until the end of February 2027. To take part, leave your details below.