Loading

Lunaria: An Open Call for Pilot Testers

Lunaria predicts the patterns in survey responses from the item wording alone, as accurately as collecting data from 71 participants. We are looking for organizations to test it during the beta.

BEH Dr. Björn E. Hommel
Dr. Björn E. Hommel Senior Scientist

magnolia psychometrics is looking for organizations to pilot test Lunaria, our first commercial model for higher accuracy and more precise decision-making in assessment development, thanks to explainable features.

Lunaria predicts how strongly people’s answers to two survey items will correlate, from the item wording alone. It needs no response data. From the same text it predicts correlations between scales and their reliability (Cronbach’s α).

Use cases

Lunaria answers your empirical questions without collecting data. Its predictions miss the observed correlations on average by about as much as a survey of 71 participants misses them through sampling error alone. Instead of surveying 71 people, you get an answer instantly, at a fraction of the cost. Before acting on an important finding, we still recommend confirming it with empirical data.

The same predictions support instrument development. Finding items that fit poorly with their scale or duplicate another item, building short forms that keep a target reliability, and spotting scales that measure the same construct under different names.

Validation

We compared Lunaria with several models on a survey held-out during training, and report the previously leading one here, SurveyBot3000.[1] Across 30,008 item pairs, Lunaria’s predictions correlate at r = .79 with the observed correlations, compared with r = .57 for SurveyBot3000 (Figure 1).

Item pairs
Lunaria
r = .79, RMSE = .13
SurveyBot3000
r = .57, RMSE = .17
Scale pairs
Lunaria
r = .88, RMSE = .15
Cronbach’s α
Real scales
Random scales
Lunaria, all scales
r = .92, RMSE = .20
Figure 1. Predicted and observed values in a held-out survey. Correlations of item pairs (3,000 of 30,008 pairs shown) and scale pairs, and Cronbach’s α of real and random scales. Points are Lunaria’s predictions, the solid lines least-squares fits of the observed on the predicted values, and the dotted diagonal marks a perfect prediction. r and RMSE are computed on all data.

Explainable features

Unlike earlier AI models that predict correlations between survey items, Lunaria not only estimates them more accurately but also explains why two items or scales correlate. It decomposes each prediction into feature categories: construct alignment, general similarity, response statistics, ratings of the wording, content categories and linguistic features. Figure 2 shows a selection of these features for two items about voting.

A I usually vote for the conservative party in every election.
B I have voted for the democrats today.
Ratings
Social desirability
A 2.9, B 3.2
Sentiment
A 3.1, B 3.3
Stability
A 4.3, B 2.0
Abstraction
A 2.6, B 1.5
Expected answers
Mean response
A 39%, B 45%
Topic
Politics
A .88, B .76
Relations
A .00, B .00
Work
A .00, B .00
Consumer behavior
A .02, B .02
Leisure
A .05, B .12
Health
A .00, B .07
Personality
A .00, B .00
Lifestyle
A .03, B .00
Other
A .03, B .03
Psychological content
Affect
A .03, B .03
Behavior
A .73, B .77
Cognition
A .22, B .20
Desire
A .00, B .00
None
A .01, B .00
Statement type
Evaluative belief
A .03, B .02
Evaluation
A .04, B .02
Judgment
A .04, B .00
Feeling
A .00, B .03
Importance
A .00, B .00
Preference
A .01, B .12
Relationship
A .04, B .07
Norm
A .02, B .00
Policy
A .00, B .00
Right
A .00, B .04
Expectation
A .01, B .02
Action tendency
A .04, B .00
Behavior
A .78, B .69
Figure 2. A selection of Lunaria’s features for two items. Ratings of the wording and the expected mean answer (top), and the probability of every class in the three content categories (bottom). Lunaria v0 (beta).

Both items concern voting, from opposite sides of the political spectrum. Item A describes a lasting habit, item B a single act. Accordingly, Lunaria rates A as a stable trait and B as a passing state, and B as more concrete. Lunaria attempts to tell apart two questions that one correlation blends together. Construct alignment says whether two items address the same construct, and from which end. The predicted correlation says how strongly their answers will relate. Two items can be clearly aligned and still correlate only modestly.

Pilot testing

Lunaria v0 is in beta. We are looking for organizations that develop or maintain assessments and want to test it on their own items. Pilot partners get early access with generous rate limits for a pilot phase of about three months, and help shape the model with their feedback.

Lunaria’s predictions of empirical correlations and reliabilities are well tested and robust. Some of the explainable features are still experimental, and feedback from the pilot phase will help refine them.

Registration is open until the end of February 2027. To take part, leave your details below.

Registration for the pilot phase closed at the end of February 2027. For questions, email us at hello@magnolia-pm.com
Please fill in this field
Please fill in this field Please enter a valid email address
Please fill in this field
Please fill in this field
Thank you. We will get back to you about the pilot phase.

References

  1. Hommel, B. E., & Arslan, R. C. (2025). Language models accurately infer correlations between psychological items and scales from text alone. Advances in Methods and Practices in Psychological Science, 8(4), 25152459251377093. https://doi.org/10.1177/25152459251377093

Author
BEH Dr. Björn E. Hommel
Dr. Björn E. Hommel Senior Scientist

Björn has worked in psychological assessment for more than a decade, in both practice and research. In his academic work, he studies how language models and AI can improve the quality of psychological measurement. At magnolia psychometrics, he builds these methods into practical tools that help organizations conduct more reliable, data-driven assessments.

Cite as

Hommel, B. E. (2026, November 22). Lunaria: An open call for pilot testers. magnolia psychometrics. https://magnolia-psychometrics.com/blog/lunaria-pilot-testers

@misc{Hommel2026Lunaria,
    author       = {Hommel, Bj{\"o}rn E.},
    title        = {{Lunaria: An Open Call for Pilot Testers}},
    howpublished = {magnolia psychometrics},
    year         = 2026,
    month        = nov,
    url          = {https://magnolia-psychometrics.com/blog/lunaria-pilot-testers},
    note         = {Blog post}
}
Share
  1. Synthometricon: A Search Engine for Navigating Assessment Space

    With the release of Synthometricon, researchers and practitioners get a single web app to search tens of thousands of questionnaires, surveys and tests, whether the goal is to find an established scale or to see where a new item pool repeats existing work.

    Project Announcement Oct 14, 2026 1 min read

  2. Career Atlas: Mapping Career Options from Self-Description

    The Career Atlas is a free web app for career orientation. Users describe their interests, strengths and occupational preferences in their own words, and a language model places more than 3,200 occupations on a personal map of options, as a starting point for self-exploration and career counseling.

    Project Announcement Jan 29, 2025 5 min read

Diese Seite gibt es auch auf Deutsch. Auf Deutsch lesen

No tracking cookies, no cookie banner. Our statistics are anonymous and stay on our own servers in Germany. Privacy policy