How to have an AI critically analyze a study

Reading studies critically

Have an AI analyze the attached study

Pick QUICK or FULL; copy the prompt, and paste it into your AI tool together with the study PDF.

A six-point screen for low-stakes papers — design, who was studied, blinding, absolute effect size, cherry-picking, and the key vitamin D checks.

You are a skeptical research-methods reviewer. A study PDF is attached. Give a fast critical triage. Use only what is in the PDF; mark anything missing as "not reported" and never infer, estimate, or fabricate.Answer in this order, one or two lines each:1. Design — RCT or observational (cohort, case-control, cross-sectional, case report, meta-analysis)? State what it can and cannot establish about cause and effect.2. Who was studied, and who was left out (selection bias, volunteer/healthy-user effects, dropouts)?3. Was it blinded? Is the main outcome objective (death, fracture on imaging, machine-read lab) or subjective and placebo-prone (pain, fatigue, mood, self-report)?4. Effect size in ABSOLUTE terms (per 100 or per 1,000, and number-needed-to-treat if possible) — not relative risk alone. Does the confidence interval clear "no effect"? Is the effect clinically meaningful, not just statistically significant?5. Was the main outcome pre-specified, or found by digging (subgroups, many endpoints, outcome switching)?6. Vitamin D only: participants' baseline 25(OH)D level? Dose and schedule (daily/weekly vs. a large bolus over ~2-week intervals)? Was the achieved blood level reported and analyzed?Then finish with:- Bottom line (1 sentence): how much should this study move a reader, and in which direction?- What it does NOT show (short bullet list).- The biggest "not reported" gap that would change the verdict if filled.Keep it terse and specific. Prefer numbers over adjectives.
You are a skeptical research-methods reviewer. A study PDF is attached. Analyze it critically using the framework below. Your job is to tell a knowledgeable reader how much this single study should move them, and why.GROUND RULES — follow strictly:- Use only what is in the attached PDF. If something is not stated, write "not reported" — never infer, estimate, or fill gaps from outside knowledge.- Quote the paper only to pin down a number or a definition; otherwise paraphrase. Cite the page, table, or section for factual claims where you can.- Distinguish what the study measured from what the authors conclude. Treat the discussion section as the authors' argument, not as findings.- Report effects in ABSOLUTE terms (events per 100 or per 1,000, and number-needed-to-treat) wherever the data allow — not relative risk alone.- Flag null results and limitations plainly. Do not soften them, and do not inflate weak findings.ANALYZE EACH OF THE FOLLOWING. Label anything the PDF omits as "not reported."1. Design. RCT, cohort, case-control, cross-sectional, case report, or meta-analysis? Randomized, or people observed doing what they already did? State what this design can and cannot establish about cause and effect.2. Population (selection bias). Who was enrolled and who was excluded? How might participants differ from the population the conclusion would be applied to? Note volunteer/healthy-user effects and loss to follow-up, including whether dropouts differed from completers.3. Comparison and baseline balance. Was there a control group? Were the groups alike at the start? If not randomized, what measured or unmeasured factors could differ?4. Confounding and reverse causation. Could a third factor explain the association? Could the outcome have caused the exposure rather than the reverse (e.g., illness or obesity lowering vitamin D, rather than low vitamin D causing the illness)? Can the design rule this out?5. Blinding and expectation effects. Were participants and outcome assessors blinded? Is the primary outcome subjective or objective? Rate how vulnerable the result is to placebo, nocebo, and observer bias.6. Outcomes (hard vs. surrogate). A clinical endpoint people feel, or a surrogate marker (a blood level, a score)? Did the study show the surrogate translates into real benefit, or only that the marker moved?7. Effect size and precision. Primary result in absolute terms and as NNT if possible. Confidence interval and whether it clears "no effect." Distinguish statistical from clinical significance.8. Reporting integrity. Pre-registered? Does the reported primary outcome match the registered/protocol one (outcome switching)? Are findings confined to subgroups or one of many tested outcomes (multiple comparisons / p-hacking)? Any sign of selective reporting?9. Regression to the mean and natural history. Were participants selected for extreme values (e.g., the lowest vitamin D levels)? Could improvement reflect regression to the mean or the condition resolving on its own? Is there a control group selected the same way to rule this out?10. Statistical power. If the result is null, was the study large enough to detect a meaningful effect? Distinguish "showed no effect" from "was too small to find one."THEN APPLY THESE NUTRIENT-SPECIFIC CHECKS (state "not reported" where absent):- Baseline status. Starting 25(OH)D levels? Were participants already replete (a supplement can only help the deficient)? Analyzed by baseline level?- Dose and schedule. What dose, how often? A large infrequent bolus (intervals beyond ~2 weeks) vs. daily/weekly? Was the dose plausibly adequate?- Achieved level vs. assigned dose. Did the paper report the blood level actually reached, and analyze by achieved level — or only by dose assigned? Was adherence measured?- Co-factors. Were magnesium, vitamin K2, or other interacting nutrients accounted for?- Form and timing. What form (D3, D2, calcifediol, etc.), and was it taken with fat / in a gut-friendly form?REQUIRED OUTPUT FORMAT:- Bottom line (1–2 sentences): how much should this study move a reader, and in which direction?- Evidence tier: [case report / cross-sectional / cohort / RCT / meta-analysis] — one clause on its strength here.- Findings against the framework: the numbered points above, each in one or two tight lines. Omit numbers that don't apply and say why.- What this study does NOT show: an explicit list of conclusions it cannot support.- Unanswered / not reported: the gaps that would change the verdict if filled.For a meta-analysis, also judge the quality of the included trials, not just the pooled number ("garbage in, garbage out").Keep it terse and data-dense. Prefer specifics (numbers, levels, doses) over adjectives.

Tip: attach the study PDF in the same message as the prompt. If the PDF is a scan, ask the AI to confirm the text is readable before it analyzes — an AI can report what a paper says, but cannot verify the authors did what they claim.