Evals Hub

This page organizes Dramatica evaluations as benchmark cards so researchers and studio technical teams can inspect signal quality quickly.

Framing

Each benchmark card is reported with:

Constraints Layer

Benchmark Card: Storyform Constraint Compliance

Benchmark Card: Throughline Role Integrity

Alignment Layer

Finding: StoryBeat Alignment Across Domains

Dramatica evaluates StoryBeat generation for structural alignment, not just style or plausibility. A generated StoryBeat should illustrate the requested Throughline, Domain, Signpost, and scope while still reading as usable storytelling.

In internal evaluation runs, Objective Story Signposts passed StoryBeat alignment checks across all four Domains. Each run was tested both without extra notes and with ordinary writer-provided ideas, such as a concrete event, discovery, split, or reversal that needed to be preserved inside the Storyform.

Objective Story Domain Signpost Sequence Result
Physics Understanding, Learning, Doing, Obtaining Passed internal alignment evaluation
Psychology Conceptualizing, Being, Becoming, Conceiving Passed internal alignment evaluation
Universe Past, Progress, Future, Present Passed internal alignment evaluation
Mind Memory, Preconscious, Subconscious, Conscious Passed internal alignment evaluation

This matters because a StoryBeat can sound dramatic while still doing the wrong structural job. Alignment checks help separate "this reads like a scene" from "this beat is carrying the intended Storyform pressure."

Writer notes remain creative input. A note like "they escape" or "the family splits in two" is treated as story material, not as a replacement for the Storyform. The alignment pass preserves the idea while checking whether the resulting beat still performs the correct narrative function.

Writing use: When reviewing generated Storybeats, ask whether the beat changes the story through the intended structural pressure, not merely whether it mentions the right Dramatica term.

Benchmark Card: Throughline Voice Classification

Benchmark Card: Dynamic Consistency

Benchmark Card: Storybeat Advancement

Quality and Diversity Layer

Benchmark Card: Diverse Valid Candidates

Benchmark Card: Human Selection Lift

Pipeline summary

Storyform Spec -> Generator -> Dramatica Verifier (constraints + scores) -> Best-of-N selection -> Human choice

The verifier narrows outputs to meaning-correct candidates. Human reviewers choose voice and taste.