# Diagnostics and Grounding

*Teaching errors, source locations, the rendered program, the ground
program, and where a grounding's size comes from.*

When something goes wrong — a rule refused, a surprise UNSAT, a grounding
that takes forever — this page is the toolbox. Half of it is errors that
come to you: validation at construction, diagnostics mapped back to your
Python source. The other half is surfaces you go to: the rendered ASP, the
ground program exactly as clasp receives it, and a profile of where the
grounding's size comes from.

## Errors that teach

Rules are validated as they are constructed, so errors land on the Python
line at fault — not at solve time, three files away — and every message says
what you probably meant instead. The net is deliberately one-sided: every
refusal is a certain clingo rejection or a near-certain mistake, so valid
programs never fight the validator. The deliberate exclusions, and the
reasoning behind each, live in [What We Don't Support](unsupported.md).

Here is one, fired on purpose. We build a small scheduling program, then
try to state a "fact" that still contains a variable:

```python
from aspalchemy import ASPProgram, Choice, Field, Predicate, Variable

class Task(Predicate, show=False):
    name: Field[str]

class Slot(Predicate, show=False):
    hour: Field[int]

class Scheduled(Predicate):
    task: Field[str]
    hour: Field[int]

program = ASPProgram()
program.fact(Task(name="brew"), Task(name="bottle"))
program.fact(*(Slot(hour=h) for h in (9, 10, 11)))
T, H = Variable("T"), Variable("H")
program.when(Task(name=T)).derive(
    Choice(Scheduled(task=T, hour=H), condition=Slot(hour=H)).exactly(1)
)
```

A "fact" with a variable in it is not a fact — and the error says so:

```python
>>> program.fact(Scheduled(task=T, hour=9))
Traceback (most recent call last):
  ...
ValueError: fact() requires grounded predicates, but scheduled(T, 9) contains variable(s) T. Use when(*conditions).derive(...) to derive predicates.
```

It names the offending atom, the variable, and the construct you probably
wanted. Every refusal in the library aims for this shape: what broke, where,
and what to write instead.

## Source locations

Every statement remembers the Python line that authored it (this is the
default — `ASPProgram(source_locations=False)` switches capture off, which
you may want when generating rules in bulk; capture is cheap either way, as
it reads a few frame attributes and never touches source files). Three
consumers use the recorded locations: the dangling-`when()` report below,
[`render(annotate=True)`](#seeing-the-asp), and the "generated by" notes on
[grounding diagnostics](#clingos-messages).

The dangling-`when()` report is the first one you are likely to meet. A
`when()` must be completed by exactly one closer (`derive`, `choose`,
`require`, `forbid`, or `penalize`); one left unclosed fails at render —
and the error names the file and line where it was *opened*, which is where
the fix goes:

```python
scratch = ASPProgram()
pending = scratch.when(Slot(hour=H))  # opened, never closed
```

```python
>>> scratch.render()
Traceback (most recent call last):
  ...
ValueError: Segment 'Rules' has incomplete when() statements: when(slot(H)) opened at ... Complete each with .derive/.choose/.require/.forbid/.penalize.
```

The "opened at" names the file and the line where the `when()` was written.
A held reference can still be completed — and the report clears:

```python
class OpenSlot(Predicate):
    hour: Field[int]

pending.derive(OpenSlot(hour=H))
scratch.render()  # renders cleanly now
```

By default, the captured line is the first stack frame outside ASPAlchemy
itself — exactly right for a script, wrong for a framework built on top of
ASPAlchemy, whose own plumbing would soak up every location. Framework
authors get a kit of three tools:

- **`register_skip_package("myfw")`** declares a whole package plumbing, so
  captured locations point at *its* caller's code. A package that also
  contains authored code registers just its plumbing modules by dotted
  prefix (`"myfw.core"`, `"myfw.solvers.base"`), leaving the authored
  modules' lines intact. Registration is process-wide and permanent —
  register only names you own (`"__main__"` is rejected: that is the user's
  own script, never plumbing).
- **`@attribute_to_caller`** marks a single helper function that emits rules
  on its caller's behalf: the interesting line is the call site, not the
  helper's interior. The mark is per code object, so lambdas and
  comprehensions defined inside the helper are not covered (registering the
  module covers those too).
- **`location_override(loc)`** attributes everything created in the block to
  a location you supply — for code where no stack frame is the honest answer,
  such as a framework module emitting rules during a finalize pass, where the
  meaningful location is a construction site captured earlier. The
  `SourceLocation` it consumes is produced by **`capture_location()`**, which
  walks the stack outward and returns the first frame that is not plumbing.

This is how [aspuzzle](https://github.com/jolyonb/aspuzzle), a grid-puzzle
framework built on ASPAlchemy, keeps diagnostics landing on each puzzle's
own definition rather than on framework internals.

## Seeing the ASP

`program.render()` is the program text — the clingo you would have written
by hand. The [tutorial's receipts moment](getting-started.md#see-the-clingo-you-wrote)
shows it in full; for the scheduling program above it is:

```python
>>> print(program.render())
% Generated by aspalchemy ...
task("brew").
task("bottle").
slot(9).
slot(10).
slot(11).
{ scheduled(T, H) : slot(H) } = 1 :- task(T).

#show.
#show scheduled/2.
```

`render(annotate=True)` appends a trailing `% file:line` comment to each
statement line, naming the Python line that authored it. Line numbering is
unchanged from the plain render, which is the point: the annotated text
doubles as a reverse map, so when raw clingo (or any external tool)
complains about line *N*, the comment on line *N* of the annotated render is
the explanation.

Here is the same program, rebuilt with `location_override` so that the
comments below name a stable file instead of this page — in your program
they simply name your own files and lines:

```python
from aspalchemy import SourceLocation, location_override

brewery = ASPProgram()
with location_override(SourceLocation("brewery.py", 14)):
    brewery.fact(Task(name="brew"), Task(name="bottle"))
with location_override(SourceLocation("brewery.py", 15)):
    brewery.fact(*(Slot(hour=h) for h in (9, 10, 11)))
with location_override(SourceLocation("brewery.py", 19)):
    brewery.when(Task(name=T)).derive(
        Choice(Scheduled(task=T, hour=H), condition=Slot(hour=H)).exactly(1)
    )
```

```python
>>> print(brewery.render(annotate=True))
% Generated by aspalchemy ...
task("brew").  % brewery.py:14
task("bottle").  % brewery.py:14
slot(9).  % brewery.py:15
slot(10).  % brewery.py:15
slot(11).  % brewery.py:15
{ scheduled(T, H) : slot(H) } = 1 :- task(T).  % brewery.py:19

#show.
#show scheduled/2.
```

Annotation is off by default for a practical reason: annotated output churns
on unrelated edits (every line move rewrites the comments), so keep
checked-in or golden-compared renders unannotated.

## Organizing output

The render you just saw is meant to be *read* — in review, in diagnostics, in these docs — and a few
formatting verbs shape it. `comment(text)` emits a `%` comment, `blank_line()` drops a separator, and
`section(title)` does both at once. They cost nothing at solve time and pay for themselves the first time
you read back a 200-line render.

For programs big enough to have chapters, **segments** are the chapters. Every program starts with one
default segment that the program-level verbs write to; `add_segment("name")` adds a named one, fixes its
position in the output, and returns a handle carrying the full set of statement verbs. You can read a
segment back later with `program["name"]`. Once named segments exist, each renders under a
`% ===== name =====` banner:

```python
Wall = Predicate.define("wall", ["at"], show=False)

house = ASPProgram()
board_seg = house.add_segment("board")
board_seg.section("Board geometry")
board_seg.fact(Wall(at=1))

house.comment("program-level verbs write to the default segment")
house.fact(Wall(at=2))
```

```python
>>> print(house.render())
% Generated by aspalchemy ...

% ===== board =====

% Board geometry
wall(1).


% ===== Rules =====

% program-level verbs write to the default segment
wall(2).

#show.

>>> house["board"] is board_seg  # the handle reads back by name
True
```

Segments render in the order they were *added*, no matter when statements land in them — so a framework
can reserve a "definitions" segment up front and fill it as facts arrive. That's as deep as most programs
need; the framework-author machinery — attributing generated rules to the right source lines across
segment boundaries — builds on [source locations](#source-locations).

## Seeing the ground program

`render()` shows the program you wrote; the solver runs the *ground*
program — every rule instantiated over the actual data — and when grounding
surprises you, that is the text to read. A
[`GroundedProgram`](solving.md#ground-once-solve-many) exposes it in both of
gringo's own output formats:

- **`ground_text()`** is gringo's `--text` rendering: every rule
  instantiated, atom names intact — what humans read, the programmatic twin
  of dumping a ground file when debugging grounding size or content.
- **`aspif()`** is the exact statement stream clasp receives, byte-for-byte
  what `clingo --mode=gringo` emits. This is the honest measure for
  grounding-size questions: pretty-printed text re-prints the shared
  elements of a [banded aggregate](choices-and-aggregates.md#guards-done-right)
  and looks duplicated, while aspif shows what clasp actually receives.

```python
from aspalchemy import RangePool

class Pick(Predicate, show=False):
    n: Field[int]

class Big(Predicate):
    n: Field[int]

X = Variable("X")
pick_program = ASPProgram()
pick_program.choose(Choice(Pick(n=RangePool(1, 3))))
pick_program.when(Pick(n=X), X > 1).derive(Big(n=X))

grounded = pick_program.ground()
```

The source program has one choice rule and one rule with a variable;
`ground_text()` shows what they became — instantiated, atom names intact:

```python
>>> print(grounded.ground_text())
{pick(1)}.
{pick(2)}.
{pick(3)}.
big(2):-pick(2).
big(3):-pick(3).
#show.
#show big/1.
```

while `aspif()` is the same program as clasp sees it — numbered literals,
rules, and the output table:

```python
>>> print(grounded.aspif())
asp 1 0 0 incremental
1 1 1 1 0 0
1 1 1 2 0 0
1 1 1 3 0 0
1 0 1 4 0 1 2
1 0 1 5 0 1 3
4 6 big(2) 1 4
4 6 big(3) 1 5
0
```

Both are generated on demand: each call re-grounds this handle's stored
source through clingo's own application, so the outputs come from clingo's
formatters, never from a hand-maintained serialization — byte-identity
against `clingo --mode=gringo` is pinned by the test suite in CI. For
groundings made with `ground(context=...)`, the re-ground runs in process so
the stored [@-function context](escape-hatches.md#calling-python-during-grounding)
still answers; a *stateful* context whose functions answer differently per
call may make the re-ground diverge from the original grounding.

## Profiling grounding

When a grounding is slow or huge, `grounded.analyze_grounding()` is the
first stop: ground-atom counts per signature, largest first, each joined
back to the Python lines that derive it. The top rows name the lines to
rethink.

Here is a program with a deliberate cross-product — 30 items, and a rule
that derives every ordered pair of them. (The statements are pinned to named
locations with `location_override` so the report below is reproducible; in
your program the rows simply name your files and lines.)

```python
class Item(Predicate, show=False):
    n: Field[int]

class Pair(Predicate):
    a: Field[int]
    b: Field[int]

A, B = Variable("A"), Variable("B")
prof_program = ASPProgram()
with location_override(SourceLocation("puzzle.py", 12)):
    prof_program.fact(*(Item(n=i) for i in range(1, 31)))
with location_override(SourceLocation("puzzle.py", 13)):
    prof_program.when(Item(n=A), Item(n=B)).derive(Pair(a=A, b=B))
```

```python
>>> report = prof_program.ground().analyze_grounding()
>>> print(report)
Grounding profile: 930 ground atoms across 2 signatures
  pair/2: 900 atoms — derived at puzzle.py:13
  item/1: 30 atoms — derived at puzzle.py:12
```

The top row is the 30 x 30 cross-product: the line to rethink. Counts are gringo's ground truth: atoms simplified away during grounding are
already gone, and both signs of a signature count together. When you want
the same data structured rather than printed, `grounding_profile()` returns
one typed `SignatureGrounding` row per signature — `name`, `arity`,
`atom_count`, and the `derived_at` locations:

```python
>>> profile = prof_program.ground().grounding_profile()
>>> profile[0].name, profile[0].atom_count
('pair', 900)
```

One blind spot, stated plainly: the report joins on rule *heads*, so a
statement whose body grounds large without deriving new atoms — a wide join
feeding a small head — shows up under its head's count, not as its own row.
The statement profile below closes it.
Grounding size is also only half the scale story; the other half, reading
large models back, has [its own honest paragraph](solving.md#a-note-on-scale).

## Profiling by statement

The signature profile counts the atoms each signature ends up with — but a
*constraint* has no head to charge, and the costliest statements in a
program are often exactly the headless ones. `analyze_statements()` charges
every statement its own row: ground instantiation counts, largest first,
each with its authoring line. Here is the cross-product again, this time
built from a pooled range and joined by a constraint:

```python
X = Variable("X")
stmt_program = ASPProgram()
with location_override(SourceLocation("puzzle.py", 12)):
    stmt_program.when(X.in_(RangePool(1, 30))).derive(Item(n=X))
with location_override(SourceLocation("puzzle.py", 13)):
    stmt_program.when(Item(n=A), Item(n=B)).derive(Pair(a=A, b=B))
with location_override(SourceLocation("puzzle.py", 14)):
    stmt_program.forbid(Pair(a=A, b=B), Pair(a=B, b=A), A != B)
```

```python
>>> print(stmt_program.ground().analyze_statements())
Statement profile: 1800 ground instantiations across 3 statements
         900  pair(A, B) :- item(A), item(B).  — puzzle.py:13
         870  :- pair(A, B), pair(B, A), A != B.  — puzzle.py:14
          30  item(X) :- X = 1..30.  — puzzle.py:12
```

The constraint — invisible to the signature profile — is the second-largest
row here, and in real programs it is routinely the first. When you want the
same data structured, `statement_profile()` returns one typed
`StatementGrounding` row per statement — `statement`, `location`,
`instances`:

```python
>>> row = stmt_program.ground().statement_profile()[1]
>>> row.statement, row.instances
(':- pair(A, B), pair(B, A), A != B.', 870)
```

How it counts, stated plainly. Each call re-grounds an *instrumented* copy
of the program, and a ground-program observer tallies a reserved marker
literal (`__aspalchemy_stmt/1`) per statement — no text is ever parsed.
Statements are classified structurally from the render itself, so a comment
or directive can never be mistaken for one. The instrumentation is chosen
per statement kind so the copy grounds as faithfully as possible:

- **Facts stay real facts** — they are counted by a companion rule whose
  pool expansion instantiates once per fact instance — so aggregate
  evaluation over facts, and ground-evaluable conditions generally,
  resolve exactly as in the true grounding.
- **Rules** carry the marker in their bodies (joined with `;`, which a
  trailing conditional literal's condition cannot absorb); the marker
  keeps each instantiation a rule, so instances the real grounding would
  fold into facts are still counted. Bodiless choice statements gate on
  the marker instead (choice atoms are open in the real grounding too),
  as do raw statements whose heads the companion cannot wrap —
  disjunctions, conditional heads, and cardinality-bounded choices.
- **Weak constraints are not touched at all** — their bodies ground
  exactly as the real program; the tail's priority is redirected to a
  reserved value per statement, and gringo's minimize callback reports
  one entry per ground tuple there.
- **Anonymous variables are renamed apart** in the instrumented copy:
  gringo's projection rewrite would otherwise hide a statement's join work
  from the marker (`p(_, C)` counting distinct `C` instead of the join).
  Occurrences under default negation are left alone — renaming them would
  be unsafe, and a negated literal carries no join work. The
  `ASPALCHEMY_ANON*` variable names are reserved for this, and fresh
  indices start past any already present in the program.

Three consequences worth knowing:

- Counts are the instantiations gringo emits for each statement, kept
  visible past the final fact-folding. `minimize()`/`maximize()`
  directives are the one excluded statement kind — a wide-bodied
  objective's grounding work is invisible here, as it is in the
  signature profile.
- Each call pays a full in-process re-ground, with `ground_text()`'s
  stateful-context caveat (a context answering differently on the second
  call skews or breaks the profile) — but none of its threading caveat.
- Every rendered ASPAlchemy statement except optimization directives is
  counted. Inside `raw_asp()` text, attribution is line-based, and lines
  the pass cannot honestly attribute pass through uncounted: statements
  spanning several lines (including a `1..\n3` range split), several
  statements sharing one line, a statement sharing its line with a block
  comment, and raw weak constraints and directives. A bare choice over a
  pool counts gringo's per-element ground rules.
- One honest upper bound: a rule derived purely from facts is OPEN in
  the instrumented copy, so an assignment aggregate over such derived
  atoms grounds over its achievable range rather than evaluating —
  those aggregate rows charge an upper bound, not the single emitted
  rule. Aggregates directly over facts are exact.

## Profiling recursion

Recursive grounding has its own cost model: gringo grounds each strongly
connected component of the positive predicate dependency graph as one
fixpoint, re-evaluating the component's rules across the iteration. A
statement inside that fixpoint that need not feed the recursion — a
derivation restatable as a requirement — is the classic slow-grounding
finding, and it is visible from the program's *structure* alone.
`analyze_recursion()` reports it before any grounding is paid:

```python
X, Y, Z = Variable("X"), Variable("Y"), Variable("Z")

class Edge(Predicate, show=False):
    a: Field[int]
    b: Field[int]

class Reach(Predicate):
    a: Field[int]
    b: Field[int]

tc_program = ASPProgram()
with location_override(SourceLocation("paths.py", 7)):
    tc_program.fact(Edge(a=1, b=2), Edge(a=2, b=3))
with location_override(SourceLocation("paths.py", 8)):
    tc_program.when(Edge(a=X, b=Y)).derive(Reach(a=X, b=Y))
with location_override(SourceLocation("paths.py", 9)):
    tc_program.when(Reach(a=X, b=Y), Edge(a=Y, b=Z)).derive(Reach(a=X, b=Z))
```

```python
>>> print(tc_program.analyze_recursion())
Recursion profile: 1 recursive component
  reach/2
    reach(X, Z) :- reach(X, Y), edge(Y, Z).  — paths.py:9
```

The base rule seeds the component but is not listed: only rules that
depend on the component from within it — positively, or through the
negation that makes a component unstratified — are part of its cycle. `recursion_profile()` returns the same data structured — one
`RecursiveComponent` per component: `signatures`, `statements` (rendered
text with authoring locations), and `unstratified` — True when some rule
of the component also reaches it through default negation, a `not p`
whose `p` is in the component. Components are the strongly connected
components of the FULL dependency graph — positive and default-negated
edges together, the textbook object of stratification — so a cycle
running entirely through negation (`p :- not q. q :- not p.`), or two
positive fixpoints entangled by mutual negation, is reported and
flagged. An unstratified component means strongly circular rules —
defined in terms of their own absence: gringo cannot settle them and
clasp is relegated to guess-and-check search. A well-shaped encoding is
a tower — base data, then choices, then rules on top, each stratum
negating only what is settled below — and an unstratified component is
the tower folding back on itself: expensive to solve and almost always
unintended. Stated
plainly: the analysis is static (no grounding runs), and `raw_asp()`
text is invisible here — its `predicates=` declarations give existence,
not structure.

## Clingo's messages

Some mistakes only grounding can catch, and clingo reports them as log
messages rather than exceptions: `LogLevel.INFO`, `WARNING`, or `ERROR`
(`CRITICAL` exists only as a threshold meaning "never halt"). `ground()` —
and every verb that grounds: `solve`, `brave`, `cautious`, `optimize` —
takes `stop_on_log_level`, and any message at or above the threshold raises
`GroundingError`. The default is `INFO`, the strictest setting, because
clingo's quietest messages are often the loudest problems.

The flagship case is [division by zero](math.md#where-clingo-and-python-disagree):
raw clingo *deletes* any rule instance whose arithmetic is undefined,
emitting only an info-level message, and your program silently shrinks.
Under ASPAlchemy's default threshold that becomes a loud failure — and the
diagnostic carries a "generated by file:line" note joining the offending ASP
back to the Python that authored it, so even errors only gringo can catch
land on your source. (Again pinned with `location_override` so the receipt
below is stable.)

```python
from aspalchemy import GroundingError, LogLevel

class Amount(Predicate, show=False):
    n: Field[int]

class Ratio(Predicate):
    n: Field[int]

ratio_program = ASPProgram()
with location_override(SourceLocation("recipe.py", 30)):
    ratio_program.fact(Amount(n=0), Amount(n=2))
    ratio_program.when(Amount(n=X)).derive(Ratio(n=10 // X))  # 10 // 0 is undefined
```

The formatted diagnostic quotes the rendered ASP with a caret under the
offending expression — Python's `//` renders as clingo's `/` — then closes
the loop back to Python:

```python
>>> ratio_program.solve().first()
Traceback (most recent call last):
  ...
aspalchemy.exceptions.GroundingError: Grounding produced INFO level messages (stop threshold: INFO).
...
INFO: operation undefined:
  (10/X)

  at line 4, columns 7-13

  in block
   3 | amount(2).
   4 | ratio(10 / X) :- amount(X).
             ^^^^^^
  generated by recipe.py:30
...
```

The same structured diagnostics ride the exception as `e.messages`, a tuple
of `ClingoMessage` values with the severity, position, and text as fields —
assert or branch on those rather than on the prose:

```python
>>> try:
...     ratio_program.solve().first()
... except GroundingError as e:
...     message = e.messages[0]
>>> message.severity
<LogLevel.INFO: 10>
```

The "generated by" note works for every
statement with a recorded location, including errors inside
[`raw_asp()`](escape-hatches.md) text: the raw block's internals get no
construction-time validation, but grounding diagnostics still map back to
the `raw_asp` call's line.

The INFO promotion you will actually meet first is quieter: **"atom does
not occur in any rule head"**. A predicate used only under
[default negation](statements.md#default-negation) — the everyday shape being
input facts loaded from data that happens to be empty today — has no head
anywhere in the program, gringo flags it at INFO, and under the default
threshold `solve()` raises. The program below is perfectly sensible; it
just has no bookings yet:

```python
class Room(Predicate, show=False):
    number: Field[int]

class Booked(Predicate, show=False):
    room: Field[int]

class Free(Predicate):
    room: Field[int]

R = Variable("R")
free_program = ASPProgram()
free_program.fact(Room(number=101), Room(number=102))
# ...booking data would be loaded here — and today there is none
free_program.when(Room(number=R), ~Booked(room=R)).derive(Free(room=R))
```

```python
>>> free_program.solve().first()
Traceback (most recent call last):
  ...
aspalchemy.exceptions.GroundingError: Grounding produced INFO level messages (stop threshold: INFO).
...
INFO: atom does not occur in any rule head:
  booked(R)
...
   4 | free(R) :- room(R), not booked(R).
                               ^^^^^^^^^
  generated by ...
```

gringo's caution is not misplaced — an atom nothing can derive is often a
typo for one that can — but here the absence is the point. Three remedies,
in order of preference: tell clingo the predicate is deliberately
head-less, give it a head occurrence somewhere, or raise the threshold
(below, with its cost). The first is one clingo directive, `#defined`,
delivered through [`raw_asp()`](escape-hatches.md):

```python
>>> free_program.raw_asp("#defined booked/1.")
>>> free = free_program.solve().first()
>>> sorted(f.room for f in free.atoms(Free))  # no bookings: all free
[101, 102]
```

Raising the threshold tolerates the message — and buys exactly the silent
behavior the default protects you from:

```python
>>> models = list(ratio_program.solve(stop_on_log_level=LogLevel.WARNING))
>>> len(models)  # facts and one rule: exactly one answer set
1
>>> [r.n for r in models[0].atoms(Ratio)]  # ratio(10/0) is just... gone
[5]
```

`ratio(5)` survives; the `Amount(n=0)` instance vanished without a trace.
Relax the threshold only when you have read the message and decided the
deletion is what you want. Messages from *solving* (as opposed to
grounding) are collected too — each model carries them as `.messages`; see
[Statistics and messages](solving.md#statistics-and-messages).