Collect Constraints Once. Inject Them on Every Request.

Series: Building Backarch, engineering decisions from building backarch.com

A user types "should I use Kafka?" into an AI chat panel. The AI says "it depends: on your budget, your team size, your throughput requirements." And the user stares at the screen because they already set their budget and constraints on this diagram, and the AI just asked for them again.

"It depends" is a cop-out when the information is knowable. Constraint injection is how you make it stop happening.

Why the AI keeps forgetting

Most LLM applications send a fresh, blank context on every request. The model answers the question in front of it. It doesn't know anything about the user unless the frontend passes that information in the message, and if the frontend doesn't, the model asks.

Even when the frontend does pass context, it's often in the user message: "I'm on AWS, budget $500/month, 3-person team, should I use Kafka?" Now the user is repeating themselves every time they start a new topic. The context window is being spent on information that doesn't change between messages.

Constraint injection separates stable diagram knowledge from dynamic conversation. Stable facts, cloud provider, budget range, target RPS, team size range, compliance requirements, are set once on the diagram and injected into every system prompt for that diagram. They never need to appear in the conversation again unless the user explicitly updates them.

The system prompt is two cacheable blocks, not five layers

The prompt splits into a static half and a dynamic half, and that split exists specifically so the static half can be cached:

Block                       Tokens    Changes when?
──────────────────────────────────────────────────
Persona (static, cached)    ~1,600    Never (literal string constant)
Tool schemas (cached)       ~3,300    Never (registered once)
──────────────────────────────────────────────────
Constraints (dynamic)       ~150      Diagram or request-level edit
Canvas state (dynamic)      ~2,000    Per message (current nodes/edges)
Mode instructions (dynamic) ~200      Chat vs. review
──────────────────────────────────────────────────
Conversation history        ~3,000    Per message
User message                ~200      Per message
──────────────────────────────────────────────────
Total (first turn)          ~10,500   (200k window, ~5% utilised)

Token estimates here are for a 200k-token model (Claude). GPT-4o's context is 128k; Gemini 1.5 goes to 1M+. The proportions hold regardless, at a few thousand tokens of injected context, you're using a small fraction of any modern model's window.

The persona, the AI's character, reasoning framework, and communication style, is a literal Python string constant, not a file read at request time:

# app/services/ai/canvas_assistant.py
_PERSONA = """\
...
"""
 
def _build_system_blocks(
    effective_constraints: EffectiveConstraints,
    diagram_name: str,
    nodes: list[dict],
    edges: list[dict],
    mode: str,
) -> list[dict]:
    return [
        {"type": "text", "text": _PERSONA, "cache_control": {"type": "ephemeral"}},
        {
            "type": "text",
            "text": _build_request_prompt(effective_constraints, diagram_name, nodes, edges, mode),
        },
    ]

At backarch.com, this structure means the assistant starts every conversation knowing the diagram's cloud provider, budget, compliance requirements, and team size, while paying full price for only the ~1,600-token persona once per cache window instead of on every single request.

The persona is the character. The constraints block is the brief. Everything below it is dynamic.

Two constraint sources, one merged result

Constraints come from two places: DiagramVersion.constraints, the persisted, saved-on-the-diagram values, and per-request constraints, unsaved edits the frontend can pass for a single AI turn, useful for trying "what if my budget were $200" without committing to it.

When both exist, the request-level value wins:

async def build_effective_constraints(
    db: AsyncSession, user_id: UUID, diagram_id: UUID | None,
    request_constraints: dict | None = None,
) -> EffectiveConstraints:
    """Load and merge constraints with precedence: diagram < request."""
    normalized_diagram = {}
    if diagram_id:
        row = await canvas_repo.get_diagram_with_version(db, diagram_id)
        if row:
            _, version = row
            normalized_diagram = normalize_diagram_constraints(version.constraints)
 
    normalized_request = {}
    if request_constraints:
        normalized_request = normalize_diagram_constraints(request_constraints)
        # Re-labelled "request" so the evidence model can tell unsaved chip
        # edits apart from persisted diagram constraints.
 
    merged = _merge_flat(normalized_diagram, normalized_request, "request")
    return _to_effective(merged)

The output is an EffectiveConstraints object with a source_map:

{
  "cloud": "aws",
  "budget_max_usd_month": 500,
  "team_size_max": 3,
  "compliance": ["SOC2"],
  "source_map": {
    "cloud": "diagram",
    "budget_max_usd_month": "request",
    "team_size_max": "diagram",
    "compliance": "diagram"
  }
}

The source_map is the evidence layer. When the AI says "I'd avoid MSK Kafka given your $500/month budget," the UI can show whether that $500 is the diagram's saved budget or a one-off value the user is trying out for this turn. Engineers debug by tracing causality, give them the trace.

Derived rules are generated in code, not by the LLM

Below the raw constraint values, the constraints block includes derived rules:

RULES DERIVED FROM CONSTRAINTS:
  - Never suggest MSK Kafka (est. $350/mo), suggest SQS instead
  - Avoid multi-region setup for a team of 3
  - Flag anything pushing estimated cost above $500/mo
  - SOC2 compliance required, flag any unencrypted external edges

These rules are generated in application code from the constraint values, not by the LLM. The LLM doesn't decide the rules; the rules inform the LLM. This keeps them deterministic and testable. A rule change is a code change you can unit-test. A rule change that depended on LLM reasoning would be probabilistic and unverifiable.

The same constraints object also carries the mode instructions. In review mode, the system prompt tells the model to call health_score immediately, then adr_search for any key technology choices, surface the top few findings by severity, and call draft_adr if the review settles a decision. In chat mode, it tells the model to check adr_search before starting an architecture-choice conversation and to call draft_adr as soon as a decision is settled, without waiting to be asked.

What happens when a constraint is missing

Constraints are optional. A user can go straight to the canvas without setting a budget, a region, or a team size at all.

When a constraint is missing, the right move is to ask one targeted question, not a generic "tell me about your architecture":

Before I recommend Kafka vs SQS, do you require strict ordering per customer?
That will determine whether the cost premium is justified.

When two or more constraints are missing, prioritise asking for the one that would most change the recommendation. For a cost-sensitive decision, budget matters more than team size. For a compliance question, the compliance requirements matter more than target RPS.

The system still proceeds on partial information, it doesn't block on incomplete context. Missing constraints become explicit assumptions surfaced in the response, not silent gaps. Budget and compliance constraints get priority because they eliminate entire categories of services. Team size and region affect trade-off priorities but rarely change the fundamental recommendation.

This is better than a list of trade-offs with no opinion. The whole point of constraint injection is to make the AI opinionated. A missing constraint is the only time it should pause. One question, targeted, actionable.

The difference from few-shot prompting

Few-shot prompting puts examples in the prompt to guide the model's output style or format. Constraint injection puts facts that are treated as hard requirements. The model doesn't avoid suggesting Kafka because it learned from examples that you prefer cheaper options, it avoids it because the system prompt explicitly says a rule derived from the stated budget, and the estimated Kafka cost exceeds it.

Deterministic, not probabilistic. Consistent across every model version, every conversation, every user. The constraint is structural, not learned.

At backarch.com, constraint injection was one of the first AI features shipped, before the prompt caching work, before tool use expanded past a handful of calls. Without it, the AI assistant was a generic chat interface that happened to sit next to a diagram. With it, a response to "add a message queue" can recommend SQS over Kafka without the user restating their budget, because the diagram already carries it.

The question worth asking before building any AI feature: how much does the AI already know about the user before they type their first message? If the answer is "nothing," you're asking the user to carry context that the system should be holding.