Saiku semantic annotations

Mondrian’s generic <Annotation> block is a free-form key/value bag attached to any schema element (Cube, Dimension, Hierarchy, Level, Measure, …). Saiku reserves the saiku.semantic.* namespace inside that bag for a small typed vocabulary that drives:

1. The AI Ask layer — descriptions and synonyms give the LLM real business context; aggregation kinds, grains, and required filters keep its query suggestions sane.
2. PII protection — the saiku.semantic.pii=true marker flips a level or measure into “structurally present but opaque” mode in agent-facing surfaces and blocks drillthrough, share/embed exposure, and download paths from leaking the underlying values.

All keys are optional. A schema without any saiku.semantic.* annotations runs identically to one without the annotation block; the namespace is purely additive.

At a glance

Annotation key	Valid on	Type / values	Purpose
saiku.semantic.description	Dimension · Measure · Level	free text	Business description; surfaced in /ai/schema so the LLM has context
saiku.semantic.synonyms	Dimension · Measure · Level	CSV list	Alternative names so the AI can resolve “country” → “Store Country”
saiku.semantic.unit	Measure	free text (e.g. USD, kWh, count)	Unit string; flows into the AI cell {value, formatted, unit} shape
saiku.semantic.currency	Measure	ISO 4217 code (e.g. USD, EUR)	Currency hint for monetary measures
saiku.semantic.aggregation_kind	Measure	enum: sum · count · distinct-count · non-additive	Aggregation semantics. Drives sort/total semantics in the AI layer.
saiku.semantic.cardinality	Level	enum: low · medium · high	Member-count hint. Drives picker UX and AI cost hints.
saiku.semantic.grain	Level	enum: year · quarter · month · week · day · hour · minute	Time grain for date-filter modal and time-series chart inference
saiku.semantic.required_filters	Level	CSV of (hierarchy, level) pairs	Levels the AI must include in filters whenever this level is touched
saiku.semantic.pii	Level · Measure	boolean — true yes y on 1 / false no n off 0	PII marker — see PII protection below

Unknown values for the enum-typed fields are logged at WARN with the allowed list and otherwise ignored — a typo doesn’t break the schema.

The same applies to saiku.semantic.pii: an unrecognised value means not PII, and is logged at WARN naming the value and the accepted spellings. The trade-off is deliberate — for an analytics product, masking working data because of a slip is a worse everyday failure than a schema author correcting a value. It does mean a typo leaves the column unprotected, so treat that WARN as actionable.

Accepted spellings widened in 4.7.1

Before 4.7.1 the marker matched only the literal string true. pii="yes", pii="1" and pii="on" were silently read as false — the column was not protected, and nothing was logged. If your schemas predate 4.7.1, grep them for saiku.semantic.pii and confirm every marker actually took effect.

Where to put them

Annotations attach to any schema element via a child <Annotations> block (XML) or an annotations: key (YAML).

XML

<Measure name="Quantity" column="quantity_units" aggregator="sum" formatString="#,##0">
  <Annotations>
    <Annotation name="saiku.semantic.description">Total units prescribed (across all dispensings).</Annotation>
    <Annotation name="saiku.semantic.synonyms">units, quantity, scripts</Annotation>
    <Annotation name="saiku.semantic.unit">count</Annotation>
    <Annotation name="saiku.semantic.aggregation_kind">sum</Annotation>
  </Annotations>
</Measure>

YAML

measures:
  - name: Quantity
    column: quantity_units
    aggregator: sum
    format_string: "#,##0"
    annotations:
      saiku.semantic.description: "Total units prescribed (across all dispensings)."
      saiku.semantic.synonyms: "units, quantity, scripts"
      saiku.semantic.unit: count
      saiku.semantic.aggregation_kind: sum

Levels accept the same <Annotations> shape:

XML

<Level attribute="Year">
  <Annotations>
    <Annotation name="saiku.semantic.description">Calendar year.</Annotation>
    <Annotation name="saiku.semantic.synonyms">annual, yearly, fiscal year, y</Annotation>
    <Annotation name="saiku.semantic.cardinality">low</Annotation>
    <Annotation name="saiku.semantic.grain">year</Annotation>
  </Annotations>
</Level>

YAML

levels:
  - attribute: "Year"
    annotations:
      saiku.semantic.description: "Calendar year."
      saiku.semantic.synonyms: "annual, yearly, fiscal year, y"
      saiku.semantic.cardinality: low
      saiku.semantic.grain: year

In YAML, a level with nothing to annotate is just the attribute name as a bare string (- "Year"); the map form is what you reach for once it needs a key of its own. See YAML schemas for the whole format.

PII protection

Marking a level or measure with saiku.semantic.pii=true opts it into Saiku’s PII enforcement stack. The annotation is the declaration — Saiku layers four enforcement points on top of it.

1. AI schema projection — descriptive metadata stripped

The /ai/schema/{connection/catalog/schema/cube} endpoint feeds the LLM the cube’s structure. For any PII-flagged element it strips:

Stripped	Kept
displayName	name
description	uniqueName
synonyms	pii (so downstream layers still see the flag)
unit, currency (measures)	cardinality, grain, requiredFilters (levels)
sample members — replaced with a single [REDACTED] sentinel, so the shape “this level has members” survives without leaking captions

Aliases are stripped too: any display-name alias resolving to a PII-flagged measure or level is dropped from the alias maps, so a lookup can’t reach a redacted slot under another name.

The level or measure still appears structurally — the agent knows it exists and can reference it by name — but the descriptive metadata and values stay opaque. That is deliberate: an agent that cannot see the column at all tends to invent one.

The identifier is not secret — design accordingly

name and uniqueName survive redaction by design. An agent calling /ai/schema on a cube with a PII measure called Salary still learns that a measure named Salary, with unique name [Measures].[Salary], exists.

For value-sensitive columns (Salary, Account Number) that is the right trade — the name is unremarkable, the values are what matter.

For existence-sensitive columns, where the column’s very presence is the disclosure (HIV Status, Bankruptcy Flag, Safeguarding Alert), the annotation does not hide it. Keep those out of the cube the AI surface is pointed at rather than relying on pii=true alone.

2. Drillthrough returns= blocked

Drillthrough returns raw fact rows, so this is the last gate between an agent’s projection list and unaggregated personal data.

Drillthrough requests carry a returns= parameter listing the columns to export. If any token resolves to a PII-flagged level or measure, the resolver throws AiPiiException and the endpoint returns 403 with the offending column named. The agent can retry with a narrower projection.

The gate is on the column, not the spelling. Both of these are refused for the same flagged column:

returns=Salary                    ← bare caption
returns=[Measures].[Salary]       ← fully-qualified MDX

HTTP 403
{
  "field": "returns",
  "error": "Column '[Measures].[Salary]' is annotated PII (saiku.semantic.pii=true) and cannot be
            projected through DRILLTHROUGH ... RETURN. Pick a non-PII column from the candidate list.",
  "available": ["Unit Sales", "Store Cost", "City"]
}

The available list deliberately excludes every PII-flagged column, so an agent cannot mine the self-correction hint to enumerate what is protected.

A bracketed identifier the schema does not recognise is still passed through untouched — Mondrian remains the judge of MDX it alone understands. The gate adds a refusal, never a new rejection.

Qualified MDX was not gated before 4.7.1

Until 4.7.1 the check ran only on the bare-caption path. A column refused as Salary was projected without complaint as [Measures].[Salary] — and because unique names are visible in /ai/schema, and the refusal message itself recommends the qualified form, an agent following its own self-correction hint walked around the gate. If you run an older build with PII annotations and drillthrough enabled, treat that as an exposure and upgrade.

3. Embed/share tokens auto-redact

When you mint an embed or share token bound to a query that touches a PII-flagged level, the inspector escalates the token’s redaction policy to FORCE_ON regardless of the operator’s default. Recipients of the token see redacted cells even if the embed wasn’t explicitly configured for redaction. Eliminates the “I forgot to turn on redaction” footgun on shared links.

4. AI data policy gating

PII annotations are cube-level facts. They pair with the SAIKU_AI_POLICY environment variable, which is the deployment-level dial:

SAIKU_AI_POLICY	What may cross the trust boundary
schema-only	Schema metadata and sample members only. No cell values at all.
aggregated	Adds aggregated result values (the typed {value, formatted, unit} cells).
full	Adds raw fact rows from drillthrough. PII annotations still apply.

Set via the SAIKU_AI_POLICY environment variable or the ai.policy system property — the environment wins. Values are case-insensitive and accept either separator, so schema-only, SCHEMA_ONLY and Schema_Only are equivalent.

The tiers are a ladder: each permits everything below it. The annotation says “this column is sensitive”; the policy says “this is how much data of any kind we let out”. Both must allow a thing for it to leave.

The default is schema-only, so an unconfigured deployment leaks no values at all — you opt in to looser tiers, never out of them.

row-level is not a valid value

Earlier revisions of this page listed a row-level tier. There is no such tier — the third value is full. An invalid value is not ignored: parsing throws at startup by design, so a policy typo can’t silently run looser than intended, and the server will not come up.

Invalid ai.policy / SAIKU_AI_POLICY value 'row-level' — expected one of: schema-only, aggregated, full

If you copied row-level from these docs, change it to full.

There is a second, independent dial for the same question one layer out: SAIKU_AI_LLM_EGRESS / ai.llm.egress uses the same three tiers and answers “may cell data reach a third-party LLM vendor?” separately from “may it return to the authenticated caller?”. It also defaults to schema-only.

Pre-flight scanner

PiiScanner runs over a cube’s schema at build time and logs WARN lines for any measure or level whose name looks PII-shaped but carries no annotation. The warning includes a paste-ready XML snippet so the schema author can opt the column in without hunting through the spec:

WARN  [PiiScanner] Likely PII column 'Customer Email' on level [Customer].[Customer].[Email]
      is not annotated. Add:
  <Annotations>
    <Annotation name="saiku.semantic.pii">true</Annotation>
  </Annotations>

The scanner never mutates the schema and never auto-applies the flag — it surfaces candidates, and the schema author decides. A column you have already annotated is never re-suggested, so the warnings stay signal.

What it actually matches

Rule	Matches
email	email, e_mail, mail
ssn	ssn, social_security, nin, sin
phone	phone, mobile, tel, tel_no, tel_number
name	name, first_name, last_name, surname, given_name, full_name, customer_name
dob	dob, date_of_birth, birth_date, birthdate
address	address, street, postcode, postal_code, zip, zip_code
medical_record	nhs, medical_record, mrn
financial	account_number, account_no, iban, swift, bic, ccn, credit_card

Matching is case-insensitive and word-boundary anchored. That boundary is deliberate — it is what stops email_count (a COUNT(emails) aggregate, not personal data) from firing on the email rule.

The boundary cuts both ways

Word boundaries also mean a run-together column name is not matched. prescribername does not fire the name rule; prescriber_name and Prescriber Name do. Neither does npi — there is no NPI rule, so healthcare identifiers of that shape are not detected.

The scanner is a safety net for common shapes, not an inventory. It finding nothing is not evidence that a cube is clean. Annotate deliberately; treat the scanner as a second pair of eyes, never the first.

Worked example — Pharma demo cube

The Pharma demo schema ships with PII annotations on prescriber name + NPI, and leaves specialty + decile un-annotated (those are operationally safe to share). Excerpt from saiku-home/data/Pharma.xml:

<Dimension name="Prescriber" table="dim_prescriber" key="Prescriber">
  <Attributes>
    <Attribute name="Specialty"  keyColumn="specialty"     hasHierarchy="false"/>
    <Attribute name="Decile"     keyColumn="decile"        hasHierarchy="false"/>
    <Attribute name="Prescriber" keyColumn="prescriberkey"
               nameColumn="prescribername" hasHierarchy="false"/>
    <Attribute name="NPI"        keyColumn="prescribernpi" hasHierarchy="false"/>
  </Attributes>
  <Hierarchies>
    <Hierarchy name="Prescriber" allMemberName="All Prescribers">
      <Level attribute="Specialty"/>
      <Level attribute="Decile"/>
      <Level attribute="Prescriber">
        <Annotations>
          <Annotation name="saiku.semantic.pii">true</Annotation>
        </Annotations>
      </Level>
    </Hierarchy>
    <Hierarchy name="NPI" allMemberName="All NPIs">
      <Level attribute="NPI">
        <Annotations>
          <Annotation name="saiku.semantic.pii">true</Annotation>
        </Annotations>
      </Level>
    </Hierarchy>
  </Hierarchies>
</Dimension>

Note where the annotation goes: on the <Level>, not the <Attribute> that supplies its column. The level is what a query names, so that’s what the policy layer checks.

With this in place, an AI agent asking “break down Quantity by Specialty” succeeds normally, but “break down Quantity by Prescriber and download the rows” either refuses (drillthrough block) or returns aggregated buckets with prescriber names suppressed (depending on the AI policy tier).

Note that neither of these columns would have been suggested by the pre-flight scanner: prescribername is run-together so it misses the word-boundary-anchored name rule, and there is no NPI rule at all. They are annotated because someone decided they were sensitive — which is the intended workflow. The scanner catches common shapes; domain identifiers are yours to know.

Worked example — FoodMart cube

FoodMart ships with the full descriptive annotation set (no PII columns — synthetic data). Lifted from saiku-home/data/FoodMart4.xml:

<Level attribute="Store Country">
  <Annotations>
    <Annotation name="saiku.semantic.description">Store's country.</Annotation>
    <Annotation name="saiku.semantic.synonyms">nation, country code</Annotation>
    <Annotation name="saiku.semantic.cardinality">low</Annotation>
  </Annotations>
</Level>
<Level attribute="Year">
  <Annotations>
    <Annotation name="AnalyzerDateFormat">[yyyy]</Annotation>
    <Annotation name="saiku.semantic.description">Calendar year.</Annotation>
    <Annotation name="saiku.semantic.synonyms">annual, yearly, fiscal year, y</Annotation>
    <Annotation name="saiku.semantic.cardinality">low</Annotation>
    <Annotation name="saiku.semantic.grain">year</Annotation>
  </Annotations>
</Level>

The same pattern scales across every dim + level. Each line is independently optional, so you can roll the annotations out incrementally.

Validation and unknown values

• Enum-typed fields (aggregation_kind, cardinality, grain) validate against their allowed list. Unknown values log a WARN with the offending value + the allowed list and the field defaults to unset.
• Boolean fields (pii) accept true / false case-insensitively after trim. Anything else is false. Schema authors get the conservative default if they mistype.
• Free-text fields (description, unit, currency) are trimmed; empty strings become null.
• synonyms and required_filters parse as CSV; empty entries are dropped.

Related

• Exporting to Apache Ossie — how the saiku.semantic.* annotations survive round-trip into portable Ossie YAML (description + synonyms lift into ai_context; everything else rides in custom_extensions as SAIKU vendor data).
• Well-known Ossie extensions — the Ossie-side equivalent (saiku.display, saiku.roles, saiku.pii) for YAML models. Same intent, different file format.
• Extensions: functions and formatters — the generic <Annotation> block on which saiku.semantic.* builds (and its locale-metadata convention).
• YAML schemas — full YAML reference, including annotations: on every element.
• Access control and roles — schema-level access enforcement (complements PII annotations: ACLs gate who can query a cube; PII annotations gate what the result exposes once they can).