Gating Schema Changes in a CI Pipeline

This guide is the concrete pipeline that stops a breaking JanusGraph schema change from reaching production by catching it in a pull request instead of at query time. It sits under the Schema Evolution and CI Gating reference, which establishes why the schema must be treated as a versioned artifact, and it narrows that principle to a runnable job: encode the schema as code, diff the proposal against the live graph, dry-run it against a throwaway JanusGraph instance, and fail the build on any forbidden mutation before a human can merge it. The specific failure it prevents is the one that commits without an error — a property key silently retyped, an index left off a high-cardinality field — and only surfaces days later as a full-graph scan nobody can trace back to the merge that caused it.

CI gating flow that diffs a schema-as-code proposal against the live graph and blocks forbidden changes A committed schema.yaml file feeds a CI job that boots an ephemeral JanusGraph instance on BerkeleyJE, then diffs the proposed schema against the live graph. A decision asks whether the diff contains a forbidden change — a rename, retype, drop, or unindexed high-cardinality property. If yes, the build fails and a migration plan is required, a terminal block. If no, the change is dry-run validated on the ephemeral instance and the merge gate passes. schema.yaml schema as code Ephemeral JanusGraph BerkeleyJE · in CI Diff vs live graph schema Forbidden change? Dry-run validate merge gate passes Fail build require migration plan no yes
A committed schema.yaml boots an ephemeral JanusGraph in CI, the diff against the live graph is classified, and a forbidden change fails the build and demands a migration plan while a safe change is dry-run validated and allowed to merge.

Prerequisites

Confirm each item before adding the gate. The most common way this pipeline gives false confidence is an ephemeral instance whose configuration diverges from production, so the storage and index settings it boots with must mirror the live graph’s contract.

  • JanusGraph 0.6.x or 1.0.x with a CQL backend in production, and the janusgraph-berkeleyje bundle available to the CI image for the throwaway instance — BerkeleyJE needs no external cluster and boots in seconds.
  • gremlinpython matching your server’s TinkerPop line, plus PyYAML, in the CI runner image, for the diff and dry-run scripts.
  • Read access to the live graph schema from the CI network — a read-only Gremlin endpoint is enough; the job never writes to production.
  • A committed schema definition as the single source of truth. If you have not yet extracted the schema from the running graph, do that first with mgmt.printSchema() and check the result into version control.
  • Branch protection on the target branch so the gate can be a required status check. The gate only prevents a merge if the merge is not allowed to proceed while the check is red. The forbidden-change rules below assume the constraints in Vertex and Edge Validation and the index contract from Property Indexing Rules already hold.

Step 1 — Encode the schema as code

The diff can only be computed against a declarative source of truth, so express every property key, label, and index as data — not as an imperative Groovy script whose intent a machine cannot compare. Keep this file in the repository next to the application it describes.

yaml 
# schema/graph-schema.yaml — the single source of truth the diff runs against
property_keys:
  - name: account_id
    data_type: String
    cardinality: SINGLE
  - name: created_at
    data_type: Date
    cardinality: SINGLE
  - name: email
    data_type: String
    cardinality: SINGLE
vertex_labels:
  - name: account
  - name: transaction
edge_labels:
  - name: owns
    multiplicity: ONE2MANY
composite_indexes:
  - name: byAccountId
    key: account_id
    unique: true
mixed_indexes:
  - name: accountSearch
    backend: search
    keys: [email, created_at]

Treat the file as append-mostly: adding a key, label, or index is a safe evolution, while editing an existing entry’s data_type, cardinality, or index membership is exactly the class of change the gate exists to catch.

Verify: confirm the file parses and every property key referenced by an index actually exists.

bash 
python -c "
import yaml, sys
s = yaml.safe_load(open('schema/graph-schema.yaml'))
keys = {k['name'] for k in s['property_keys']}
for idx in s.get('composite_indexes', []) + s.get('mixed_indexes', []):
    refs = [idx.get('key')] if 'key' in idx else idx['keys']
    missing = [r for r in refs if r not in keys]
    assert not missing, f\"{idx['name']} references undefined keys: {missing}\"
print('schema-as-code OK')
"

Step 2 — Add the CI job that boots an ephemeral JanusGraph

Give the pipeline a disposable graph to validate against, isolated from production. BerkeleyJE runs in-process with no external dependency, so the job spins up a clean instance, applies the proposal, and throws it away — the dry-run never risks the live cluster.

yaml 
# .github/workflows/schema-gate.yml
name: schema-gate
on:
  pull_request:
    paths: ["schema/**"]
jobs:
  gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - name: Install deps
        run: pip install gremlinpython pyyaml
      - name: Boot ephemeral JanusGraph (BerkeleyJE)
        run: |
          docker run -d --name jg-ephemeral -p 8182:8182 \
            -e janusgraph.storage.backend=berkeleyje \
            -e janusgraph.storage.directory=/tmp/jg \
            -e janusgraph.index.search.backend=lucene \
            -e janusgraph.index.search.directory=/tmp/jg-idx \
            janusgraph/janusgraph:1.0.0
          for i in $(seq 1 30); do
            docker exec jg-ephemeral bin/gremlin.sh -e /dev/stdin <<< "1+1" && break
            sleep 2
          done
      - name: Diff proposed schema against live graph
        run: python schema/diff_schema.py --live "${{ secrets.LIVE_GREMLIN_URL }}"
      - name: Dry-run apply on ephemeral instance
        run: python schema/dry_run.py --target ws://localhost:8182/gremlin
      - name: Require migration plan for breaking changes
        run: test ! -f .schema-blocked || (echo "migration plan required" && exit 1)

The lucene index backend stands in for the production Elasticsearch mixed index: it exercises the same INSTALLED → REGISTERED → ENABLED lifecycle locally without a search cluster. Boot BerkeleyJE with a temporary directory so each run starts from an empty graph.

Verify: confirm the instance is reachable before the diff runs.

bash 
docker exec jg-ephemeral bin/gremlin.sh -e /dev/stdin <<< \
  "graph = JanusGraphFactory.open('/tmp/jg.properties'); g = graph.traversal(); g.V().limit(1).count().next()"
# Expect 0 on a fresh instance, and no connection error.

Step 3 — Diff the proposed schema against the live graph

Read the live schema over a read-only Gremlin connection, compare it against graph-schema.yaml, and classify every difference. An additive change — a new key, label, or index — is safe; a mutation of an existing entry is forbidden and marks the build for blocking.

python 
# schema/diff_schema.py
import argparse, yaml, sys
from gremlin_python.driver.client import Client

FORBIDDEN = []


def live_property_keys(client: Client) -> dict:
    script = """
        mgmt = graph.openManagement()
        try {
            mgmt.getRelationTypes(PropertyKey.class).collectEntries {
                [(it.name()): [dataType: it.dataType().simpleName,
                               cardinality: it.cardinality().name()]]
            }
        } finally { mgmt.rollback() }
    """
    return client.submit(script).all().result()[0] or {}


def diff(proposed: dict, live: dict) -> None:
    prop = {k["name"]: k for k in proposed["property_keys"]}
    for name, live_def in live.items():
        if name not in prop:
            continue  # dropped from source but still live — flag, do not auto-remove
        want = prop[name]
        if want["data_type"] != live_def["dataType"]:
            FORBIDDEN.append(f"retype {name}: {live_def['dataType']} -> {want['data_type']}")
        if want["cardinality"] != live_def["cardinality"]:
            FORBIDDEN.append(f"cardinality change on {name}")
    # A live key absent from the proposal is a rename/drop — never silent.
    for name in live:
        if name not in prop:
            FORBIDDEN.append(f"drop/rename of live key {name}")


if __name__ == "__main__":
    ap = argparse.ArgumentParser()
    ap.add_argument("--live", required=True)
    args = ap.parse_args()
    proposed = yaml.safe_load(open("schema/graph-schema.yaml"))
    client = Client(args.live, "g")
    try:
        diff(proposed, live_property_keys(client))
    finally:
        client.close()
    if FORBIDDEN:
        open(".schema-blocked", "w").write("\n".join(FORBIDDEN))
        print("FORBIDDEN changes detected:\n  " + "\n  ".join(FORBIDDEN))
        sys.exit(1)
    print("diff OK — additive only")

A retype, a cardinality narrowing, or the disappearance of a live key are all forbidden because JanusGraph treats a registered key’s type and cardinality as immutable — the only correct path for those is a new key plus a migration, the pattern documented in Schema Migration Patterns.

Verify: run the diff against a scratch copy and confirm a deliberate retype is caught.

bash 
python schema/diff_schema.py --live "$LIVE_GREMLIN_URL"; echo "exit=$?"
# A forbidden change writes .schema-blocked and exits 1; additive-only exits 0.

Step 4 — Dry-run apply and reject unindexed high-cardinality keys

An additive diff still has to prove it applies cleanly and does not introduce a query-time landmine. Apply the proposal to the ephemeral instance inside a transaction, then assert that every high-cardinality property that will be queried is backed by an index — an unindexed high-cardinality key is the classic cause of a silent full-graph scan.

python 
# schema/dry_run.py
import argparse, yaml, sys
from gremlin_python.driver.client import Client

HIGH_CARDINALITY = {"account_id", "email", "transaction_id"}


def indexed_keys(proposed: dict) -> set:
    keys = set()
    for idx in proposed.get("composite_indexes", []):
        keys.add(idx["key"])
    for idx in proposed.get("mixed_indexes", []):
        keys.update(idx["keys"])
    return keys


def dry_run(client: Client, proposed: dict) -> None:
    # Build an idempotent Groovy body from the declarative schema.
    lines = ["mgmt = graph.openManagement()", "try {"]
    for k in proposed["property_keys"]:
        lines.append(
            f"  if (!mgmt.containsPropertyKey('{k['name']}')) "
            f"mgmt.makePropertyKey('{k['name']}')"
            f".dataType({k['data_type']}.class)"
            f".cardinality(Cardinality.{k['cardinality']}).make()"
        )
    lines += ["  mgmt.commit(); 'APPLIED'", "} catch (e) { mgmt.rollback(); throw e }"]
    result = client.submit("\n".join(lines)).all().result()
    assert result and result[0] == "APPLIED", "dry-run apply did not commit"


if __name__ == "__main__":
    ap = argparse.ArgumentParser()
    ap.add_argument("--target", required=True)
    args = ap.parse_args()
    proposed = yaml.safe_load(open("schema/graph-schema.yaml"))
    unindexed = HIGH_CARDINALITY & (
        {k["name"] for k in proposed["property_keys"]} - indexed_keys(proposed)
    )
    if unindexed:
        print(f"BLOCK: high-cardinality keys without an index: {sorted(unindexed)}")
        sys.exit(1)
    client = Client(args.target, "g")
    try:
        dry_run(client, proposed)
    finally:
        client.close()
    print("dry-run OK — applied cleanly, all high-cardinality keys indexed")

The Groovy body guards every makePropertyKey behind a containsPropertyKey check so the dry-run is idempotent and a retried job is a no-op, and it rolls back on any exception so a failed apply never leaves the ephemeral schema half-written — the same transactional discipline the parent reference applies to production.

Verify: confirm a clean run applies and an unindexed high-cardinality key is rejected.

bash 
python schema/dry_run.py --target ws://localhost:8182/gremlin; echo "exit=$?"
# Clean additive schema exits 0; an unindexed high-cardinality key exits 1 before applying.

Step 5 — Require a migration plan and gate the merge

A forbidden change must not be merge-able just because someone is in a hurry. When the diff writes .schema-blocked, the job fails and the required status check goes red; the only way forward is a committed migration plan the reviewers can inspect, which turns an irreversible production mistake into a deliberate, reviewed decision.

yaml 
# schema/migration-plan.yaml — required in the PR when a forbidden change is proposed
change: "retype account_id String -> Long"
strategy: "dual-write"
steps:
  - "Register new key account_id_v2 (Long) — additive, gated normally"
  - "Backfill account_id_v2 from account_id in a batch job"
  - "Cut reads over to account_id_v2 behind a flag"
  - "Deprecate account_id after a full retention window"
rollback: "Reads fall back to account_id; account_id_v2 is additive and safe to drop"
approved_by: "graph-oncall"

Wire the plan into the gate so the build only goes green when the block is accompanied by a plan: the workflow’s final step fails while .schema-blocked exists unless schema/migration-plan.yaml is present and references the same change. With branch protection making schema-gate a required check, no forbidden change reaches the default branch without a reviewed plan.

Verify: confirm branch protection enforces the check and a plan clears a blocked build.

bash 
gh api "repos/$GH_REPO/branches/main/protection/required_status_checks" \
  | jq '.contexts'      # must include "schema-gate"
# With .schema-blocked present, the build stays red until migration-plan.yaml is committed.

Fallback and rollback procedures

Each stage fails independently. Fix the failing stage in isolation rather than disabling the gate to unblock a merge.

If Step 1 fails to parse. A malformed graph-schema.yaml blocks every subsequent stage, which is correct — an unparseable source of truth is worse than none. Fix the YAML and re-run; never merge a schema file the validator rejects.

If Step 2’s ephemeral instance never boots. The dry-run cannot run without it, so the job should fail closed, not skip. Check the container logs (docker logs jg-ephemeral); the usual cause is a version skew between the image tag and the gremlinpython line. Pin both to the production TinkerPop version and re-run.

If Step 3 cannot reach the live graph. A read failure against production must fail the build, not pass it — a diff computed against nothing would wave through any change. Confirm the CI network can reach the read-only Gremlin endpoint and that the credential is valid; treat an unreachable live graph as a hard block until connectivity is restored.

If Step 4 blocks a legitimate high-cardinality key. The rejection means the key is queryable but unindexed. The fix is to add the index to graph-schema.yaml, not to remove the key from HIGH_CARDINALITY — that list is the guardrail. Re-run once the index is declared and let the dry-run confirm it applies.

If a forbidden change was already merged before the gate existed. Roll it back through the migration path, not by editing the live key in place, which JanusGraph forbids: register a new key, backfill, cut reads over, and deprecate the old key. Wire the stalled-schema and violation signals into the notification path from Alert Routing for Violations so the damage pages on-call instead of waiting to be found as latency.

Frequently Asked Questions

Why diff against the live graph instead of the previous version of the schema file? Because the file and reality drift. A schema mutation applied out-of-band, a partially failed migration, or a hand-run Groovy script can leave the live graph in a state the committed file no longer describes. Diffing the proposal against the running schema catches a forbidden change relative to what is actually deployed, which is the only comparison that prevents a production incident.

What exactly counts as a forbidden change the build should block? Retyping a property key, narrowing or changing its cardinality, dropping or renaming an existing key or label, and adding a queryable high-cardinality property with no backing index. JanusGraph treats a registered key’s type and cardinality as immutable, so these cannot be edited in place — they require a new key and a migration, which is why the gate blocks them rather than applying them.

Why use an ephemeral BerkeleyJE instance rather than a shared staging cluster? Isolation and speed. BerkeleyJE boots in-process with no external dependency, so every pull request gets a clean graph that starts empty and is discarded, with no risk of one job’s dry-run polluting another’s or of a mistake touching production. A shared staging cluster reintroduces exactly the shared-state hazard the gate exists to remove.

How does requiring a migration plan actually stop a bad merge? The diff writes a block marker for any forbidden change, which fails the required schema-gate status check. With branch protection, a red required check makes the merge button unavailable until the check passes, and it only passes when a committed migration plan accompanies the change — turning an irreversible edit into a reviewed, deliberate decision.