Choosing Cassandra, ScyllaDB, or Bigtable

Deciding between Cassandra, ScyllaDB, and Bigtable for a JanusGraph deployment is a scoring exercise, not a gut call: each backend wins a different combination of write rate, tail-latency budget, multi-region need, and appetite for operations, and the right answer falls out of weighing those against your actual constraints. This guide sits under the Storage Backend Selection and Sizing reference and turns that comparison into a repeatable procedure — score the workload, map the winner to a concrete storage.backend and driver profile, then prove the choice with a pilot before any data commits to it. The failure it prevents is the expensive reversal — a backend picked from a benchmark blog post that turns out wrong for your write pattern or budget, discovered only after production data has landed and a full re-platforming is the only exit. Each step below produces evidence you can put in a decision record, and the last step tells you how to reverse a wrong pick with the least data movement.

Prerequisites

Gather the inputs before scoring — a decision made without numbers just re-encodes a preference.

• JanusGraph 0.6.x or 1.0.x and the ability to stand up a throwaway pilot against each candidate backend.
• Measured workload numbers: peak and sustained write rate (mutations/sec), the read P99 SLO you must hold, region topology, and a monthly budget ceiling.
• A CQL-capable environment (Cassandra or ScyllaDB nodes, or ScyllaDB Cloud) and, if Bigtable is a candidate, a GCP project with the Bigtable API enabled and the Bigtable HBase client available.
• A representative benchmark script — the pilot must replay your real read/write mix, not a generic YCSB profile, because graph traversals stress partitions differently than key-value loads.
• A decided replication and consistency target. The ScyllaDB Migration guide and the sizing math in Sizing Cassandra Clusters for JanusGraph give the parameters the pilot will use.

Step 1 — Score your workload against the criteria

Weight the five criteria by how much each one hurts if you get it wrong, then score each backend 1–5. Multiply, sum, and let the totals rank the candidates before you touch a benchmark.

# weight each criterion 0-1 by importance to YOUR workload, then score 1-5.
weights = {
    "write_rate": 0.25,
    "tail_latency": 0.30,   # tighten this if you have a hard read P99 SLO
    "multi_region": 0.15,
    "managed_ops": 0.20,    # raise if you have no dedicated DBA capacity
    "budget_steady": 0.10,
}
scores = {
    "cassandra": {"write_rate": 4, "tail_latency": 3, "multi_region": 5, "managed_ops": 2, "budget_steady": 5},
    "scylladb":  {"write_rate": 5, "tail_latency": 5, "multi_region": 4, "managed_ops": 3, "budget_steady": 5},
    "bigtable":  {"write_rate": 4, "tail_latency": 4, "multi_region": 4, "managed_ops": 5, "budget_steady": 3},
}
ranked = sorted(
    ((sum(weights[k] * s[k] for k in weights), name) for name, s in scores.items()),
    reverse=True,
)
for total, name in ranked:
    print(f"{name:10s} {total:.2f}")

The template scores are a starting bias from the comparison below; replace them with your own judgement. The weights matter more than the scores — a hard tail-latency SLO pushes ScyllaDB to the top, while zero DBA capacity pushes Bigtable there.

Verify: the ranking is stable — nudge each weight by ±0.05 and confirm the top choice does not flip. If it does, the decision is genuinely close and the pilot in Step 3 must break the tie.

python3 score.py           # run the scorer above
# then re-run with perturbed weights and diff the top line

Step 2 — Map the winner to janusgraph.properties

Translate the top-ranked backend into a concrete configuration. Cassandra and ScyllaDB share the cql adapter and differ only in driver multiplexing; Bigtable uses the hbase adapter through the Bigtable HBase client, which is a different property namespace entirely.

# --- Cassandra (cql) ---
storage.backend=cql
storage.hostname=10.0.1.10,10.0.1.11,10.0.1.12
storage.cql.keyspace=janusgraph_prod
storage.cql.local-datacenter=us-east-1
storage.cql.max-requests-per-connection=1024      # Cassandra-tuned multiplexing

# --- ScyllaDB (cql, same adapter, retuned driver) ---
storage.backend=cql
storage.hostname=10.0.2.10,10.0.2.11,10.0.2.12
storage.cql.keyspace=janusgraph_prod
storage.cql.local-datacenter=us-east-1
storage.cql.max-requests-per-connection=4096      # shard-per-core rewards higher multiplexing

# --- Bigtable (hbase adapter via Bigtable HBase client) ---
storage.backend=hbase
storage.hbase.ext.google.bigtable.project.id=my-gcp-project
storage.hbase.ext.google.bigtable.instance.id=janusgraph-instance
storage.hbase.ext.hbase.client.connection.impl=com.google.cloud.bigtable.hbase2_x.BigtableConnection

The critical differences to internalize: the CQL backends key off storage.cql.* and a list of contact-point hosts, while Bigtable keys off storage.hbase.ext.* project and instance IDs and swaps the HBase connection implementation for Bigtable’s. The connection-budget keys that carry every mutation differ accordingly — reconcile them through Connection Pooling once the backend is chosen.

Verify: boot JanusGraph against the chosen config and confirm the adapter it actually loaded matches your intent.

grep -E "storage.backend|Bigtable|CQLStoreManager|HBaseStoreManager" \
  /var/log/janusgraph/server.log | tail -5

Step 3 — Pilot and benchmark against your real mix

A score is a hypothesis; the pilot is the test. Stand up a small instance of the top candidate (and the runner-up if the ranking was close), replay your real read/write mix, and record write throughput and read P99 — not averages, which hide the tail the SLO cares about.

import time, statistics
from gremlin_python.driver.driver_remote_connection import DriverRemoteConnection
from gremlin_python.process.anonymous_traversal import traversal


def benchmark(ws_endpoint, reads=5000):
    conn = DriverRemoteConnection(ws_endpoint, "g")
    g = traversal().withRemote(conn)
    latencies = []
    for i in range(reads):
        t0 = time.perf_counter()
        g.V().has("entity", "k", i % 1000).out("links").limit(20).toList()
        latencies.append((time.perf_counter() - t0) * 1000)
    conn.close()
    latencies.sort()
    p99 = latencies[int(0.99 * len(latencies)) - 1]
    print(f"p50={statistics.median(latencies):.1f}ms  p99={p99:.1f}ms")
    return p99


# benchmark("ws://scylla-pilot:8182/gremlin")

Run identical load against each candidate on comparably sized hardware — an unfair node count invalidates the comparison. Hold the pilot long enough to trigger compaction (CQL) or a scaling event (Bigtable), because tail latency during those events is exactly what separates the backends.

Verify: the winning backend’s measured P99 clears your SLO with margin, and write throughput meets peak with headroom.

# confirm the pilot sustained the target write rate without error
grep -c -E "Timeout|NoHostAvailable|DEADLINE_EXCEEDED" /var/log/janusgraph/janusgraph.log

Step 4 — Decide and record

Make the call from evidence, not the initial score, and write it down so the reasoning survives.

1. Confirm the pilot agrees with the score. If the benchmark contradicts the ranking (common when a workload has an unusual partition shape), trust the pilot.
2. Record the decision with the weights, scores, and measured P99/throughput in a decision record, so a future re-evaluation starts from data rather than reopening the debate.
3. Provision at the sized node count from the sizing procedure and roll out — do not resize the pilot in place, which conflates benchmark and production topology.

Verify: the production config matches the piloted config exactly on every node.

for host in node1 node2 node3; do
  ssh "$host" "grep -E 'storage.backend|max-requests-per-connection|bigtable.instance' \
    /etc/janusgraph/janusgraph.properties"
done

Fallback and rollback procedures

Because a backend switch is a data migration, the cheapest reversal is the one you plan before committing. Keep the pilot artifacts until production is stable.

If Step 1’s ranking is a near-tie. Do not force it — let Step 3’s pilot decide, and weight tail latency higher, since a missed P99 SLO is the failure most likely to page you.

If Step 3’s pilot fails the SLO on the top candidate. Fall back to the runner-up’s pilot rather than tuning the loser indefinitely. If both CQL backends miss the SLO, the constraint is usually schema (a supernode or hot partition), not the backend — fix the model before re-benchmarking.

If you must reverse a wrong choice after production data has landed. Between the two CQL backends (Cassandra ↔ ScyllaDB) the migration is a keyspace-level data copy that reuses the same adapter; follow the ScyllaDB Migration procedure. Moving to or from Bigtable crosses the cql/hbase adapter boundary, so it is a full export-and-reload:

# adapter-crossing reversal: export via a Gremlin/Spark bulk read, reload into the new backend
# 1. freeze writes at the application boundary (drain the ingestion queue)
# 2. bulk-export the graph (GraphSON/CSV) from the wrong backend
# 3. stand up the correct backend, reload, and re-point storage.backend
# 4. re-run the Step 3 benchmark before unfreezing writes

If the rollback itself stalls. Never delete the source backend until the target passes the Step 3 benchmark and a read-consistency check on a sample of vertices. Keep the source read-only as the authoritative copy until then.

Related

• Up a level: Storage Backend Selection and Sizing — the reference that frames this scored decision alongside capacity math.
• Sizing Cassandra Clusters for JanusGraph — the node-count and heap procedure to run once you have chosen a CQL backend.
• ScyllaDB Migration — the procedure for reversing a Cassandra-versus-ScyllaDB decision with the shared cql adapter.
• Connection Pooling — the driver connection budget that differs between the cql and hbase adapters you configure here.