Elasticsearch Multi-Tenancy: Index per Tenant vs Shared Index

Every SaaS team that puts customer data into Elasticsearch faces the same choice: give each tenant its own index, or put many tenants in shared indices and separate them with a tenant field. Index per tenant gives clean isolation and trivial deletion but multiplies shards until the cluster spends its time managing metadata. Shared indices scale to any number of tenants but make isolation a query-time promise and let one large tenant hurt everyone. The answer for most systems is neither extreme: a shared, routed index for the long tail of small tenants, dedicated indices for the few large ones, and an alias per tenant so the application never knows which is which. This article gives you the cost model for each pattern, the exact configuration for routing, filtered aliases and document level security, and the failure modes that show up at year two.

The mental model: what a tenant costs the cluster

Three resources decide which pattern works.

Shards and cluster state. Every index has at least one primary shard, every shard has replicas, and every shard is tracked in the cluster state that the elected master maintains and publishes to every node. Each index also carries its mappings in that state. A thousand tiny indices cost far more to manage than one index with the same total data: more cluster state to publish on every change, more shard-level overhead on the data nodes, more recoveries when a node restarts. Elasticsearch enforces a soft ceiling with cluster.max_shards_per_node, default 1,000 shards per non-frozen data node, counted across the cluster (the limit times the number of eligible data nodes). See shard sizing for the full capacity model.

Mappings. Each index's mapping lives in cluster state and costs heap on data nodes. Elastic's sizing guidance for 8.x expresses data node heap overhead per mapped field per index rather than per shard. A schema of 500 fields is cheap once and expensive when copied into 5,000 indices.

Shared execution. All indices on a node share its thread pools (search, write), its heap, its page cache and its disk IO. No index-level setting gives a tenant a guaranteed slice of CPU. Isolation in Elasticsearch is achieved by placement (which nodes hold which shards) or by separate clusters, never by priority settings.

Routing ties these together. A document's shard is chosen by:

shard_num = hash(_routing) % num_primary_shards

where _routing defaults to the document _id. If you set the routing to the tenant id, all of a tenant's documents land on one shard, and a search with that routing value touches only that shard instead of all of them.

The three patterns

Worked example: the shard math

Assume 4,000 tenants, one primary and one replica each, on data nodes allowed 1,000 shards each.

• Index per tenant: 4,000 * 2 = 8,000 shards. 8,000 / 1,000 = 8 data nodes minimum just to satisfy the shard limit, whatever the data volume. If the tenants hold 2 TB of primary data in total, that is 2,000 GB / 4,000 primaries = an average of 500 MB per shard, far below the tens of gigabytes per shard that Elastic's guidance targets.
• Add a time dimension (one index per tenant per month, kept for 12 months): 4,000 * 12 * 2 = 96,000 shards. 96 nodes for the limit alone. This is the shape of cluster that ends in a master node struggling to publish cluster state.
• Shared index: the same 2 TB of primary data, with 1 replica, at a target of roughly 40 GB per primary means 2,000 / 40 = 50 primaries, 100 shards in total. The tenant count no longer appears in the arithmetic.

Raising cluster.max_shards_per_node makes the error go away and leaves the cost in place. Treat the limit as a warning about the design.

Pattern one: index per tenant

It is the right choice when tenants are few and large, need different mappings, have contractual isolation or deletion requirements, or need different retention.

Use an index template so every tenant index gets the same settings and mappings:

PUT _index_template/tenant-template
{
  "index_patterns": ["tenant-*"],
  "priority": 200,
  "template": {
    "settings": { "number_of_shards": 1, "number_of_replicas": 1 },
    "mappings": {
      "dynamic": "strict",
      "properties": {
        "title":      { "type": "text" },
        "created_at": { "type": "date" },
        "attributes": { "type": "flattened" }
      }
    }
  }
}

Keep tenant indices at one primary unless a tenant is large enough to need more; most tenant indices are small. Put an alias in front of each (tenant-acme pointing at tenant-acme-v1) so a mapping change can be done with a reindex and alias swap, as described in zero-downtime reindexing.

Security is straightforward: a role per tenant, or one role with an index pattern, restricted to that tenant's indices.

POST _security/role/tenant_acme_reader
{
  "indices": [
    { "names": ["tenant-acme*"], "privileges": ["read"] }
  ]
}

The weakness is cost at scale, and one subtler problem: with many indices, a mapping update applied to all of them (through a template change plus a rollover, or a PUT _mapping across a pattern) becomes a large cluster state change.

Pattern two: shared index with routing and filtered aliases

For thousands of small tenants, put them in a shared index with a tenant_id keyword field and route by tenant.

PUT saas-shared-v1
{
  "settings": {
    "number_of_shards": 12,
    "number_of_replicas": 1,
    "index.routing_partition_size": 3
  },
  "mappings": {
    "_routing": { "required": true },
    "properties": {
      "tenant_id":  { "type": "keyword" },
      "title":      { "type": "text" },
      "created_at": { "type": "date" },
      "attributes": { "type": "flattened" }
    }
  }
}

Then give each tenant a filtered alias that applies both the filter and the routing:

POST _aliases
{
  "actions": [
    {
      "add": {
        "index": "saas-shared-v1",
        "alias": "t-acme",
        "filter": { "term": { "tenant_id": "acme" } },
        "routing": "acme"
      }
    }
  ]
}

What this buys:

• The application writes to and searches t-acme without adding routing or a tenant filter itself. Writes through the alias get the routing value. Searches get the routing and the filter.
• A search touches only the shards that the routing value maps to, not all 12.
• Moving a tenant later from the shared index to its own index is an alias change, invisible to the application.

"_routing": {"required": true} makes Elasticsearch reject index, update and delete requests without a routing value, which prevents one class of bug: a document written without routing lands on the wrong shard, and a later delete or get with routing cannot find it.

routing_partition_size: avoiding the hot shard

With plain tenant routing, a tenant's documents all go to one shard. A large tenant makes that shard huge and makes every one of its searches hit a single shard. index.routing_partition_size spreads each routing value over a subset of shards:

shard_num = (hash(_routing) + hash(_id) % routing_partition_size) % num_primary_shards

With 12 primaries and a partition size of 3, each tenant spans 3 shards. Searches for a tenant hit 3 shards instead of 1 or 12, and a large tenant's data is split three ways. The setting is fixed at index creation, must be greater than 1 and less than the number of primary shards, requires _routing to be required in the mapping, and cannot be combined with join fields.

Worked example: 12 shards, partition size 3. A search for one tenant fans out to 3 / 12 = 25% of the shards. Without partitioning it would be 1 / 12, about 8%, but the largest tenant would have a whole shard of its own data and potentially a shard several times larger than the others.

The shared-index trade-offs to accept consciously

• Term statistics are shared. BM25 IDF is computed per shard over all tenants' documents on it. A word common in one big tenant's documents becomes "common" for every tenant on that shard. For most applications this is invisible. For search-quality-sensitive products, it is a reason to give large tenants their own index. See relevance tuning.
• One mapping for everyone. Tenant-specific custom fields cannot each become a mapped field, or you hit index.mapping.total_fields.limit (default 1,000) and a mapping explosion. Put tenant-defined attributes in a flattened field, or in a key/value nested structure, and see mapping design.
• Deleting a tenant is expensive. POST saas-shared-v1/_delete_by_query with a term on tenant_id and the tenant's routing marks the documents deleted; the space returns only as segments merge. For regulated deletion, record when the delete ran and understand that deleted documents physically persist in segments until merged away.
• Alias count. Thousands of filtered aliases are themselves cluster state. A common alternative is to have the application inject the routing and filter itself and reserve aliases for the tenants that need them, or rely on document level security instead.

Document level security as the enforcement layer

Filtered aliases are a convenience, not a security boundary: anyone with read access to the underlying index can bypass the alias. When tenants or tenant-facing services query Elasticsearch with their own credentials, enforce the separation with document level security (DLS), which applies a query to every search a role makes. DLS and field level security are features of the paid subscriptions, so check your license before designing around them.

A single role can serve every tenant with a templated query that reads the tenant from the user's metadata:

POST _security/role/tenant_reader
{
  "indices": [
    {
      "names": ["saas-shared-*"],
      "privileges": ["read"],
      "query": {
        "template": {
          "source": "{\"term\": {\"tenant_id\": \"{{_user.metadata.tenant_id}}\"}}"
        }
      }
    }
  ]
}

POST _security/user/acme-svc
{
  "password": "use-a-real-secret",
  "roles": ["tenant_reader"],
  "metadata": { "tenant_id": "acme" }
}

With API keys you can achieve the same per key, by creating the key with role_descriptors that contain the tenant's query directly. The key can only see what its descriptors allow, intersected with the creating user's privileges.

Three things to know about DLS in production:

• It adds a filter to every query; the filter on a keyword field is cheap and cacheable, but it is still evaluated per shard.
• It does not apply routing. A DLS-restricted search without routing still fans out to every shard of the index. Pass routing from the application when you can.
• DLS is designed for read access. Do not rely on it to validate what a tenant writes. If tenants write with their own credentials, validate tenant_id on the way in, for example in an ingest pipeline set as the index's final_pipeline or in your API layer. See security architecture.

Noisy neighbors

The hardest multi-tenant problem is not data separation. It is one tenant's expensive query or bulk import degrading everyone else's latency.

How to see it

• Attribute requests. Have the application send an X-Opaque-Id header with the tenant id on every request. It appears in the tasks API and in slow logs, so GET _tasks?actions=*search*&detailed shows which tenant's searches are running now.
• Slow logs. Enable search slow logs on the shared index at sensible thresholds and look for one tenant id dominating. See search latency diagnosis.
• Thread pools. Rising search or write queue sizes and rejections on the nodes holding one tenant's shards point to a hot spot. See thread pools and rejections.
• Shard sizes. GET _cat/shards/saas-shared-v1?v&s=store:desc shows whether one shard is much larger than its siblings, the signature of a big tenant concentrated by routing.

How to contain it

Allocation filtering uses custom node attributes. Tag the dedicated nodes in elasticsearch.yml with node.attr.tenant_pool: dedicated, then:

PUT tenant-bigcorp-v1/_settings
{
  "index.routing.allocation.require.tenant_pool": "dedicated"
}

Shared indices get the opposite requirement (for example "index.routing.allocation.require.tenant_pool": "shared") so the general population never lands on the dedicated nodes. If the dedicated nodes cannot hold the shards (disk, or one node too few for a replica), the shards stay unassigned, and the allocation explain API will name the filter decider as the reason. See unassigned shards.

The hybrid pattern in practice

The pattern that ages best:

1. All tenants start in a shared, routed index behind a per-tenant alias, or behind application-side routing plus DLS.
2. A tenant whose data passes a threshold (a rule of thumb: when its share would be a meaningful fraction of a target shard size, or when it needs its own mapping or retention) is promoted to a dedicated index.
3. Promotion is a reindex of that tenant's documents and an alias swap.
4. Very large or regulated tenants get dedicated nodes through allocation filtering, or a dedicated cluster.

Promotion, step by step:

PUT tenant-bigcorp-v1
{
  "settings": { "number_of_shards": 2, "number_of_replicas": 0, "refresh_interval": "-1" }
}

POST _reindex?wait_for_completion=false&slices=auto
{
  "source": {
    "index": "saas-shared-v1",
    "query": { "term": { "tenant_id": "bigcorp" } }
  },
  "dest": { "index": "tenant-bigcorp-v1", "routing": "discard" }
}

Then restore replicas and the refresh interval, compare document counts, and swap atomically:

POST _aliases
{
  "actions": [
    { "remove": { "index": "saas-shared-v1", "alias": "t-bigcorp" } },
    { "add":    { "index": "tenant-bigcorp-v1", "alias": "t-bigcorp" } }
  ]
}

Writes arriving during the copy must be handled, either by pausing that tenant's writes, dual-writing, or a catch-up reindex of documents changed since the copy started. The full method is in zero-downtime reindexing. Only after the swap is verified do you delete the tenant's documents from the shared index.

Note the "routing": "discard" in the destination. Reindex keeps each document's routing value by default (dest.routing accepts keep, discard or =value). Kept, the value bigcorp would send every document to one shard of the new two-shard index, which defeats the point of giving the tenant its own index.

Failure modes and anti-patterns

Decision checklist

Run through these when choosing or revisiting the model:

1. How many tenants now, and in three years? Above a few hundred, index per tenant needs a hard justification.
2. What is the size distribution? If the largest tenant is orders of magnitude bigger than the median, plan the hybrid pattern from day one.
3. Do tenants need different mappings, analyzers or retention? Those push toward dedicated indices for the tenants that need them.
4. Who queries Elasticsearch directly: only your service, or tenant-scoped credentials? The second requires DLS or API keys with role descriptors.
5. What does tenant deletion have to guarantee, and how fast?
6. Compute shards: tenants * indices per tenant * (1 + replicas). Compare with 1,000 per data node.
7. Decide how you will attribute load to tenants (X-Opaque-Id, slow logs) before the first noisy neighbor incident, not during it.
8. Write down the promotion threshold and the promotion procedure.

Key takeaways

• Index per tenant multiplies shards and mappings; the arithmetic, tenants times indices times copies, decides whether it is viable.
• Shared indices with tenant routing keep the shard count independent of tenant count; require _routing and consider routing_partition_size for uneven tenants.
• Filtered aliases are a convenience and an indirection layer, not a security boundary; use DLS or scoped API keys to enforce isolation.
• Noisy neighbors are contained by placement, by promotion to dedicated indices, and by limits in your API layer; Elasticsearch has no per-tenant CPU quota.
• The hybrid pattern, shared tail and dedicated head behind per-tenant aliases, ages best, because moving a tenant becomes a reindex and an alias swap.

How Sparced helps

Sparced reads the cluster's shard, index and node state in one pass, so the symptoms of a multi-tenant design drifting into trouble are in front of you: total shards against the per-node limit, the shard sizes per index (where one routed shard much larger than its siblings points to a heavy tenant), and search or write thread pool rejections per node. When allocation filtering for dedicated tenant nodes leaves shards unassigned, it calls _cluster/allocation/explain for each stuck shard and prints the decider's own explanation, a fix command with the index and node names filled in, and a check to confirm it. It does not see your application's tenant ids or rate limits, and it writes nothing to the cluster.