Fix Unassigned Shards in Elasticsearch with Allocation Explain

An unassigned shard is the cluster telling you that it has a shard copy it wants to place and no node it is allowed to place it on. The cluster always knows why. The _cluster/allocation/explain API returns that reason, decider by decider and node by node, and in almost every case it names the exact setting, threshold or condition that is blocking the shard. The work is reading it correctly and picking the fix that matches. This article explains how the allocator decides, walks through every common decider with its typical explanation and fix, covers delayed allocation and the retry_failed reroute, and ends with a runbook you can follow on a yellow cluster at 3 a.m.

Yellow, red, and what unassigned means

Every index has primaries and, usually, replicas. Cluster health is the worst state of any index:

• Green: every primary and every replica is assigned and started.
• Yellow: every primary is assigned, at least one replica is not. All data is searchable and writable, but you have lost redundancy for some shards.
• Red: at least one primary is unassigned. Some data is unavailable: searches touching that shard return partial results, and writes routed to it fail.

Red is an incident with its own procedure; see /guides/elasticsearch-red-cluster-runbook/. This article is mostly about the yellow case and the primary cases that are really allocation problems rather than lost data.

Start by locating the shards:

GET _cluster/health?filter_path=status,number_of_nodes,number_of_data_nodes,unassigned_shards,delayed_unassigned_shards,initializing_shards,relocating_shards

GET _cat/shards?v&h=index,shard,prirep,state,node,unassigned.reason,unassigned.at,unassigned.details&s=state

The unassigned.reason column tells you why the shard became unassigned in the first place. That is history, not the current blocker. Common values:

delayed_unassigned_shards in cluster health counts shards waiting out the node-left delay. Those are expected to recover on their own; do not "fix" them.

The mental model: deciders vote, the allocator obeys

Shard allocation is run by the elected master. For each unassigned copy, it asks a chain of allocation deciders, for each candidate node, whether the copy may go there. Each decider answers YES, NO or THROTTLE. The combined answer for a node is:

• NO if any decider says NO.
• THROTTLE if none says NO and at least one says THROTTLE: allowed, but not right now.
• YES otherwise.

If at least one node returns YES, the copy is assigned (subject to balancing). If every node returns NO, the copy stays unassigned, and the allocator re-checks every time the cluster state changes. That is why many unassigned shards fix themselves once you remove the cause: you do not need to "trigger" anything. The exception is max_retry, covered below, which needs a manual retry.

Primaries have one additional rule: an existing primary can only be allocated to a node that holds a valid, in-sync copy of the data. The allocator does not invent a primary from nothing. If no node has an in-sync copy, the answer is no_valid_shard_copy, which is the red-cluster scenario.

Replicas are rebuilt from the primary, so they can go to any node that the deciders allow.

Reading allocation explain

Call it with a specific shard:

GET _cluster/allocation/explain
{
  "index": "logs-app-2026.10.20",
  "shard": 2,
  "primary": false
}

Called with no body, it explains the first unassigned shard it finds, and returns an error if none is unassigned. With many indices, always name the shard so you know which one you are looking at.

A trimmed response for a replica that cannot be placed:

{
  "index": "logs-app-2026.10.20",
  "shard": 2,
  "primary": false,
  "current_state": "unassigned",
  "unassigned_info": {
    "reason": "NODE_LEFT",
    "at": "2026-10-21T02:14:07.112Z",
    "details": "node_left [h3Kq...]",
    "last_allocation_status": "no_attempt"
  },
  "can_allocate": "no",
  "allocate_explanation": "Elasticsearch isn't allowed to allocate this shard to any of the nodes in the cluster...",
  "node_allocation_decisions": [
    {
      "node_name": "data-1",
      "node_decision": "no",
      "deciders": [
        {
          "decider": "same_shard",
          "decision": "NO",
          "explanation": "a copy of this shard is already allocated to this node [...]"
        }
      ]
    },
    {
      "node_name": "data-2",
      "node_decision": "no",
      "deciders": [
        {
          "decider": "disk_threshold",
          "decision": "NO",
          "explanation": "the node is above the low watermark cluster setting [cluster.routing.allocation.disk.watermark.low=85%] ..."
        }
      ]
    }
  ]
}

The fields that matter:

• can_allocate: the overall answer. Values you will see include no (every node refuses), throttled (allowed but waiting for recovery slots), allocation_delayed (waiting out the node-left delay), no_valid_shard_copy (a primary with no usable copy), awaiting_info (the master is still fetching shard store information from nodes, usually right after a restart), and yes.
• unassigned_info.reason and details: why it became unassigned; for ALLOCATION_FAILED, details holds the last failure message, which is often the real story (for example a corrupt file or a mapping failure on the target).
• node_allocation_decisions[].deciders: the per-node blockers. Only NO and THROTTLE decisions appear by default. Add ?include_yes_decisions=true to see every decider.
• ?include_disk_info=true adds the master's view of disk usage per node, which is what the disk decider actually used.

The exact wording of allocate_explanation and the decider explanations differs between versions. The decider names and the settings they cite are stable; read those.

The one-line triage

Group the decisions by decider before reading any node in detail. If every node says NO for the same decider, that decider is the cause. If each node says NO for a different reason, you have several independent blockers, and fixing one may not be enough. A useful pattern: one node says same_shard (it holds the primary), every other node says disk_threshold. Then the fix is disk, and same_shard is just the normal rule.

Every common decider, with its fix

same_shard

What it enforces: two copies of the same shard never go on the same node. This also covers nodes on the same host when cluster.routing.allocation.same_shard.host is enabled (it is false by default).

Typical cause: fewer data nodes than copies. A single-node cluster can never place replicas: one node, one primary, same_shard refuses the replica. A two-node cluster with number_of_replicas: 2 has the same problem for the second replica.

Fix: add nodes, or match replicas to the node count.

PUT logs-app-*/_settings
{ "index.number_of_replicas": 0 }

For development clusters, "index.auto_expand_replicas": "0-1" makes the replica count follow the node count. Do not use 0 replicas in production unless the data is reproducible from elsewhere.

disk_threshold

What it enforces: the disk watermarks. Defaults:

• cluster.routing.allocation.disk.watermark.low: 85%. No new shards are allocated to a node above it. It does not apply to primaries of newly created indices.
• cluster.routing.allocation.disk.watermark.high: 90%. The cluster tries to move shards off a node above it.
• cluster.routing.allocation.disk.watermark.flood_stage: 95%. Every index with a shard on that node gets the index.blocks.read_only_allow_delete block; in recent versions the block is removed automatically when usage falls back below the high watermark.

In recent 8.x releases, each watermark also has a max_headroom companion that caps the free space required on very large disks when the watermark is a percentage (by default 200 GB for low, 150 GB for high and 100 GB for flood stage). A 20 TB disk therefore hits the low watermark when free space drops below 200 GB, not at 85% used. These headroom defaults apply only while the matching watermark is left at its default; if you set a watermark explicitly, set its max_headroom too if you want one.

Typical cause: the remaining nodes are too full to take the copies of a node that left, or a tier is simply full.

Confirm:

GET _cat/allocation?v&s=disk.percent:desc

GET _cluster/allocation/explain?include_disk_info=true
{ "index": "logs-app-2026.10.20", "shard": 2, "primary": false }

Fix: free disk (delete or snapshot-and-delete old indices, reduce replicas on old data, force merge read-only indices that have many deleted documents), or add capacity. Raising the watermarks is a short-term lever only; it trades a yellow cluster for the risk of a full disk. Details in /guides/elasticsearch-disk-watermarks/.

filter

What it enforces: allocation filters on the index or the cluster: index.routing.allocation.require.*, include.*, exclude.*, and the cluster-level cluster.routing.allocation.require.*, include.*, exclude.*. Attributes include _name, _ip, _host, _id and any custom node.attr.*.

Typical causes:

• A shrink that was abandoned and left index.routing.allocation.require._name pointing at one node. If that node leaves, the index cannot allocate anywhere.
• A node decommissioned with cluster.routing.allocation.exclude._name and never removed from the exclusion. When the node comes back, nothing is allocated to it.
• Old attribute-based hot/warm settings (require.box_type: warm) on indices, after the nodes were reconfigured to tier roles without that attribute.

Confirm: the explanation names the setting and its value. Then:

GET logs-app-2026.10.20/_settings/index.routing.allocation*?flat_settings=true

GET _cluster/settings?filter_path=*.cluster.routing.allocation

Fix: remove or correct the filter by setting it to null:

PUT logs-app-2026.10.20/_settings
{ "index.routing.allocation.require._name": null }

data_tier

What it enforces: index.routing.allocation.include._tier_preference. The shard may only go to a node in the first listed tier that exists.

Typical cause: ILM moved an index to data_warm and the warm tier has no room, or a node was given the wrong role. The explanation names the preferred tiers.

Fix: add capacity to the tier, or correct node roles. See /guides/elasticsearch-data-tiers-ilm/.

awareness

What it enforces: shard allocation awareness. With cluster.routing.allocation.awareness.attributes: zone, copies of a shard are spread across zone values. With forced awareness (cluster.routing.allocation.awareness.force.zone.values), the allocator will not put more copies into the remaining zones than they would hold if every listed zone were present.

Typical cause: a zone is down and forced awareness is doing exactly its job; or a node is missing its node.attr.zone value; or the number of copies is larger than the number of zone values allows to spread.

Confirm: the explanation states the attribute, the number of copies and the per-value target. Check every node has the attribute:

GET _cat/nodeattrs?v&h=node,attr,value&s=attr

Fix: if a zone is down, bringing it back is the fix; yellow is the intended state meanwhile. If a node is missing the attribute, set it in elasticsearch.yml and restart that node. See /guides/elasticsearch-cluster-topology/.

max_retry

What it enforces: after a shard fails to allocate index.allocation.max_retries times in a row (default 5), the allocator stops trying. The explanation says the shard has exceeded the maximum number of retries and tells you to call the reroute API with retry_failed.

Typical causes: something on the target node made recovery fail five times: a transient disk-full during recovery, a mapping conflict, a corrupted file on the source, a circuit breaker trip, a plugin or analyzer missing on a node. The decider is a symptom; the real cause is in unassigned_info.details (the last failure) and in the node logs.

Confirm:

GET _cluster/allocation/explain
{ "index": "orders-v7", "shard": 0, "primary": false }

Read unassigned_info.details carefully. failed_allocation_attempts shows the count.

Fix: fix the underlying cause first, then retry:

POST _cluster/reroute?retry_failed=true

This resets the failure counter for all shards that hit the limit and lets the allocator try again. Retrying without fixing the cause gets you five more failures and the same state, plus extra load from five more recoveries.

enable

What it enforces: cluster.routing.allocation.enable (values all, primaries, new_primaries, none) and the index-level index.routing.allocation.enable.

Typical cause: a rolling restart procedure set allocation to primaries and the last step, setting it back, was forgotten. Every replica of every node restarted since stays unassigned.

Confirm: the explanation names the setting and the value.

Fix:

PUT _cluster/settings
{ "persistent": { "cluster.routing.allocation.enable": null } }

Setting it to null restores the default, all. Check transient as well as persistent; transient cluster settings are deprecated but may still exist on clusters upgraded from older versions.

shards_limit

What it enforces: index.routing.allocation.total_shards_per_node (per index) and cluster.routing.allocation.total_shards_per_node (all shards). These are hard caps.

Typical cause: a cap that fits the healthy cluster exactly. An index with 10 shard copies and total_shards_per_node: 1 on 10 nodes cannot place the copies of a failed node anywhere.

Fix: raise the cap by one, or remove it, at least until the node returns.

PUT logs-app-2026.10.20/_settings
{ "index.routing.allocation.total_shards_per_node": 2 }

Node left and delayed allocation

When a node leaves, its replicas become unassigned with reason NODE_LEFT. Elasticsearch does not immediately rebuild them elsewhere. It waits index.unassigned.node_left.delayed_timeout, which defaults to 1 minute, in case the node comes back. If it returns in time, its copies can be reused, and only the operations missed meanwhile need to be replayed. If not, new replicas are built from the primaries on other nodes, which costs a full copy over the network for every affected shard.

While waiting, allocation explain reports can_allocate: allocation_delayed with the configured and remaining delay:

{
  "can_allocate": "allocation_delayed",
  "configured_delay_in_millis": 60000,
  "remaining_delay_in_millis": 41230
}

For planned maintenance that takes longer than a minute, raise the delay beforehand:

PUT _all/_settings
{ "settings": { "index.unassigned.node_left.delayed_timeout": "10m" } }

Arithmetic for whether that is worth it: a node holding 3 TB of replicas, rebuilt at an aggregate 1 GB/s across the cluster, takes about 3,000 seconds, or 50 minutes, of network and disk load. Then when the node returns, the cluster rebalances again. If the node is back in 8 minutes, a 10-minute delay avoids both moves.

The delay applies to replicas only. A primary on the departed node is replaced immediately by promoting an in-sync replica, so writes continue.

Throttling is not a failure

THROTTLE decisions come from the recovery concurrency limits: cluster.routing.allocation.node_concurrent_recoveries (default 2 concurrent incoming and outgoing recoveries per node) and cluster.routing.allocation.node_initial_primaries_recoveries (default 4, for primaries recovering from local disk after a restart). A shard reported as throttled will be assigned when a slot frees. After a full restart, many throttled shards are normal.

Bandwidth for peer recovery is limited by indices.recovery.max_bytes_per_sec, 40mb by default, except on dedicated cold or frozen nodes, where the default scales with the node's total memory (up to 250 MiB/s on nodes with more than 32 GiB). Raising it speeds recovery at the cost of I/O for searches and indexing. Watch active recoveries:

GET _cat/recovery?v&active_only=true&h=index,shard,stage,source_node,target_node,bytes_percent,time

Other deciders you will meet

• replica_after_primary_active: a replica waits until its primary is started. Fix the primary first.
• node_version: a replica cannot go to a node running an older version than its primary. Normal mid-upgrade; finish the upgrade. See /guides/elasticsearch-rolling-upgrades/.
• snapshot_in_progress and restore_in_progress: allocation interacts with running snapshots and restores; the explanation names which.
• resize: shrink, split and clone targets must be allocated on nodes holding the source shards.

Decision table

Failure modes and anti-patterns

Running allocate_empty_primary to turn a cluster green. This creates an empty primary and discards whatever data the shard had. It is never the fix for a replica problem, and for a primary problem it is a deliberate data-loss decision. See /guides/elasticsearch-red-cluster-runbook/.

Calling retry_failed in a loop. Symptom: recoveries start, fail, start again; node logs repeat the same exception. Fix: read unassigned_info.details, fix the cause (disk, mapping, missing plugin, corruption), then retry once.

Treating same_shard as the problem. same_shard appears on the node holding the primary for every unassigned replica. It is only the problem when it is the reason on every node, which means there are not enough nodes.

Raising watermarks to make yellow go away. Symptom: cluster green, then a node fills to 100% during the next merge or relocation, and indexing stops on every index with a shard there. Fix: free or add disk instead.

Forgetting to re-enable allocation after maintenance. Symptom: yellow for days after a rolling restart; every unassigned shard says enable. Fix: reset cluster.routing.allocation.enable to null, and put the reset step in the runbook with a check.

Ignoring a node that is "flapping" in and out. Symptom: repeated NODE_LEFT reasons, constant recoveries. Confirm: master logs show the node leaving and rejoining; check network, long GC pauses (_nodes/stats/jvm), and disk. Fix: the node, not the allocation settings.

Runbook: a yellow cluster

1. Scope it. GET _cluster/health. Note unassigned_shards, delayed_unassigned_shards, initializing_shards, number_of_data_nodes. Is a node missing compared with what you expect?
2. If shards are delayed or initializing, wait one delay period (1 minute by default) and re-check. Most yellow states after a node restart resolve here.
3. List the unassigned shards and their reasons. GET _cat/shards?v&h=index,shard,prirep,state,unassigned.reason,unassigned.details&s=state
4. Explain one shard per distinct reason. GET _cluster/allocation/explain with index, shard, primary.
5. Group the decider NOs. Identify the decider that blocks on every candidate node.
6. Apply the matching fix from the decision table. Prefer the fix that removes the cause (disk, filter, roles, missing node) over one that loosens a safety setting.
7. For max_retry only, after the cause is fixed: POST _cluster/reroute?retry_failed=true.
8. Watch recovery. GET _cat/recovery?v&active_only=true. Throttling is normal.
9. Confirm green. GET _cluster/health?wait_for_status=green&timeout=5m.
10. Record the cause and add a check (a disk alert, a post-maintenance allocation check) so the same decider does not surprise you again.

Key takeaways

• Yellow is an unassigned replica, red an unassigned primary; unassigned.reason is history, the deciders are the current blocker.
• _cluster/allocation/explain with a named shard returns can_allocate and per-node decider decisions; group the NOs by decider to find the cause.
• Most deciders self-heal once the cause is removed; max_retry needs POST _cluster/reroute?retry_failed=true after the cause is fixed.
• Disk watermark defaults are 85%, 90% and 95%, with max_headroom caps on large disks in recent 8.x releases.
• Node-left allocation is delayed by 1 minute by default; raise index.unassigned.node_left.delayed_timeout before long maintenance.
• Never use allocate_empty_primary or allocate_stale_primary to fix a replica problem.

How Sparced helps

When the cluster is yellow or red, Sparced calls _cluster/allocation/explain for each stuck shard, so every unassigned copy is listed with the cluster's own reason: the decider, the setting it cites and the nodes it applies to. It reads cluster health, shards, nodes and allocation in the same pass, so the disk usage behind a disk_threshold refusal or the missing node behind a NODE_LEFT appears next to the shard. Each finding comes with a fix command with your index, shard and node names filled in, such as removing a leftover allocation filter or running retry_failed after the cause is fixed, and a check to confirm the shard was assigned. It never invents a cause: the reason shown is the one the cluster returned, and it does not change any allocation setting itself.