Zero-Downtime Reindexing in Elasticsearch: Aliases and Cutover

You cannot change the type or analyzer of an existing field in Elasticsearch. The inverted index, doc values and norms for that field were built at write time, and rewriting them means rewriting the documents. So every meaningful mapping change is a reindex into a new index. Done naively, that means downtime or lost writes. Done properly, it is invisible to users: the application only ever talks to an alias, you build the new index beside the old one with a sliced, throttled reindex, you catch up the writes that arrived during the copy, you verify, and you swap the alias in one atomic action, keeping the old index as the rollback. This article covers which changes need a reindex at all, the reindex API in detail with the throttling arithmetic, three ways to handle live writes, and the cutover and rollback steps.

The mental model: why mappings are frozen

A field's mapping decides how its values were turned into Lucene structures when each document was indexed: the analyzer produced the terms in the inverted index, the type decided the doc values encoding and the points structure for numbers and dates. Those structures live in immutable segments. Changing "type": "keyword" to "type": "long" would require reading every document's original value and building new structures, which is what a reindex is.

Reindex works only because of _source. It reads the original JSON of each document from _source (through scroll-style batches over the source index) and indexes it again into the destination, which applies the destination's mappings and, optionally, an ingest pipeline. If _source is disabled on the source index, reindex is impossible and you must re-ingest from the system of record. Synthetic _source reconstructs the document from doc values and stored fields, which reindex can use, with the caveats that come with synthetic source (for example, arrays may come back sorted and deduplicated for some types).

Which changes need a reindex

The in-place backfill with update_by_query

Adding a multi-field is the most common no-reindex change. Say title is text and you want sortable exact values:

PUT products/_mapping
{
  "properties": {
    "title": {
      "type": "text",
      "fields": { "raw": { "type": "keyword", "ignore_above": 256 } }
    }
  }
}

Existing documents are not in title.raw until they are reindexed in place:

POST products/_update_by_query?conflicts=proceed&slices=auto&wait_for_completion=false
{
  "query": { "bool": { "must_not": { "exists": { "field": "title.raw" } } } }
}

_update_by_query with no script simply re-indexes each matching document from its _source, picking up the new mapping. conflicts=proceed counts version conflicts (documents changed by the application during the run) instead of aborting, and is safe here because the application's own write already indexed the latest version under the new mapping. Note the query: an exists query on the new subfield finds the documents not yet backfilled, so the job can be restarted safely if it is interrupted. Documents whose title exceeds ignore_above never get a title.raw value, so a restarted job will process them again; that is harmless, just wasted work.

Alias-first design: do this before you need it

Zero-downtime reindexing is only possible if the application never addresses the concrete index. Create every index with a versioned name and an alias:

PUT products-v1
{
  "aliases": { "products": {} },
  "mappings": { "properties": { "title": { "type": "text" } } }
}

The application reads and writes products. When an alias points to exactly one index, writes through it work. When you need an alias pointing at several indices that still accepts writes, mark one with "is_write_index": true.

If you inherited an application writing to a concrete index called products, you cannot create an alias with the same name as an existing index. The migration is one short interruption or a two-step change: create products-v2, reindex, then in one _aliases request use the remove_index action on products and add the alias products to products-v2. Deleting the index and adding the alias happen atomically, but remove_index deletes the old data, so you lose the rollback unless you snapshot first. Do it once, and never again.

Data streams are a special case. Their backing indices are created by the template on rollover, so most mapping changes are applied by updating the index template and rolling over (POST my-logs/_rollover): the new backing index has the new mapping, old ones keep theirs. Only if you need the change on historical backing indices do you reindex those, with op_type: create, which data streams require. See index templates and data streams.

The reindex API in detail

Prepare the destination

Create the destination explicitly with its final mappings and settings. Never let reindex create it implicitly with dynamic mappings, which is how type mismatches survive a migration.

PUT products-v2
{
  "settings": {
    "number_of_shards": 6,
    "number_of_replicas": 0,
    "refresh_interval": "-1"
  },
  "mappings": {
    "dynamic": "strict",
    "properties": {
      "title":    { "type": "text", "fields": { "raw": { "type": "keyword" } } },
      "sku":      { "type": "keyword" },
      "price":    { "type": "scaled_float", "scaling_factor": 100 },
      "updated_at": { "type": "date" }
    }
  }
}

Replicas at 0 and refresh disabled make the bulk copy cheaper (see indexing performance). You will restore both before cutover. Make sure no index template with a matching pattern adds settings you did not expect; check with POST _index_template/_simulate_index/products-v2.

Run it as a task

POST _reindex?slices=auto&wait_for_completion=false&requests_per_second=2000
{
  "source": {
    "index": "products-v1",
    "size": 1000
  },
  "dest": {
    "index": "products-v2",
    "version_type": "external"
  },
  "conflicts": "proceed"
}

The response is a task id:

{ "task": "oTUltX4IQMOUUVeiohTt8A:12345" }

Key parameters:

• wait_for_completion=false returns immediately and runs the job as a task. The result is stored in the .tasks system index, so you can read it after completion. Always use it for anything that takes more than seconds.
• slices parallelizes the job. slices=auto lets Elasticsearch choose, typically one slice per shard of the source index up to a limit. A sensible manual value is the number of source shards; more slices than shards adds overhead without gain.
• source.size is the batch size per scroll request, default 1,000 documents. Larger batches help with small documents; with large documents keep the batch modest so each bulk request stays at a reasonable size.
• requests_per_second throttles the job. Default is no throttle (-1).
• conflicts: proceed counts version conflicts instead of failing.
• dest.version_type: external copies each document with the source's version number and only overwrites a destination document whose version is lower. Using it on the main copy as well as on the catch-up passes keeps the version numbers comparable between the two indices, which is what makes the catch-up safe, as below.
• dest.op_type: create only creates documents that do not exist in the destination; existing ones count as conflicts. Data streams require it.
• dest.pipeline runs an ingest pipeline on each document, useful for transforming fields during the move.
• script in the body can transform ctx._source per document; prefer an ingest pipeline for anything non-trivial because it is testable with the simulate API.

Monitor and control

GET _tasks/oTUltX4IQMOUUVeiohTt8A:12345

The fields that matter in task.status: total, created, updated, deleted, batches, version_conflicts, noops, retries (bulk and search), throttled_millis and requests_per_second. total is the number of operations the job expects to perform, and the sum of created, updated and deleted against it gives a progress estimate (version conflicts and noops also consume documents without appearing in that sum). For a sliced job the parent task's status is built from finished slices, so it can lag; list the per-slice child tasks with GET _tasks?parent_task_id=oTUltX4IQMOUUVeiohTt8A:12345&detailed to see live progress. When completed is true, check response.failures: an empty array means every batch succeeded.

Change the throttle without restarting:

POST _reindex/oTUltX4IQMOUUVeiohTt8A:12345/_rethrottle?requests_per_second=5000

Speeding up takes effect immediately; slowing down takes effect after the current batch. Cancel with:

POST _tasks/oTUltX4IQMOUUVeiohTt8A:12345/_cancel

A cancelled reindex leaves the documents already copied in place. That is fine for a destination you will either finish or delete.

The throttling arithmetic

Throttling works by padding each batch with a wait. Elastic documents it this way, with a batch size of 1,000 and requests_per_second=500:

target_time = batch_size / requests_per_second = 1000 / 500 = 2 seconds
wait_time   = target_time - write_time

If writing the batch took 0.5 seconds, the job waits 1.5 seconds before the next batch. The rate applies per slice as a share of the total: with slices, the requests per second is split across them.

Use it to plan the copy. Worked example: 120 million documents, and you can spare about 8,000 documents per second of indexing on the cluster without hurting production traffic (find that number by watching write thread pool queues and indexing latency while you raise the throttle).

• Duration: 120,000,000 / 8,000 = 15,000 seconds, about 4.2 hours.
• At 4,000 per second: 8.3 hours. At 20,000: 1.7 hours, if the cluster can sustain it.

Add the catch-up and verification time to that before you schedule the cutover window. Watch write thread pool rejections during the run; rising rejections mean the throttle is too high. See thread pools and rejections.

Handling writes that arrive during the copy

Reindex copies a point-in-time view taken at the start. Writes to the old index after that moment are not copied. Choose one of three strategies before you start.

Catch-up reindex with a timestamp

If every document has a trustworthy updated_at that the application sets on every write:

1. Record T0, a time a few minutes before you start the main reindex (the margin covers clock skew and in-flight writes).
2. Run the full reindex.
3. Run a delta reindex for everything changed since T0:

POST _reindex?slices=auto&wait_for_completion=false
{
  "source": {
    "index": "products-v1",
    "query": { "range": { "updated_at": { "gte": "2026-10-21T08:55:00Z" } } }
  },
  "dest": { "index": "products-v2", "version_type": "external" },
  "conflicts": "proceed"
}

version_type: external preserves the source document's version and only overwrites a destination document if the source version is higher, so a re-run never replaces newer data with older data. Repeat the delta with a later T each time; each pass is smaller. When the delta is small enough to finish in seconds, block writes briefly, run the last delta, swap the alias, and unblock.

Deletes are the weakness: a document deleted from the old index after the copy started is still in the new one. Either the application soft-deletes (sets a flag and updated_at), or it logs deletes so you can replay them, or you accept a short write freeze for the final pass. Compare document counts to detect the gap.

To block writes for the final pass:

PUT products-v1/_block/write

Writes are rejected while the block is in place, so the application must queue or retry them. Remove the block afterward with PUT products-v1/_settings and {"index.blocks.write": false}.

Verification and cutover

Before swapping, restore the production settings on the new index:

PUT products-v2/_settings
{
  "index": { "number_of_replicas": 1, "refresh_interval": "1s" }
}

Use your normal refresh interval if it is not the 1s default. Wait for the replicas:

GET _cluster/health/products-v2?wait_for_status=green&timeout=30m

Then verify:

1. Counts. GET products-v1/_count and GET products-v2/_count should match after a refresh, once writes are blocked or caught up.
2. Samples. Fetch a few hundred random ids from both and compare _source.
3. The point of the change. Run the queries or aggregations the migration was for against products-v2 directly.
4. Relevance. If analyzers changed, run your judgment set with the rank_eval API against both. See relevance tuning.
5. Latency. Run representative queries against the new index; new indices start with cold caches.

Then swap atomically:

POST _aliases
{
  "actions": [
    { "remove": { "index": "products-v1", "alias": "products" } },
    { "add":    { "index": "products-v2", "alias": "products" } }
  ]
}

Both actions apply as a single cluster state change: there is no moment where products points at nothing or at both. If the alias was a write alias, re-specify is_write_index on the add.

Rollback

The rollback is the reverse swap, and it is only clean if you plan for it:

• Keep the old index for a defined period. Do not delete it in the same change window.
• Writes after cutover go only to the new index. If you roll back after an hour, the old index lacks that hour. Either keep dual-writing after the cutover (now with the new index as primary) until you are confident, or plan a reverse catch-up reindex from new to old with the same timestamp method.
• Take a snapshot of the old index before deleting it, so even a late rollback is possible. See snapshot and restore.

POST _aliases
{
  "actions": [
    { "remove": { "index": "products-v2", "alias": "products" } },
    { "add":    { "index": "products-v1", "alias": "products" } }
  ]
}

Reindex from remote

For migrations between clusters (a new cluster, a major version jump, a different region), reindex can pull from another cluster over HTTP. The destination cluster must allowlist the remote host in elasticsearch.yml on its nodes with the reindex.remote.whitelist setting, for example reindex.remote.whitelist: "old-cluster.internal:9200", which requires a restart of those nodes.

POST _reindex?wait_for_completion=false
{
  "source": {
    "remote": {
      "host": "https://old-cluster.internal:9200",
      "username": "reindex_user",
      "password": "use-a-secret"
    },
    "index": "products-v1",
    "size": 500
  },
  "dest": { "index": "products-v2" }
}

Constraints that change the plan:

• No slicing. Reindex from remote does not support automatic or manual slicing. Parallelize yourself by running several jobs, each with a query over a disjoint range (by date, or by a numeric field).
• Buffer size. Batches from the remote are buffered on heap with a limit of around 100 MB, so with large documents use a smaller size.
• Network and TLS. The destination must trust the remote's certificate; configure the SSL settings for reindex on the destination nodes.
• Version range. Reindex from remote can read from older major versions, which makes it a standard path for migrations where an in-place upgrade is impractical. See rolling upgrades.

The catch-up and cutover method is the same; the alias swap happens on the new cluster, and the application's endpoint changes at cutover, so plan that switch (DNS, load balancer, config) as part of the window.

Changing shard counts without a full reindex

If the only change is the number of primary shards, _split and _shrink are much faster than reindex because they work at the file level:

• _split creates an index with more primaries. The source must be write-blocked, and the target count must be a multiple of the source count (more precisely, compatible with the index's routing shard count).
• _shrink creates an index with fewer primaries. The source must be write-blocked, a copy of every shard must sit on one node, and the target count must be a factor of the source count.

Both create a new index, so the alias swap pattern applies. Both require write blocks, so they suit read-mostly indices or a short write freeze. For time-series data, it is usually simpler to change the shard count in the template and let rollover apply it. See shard sizing.

Failure modes

The disk point is easy to miss: during the migration both indices exist in full, and the new one gains its replicas before you delete the old one. If the old index with replicas uses 600 GB, plan for at least 1.2 TB, and check that the data nodes stay below the low watermark (85% by default) throughout.

Runbook

1. Confirm the change needs a reindex (see the table). If it is a new field or multi-field, use _mapping plus _update_by_query instead.
2. Confirm the application uses an alias. If not, schedule the one-time conversion.
3. Check disk headroom for a second full copy including replicas.
4. Create the destination explicitly: final mappings, dynamic: strict, replicas 0, refresh -1. Simulate templates against its name.
5. Choose the live-write strategy: pause, dual-write, or catch-up with updated_at. Record T0 if catching up.
6. Start the reindex as a task with slices=auto and a conservative requests_per_second. Record the task id.
7. Watch the task status and the write thread pools; rethrottle up to the spare capacity.
8. When complete, check failures is empty.
9. Run delta passes until small; block writes; final delta.
10. Restore replicas and refresh interval; wait for green on the new index.
11. Verify counts, samples, the target queries, relevance and latency.
12. Swap the alias atomically; unblock writes.
13. Watch error rates and latency. Keep the old index and a snapshot for the agreed rollback period.
14. Delete the old index only after the rollback period ends.

Key takeaways

• Field types and index-time analyzers cannot change in place; only additions and a few parameters can, and multi-fields need an _update_by_query backfill.
• Applications must talk to aliases from day one; the alias swap is atomic and is the entire cutover.
• Create the destination explicitly with strict mappings, replicas 0 and refresh disabled, then restore both before cutover.
• Run reindex as a task with slices and a throttle sized to measured spare capacity, and rethrottle live.
• Decide how to handle live writes before starting; for catch-up passes use version_type: external so older data never overwrites newer.
• Rollback is a reverse alias swap, but only if the old index is kept and still receives, or can be caught up with, post-cutover writes.

How Sparced helps

Sparced does not run reindexes or swap aliases. What it does is show the cluster state that decides whether a migration is safe to start and whether it is hurting production while it runs: disk usage on each node against the watermarks before you create a second full copy, write and search thread pool rejections during the copy, and heap pressure on the data nodes. If restoring replicas on the new index leaves shards unassigned, it calls _cluster/allocation/explain for each stuck shard and prints the cluster's own explanation, a fix command with the index and node names filled in, and a check to confirm it. It writes nothing to the cluster, so it is safe to run in the middle of a cutover window.