Elasticsearch Mapping Design for Production: Types and Limits

A mapping is the contract between your documents and Lucene's data structures. Every field you map decides which structures get built (an inverted index, doc values, norms, positions), how much disk and heap they cost, and which queries and aggregations are possible later. Get it wrong and the symptoms arrive months later: aggregations that need fielddata, exact-match lookups that are slower than they should be, an index that has grown to 4,000 fields because a client sent arbitrary JSON keys, or a type that cannot be changed without a reindex. The answer is to treat mappings as a design artifact: map every field you know explicitly, choose keyword or text (or both) deliberately, switch off structures you will never use, control what dynamic mapping is allowed to create, and keep the field count well below the limits. This article covers the model, each decision, and how to recover when a mapping has already gone wrong.

The mental model: what a mapping turns into

For each mapped field, Elasticsearch can build several independent Lucene structures. Which ones exist depends on the field type and its parameters.

Two facts make this table actionable:

• Each structure costs disk at index time and I/O at query time, whether you use it or not. A field that is only ever displayed does not need an inverted index or doc values.
• Most of these decisions are fixed once data is indexed. You can add fields and add multi-fields to an existing field. You cannot change a field's type, or turn indexing or doc values on for data already written, without reindexing. See zero-downtime reindexing.

text versus keyword

This is the most consequential choice in most mappings.

text is analyzed. The value goes through an analyzer (by default the standard analyzer: tokenize on word boundaries, lowercase), and each token is indexed. It supports full-text queries (match, match_phrase) and relevance scoring. It has no doc values, so it cannot be sorted or aggregated on without enabling fielddata, which loads the terms into the heap. Do not enable it: see JVM heap and circuit breakers.

keyword is not analyzed. The whole value is one term (optionally passed through a normalizer, for example to lowercase it). It supports exact-match term and terms queries, prefix queries, sorting and aggregations through doc values. It does not support full-text matching of words inside the value.

What dynamic mapping does to strings

If a string field arrives without a mapping and dynamic mapping is on, Elasticsearch maps it as text with a keyword sub-field named keyword, with ignore_above: 256:

{
  "message": {
    "type": "text",
    "fields": {
      "keyword": { "type": "keyword", "ignore_above": 256 }
    }
  }
}

That default is safe for exploration and wasteful in production. Every string gets both an analyzed index and a keyword index plus doc values, even fields that are only ever one or the other. A 30-field log document with 20 string fields carries 20 structures it may never use. For a high-volume index, map strings explicitly.

ignore_above means values longer than 256 characters are not indexed in the keyword sub-field (they remain in _source). Queries and aggregations on message.keyword silently skip those documents.

Multi-fields: one value, several structures

A multi-field indexes the same source value in more than one way:

{
  "mappings": {
    "properties": {
      "product_name": {
        "type": "text",
        "fields": {
          "raw": { "type": "keyword", "ignore_above": 512 },
          "autocomplete": { "type": "search_as_you_type" }
        }
      }
    }
  }
}

Each sub-field is a separate field in the mapping, with its own structures, its own disk cost, and its own entry in the field count. Add them only for query patterns you actually serve.

One useful property: you can add a multi-field to an existing field without reindexing. Documents indexed before the change will not have it populated until they are reindexed or rewritten, which you can do in place:

POST products/_update_by_query?conflicts=proceed

That rewrites every document, so throttle it on a large index.

Switching off what you do not need

doc_values: false

Doc values are on by default for every field type that supports them except text. They are what makes sorting and aggregations cheap. If a field is only ever filtered on and never sorted, aggregated or accessed by scripts, you can turn them off to save disk:

{ "session_token": { "type": "keyword", "doc_values": false } }

The trade-off is permanent for indexed data: to aggregate on that field later, you must reindex. Turn doc values off only for fields where you are confident.

index: false

index: false skips the inverted index or points. The field cannot be searched efficiently. In 8.x, fields of types such as keyword, numerics and date that have index: false but keep doc values can still be queried, using the doc values, but much more slowly; treat that as an escape hatch, not a design. Use index: false for fields you aggregate or sort on but never filter by, or with doc_values: false as well for fields you only return in results.

enabled: false

For an object you want to store and return but never search or aggregate, such as a raw payload or a blob of vendor metadata, set enabled: false on the object. Elasticsearch skips parsing its contents entirely: no fields inside it are mapped, so its contents add nothing to the field limit (the object itself counts as one field), and the object stays in _source.

{
  "mappings": {
    "properties": {
      "vendor_payload": { "type": "object", "enabled": false }
    }
  }
}

This is one of the most effective tools against mapping explosion from data you do not control.

norms: false

Norms store per-document length information for scoring. On text fields that are only filtered on (never scored), norms: false saves disk. keyword fields already have norms off by default.

Numeric types and identifiers

Choose the smallest type that will always hold the value. Elasticsearch offers long, integer, short, byte, double, float, half_float, scaled_float (a long with a scaling_factor, good for prices and percentages with fixed precision) and unsigned_long.

The more important decision is whether a number is really a number. Numeric fields are indexed as points, a structure optimized for range queries. Exact term lookups on points work but are not what the structure is designed for. Keyword fields are indexed in the inverted index, which is optimized for exact term lookups.

Elastic's search tuning guidance says it directly: consider mapping identifiers as keyword. An order ID, a customer number, an HTTP status code you only filter on, or a ZIP code is an identifier, not a quantity. You will query it with term, not range, and you will never sum it.

If you need both, map one as the primary and the other as a multi-field.

Controlling dynamic mapping

The dynamic parameter decides what happens when a document contains a field that is not in the mapping. It can be set at the top level and overridden on any object.

A pattern that works well for application logs is strict at the top level, so a typo or an unannounced field in your own schema fails loudly, combined with false or enabled: false on a specific object reserved for free-form context:

{
  "mappings": {
    "dynamic": "strict",
    "properties": {
      "@timestamp": { "type": "date" },
      "service": { "type": "keyword" },
      "level": { "type": "keyword" },
      "message": { "type": "match_only_text" },
      "trace_id": { "type": "keyword" },
      "context": { "type": "object", "dynamic": false }
    }
  }
}

The consequence of strict is operational: a producer that adds a field gets mapping errors (strict_dynamic_mapping_exception) until the mapping is updated. That is the point, but the ingest path must route those failures somewhere visible rather than drop them.

Two related defaults: date_detection is on by default, so strings that look like dates in the default formats are mapped as date; and numeric_detection is off by default, so numbers sent as strings are mapped as strings. A field whose first value was "2026-10-21" becomes a date, and a later value of "unknown" will then fail to index. If you do not want that, set date_detection: false or map the field explicitly.

Dynamic templates

When you cannot list every field in advance, dynamic templates let you decide what dynamic mapping creates. Each template matches new fields by detected JSON type, by name, or by path, and supplies the mapping to use. Templates are evaluated in order and the first match wins.

{
  "mappings": {
    "dynamic_templates": [
      {
        "labels_as_keywords": {
          "path_match": "labels.*",
          "mapping": { "type": "keyword", "ignore_above": 1024 }
        }
      },
      {
        "ip_suffix_as_ip": {
          "match": "*_ip",
          "match_mapping_type": "string",
          "mapping": { "type": "ip" }
        }
      },
      {
        "strings_as_keywords": {
          "match_mapping_type": "string",
          "mapping": { "type": "keyword", "ignore_above": 1024 }
        }
      }
    ]
  }
}

The matching parameters are match_mapping_type (the detected JSON type, such as string, long, double, boolean, object), match and unmatch (field name patterns), and path_match and path_unmatch (full dotted path patterns). Order matters here: the *_ip template must come before the catch-all string template, or it would never be reached. The last template changes the default string behavior: new strings become keyword only, with no text field, which suits most structured logging.

A dynamic template does not limit how many fields are created; it only decides their type. It is not protection against mapping explosion by itself.

Put mappings and dynamic templates in component templates, and compose them into index templates, so every new index and data stream backing index gets them. See index templates and data streams.

Runtime fields

A runtime field is computed at query time, usually by a Painless script reading doc values or _source. It is defined in the mapping's runtime section or per request in runtime_mappings:

GET web-logs/_search
{
  "runtime_mappings": {
    "response_class": {
      "type": "keyword",
      "script": "emit(doc['http.status'].value.substring(0, 1) + 'xx')"
    }
  },
  "size": 0,
  "aggs": {
    "by_class": { "terms": { "field": "response_class" } }
  }
}

That script assumes http.status is a keyword field, as recommended above. Runtime fields cost nothing at index time and no disk, and they can be added and changed freely. The price is query time: the script runs for every document the query touches. They are ideal for exploring a field before committing to a mapping, for fixing a mistake while a reindex runs, and for rarely used derived values. They are a poor choice for a field on a hot dashboard. Once a runtime field proves useful, index it.

Mapping explosion and the field limits

Mapping explosion is what happens when an index keeps acquiring fields, typically because documents use data as keys: user IDs, URLs, timestamps or arbitrary labels as field names. Each new field updates the mapping, which is part of the cluster state. The master publishes the updated cluster state to every node. With thousands of fields across many indices, cluster state grows, mapping updates queue on the master, indexing stalls waiting for them, and heap usage rises on every node.

Elasticsearch guards against this with per-index limits, all index settings:

The total fields limit counts more than leaf fields: object fields (each level of a dotted path), multi-fields, field aliases and mapped runtime fields all count. Metadata fields such as _id and _source do not. A document with 300 leaf strings mapped dynamically uses about 600 of the 1,000, because each has a .keyword multi-field.

When the limit is reached, documents that would add a new field are rejected with an error like:

Limit of total fields [1000] has been exceeded while adding new fields [3]

Recent releases add an index setting, index.mapping.total_fields.ignore_dynamic_beyond_limit (default false). When true, a document that would add dynamic fields beyond the limit is still indexed: the excess fields are left out of the mapping, as with dynamic: false, and their names are recorded in the _ignored metadata field. Recent versions turn it on by default for logsdb indices. It trades a loud failure for silently unsearchable fields, so use it knowingly.

Raising the limit is usually the wrong fix

The limit is a symptom detector. Raising it to 5,000 makes the error go away and leaves the cause in place: the field count keeps growing until the cluster state and the master feel it. Before raising it, find out which fields are growing:

GET my-index/_mapping

Look for families of fields with data-like names: labels.user_88231457, headers.x-request-..., metrics.<url>. Then fix the shape of the data:

The flattened type

flattened maps an entire object as a single field. All its leaf values are indexed as keywords, and they can be queried by path:

{
  "mappings": {
    "properties": {
      "labels": { "type": "flattened" }
    }
  }
}

GET my-index/_search
{
  "query": {
    "term": { "labels.team": "payments" }
  }
}

The whole object counts as one field toward the limit, however many keys it has. The limits of the type are the price: every value is treated as a keyword, so range queries and sorting compare strings lexicographically ("10" sorts before "9"), there is no full-text analysis, and highlighting is not supported. It supports term, terms, terms_set, prefix, range (on strings), match, multi_match, query_string, simple_query_string and exists queries, and sorting and simple keyword-style aggregations such as terms. It also has its own depth_limit and ignore_above parameters.

Failure modes and anti-patterns

The _field_caps API deserves a mention on its own. For a field that should have one type across a pattern of indices, it shows every type in use and which indices use each:

GET logs-*/_field_caps?fields=http.status,user.id

A field reported as both keyword and long across indices is a conflict that will break sorting and aggregations across those indices.

Checklist: reviewing a mapping before production

1. Every known field is mapped explicitly, with the type chosen for how it is queried.
2. Strings are deliberate: keyword for identifiers and enums, text (or match_only_text) for prose, both only where both are needed.
3. Identifiers are keyword, not numeric, unless you need range queries on them.
4. Unused structures are off: index: false, doc_values: false, norms: false, or enabled: false where the field's use is certain.
5. Dynamic behavior is set on purpose: strict for schemas you own, false or runtime for objects you do not, and dynamic templates for the rest.
6. Free-form keys are contained in a flattened field or a disabled object.
7. The field count has room: well under index.mapping.total_fields.limit, counting multi-fields.
8. date_detection is off or harmless for string fields with mixed contents.
9. Mappings live in component templates, so every new index and backing index inherits them.
10. _field_caps shows one type per field across the index pattern.

Key takeaways

• Each mapped field builds structures that cost disk and I/O whether used or not. Map for the queries you serve.
• Default dynamic mapping turns every string into text plus keyword. Fine for exploring, wasteful at volume.
• Map identifiers as keyword; numeric types are built for ranges.
• dynamic: strict makes the mapping a contract; enabled: false and flattened contain data you do not control.
• The 1,000-field default limit counts multi-fields, objects, aliases and runtime fields. Hitting it means the data's shape is wrong, not that the limit is too low.
• Types cannot be changed in place. Use runtime fields as a bridge and reindex for the real fix.

How Sparced helps

Sparced's pass covers cluster health, nodes, shards, allocation, thread pools, circuit breakers, ILM, snapshots and security, so the effects of a poor mapping show up where they land, for example as fielddata breaker usage or heap pressure on a node, with the node named and the numbers shown. It does not read or rewrite your mappings, and it will not tell you a field is mistyped unless the cluster's own output says so. It installs no agent and writes nothing to the cluster, so mapping changes and reindexing remain your decision, made with the commands in this article.