Appendix C: Datalog Schema Reference

This appendix is a compact reference for Datalevin Datalog schema maps. Schema maps are EDN data; projects using any Datalevin language binding can keep them in checked-in EDN files. For design guidance, see Chapter 5 and Chapter 11. For specialized index behavior, see the full-text, vector, embedding, and idoc chapters.

1. Schema Shape

A Datalevin Datalog schema is a map from attribute keywords to property maps:

(def schema
  {:user/email {:db/valueType :db.type/string
                :db/unique    :db.unique/identity
                :db/doc       "Primary account email."}
   :user/roles {:db/valueType   :db.type/ref
                :db/cardinality :db.cardinality/many}
   :post/body  {:db/valueType            :db.type/string
                :db/fulltext             true
                :db.fulltext/autoDomain true}})

Pass the schema when opening or creating a connection:

(def conn (d/get-conn "/data/app" schema))

Read the current effective schema with d/schema:

(d/schema conn)

The result includes application attributes, built-in attributes, and assigned internal attribute ids such as :db/aid. Use this for inspection, debugging, and migration checks. In Java examples, Datalevin.schema() is a builder for constructing schema input; it is not the same operation as reading the current schema from a database.

Attribute ids are encoded as 32-bit signed non-negative integers in Datalevin's Datalog indexes. This means one database can have at most about 2.1 billion distinct attribute ids, including built-in attributes. Normal application schemas will not approach this, but unbounded dynamic key spaces should be modeled as data rather than as newly minted attributes.

Use update-schema to change schema:

(d/update-schema conn schema-update)
(d/update-schema conn schema-update #{:old/attr})
(d/update-schema conn nil nil {:old/name :new/name})

2. Defaults and Built-Ins

• Schema is optional. Undefined attributes are created on write and store values as generic EDN data.
• :db/cardinality defaults to :db.cardinality/one.
• Attribute names must be keywords.
• Built-in schema includes :db/ident, which is unique and keyword-valued. It also includes internal/system attributes such as :db/created-at, :db/updated-at, :db/fn, and :db/udf.
• :db/created-at and :db/updated-at are maintained only when a database is opened with {:auto-entity-time? true}. They store epoch milliseconds as :db.type/long values and represent database mutation time, not domain event time.
• Use the store option {:closed-schema? true} to reject transactions that mention attributes not already defined in the schema.

3. Schema-Related Store Options

Some behavior that feels schema-related is controlled by connection or store options, not by entries inside the schema map:

Pass these options when opening or creating a database:

(d/get-conn "/data/app"
            schema
            {:validate-data? true
             :closed-schema? true})

4. Attribute Properties

The public schema properties that Datalevin interprets are the following:

Changing :db/noindex through schema updates requires an empty attribute. For existing values, (d/index-attr conn attr) enables value searches and removes :db/noindex, returning the updated schema. It works with local and remote connections and leaves already indexed attributes unchanged. Other secondary-index settings are independent of this flag. Chapter 11, Section 3.4 gives a complete example and advice for large databases.

The schema builders expose this setting as Schema.attribute().noIndex(true) in Java, schema_attr(no_index=True) in Python, and schemaAttr({ noIndex: true }) in JavaScript. Enable searches later with conn.indexAttr(attr) in Java, conn.index_attr(attr) in Python, or await conn.indexAttr(attr) in JavaScript.

Use :db/ensure in transaction data for entity invariants over the would-be transaction result:

[:db/ensure 'my.app.payments/bank-account? entity]

The predicate may be a function, var, or qualified symbol. It receives the would-be database after applying the transaction followed by the resolved ensure arguments, so it can check required attributes, entity-shape rules, and cross-attribute invariants after all transaction data has been expanded to datoms. :db/ensure forms do not become datoms in :tx-data; they abort the transaction before commit if a predicate returns falsey or throws.

5. Value Types

There is no :db.type/uri; store URI values as strings.

If :db/valueType is omitted, Datalevin stores values as serialized EDN data, which is flexible but not as useful for ordered range access.

A custom type's registered name can also serve as :db/valueType. Register the definition through the connection before adding the attribute to the schema:

(d/register-type conn :app/task task-type)

(d/update-schema conn
  {:task/data {:db/valueType :app/task}})

Use a namespaced keyword distinct from the built-in type names. Custom attributes support cardinality one and many, uniqueness, lookup references, joins, pull, entities, and datom reads. Exact matching and uniqueness compare complete values; indexed ranges follow the registered order function. Custom attributes cannot use :db/fulltext indexing or serve as components of :db/tupleAttrs composite tuples. Chapter 11, Section 4.3 covers modeling; Appendix F, Section 12 lists the definition fields and registration APIs.

6. Identity and Uniqueness

:db.unique/value means no two entities may have the same value for that attribute. Duplicates will be rejected with a transaction error.

Any :db/unique attribute can be used in a lookup ref. :db.unique/identity also enables upsert:

{:user/email {:db/valueType :db.type/string
              :db/unique    :db.unique/identity}}

(d/pull db '[*] [:user/email "a@example.com"])

(d/transact! conn
  [{:user/email "a@example.com"
    :user/name  "Ada"}])

If an entity with that email already exists, the map transaction updates that entity instead of creating a duplicate.

:db/ident is the built-in identity attribute for globally named entities:

(d/transact! conn [{:db/ident :order.status/shipped}])

7. References and Components

Reference attributes must declare :db/valueType :db.type/ref:

{:order/customer {:db/valueType :db.type/ref}
 :order/items    {:db/valueType   :db.type/ref
                  :db/cardinality :db.cardinality/many
                  :db/isComponent true}}

Use :db/isComponent true only for owned children. Component entities are pulled recursively by wildcard pull and are retracted with the parent when using :db/retractEntity.

:db/isComponent does not enforce unique ownership. Datalevin will allow more than one parent to refer to the same component child, but that shape is not a well-formed component model. Parent retraction with :db/retractEntity, or attribute retraction with :db.fn/retractAttribute, treats the child as owned data and retracts it. Retracting the child also retracts declared reference datoms that point to it, so any other parent loses its reference. A value-specific [:db/retract parent component-attr child] removes only that one reference and does not cascade.

:db/isComponent true without :db/valueType :db.type/ref is invalid.

8. Tuples

Datalevin has two related tuple features that should not be confused:

• Composite indexes use :db/tupleAttrs to derive an index entry from existing attributes. Application code does not transact the composite tuple value directly; Datalevin maintains it from the component attributes in the normal Datalog indexes.
• Stored tuple values use :db.type/tuple with :db/tupleType or :db/tupleTypes when the tuple itself is application data and its elements usually do not need to stand alone as separate facts.

Composite index derived from other attributes:

{:order/customer {:db/valueType :db.type/ref}
 :order/date     {:db/valueType :db.type/instant}
 :order/customer+date
 {:db/tupleAttrs [:order/customer :order/date]}}

Rules for :db/tupleAttrs:

• It must be a non-empty sequential collection.
• The tuple attribute must be cardinality one.
• It cannot depend on another tuple attribute.
• It cannot depend on a cardinality-many attribute.
• It does not require :db/unique. Add uniqueness only when the component combination is a real identity or uniqueness constraint; a non-unique tuple can be used as a repeated-query access path.
• The derived tuple uses the same tuple value encoding as a transacted typed tuple, so tuple element limitations still apply.

Datalevin stores the derived tuple in the normal EAV and AVE indexes and updates it whenever a component changes. A query must reference the tuple attribute to use that composite path. Its read advantage comes from scanning the qualifying joint combinations instead of emitting a potentially much larger intermediate relation from separate component EAV datoms. It increases write work and index storage; Chapter 11, Section 1.2 gives the IC5 cardinalities and the selection criteria.

Stored homogeneous tuple value:

{:point/xy {:db/valueType :db.type/tuple
            :db/tupleType :db.type/double}}

Stored heterogeneous tuple value:

{:image/bounds {:db/valueType  :db.type/tuple
                :db/tupleTypes [:db.type/double
                                :db.type/double
                                :db.type/long
                                :db.type/long]}}

Rules for stored typed tuples:

• Stored tuple values are transacted as non-empty vectors. An empty vector is not a valid tuple value.
• :db.type/tuple must include :db/tupleType or :db/tupleTypes when the tuple value is stored directly.
• Tuple element declarations support these scalar Datalevin value types: :db.type/string, :db.type/bigdec, :db.type/bigint, :db.type/boolean, :db.type/double, :db.type/float, :db.type/long, :db.type/ref, :db.type/instant, :db.type/uuid, :db.type/keyword, and :db.type/symbol.
• :db/tupleType must be one supported tuple element type. It describes a homogeneous tuple: one or more elements, all of the same type.
• :db/tupleTypes must contain more than one supported tuple element type. It describes a heterogeneous tuple: the vector arity must match the declaration.
• Tuple elements cannot themselves be :db.type/tuple, :db.type/vec, or :db.type/idoc.
• Variable-length tuple elements, such as strings, must fit within Datalevin's tuple element encoding limit of 255 bytes. Keep stored tuples small; use ordinary attributes or document values when the structure is open-ended.

For example, this homogeneous tuple schema is valid:

{:sf/emails {:db/valueType :db.type/tuple
             :db/tupleType :db.type/string}}

But transacting an empty tuple fails:

{:sf/foo "hi"
 :sf/emails []}

Datalevin rejects this data with Cannot store an empty homogeneous tuple for attribute :sf/emails. If the entity has no tuple value, omit :sf/emails from the transaction rather than transacting an empty vector.

9. Full-Text Schema Keys

Use :db/fulltext true for attributes that should participate in full-text search:

{:post/body {:db/valueType            :db.type/string
             :db/fulltext             true
             :db.fulltext/domains     ["posts"]
             :db.fulltext/autoDomain true}}

If no explicit domain is given, full-text attributes participate in the default "datalevin" domain. :db.fulltext/autoDomain true also creates an attribute-specific domain named by the attribute without the leading colon: :title becomes "title" and :post/body becomes "post/body". Unlike vector and embedding domains, full-text auto domains keep / in namespaced attribute names.

Store-level :search-opts and :search-domains configure search domain options such as :indexing-mode :async, :index-position? true, and :include-text? true. If a custom analyzer uses ngrams, prefer gram sizes of 3 characters or longer; shorter grams produce many low-selectivity tokens and can make the index much larger. Full-text indexes store document references by default. In Datalog, the document reference is the indexed datom itself; the index does not store a duplicate raw-text copy unless :include-text? true opts a domain into storing text for :display :texts, :display :texts+offsets, and re-indexing.

10. Vector and Embedding Schema Keys

Use :db.type/vec when your application supplies vectors:

{:item/embedding {:db/valueType   :db.type/vec
                  :db.vec/domains ["items"]}}

Every vector attribute has an attribute-specific domain derived from the attribute name with / replaced by _: :embedding becomes "embedding" and :item/embedding becomes "item_embedding". :db.vec/domains adds shared domains in addition to this attribute domain. Configure dimensions, metric, and indexing mode in :vector-opts or :vector-domains, and keep every domain that receives the same vector values compatible.

Use :db/embedding true when Datalevin should embed string datoms:

{:doc/text {:db/valueType             :db.type/string
            :db/embedding             true
            :db.embedding/domains     ["docs"]
            :db.embedding/autoDomain true}}

Embedding rules:

• The attribute must have :db/valueType :db.type/string.
• :db.embedding/domains must be a non-empty sequence of non-blank strings.
• If no explicit domain is given, the attribute participates in the default "datalevin" embedding domain.
• If :db.embedding/domains is present, the attribute participates in those domains instead of the default unless "datalevin" is listed.
• :db.embedding/autoDomain true adds an attribute-specific embedding domain using the same slash-to-underscore naming rule as vector domains.
• Changing :db/embedding, :db.embedding/domains, or :db.embedding/autoDomain on a populated attribute is rejected; rebuild explicitly instead.

Store-level :embedding-opts, :embedding-domains, and :embedding-providers configure providers, dimensions, metric, and indexing mode.

11. Idoc Schema Keys

Use :db.type/idoc for nested document values that should be indexed by path:

{:doc/json {:db/valueType  :db.type/idoc
            :db/idocFormat :json
            :db/domain     "profiles"}
 :doc/edn  {:db/valueType :db.type/idoc}
 :doc/md   {:db/valueType  :db.type/idoc
            :db/idocFormat :markdown}}

Allowed :db/idocFormat values are :edn, :json, and :markdown. The default is :edn.

Idoc input must normalize to a map or vector at the root. EDN strings may yield either shape, JSON strings may contain objects or arrays, and Markdown strings produce heading maps. Map keys must be keywords or strings. Use vectors for arrays; lists and scalar roots are rejected. nil values normalize to :json/null, and the literal value :json/null is reserved and rejected on input. Root vectors are stored intact and support membership queries and zero-based positional paths; see Chapter 14.

:db/domain names the idoc path index used for that attribute. If it is omitted, Datalevin derives the domain from the attribute name without the leading colon, so :doc/edn becomes "doc/edn". Several idoc attributes may share a domain intentionally, but keep formats coherent: if attributes with different :db/idocFormat values share a domain, Datalevin treats the domain as mixed-format.

Use :db.idoc/indexedPaths and :db.idoc/excludedPaths to control which path prefixes enter the idoc path index. Changing those keys on a populated attribute is rejected; rebuild explicitly instead. Chapter 14 covers path selectors and store-level idoc domain options.

Python and JavaScript expose convenience builders for the same maps: idoc_attr / idocAttr build :db.type/idoc schema attributes, and idoc_domain / idocDomain build idoc option maps for :idoc-opts or entries under :idoc-domains. Clojure can write the EDN maps directly; Java can use the schema builder methods shown in Chapter 14.

12. Schema Evolution

Use d/update-schema to add, change, delete, or rename schema attributes on an open connection:

(d/update-schema conn
  {:user/age {:db/valueType :db.type/long}})

(d/update-schema conn
  {:user/age {:db/doc "Age in whole years"}})

(d/update-schema conn
  {:user/age {:db/doc :db/retract}})

(d/update-schema conn
  {:new/name {:db/valueType :db.type/string}}
  #{:old/name})

(d/update-schema conn nil nil
  {:old/name :new/name})

The schema-update map and each nested property map are patches:

Existing attributes retain their internal :db/aid, and supplied ids are ignored. The complete request is validated before writing, committed atomically, and safe to retry unchanged. Conflicting renames, rename chains or cycles, duplicate targets, and conflicting patch/delete or delete/rename operations are rejected before writing.

A schema supplied while opening an existing database uses the same patch semantics and persists its changes; it is not a read-only drift check. For production evolution, open from the stored schema, apply deliberate changes through update-schema, and verify with d/schema. Chapter 11, Section 6.6 explains declarative convergence and ordered migrations.

13. Datomic and DataScript Differences

Datalevin's Datalog model is familiar to Datomic and DataScript users, but the schema mechanics differ:

• Schema is a map of maps, not a vector of transaction maps.
• Schema is passed to get-conn, create-conn, empty-db, or init-db, and later changed with update-schema.
• Undefined attributes are allowed unless :closed-schema? true is set.
• Unspecified value types are stored as EDN data.
• AVE indexing is automatic unless :db/noindex true opts out; EAV still stores every attribute. Use index-attr to enable AVE on existing values.
• Use :db/ident for named enum/system entities.
• Use :db.attr/preds for attribute-local value predicates, and use :db/ensure for explicit entity invariants over the would-be transaction result.

14. Minimal Production Template

(def schema
  {:user/id    {:db/valueType :db.type/uuid
                :db/unique    :db.unique/identity
                :db/doc       "Stable external user id."}
   :user/email {:db/valueType :db.type/string
                :db/unique    :db.unique/identity
                :db/doc       "Login and notification email."}
   :user/name  {:db/valueType :db.type/string}

   :post/id    {:db/valueType :db.type/uuid
                :db/unique    :db.unique/identity}
   :post/author {:db/valueType :db.type/ref}
   :post/body  {:db/valueType            :db.type/string
                :db/fulltext             true
                :db.fulltext/autoDomain true}
   :post/tags  {:db/valueType   :db.type/keyword
                :db/cardinality :db.cardinality/many}})

(def conn
  (d/get-conn "/data/app" schema
              {:closed-schema? true}))