BLOB: on-chain chunked binary storage
Protocol and consensus12.4.15+

BLOB: on-chain chunked binary storage

Binary files stored on chain in chunks: layout, rules and REST routes.

BLOB stores arbitrary binary data entirely on chain, split into chunks of one OP_RETURN output each, and addressed by the SHA-256 of the reassembled bytes. The node collects the chunks, reassembles a blob once every chunk is present and contiguous, checks that its SHA-256 equals the announced blob_id, and then serves the bytes over REST with the right Content-Type. It was written as the storage backend for poker NFT metadata (JSON, SVG, WebP), but nothing in the code is poker-specific. The value attached to each chunk output is unspendable (OP_RETURN) and is summed as "burn".

Status in this node #

  • Built-in module blob (class CBlobModule), always registered: src/modules/builtin_modules.cpp:34. There is no switch to turn it off.
  • Registered after eMPP, so the eMPP segments of a block are already stored when BLOB reads them (src/modules/builtin_modules.cpp:28-31).
  • Database: <datadir>/modules/blob.sqlite (src/modules/blob/blob_module.cpp:301, directory from src/modules/module_registry.cpp:196).
  • Indexing only: BLOB changes no consensus or relay rule. The policy limit that makes multi-kilobyte chunks relayable is the node-wide MAX_OP_RETURN_RELAY = 8005 bytes of scriptPubKey (src/script/standard.h:85, overridable with -datacarriersize).
  • LOKAD id: "BLOB" = 42 4c 4f 42 (src/modules/blob/blob_module.cpp:31).

Envelope #

Two transports are accepted, both decoded on every connected block (ExtractChunksFromBlock, src/modules/blob/blob_module.cpp:229-294):

TransportScript shapeDecodersource column
NativeOP_RETURN <push "BLOB"> <push blob_id> <push index> <push total> <push type> <push payload>DecodeBlobOutput, blob_module.cpp:135native
eMPPOP_RETURN OP_RESERVED <push "BLOB" + flat bytes> [more segments]eMPP module, then DecodeBlobPayloadFlat, blob_module.cpp:208empp

Any output of any transaction (coinbase included) is scanned. For eMPP, see eMPP; BLOB receives each segment's bytes after the 4-byte LOKAD id. If the same (txid, vout) was already taken by the native decoder, the eMPP row is skipped (blob_module.cpp:273); the two shapes cannot actually overlap because native requires a push right after OP_RETURN and eMPP requires OP_RESERVED (0x50).

Byte layout #

Native chunk (one output = one chunk) #

Exactly six data pushes after OP_RETURN (0x6a). All integers are big-endian.

Offset (in script)SizeTypeEndiannessFieldNotes
01opcoden/aOP_RETURN0x6a
11opcoden/apush length0x04
24bytesn/aLOKAD id42 4c 4f 42 ("BLOB")
61opcoden/apush length0x20
732bytesn/ablob_idSHA-256 of the full reassembled buffer, stored as lowercase hex in byte order
391opcoden/apush length0x04
404uint32big-endianchunk_index0-based
441opcoden/apush length0x04
454uint32big-endiantotal_chunksat least 1
491opcoden/apush length0x01
501uint8n/acontent_type_idsee the table below
511, 2, 3 or 5opcode(s)little-endian lengthpayload pushdirect push (1 to 75 bytes), PUSHDATA1, PUSHDATA2 or PUSHDATA4
nextN (at least 1)bytesn/apayloadthis chunk's slice of the blob

The header (offsets 0 to 50) is 51 bytes, so with the default 8005-byte relay ceiling a chunk can carry up to 8005 - 51 - 3 = 7951 bytes of payload (PUSHDATA2 prefix). The -datacarriersize arithmetic is policy only.

eMPP segment body ("flat" layout) #

The segment push contains "BLOB" followed by these bytes (offsets are relative to the end of the LOKAD id):

OffsetSizeTypeEndiannessFieldNotes
032bytesn/ablob_idSHA-256 of the full buffer
324uint32big-endianchunk_index0-based
364uint32big-endiantotal_chunksat least 1
401uint8n/acontent_type_id
41N (at least 1)bytesn/apayloadruns to the end of the segment

There are no inner length prefixes: the eMPP push length delimits the payload.

Content types #

The MIME type is chosen only when serving (ContentTypeName, src/modules/blob/blob_module.cpp:35-47); every byte value is accepted when indexing.

content_type_idMIME type served
0x01application/json
0x02image/svg+xml
0x03image/webp
0x04application/json+gzip (clients decompress themselves; no Content-Encoding is sent)
0x05image/png
0x06image/jpeg
0x07text/plain; charset=utf-8
0x08application/octet-stream
any otherapplication/octet-stream

Versioning #

There is no version field. A layout change would need a new LOKAD id or a new content type value.

Validation rules #

Native chunk (DecodeBlobOutput and its push reader):

  1. The script is at least 6 bytes long and its first byte is OP_RETURN (src/modules/blob/blob_module.cpp:136-139).
  2. Every element after OP_RETURN must be a data push. Direct pushes 0x01 to 0x4b are taken as-is; OP_PUSHDATA1 must carry a length of at least 0x4c, OP_PUSHDATA2 at least 0x100, OP_PUSHDATA4 at least 0x10000 (blob_module.cpp:78-123). Any other opcode, including OP_0, OP_1NEGATE and OP_1 to OP_16, rejects the output (blob_module.cpp:124). A push running past the end of the script rejects it.
  3. More than 8 elements rejects the output (blob_module.cpp:148); exactly 6 are required (blob_module.cpp:152).
  4. Element 0 is exactly the 4 bytes "BLOB" (blob_module.cpp:154-157).
  5. Sizes: blob_id 32 bytes, chunk_index 4, total_chunks 4, content_type_id 1, payload non-empty (blob_module.cpp:158-162).
  6. total_chunks is not 0 and chunk_index < total_chunks (blob_module.cpp:173-175).

eMPP chunk (DecodeBlobPayloadFlat), after the eMPP envelope itself has decoded (eMPP validation):

  1. The body is at least 42 bytes (blob_module.cpp:210).
  2. total_chunks is not 0 and chunk_index < total_chunks (blob_module.cpp:220-222).
  3. The payload after offset 41 is non-empty (blob_module.cpp:225).

Reassembly (TryCompleteBlob, run for every blob touched by a block):

  1. A chunk is stored only if no chunk with the same (blob_id, chunk_index) exists: INSERT OR IGNORE, first one wins (blob_module.cpp:536-540; primary key at blob_module.cpp:322).
  2. While fewer rows than total_chunks exist, the blob stays pending (blob_module.cpp:430-447).
  3. Once enough rows exist, they are read in chunk_index order and must be exactly 0, 1, ..., total_chunks - 1 (blob_module.cpp:450-476); otherwise nothing is written.
  4. sha256(concatenation) == blob_id sets complete = 1 and stores the buffer; a mismatch sets corrupt = 1 and stores no buffer (blob_module.cpp:478-515).

Nothing checks the output value: zero-value chunks are indexed. Nothing checks that all chunks of a blob agree on total_chunks or content_type_id (see Limits).

Indexing #

Database blob.sqlite (CreateSchema, src/modules/blob/blob_module.cpp:308-377):

blob_chunks, one row per accepted chunk:

ColumnTypeNotes
blob_idTEXT NOT NULL64 lowercase hex characters
chunk_indexINTEGER NOT NULL
total_chunksINTEGER NOT NULLas announced by this chunk
content_typeINTEGER NOT NULLcontent_type_id
txidTEXT NOT NULL
voutINTEGER NOT NULL
block_heightINTEGER NOT NULL
block_timeINTEGER NOT NULLblock header time
satsINTEGER NOT NULLburn credited to this chunk: the value of the output (native), or for eMPP the value of the whole hosting output (EmppRow::output_sats) on the first chunk of that output that gets stored, 0 on the other chunks of the same output (12.4.6: one output burns its value once, see eMPP)
payloadBLOB NOT NULLchunk bytes
sourceTEXT NOT NULL DEFAULT 'native'native or empp; added by an idempotent migration on older databases (blob_module.cpp:363-376)

Primary key (blob_id, chunk_index); index idx_bc_height on block_height DESC.

blobs, one row per blob seen:

ColumnTypeNotes
blob_idTEXT PRIMARY KEY
total_chunksINTEGER NOT NULL
content_typeINTEGER NOT NULL
byte_sizeINTEGER NOT NULL DEFAULT 0size of the reassembled buffer
completeINTEGER NOT NULL DEFAULT 01 when the hash matched
corruptINTEGER NOT NULL DEFAULT 01 when the hash did not match
reassembledBLOBthe bytes, only when complete
first_seen_heightINTEGER NOT NULLlowest chunk height
complete_heightINTEGERhighest chunk height, set at completion
total_burn_satsINTEGER NOT NULL DEFAULT 0sum of sats over the chunks (so a blob carried by one output counts that output once)

Indexes idx_blobs_height (first_seen_height DESC) and idx_blobs_complete (complete, first_seen_height DESC). Index idx_bc_output (txid, vout) on blob_chunks finds the chunks of one output. blob_meta(key, value) holds last_indexed_height and data_version (2 since 12.4.6).

Block flow:

  • Connect: decode, then in one transaction insert the chunks (value credited once per output, see sats), run TryCompleteBlob once per touched blob, set last_indexed_height (ConnectBlock, ApplyChunksLocked).
  • TryCompleteBlob rewrites the whole blobs row from the chunks present: no chunk, the row is deleted; fewer chunks than total_chunks, a pending row (complete = 0, corrupt = 0, no bytes, no complete_height, first_seen_height = lowest chunk height); all chunks, the reassembly and hash check below.
  • Disconnect: in one transaction, DELETE FROM blob_chunks WHERE block_height = h, then TryCompleteBlob for every blob that lost a chunk (back to pending, or gone when no chunk is left), last_indexed_height = h - 1 (DisconnectBlock). Connecting N blocks then disconnecting K equals connecting Nāˆ’K (blob_module_tests/connect_disconnect_symmetry_all_depths). Before 12.4.6 the blobs row was not touched: a blob completed by the disconnected block stayed complete and served.
  • Startup reconciliation: BLOB does not override IndexedTipHeight/ResetForReindex (src/modules/blob/blob_module.h:38-76), so the registry classifies it as "no tip state" and never replays or wipes it (src/modules/module_registry.cpp:70-75). Blocks connected while the module was not running are only picked up with a rescan.
  • Mempool: not indexed. A chunk appears only once its transaction is in a block.
  • Rescan (rescanmodule blob, a background job; RescanRange): reads each block from disk, decodes it outside any lock, then re-inserts its chunks (duplicates ignored, so it is idempotent) under cs_main and the module mutex after checking that the block is still on the active chain (otherwise the rescan stops, state aborted). last_indexed_height only moves forward, and only if the range starts at or below it + 1. A range above the tip does nothing.
  • Upgrade (12.4.6). blob_chunks was right in older databases except the value repeated on every chunk of a multi-chunk eMPP output; blobs could hold rows of disconnected blocks and corrupt rows created by a REST read of an unknown id. At Init, a database without data_version 2 is repaired in place in one transaction: the value is kept on the first stored chunk of each output (lowest rowid, chunks are stored in segment order) and set to 0 on the others, then every blobs row is rebuilt from its chunks with TryCompleteBlob (rows without chunks deleted). Log: BLOB: indexed data version 1 < 2: repairing chunk burns and blob rows. Test: blob_module_tests/upgrade_from_v1_repairs_in_place.

REST API #

All routes are registered through AddModuleRoute with no response-cache TTL (src/api/api_server.cpp:345-348; routes at src/modules/blob/blob_module.cpp:640-654). JSON errors use the common envelope {"error": code, "message": text, "status": n}.

MethodPathAnswer
GET/api/v1/blob/recent?limit=N{"blobs":[...]}, newest first_seen_height first; limit default 25, clamped to 1..200
GET/api/v1/blob/<sha256>the raw bytes
GET/api/v1/blob/<sha256>/infoJSON metadata
GET/api/v1/blob/statstotals
GET, POST/api/v1/blob/rescan403 rescan_moved; use the authenticated RPC rescanmodule blob [start] [end] (see module rescans)

/blob/recent row (blob_module.cpp:694-713):

JSON
{"blobId":"<64 hex>","totalChunks":1,"contentType":"application/json","contentTypeId":1,
 "byteSize":7,"complete":true,"corrupt":false,"firstSeenHeight":100,"completeHeight":100,
 "totalBurnSats":0,"source":"native"}

completeHeight is present only when set; source is native, empp or mixed (both transports in one blob).

/blob/<sha256> (blob_module.cpp:722-827):

  • The id must be exactly 64 characters of lowercase hex, otherwise 400 bad_blob_id.
  • Unknown id: the handler first tries to roll up existing chunks (TryCompleteBlob), then answers 404 not_found. With no chunk at all nothing is stored (before 12.4.6 the roll-up stored a corrupt row, the SHA-256 of nothing, for any id asked for: /blob/stats counted it).
  • Corrupt: 404 corrupt. Not complete: 404 incomplete.
  • Complete: 200 with Content-Type from the table above, Cache-Control: public, max-age=31536000, immutable, X-Blob-Id, X-Blob-Total-Burn-Sats, Access-Control-Allow-Origin: *.

/blob/<sha256>/info (blob_module.cpp:771-797): blobId, totalChunks, contentType, contentTypeId, byteSize, complete, corrupt, firstSeenHeight, completeHeight (if set), totalBurnSats, and chunksPresent when not complete.

/blob/stats (blob_module.cpp:829-866):

JSON
{"completeBlobs":0,"completeBytes":0,"totalBurnSats":0,"pendingBlobs":0,"corruptBlobs":0,"blobs":0,"chunks":0,"lastIndexedHeight":-1}

totalBurnSats counts complete blobs only. blobs (every row of blobs) and chunks (rows of blob_chunks) were added in 12.4.6.

/blob/rescan: answers 403 rescan_moved to GET and POST since 12.4.6 and runs nothing. Rescans go through rescanmodule blob [start] [end] on the authenticated JSON-RPC.

Dashboard #

Tab "BLOB" (data-tab="blob", hidden when the module is not loaded), sub-tabs Hub and Recent (share/www/dashboard/js/dashboard.js:2765-2811). The Hub shows stat cards from /blob/stats and the 10 newest blobs from /blob/recent; Recent shows 50. Each blob id links to /api/v1/blob/<id>.

The Hub cards read blobs, chunks and completeBytes of /blob/stats ("Blobs", "Chunks", "Bytes Stored"), plus totalBurnSats and lastIndexedHeight. Before 12.4.6 they read fields the API did not return and always showed 0.

Limits and edge cases #

  • First chunk wins per (blob_id, chunk_index). Anyone who knows a blob_id (it is public as soon as one chunk is broadcast) can publish a different chunk at the same index first; the blob then reassembles to the wrong hash and is marked corrupt. The genuine chunk is ignored, and there is no retry with alternative chunks.
  • Conflicting total_chunks or content_type_id across chunks of one blob: TryCompleteBlob takes those columns from an arbitrary row of an aggregate query (blob_module.cpp:412-415). The SHA-256 check still protects the bytes, but the served MIME type can be either value.
  • If chunks disagree on total_chunks, the blob can stay pending forever (rows are present but not contiguous for the chosen total).
  • /blob/<id> can write (roll-up of a blob whose chunks are present) from a GET; an unknown id writes nothing (12.4.6). Since 12.4.6 every handler runs holding m_db_mutex, so that write cannot interleave with ConnectBlock or another request.
  • Size: no limit of its own beyond relay policy and block size. Every chunk is stored, plus a second copy of the bytes for complete blobs.
  • /api/v1/modules lists blob with empty version, db path and schema 0: the module does not override Info().

Differences from upstream #

BLOB is a Lotus-only protocol; there is no upstream or eCash specification. Differences from this repository's older doc/blob.md:

  • That file says ids from 0x80 are reserved; the code has no such rule.
  • It says the decoder enforces "minimum-length opcodes". The code enforces the minimum PUSHDATA form, but rejects OP_0 and OP_1 to OP_16 entirely.
  • It says a disconnected block makes a completed blob pending again; the code does since 12.4.6 (see Indexing).
  • It mentions -blobminburnperbyte and -DENABLE_BLOB_MODULE as future options; neither exists.

Worked example #

No mainnet BLOB transaction could be fetched: on 2026-11-14 the public node answered 503 node_warming_up ("Rebuilding module indices from genesis") to every /api/v1/blob/* request. The example below is built by hand from the code and is not an on-chain transaction. There is no BLOB unit or functional test in the tree to borrow it from.

Blob: the 7 bytes {"a":1}, content type JSON, one chunk. sha256("{\"a\":1}") = 015abd7f5cc57a2dd94b7590f04ad8084273905ee33ec5cebeae62276a97f862.

Native output script (59 bytes):

Text
6a 04 424c4f42
20 015abd7f5cc57a2dd94b7590f04ad8084273905ee33ec5cebeae62276a97f862
04 00000000
04 00000001
01 01
07 7b2261223a317d
BytesMeaning
6aOP_RETURN
04 424c4f42push 4: "BLOB"
20 015a...f862push 32: blob_id
04 00000000push 4: chunk_index = 0
04 00000001push 4: total_chunks = 1
01 01push 1: content_type_id = 1 (application/json)
07 7b2261223a317dpush 7: payload {"a":1}

Once mined, the block's ConnectBlock inserts one blob_chunks row, TryCompleteBlob finds 1 of 1 chunks, the hash matches, and GET /api/v1/blob/015abd7f...f862 returns {"a":1} with Content-Type: application/json.

The same chunk over eMPP (52-byte segment):

Text
6a 50 34 424c4f42 015abd7f5cc57a2dd94b7590f04ad8084273905ee33ec5cebeae62276a97f862 00000000 00000001 01 7b2261223a317d

6a OP_RETURN, 50 OP_RESERVED, 34 push 52 bytes: "BLOB", then blob_id, index 0, total 1, type 1 and the payload. It is stored with source = 'empp'.

Source files #

  • src/modules/blob/blob_module.h, src/modules/blob/blob_module.cpp: decoder, schema, reassembly, REST.
  • src/modules/empp/empp_module.cpp (SegmentsForBlock), src/modules/empp/empp_decoder.cpp: eMPP transport.
  • src/modules/builtin_modules.cpp, src/modules/module_registry.cpp: registration order, reconciliation.
  • src/script/standard.h: MAX_OP_RETURN_RELAY.
  • share/www/dashboard/js/dashboard.js: BLOB tab.
  • src/test/empp_decoder_tests.cpp: uses "BLOB" as a LOKAD id in eMPP envelope tests only.
  • Older notes: doc/blob.md (includes the client-side blobStore() helper of the poker app), doc/lotus-poker-nft-v1.md (lotus-blob:// URIs).
Source doc/protocols/blob.md in the lotusd repository