feat(api): add HTTP Range request support for streaming by Zorlin · Pull Request #82 · durability-labs/archivist-node

Zorlin · 2025-12-26T05:41:05Z

Implements byte-range requests (RFC 7233) for file downloads:
- Add RangeStream wrapper for partial content streaming
- Parse Range header (e.g., "bytes=0-499", "bytes=500-")
- Return 206 Partial Content with Content-Range header
- Return 416 Range Not Satisfiable for invalid ranges
- Advertise Accept-Ranges: bytes on all file responses

  Enables seeking in media players for audio/video content.

  🤖 Generated with [Claude Code](https://claude.com/claude-code)

This leads to problems serializing TOML, see also: status-im/nim-serialization#104

Implements directory manifests for organizing multiple files into a tree: - New DirectoryCodec (0xCD04) for directory manifest CIDs - DirectoryManifest type with entries, totalSize, filesCount - Two-phase upload: files uploaded individually, then /directory endpoint finalizes them into a tree structure with POST JSON body - HTML directory browsing with Archivist branding at /data/{cid} - JSON directory listing when Accept header doesn't include text/html - Path resolution within directories via /data/{cid}/path?p=subdir/file - Auto-promotion of single-child root directories API changes: - POST /api/archivist/v1/directory - finalize directory from uploaded files - GET /api/archivist/v1/data/{cid} - now serves HTML for directories - Added MIME types: audio/mpeg, audio/flac, video/webm, etc. - Relaxed filename validation to allow paths like "Album/track.mp3" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

- Add HEAD endpoint for /api/archivist/v1/data/{cid} to support metadata preflight checks (content-type, size) without full download - Add HEAD endpoint for /api/archivist/v1/data/{cid}/network/stream to support jsmediatags ID3 tag reading - Fix directory traversal check to allow ".." in filenames (e.g. "F.R.E.S.H..mp3") while still blocking actual traversal attacks 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

Implements byte-range requests (RFC 7233) for file downloads: - Add RangeStream wrapper for partial content streaming - Parse Range header (e.g., "bytes=0-499", "bytes=500-") - Return 206 Partial Content with Content-Range header - Return 416 Range Not Satisfiable for invalid ranges - Advertise Accept-Ranges: bytes on all file responses Enables seeking in media players for audio/video content. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

github-actions

Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit

nph

[nph] _{reported by reviewdog 🐶}

	without subdirManifest =? (
	await fetchDirectoryManifest(node.networkStore, subdirCid)
	), err:

	entries.add(DirectoryEntry.new(
	name = name,
	cid = subdirCid,
	size = subdirManifest.totalSize,
	isDirectory = true,
	))

	entries.add(DirectoryEntry.new(
	name = f.name,
	cid = f.cid,
	size = f.size,
	isDirectory = false,
	mimetype = if f.mimetype.isSome: f.mimetype.unsafeGet() else: "",
	))

	entries.sort(proc(a, b: DirectoryEntry): int =
	if a.isDirectory and not b.isDirectory:
	return -1
	elif not a.isDirectory and b.isDirectory:
	return 1
	else:
	return cmp(a.name, b.name)

	let dirManifest = DirectoryManifest.new(
	entries = entries,
	name = dirNode.name,
	)

	resp.setHeader("Content-Disposition",
	"attachment; filename=\"" & manifest.filename.get() & "\"")

	return RestApiResponse.redirect(
	Http307, "/api/archivist/v1/data/" & $cidVal
	)

	return RestApiResponse.error(
	Http404, "Path not found: " & part, headers = headers
	)

	displayName = if entry.isDirectory: entryName & "/" else: entryName

	html &= fmt""" <div class="file-row">

	type
	RangeStream* = ref object of LPStream
	source*: SeekableStream
	rangeStart*: int # First byte of the range (inclusive)
	rangeEnd*: int # Last byte of the range (inclusive)
	bytesRemaining*: int # Bytes left to read in this range

	T: type RangeStream,
	source: SeekableStream,
	rangeStart: int,
	rangeEnd: int

	source: source,
	rangeStart: rangeStart,
	rangeEnd: rangeEnd,
	bytesRemaining: rangeLen

		ProtoBuffer, initProtoBuffer, getField, getRequiredRepeatedField,
		write, finish, ProtoResult

-  ProtoBuffer, initProtoBuffer, getField, getRequiredRepeatedField,
-  write, finish, ProtoResult
+  ProtoBuffer, initProtoBuffer, getField, getRequiredRepeatedField, write, finish,
+  ProtoResult

+                  entries.add(DirectoryEntry(
+                    name: entryName,
+                    cid: entryCid,
+                    size: size.NBytes,
+                    isDirectory: isDir != 0,
+                    mimetype: mimetype,
+                  ))

+                success DirectoryManifest(
+                  entries: entries,
+                  totalSize: totalSize.NBytes,
+                  name: name,
+                )

+                  cid = blk.cid,
+                  entries = directory.entries.len,
+                  totalSize = directory.totalSize

+                  cid*: Cid
+                  size*: NBytes
+                  isDirectory*: bool
+                  mimetype*: string  # Empty string = not set

	mimetype*: string # Empty string = not set
	mimetype*: string # Empty string = not set

+                    # Validate and normalize path
+                    var normalPath = pathStr.replace("\\", "/")
+                    while normalPath.len > 0 and normalPath[0] == '/':
+                      normalPath = normalPath[1..^1]

	normalPath = normalPath[1..^1]
	normalPath = normalPath[1 ..^ 1]

+                    for part in pathParts:
+                      if part == "..":
+                        return RestApiResponse.error(
+                          Http400, "Invalid path (directory traversal not allowed): " & pathStr,

	Http400, "Invalid path (directory traversal not allowed): " & pathStr,
	Http400,
	"Invalid path (directory traversal not allowed): " & pathStr,

Conversation

Zorlin commented Dec 26, 2025

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot Dec 26, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Dec 26, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Dec 26, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Dec 26, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Dec 26, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Dec 26, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Dec 26, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Dec 26, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Dec 26, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Dec 26, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

+                    inputEntries.add(InputEntry(
+                      path: normalPath,
+                      cid: cidVal,
+                      size: size,
+                      mimetype: mimetypeOpt,
+                    ))

+                        current.subdirs[part] = DirNode(
+                          name: part,
+                          subdirs: initTable[string, DirNode](),
+                        )

+                    current.files.add((
+                      name: pathParts[^1],
+                      cid: entry.cid,
+                      size: entry.size,
+                      mimetype: entry.mimetype,
+                    ))