summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
Diffstat
-rw-r--r--README.md2
-rw-r--r--docs/architecture.md4
-rw-r--r--docs/implementation-status.md11
-rw-r--r--scripts/python/README.md11
-rw-r--r--scripts/python/benchmark.py24
-rw-r--r--scripts/python/p2_benchmark.py180
-rw-r--r--scripts/python/p2_benchmark_10000x1.json12
-rw-r--r--scripts/python/p2_benchmark_50000x100.json12
-rw-r--r--src/CMakeLists.txt1
-rw-r--r--src/mcp/tools/common/context.c20
-rw-r--r--src/mcp/tools/common/context.h3
-rw-r--r--src/mcp/tools/common/history.c286
-rw-r--r--src/mcp/tools/common/history.h17
-rw-r--r--src/mcp/tools/read_messages.c298
-rw-r--r--src/mcp/tools/rebuild_index.c9
-rw-r--r--src/mcp/tools/write_message.c8
-rw-r--r--tests/CMakeLists.txt1
-rw-r--r--tests/function-coverage.md1
-rw-r--r--tests/mcp_protocol_test.py114
-rw-r--r--tests/storage_test.c28
20 files changed, 1005 insertions, 37 deletions
diff --git a/README.md b/README.md
index 2471d49..db9a5ff 100644
--- a/README.md
+++ b/README.md
@@ -55,7 +55,7 @@ stdoutはMCP JSON-RPC専用、stderrは診断専用です。
## MCP tools
- `write_message`: サーバ生成ULID・UTC timestamp付きメッセージを同期永続化
-- `read_messages`: sessionの有効メッセージを時系列順に取得
+- `read_messages`: sessionの有効メッセージを時系列順にpagination取得（既定100件、最大1000件）
- `search_messages`: contentとtagsをUTF-8コードポイントの2/3-gramで検索
- `delete_message`: DELレコードを同期永続化して即時不可視化
- `rebuild_index`: 有効PUTだけにcompactionしsnapshotを再生成
diff --git a/docs/architecture.md b/docs/architecture.md
index 9bd7c62..64d4b06 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -49,7 +49,7 @@ where it conflicts with this section, the implemented contract is authoritative.
- `--data-dir <path>` is mandatory and guarded by a process-exclusive lock.
The prototype remains Windows x64/MSVC and single-process. Retention,
-encryption, pagination, multiple writers, automated summarization, and secure
+encryption, search pagination, multiple writers, automated summarization, and secure
physical erasure are outside v1.
# ProtoType Architecture v0.0.1
@@ -58,7 +58,7 @@ Tiny-ConvMem-MCP: FINAL TOOL INTERFACE SPECIFICATION (v1.0)
この定義は、システムと外部（LLMプロセス）との間の「契約書」となります。これで実装に着手できますね！
No. Tool Name カテゴリ 機能概要 入力引数 (Arguments) 出力値 (Response)
1 write_message SET/WRITE 新しいメッセージイベントをログファイルへ追記し、インデックスを更新する。 session_id, role, content, timestamp status: Enum(OK, ERROR)
-2 read_messages GET/READ 特定セッションIDの全メッセージを時系列順に読み出す。（フル履歴取得） session_id: 文字列 history: 配列（メッセージオブジェクト）
+2 read_messages GET/READ 特定セッションIDのメッセージを安定順序でページ取得する。 session_id、limit、cursor history、nextCursor
3 search_messages GET/READ クエリに基づいてログファイルから関連するメッセージを特定し、返す。（インデックス利用が前提） query, session_id (任意), limit: 整数 results: 配列（メッセージオブジェクト）
4 delete_message SET/MANAGE 特定のメッセージIDを指定してログから完全に削除し、インデックスをクリーンアップする。 message_id: 唯一ID, session_id (任意) status: Enum(OK, NOT_FOUND)
5 rebuild_index SET/MANAGE インデックスファイルを再生成し、データ整合性を回復させる。（メンテナンス） なし status: Enum(OK, ERROR)
diff --git a/docs/implementation-status.md b/docs/implementation-status.md
index 52c037b..d0f6d8f 100644
--- a/docs/implementation-status.md
+++ b/docs/implementation-status.md
@@ -35,6 +35,11 @@ P1のgram bucket hash table化後、Releaseの32KiB write p50は128.242msから5
1KiB 0.217ms、8KiB 1.299ms、32KiB 3.948ms、1MiB 34.988msで、payload増加に対する
二次的悪化がないことを確認しました。
+P2の大規模履歴試験では、1 session 1万件を10ページ・34.2msで全件取得し、restart
+152.0ms、rebuild 109.0ms、peak working set 18.1MiBでした。100 session 5万件では
+100ページ・159.0ms、restart 3.22s、rebuild 868.4ms、peak working set 48.9MiBで、
+重複・欠落はなく最大応答も210,173 bytes以下でした。
+
実行コマンド:
```powershell
@@ -88,7 +93,7 @@ ctest --preset windows-vs2026-asan
| metadata `tool_call_id` | ✅ 実装済 | ⬜ 未作成 | ⬜ 未実施 | round-tripと上限境界が未確認 |
| metadata `tags` | ✅ 実装済 | ✅ 作成済 | ✅ Debug/Release | 日本語tag検索をDomainで確認。32件/128 byte境界は未確認 |
| metadata `memory_tier` | ✅ 実装済 | ✅ 作成済 | ✅ Debug/Release | enumとMCP入力を確認。自動ライフサイクルは対象外 |
-| `read_messages` 全履歴取得 | ✅ 実装済 | ✅ 作成済 | ✅ Debug/Release | 1件/0件を確認 |
+| `read_messages` keyset pagination | ✅ 実装済 | ✅ 作成済 | ✅ Debug/Release | 既定100、最大1000、8MiB応答境界、restart/rebuild cursor継続 |
| timestamp、message_id安定順序 | ✅ 実装済 | ⬜ 未作成 | ⬜ 未実施 | 複数件・同一timestampの順序試験が必要 |
| `search_messages` content検索 | ✅ 実装済 | ✅ 作成済 | ✅ Debug/Release | ASCII大小文字とMCP検索を確認 |
| 日本語・tag検索 | ✅ 実装済 | ✅ 作成済 | ✅ Debug/Release | Domainで確認 |
@@ -136,7 +141,7 @@ ctest --preset windows-vs2026-asan
| write時索引更新 | ✅ 実装済 | ✅ MCP検索で確認 | ✅ Debug/Release | index allocation失敗のfault injectionなし |
| compaction後索引再構築 | ✅ 実装済 | 🟡 rebuild後health/readのみ | 🟡 一部実施済 | rebuild直後のsearch専用試験が必要 |
| `message_id -> entry` hash map | ⬜ 未実装 | ⬜ 未作成 | ⬜ 未実施 | 現在は配列の線形探索 |
-| `session_id -> message list` index | ⬜ 未実装 | ⬜ 未作成 | ⬜ 未実施 | 現在は全entry走査 |
+| `session_id -> message list` index | ✅ 実装済 | ✅ 作成済 | ✅ Debug/Release | open addressing session mapと安定順序vector |
| Unicode normalization/full case folding | ➖ v1対象外 | ➖ 該当なし | ➖ 該当なし | NFKC、非ASCII大文字小文字変換は未対応 |
## バイナリログと耐久性
@@ -239,7 +244,7 @@ ctest --preset windows-vs2026-asan
| background worker | ➖ 非採用 | 単一stdio・単一スレッドの初期実装に限定 |
| 自動要約、tier昇格、discard削除 | ➖ 対象外 | LLMライフサイクルは6 tool完成後の別機能 |
| encryption at rest | ➖ 対象外 | 鍵管理を含む別設計が必要 |
-| pagination | ➖ 対象外 | `read_messages` は現状フル履歴契約 |
+| search pagination | ➖ 対象外 | `read_messages`のみkeyset pagination対応 |
| 複数writer/network server | ➖ 対象外 | data directoryを単一プロセスが排他所有 |
| SSD/バックアップの安全消去 | ➖ 対象外 | filesystem compactionだけでは保証不能 |
| Linux/macOS | 🟡 backend実装・未保証 | POSIX file/lock/clock/random実装あり。CI・実機検証は未実施 |
diff --git a/scripts/python/README.md b/scripts/python/README.md
index 8d8ed53..aaab4dc 100644
--- a/scripts/python/README.md
+++ b/scripts/python/README.md
@@ -106,7 +106,7 @@ write_message (small ~20B) 100 0 0.399 1.009
- `delete_message.session_id` は任意
- `inputSchema.required` とruntimeの必須条件が一致
- 成功・エラーの全応答がJSON-RPC 2.0形式で、Python MCP SDKから解釈可能
-- `read_messages` は現状フル履歴を返し、paginationは未実装
+- `read_messages` は既定100件・最大1000件で、`nextCursor`を追跡して全履歴を取得
未対応・確認していない項目:
@@ -122,5 +122,14 @@ Python MCP SDKでglobal/session-scoped検索を同一接続上で1000回交互
.venv\Scripts\python.exe p0_regression.py --server ..\..\build\windows-vs2026\src\Release\tiny-convmem-mcp.exe
```
+## P2大規模履歴ベンチマーク
+
+```powershell
+.venv\Scripts\python.exe p2_benchmark.py --server ..\..\build\windows-vs2026\src\Release\tiny-convmem-mcp.exe --messages 10000 --sessions 1 --output p2_benchmark_10000x1.json
+.venv\Scripts\python.exe p2_benchmark.py --server ..\..\build\windows-vs2026\src\Release\tiny-convmem-mcp.exe --messages 50000 --sessions 100 --output p2_benchmark_50000x100.json
+```
+
+raw stdioで全ページを検証し、restart、rebuild、peak working set、最大応答byte数を記録します。
+
従来の約254リクエストでの切断は、Windowsの符号なし`SOCKET`でstdio sentinelを
ネットワーク接続と誤判定し、connection poolを消費していたことが原因でした。
diff --git a/scripts/python/benchmark.py b/scripts/python/benchmark.py
index 5a146ce..0580287 100644
--- a/scripts/python/benchmark.py
+++ b/scripts/python/benchmark.py
@@ -274,13 +274,31 @@ async def write_message(
return msg
-async def read_messages(session: ClientSession, session_id: str) -> list[dict]:
- result = await session.call_tool("read_messages", {"session_id": session_id})
+async def read_messages_page(
+ session: ClientSession, session_id: str, limit: int = 100, cursor: str | None = None
+) -> tuple[list[dict], str | None]:
+ arguments: dict[str, Any] = {"session_id": session_id, "limit": limit}
+ if cursor is not None:
+ arguments["cursor"] = cursor
+ result = await session.call_tool("read_messages", arguments)
payload = json_result(result)
history = payload.get("history", [])
if not isinstance(history, list):
raise RuntimeError("history is not a list")
- return history
+ next_cursor = payload.get("nextCursor")
+ if next_cursor is not None and not isinstance(next_cursor, str):
+ raise RuntimeError("nextCursor is not a string")
+ return history, next_cursor
+
+
+async def read_messages(session: ClientSession, session_id: str) -> list[dict]:
+ history: list[dict] = []
+ cursor: str | None = None
+ while True:
+ page, cursor = await read_messages_page(session, session_id, cursor=cursor)
+ history.extend(page)
+ if cursor is None:
+ return history
async def search_messages(
diff --git a/scripts/python/p2_benchmark.py b/scripts/python/p2_benchmark.py
new file mode 100644
index 0000000..792673c
--- /dev/null
+++ b/scripts/python/p2_benchmark.py
@@ -0,0 +1,180 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import argparse
+import ctypes
+import json
+import shutil
+import subprocess
+import tempfile
+import time
+from pathlib import Path
+from typing import Any
+
+
+class ProcessMemoryCounters(ctypes.Structure):
+ _fields_ = [
+ ("cb", ctypes.c_ulong),
+ ("PageFaultCount", ctypes.c_ulong),
+ ("PeakWorkingSetSize", ctypes.c_size_t),
+ ("WorkingSetSize", ctypes.c_size_t),
+ ("QuotaPeakPagedPoolUsage", ctypes.c_size_t),
+ ("QuotaPagedPoolUsage", ctypes.c_size_t),
+ ("QuotaPeakNonPagedPoolUsage", ctypes.c_size_t),
+ ("QuotaNonPagedPoolUsage", ctypes.c_size_t),
+ ("PagefileUsage", ctypes.c_size_t),
+ ("PeakPagefileUsage", ctypes.c_size_t),
+ ]
+
+
+class Server:
+ def __init__(self, exe: Path, data_dir: Path) -> None:
+ self.process = subprocess.Popen(
+ [str(exe), "--data-dir", str(data_dir)],
+ stdin=subprocess.PIPE,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE,
+ text=True,
+ encoding="utf-8",
+ bufsize=1,
+ )
+ self.request_id = 1
+ self.max_response_bytes = 0
+ self.request(
+ "initialize",
+ {
+ "protocolVersion": "2024-11-05",
+ "capabilities": {},
+ "clientInfo": {"name": "p2-benchmark", "version": "1"},
+ },
+ )
+ self.notify("notifications/initialized", {})
+
+ def request(self, method: str, params: dict[str, Any]) -> dict[str, Any]:
+ assert self.process.stdin and self.process.stdout
+ request_id = self.request_id
+ self.request_id += 1
+ self.process.stdin.write(
+ json.dumps(
+ {"jsonrpc": "2.0", "id": request_id, "method": method, "params": params},
+ separators=(",", ":"),
+ )
+ + "\n"
+ )
+ self.process.stdin.flush()
+ line = self.process.stdout.readline()
+ self.max_response_bytes = max(self.max_response_bytes, len(line.encode("utf-8")))
+ response = json.loads(line)
+ if "error" in response:
+ raise RuntimeError(response)
+ return response["result"]
+
+ def call(self, name: str, arguments: dict[str, Any]) -> dict[str, Any]:
+ result = self.request("tools/call", {"name": name, "arguments": arguments})
+ payload = json.loads(result["content"][0]["text"])
+ if result.get("isError"):
+ raise RuntimeError(payload)
+ return payload
+
+ def notify(self, method: str, params: dict[str, Any]) -> None:
+ assert self.process.stdin
+ self.process.stdin.write(
+ json.dumps({"jsonrpc": "2.0", "method": method, "params": params}) + "\n"
+ )
+ self.process.stdin.flush()
+
+ def peak_working_set(self) -> int:
+ counters = ProcessMemoryCounters()
+ counters.cb = ctypes.sizeof(counters)
+ handle = ctypes.windll.kernel32.OpenProcess(0x1000 | 0x0400, False, self.process.pid)
+ try:
+ if handle and ctypes.windll.psapi.GetProcessMemoryInfo(
+ handle, ctypes.byref(counters), counters.cb
+ ):
+ return int(counters.PeakWorkingSetSize)
+ return 0
+ finally:
+ if handle:
+ ctypes.windll.kernel32.CloseHandle(handle)
+
+ def close(self) -> None:
+ if self.process.stdin:
+ self.process.stdin.close()
+ code = self.process.wait(timeout=30)
+ if code != 0:
+ raise RuntimeError(self.process.stderr.read() if self.process.stderr else code)
+
+
+def read_all(server: Server, session_id: str) -> tuple[int, int]:
+ cursor = None
+ count = pages = 0
+ while True:
+ arguments: dict[str, Any] = {"session_id": session_id, "limit": 1000}
+ if cursor:
+ arguments["cursor"] = cursor
+ payload = server.call("read_messages", arguments)
+ count += len(payload["history"])
+ pages += 1
+ cursor = payload.get("nextCursor")
+ if not cursor:
+ return count, pages
+
+
+def main() -> int:
+ parser = argparse.ArgumentParser()
+ parser.add_argument("--server", type=Path, required=True)
+ parser.add_argument("--messages", type=int, choices=(10000, 50000), required=True)
+ parser.add_argument("--sessions", type=int, choices=(1, 100), required=True)
+ parser.add_argument("--output", type=Path, required=True)
+ args = parser.parse_args()
+ data_dir = Path(tempfile.mkdtemp(prefix="tiny-convmem-p2-"))
+ try:
+ server = Server(args.server.resolve(), data_dir)
+ write_started = time.perf_counter()
+ for index in range(args.messages):
+ server.call(
+ "write_message",
+ {
+ "session_id": f"session-{index % args.sessions}",
+ "role": "user",
+ "content": f"history item {index}",
+ },
+ )
+ write_ms = (time.perf_counter() - write_started) * 1000
+ server.close()
+
+ restart_started = time.perf_counter()
+ server = Server(args.server.resolve(), data_dir)
+ restart_ms = (time.perf_counter() - restart_started) * 1000
+ read_started = time.perf_counter()
+ total = pages = 0
+ for session in range(args.sessions):
+ session_count, session_pages = read_all(server, f"session-{session}")
+ total += session_count
+ pages += session_pages
+ read_ms = (time.perf_counter() - read_started) * 1000
+ rebuild_started = time.perf_counter()
+ server.call("rebuild_index", {})
+ rebuild_ms = (time.perf_counter() - rebuild_started) * 1000
+ result = {
+ "messages": args.messages,
+ "sessions": args.sessions,
+ "messages_read": total,
+ "pages": pages,
+ "write_ms": write_ms,
+ "restart_ms": restart_ms,
+ "full_read_ms": read_ms,
+ "rebuild_ms": rebuild_ms,
+ "peak_working_set_bytes": server.peak_working_set(),
+ "max_response_bytes": server.max_response_bytes,
+ }
+ server.close()
+ args.output.write_text(json.dumps(result, indent=2), encoding="utf-8")
+ print(json.dumps(result, indent=2))
+ return 0 if total == args.messages and server.max_response_bytes <= 8 * 1024 * 1024 else 1
+ finally:
+ shutil.rmtree(data_dir, ignore_errors=True)
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/scripts/python/p2_benchmark_10000x1.json b/scripts/python/p2_benchmark_10000x1.json
new file mode 100644
index 0000000..242cde6
--- /dev/null
+++ b/scripts/python/p2_benchmark_10000x1.json
@@ -0,0 +1,12 @@
+{
+ "messages": 10000,
+ "sessions": 1,
+ "messages_read": 10000,
+ "pages": 10,
+ "write_ms": 3506.664600019576,
+ "restart_ms": 151.98930000769906,
+ "full_read_ms": 34.194600011687726,
+ "rebuild_ms": 109.04509999090806,
+ "peak_working_set_bytes": 18960384,
+ "max_response_bytes": 210173
+} \ No newline at end of file
diff --git a/scripts/python/p2_benchmark_50000x100.json b/scripts/python/p2_benchmark_50000x100.json
new file mode 100644
index 0000000..5db0d57
--- /dev/null
+++ b/scripts/python/p2_benchmark_50000x100.json
@@ -0,0 +1,12 @@
+{
+ "messages": 50000,
+ "sessions": 100,
+ "messages_read": 50000,
+ "pages": 100,
+ "write_ms": 41016.715099976864,
+ "restart_ms": 3215.801200014539,
+ "full_read_ms": 159.0429000207223,
+ "rebuild_ms": 868.445400003111,
+ "peak_working_set_bytes": 51240960,
+ "max_response_bytes": 105980
+} \ No newline at end of file
diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index 66ef748..1998150 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -29,6 +29,7 @@ add_executable(tiny-convmem-mcp
mcp/server.c
mcp/tools/common/common.c
mcp/tools/common/context.c
+ mcp/tools/common/history.c
mcp/tools/common/registry.c
mcp/tools/delete_message.c
mcp/tools/get_system_health.c
diff --git a/src/mcp/tools/common/context.c b/src/mcp/tools/common/context.c
index 1a469e0..d79b5de 100644
--- a/src/mcp/tools/common/context.c
+++ b/src/mcp/tools/common/context.c
@@ -42,8 +42,11 @@ tiny_mcp_tools_context_new(tiny_store_t *store)
}
context->store = store;
context->search_index = tiny_mcp_tools_build_search_index(store);
- if (!context->search_index)
+ context->history_index = tiny_history_index_build(store);
+ if (!context->search_index || !context->history_index)
{
+ tiny_search_index_free(context->search_index);
+ tiny_history_index_free(context->history_index);
free(context);
return NULL;
}
@@ -58,6 +61,7 @@ tiny_mcp_tools_context_free(tiny_mcp_tools_context_t *context)
return;
}
tiny_search_index_free(context->search_index);
+ tiny_history_index_free(context->history_index);
free(context);
}
@@ -86,3 +90,17 @@ tiny_mcp_tools_context_current(void)
{
return active_context;
}
+
+bool
+tiny_mcp_tools_ensure_history_index(tiny_mcp_tools_context_t *context)
+{
+ if (!context)
+ {
+ return false;
+ }
+ if (!context->history_index)
+ {
+ context->history_index = tiny_history_index_build(context->store);
+ }
+ return context->history_index != NULL;
+}
diff --git a/src/mcp/tools/common/context.h b/src/mcp/tools/common/context.h
index 0847d4b..2a6650a 100644
--- a/src/mcp/tools/common/context.h
+++ b/src/mcp/tools/common/context.h
@@ -2,12 +2,14 @@
#define TINY_CONVMEM_MCP_TOOLS_CONTEXT_H
#include "domain/search.h"
+#include "mcp/tools/common/history.h"
#include "storage/store.h"
typedef struct tiny_mcp_tools_context
{
tiny_store_t *store;
tiny_search_index_t *search_index;
+ tiny_history_index_t *history_index;
} tiny_mcp_tools_context_t;
tiny_mcp_tools_context_t *tiny_mcp_tools_context_new(tiny_store_t *store);
@@ -16,5 +18,6 @@ bool tiny_mcp_tools_context_activate(tiny_mcp_tools_context_t *context);
void tiny_mcp_tools_context_deactivate(tiny_mcp_tools_context_t *context);
tiny_mcp_tools_context_t *tiny_mcp_tools_context_current(void);
tiny_search_index_t *tiny_mcp_tools_build_search_index(const tiny_store_t *store);
+bool tiny_mcp_tools_ensure_history_index(tiny_mcp_tools_context_t *context);
#endif
diff --git a/src/mcp/tools/common/history.c b/src/mcp/tools/common/history.c
new file mode 100644
index 0000000..6254d8c
--- /dev/null
+++ b/src/mcp/tools/common/history.c
@@ -0,0 +1,286 @@
+#include "mcp/tools/common/history.h"
+
+#include <stdint.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "base/allocator.h"
+
+typedef struct history_bucket
+{
+ uint64_t hash;
+ char *session_id;
+ size_t *entries;
+ size_t count;
+ size_t capacity;
+} history_bucket_t;
+
+struct tiny_history_index
+{
+ history_bucket_t *buckets;
+ size_t count;
+ size_t capacity;
+ size_t *slots;
+ size_t slot_capacity;
+};
+
+#define EMPTY_SLOT SIZE_MAX
+
+static uint64_t
+session_hash(const char *value)
+{
+ uint64_t hash = UINT64_C(1469598103934665603);
+ while (*value)
+ {
+ hash ^= (unsigned char)*value++;
+ hash *= UINT64_C(1099511628211);
+ }
+ return hash;
+}
+
+static size_t
+start_slot(uint64_t hash, size_t capacity)
+{
+ return (size_t)(hash ^ (hash >> 32)) & (capacity - 1);
+}
+
+static history_bucket_t *
+find_bucket(const tiny_history_index_t *index, const char *session_id)
+{
+ uint64_t hash;
+ size_t slot, bucket;
+ if (!index || !session_id || !index->slot_capacity)
+ {
+ return NULL;
+ }
+ hash = session_hash(session_id);
+ slot = start_slot(hash, index->slot_capacity);
+ while ((bucket = index->slots[slot]) != EMPTY_SLOT)
+ {
+ if (bucket >= index->count)
+ {
+ return NULL;
+ }
+ if (index->buckets[bucket].hash == hash
+ && strcmp(index->buckets[bucket].session_id, session_id) == 0)
+ {
+ return &index->buckets[bucket];
+ }
+ slot = (slot + 1) & (index->slot_capacity - 1);
+ }
+ return NULL;
+}
+
+static void
+insert_slot(size_t *slots, size_t capacity, const history_bucket_t *buckets, size_t bucket)
+{
+ size_t slot = start_slot(buckets[bucket].hash, capacity);
+ while (slots[slot] != EMPTY_SLOT)
+ {
+ slot = (slot + 1) & (capacity - 1);
+ }
+ slots[slot] = bucket;
+}
+
+static bool
+reserve_table(tiny_history_index_t *index, size_t required)
+{
+ history_bucket_t *buckets;
+ size_t *slots, capacity, position;
+ if (required > index->capacity)
+ {
+ capacity = index->capacity ? index->capacity : 32;
+ while (capacity < required)
+ {
+ if (capacity > SIZE_MAX / 2)
+ {
+ return false;
+ }
+ capacity *= 2;
+ }
+ if (capacity > SIZE_MAX / sizeof(*buckets))
+ {
+ return false;
+ }
+ buckets = realloc(index->buckets, capacity * sizeof(*buckets));
+ if (!buckets)
+ {
+ return false;
+ }
+ index->buckets = buckets;
+ index->capacity = capacity;
+ }
+ capacity = index->slot_capacity ? index->slot_capacity : 64;
+ while (required > (capacity / 10) * 7)
+ {
+ if (capacity > SIZE_MAX / 2)
+ {
+ return false;
+ }
+ capacity *= 2;
+ }
+ if (capacity == index->slot_capacity)
+ {
+ return true;
+ }
+ slots = malloc(capacity * sizeof(*slots));
+ if (!slots)
+ {
+ return false;
+ }
+ for (position = 0; position < capacity; ++position)
+ {
+ slots[position] = EMPTY_SLOT;
+ }
+ for (position = 0; position < index->count; ++position)
+ {
+ insert_slot(slots, capacity, index->buckets, position);
+ }
+ free(index->slots);
+ index->slots = slots;
+ index->slot_capacity = capacity;
+ return true;
+}
+
+static int
+compare_entry(const tiny_store_t *store, size_t left, size_t right)
+{
+ const tiny_message_t *a = &tiny_store_entry(store, left)->message;
+ const tiny_message_t *b = &tiny_store_entry(store, right)->message;
+ if (a->timestamp_ms < b->timestamp_ms)
+ {
+ return -1;
+ }
+ if (a->timestamp_ms > b->timestamp_ms)
+ {
+ return 1;
+ }
+ return strcmp(a->message_id, b->message_id);
+}
+
+bool
+tiny_history_index_add(tiny_history_index_t *index, const tiny_store_t *store, size_t entry_index)
+{
+ const tiny_store_entry_t *entry = tiny_store_entry(store, entry_index);
+ history_bucket_t *bucket;
+ size_t low, high, capacity;
+ if (!index || !entry || !entry->message.session_id)
+ {
+ return false;
+ }
+ bucket = find_bucket(index, entry->message.session_id);
+ if (!bucket)
+ {
+ char *session = NULL;
+ size_t *entries = malloc(8 * sizeof(*entries));
+ size_t length = strlen(entry->message.session_id);
+ if (!entries || !(session = malloc(length + 1)) || index->count == SIZE_MAX
+ || !reserve_table(index, index->count + 1))
+ {
+ free(entries);
+ free(session);
+ return false;
+ }
+ memcpy(session, entry->message.session_id, length + 1);
+ bucket = &index->buckets[index->count];
+ memset(bucket, 0, sizeof(*bucket));
+ bucket->hash = session_hash(session);
+ bucket->session_id = session;
+ bucket->entries = entries;
+ bucket->capacity = 8;
+ insert_slot(index->slots, index->slot_capacity, index->buckets, index->count++);
+ }
+ if (bucket->count == bucket->capacity)
+ {
+ size_t *resized;
+ if (bucket->capacity > SIZE_MAX / 2
+ || (capacity = bucket->capacity * 2) > SIZE_MAX / sizeof(*resized)
+ || !(resized = realloc(bucket->entries, capacity * sizeof(*resized))))
+ {
+ return false;
+ }
+ bucket->entries = resized;
+ bucket->capacity = capacity;
+ }
+ low = 0;
+ high = bucket->count;
+ while (low < high)
+ {
+ size_t middle = low + (high - low) / 2;
+ if (compare_entry(store, bucket->entries[middle], entry_index) <= 0)
+ {
+ low = middle + 1;
+ }
+ else
+ {
+ high = middle;
+ }
+ }
+ memmove(bucket->entries + low + 1,
+ bucket->entries + low,
+ (bucket->count - low) * sizeof(*bucket->entries));
+ bucket->entries[low] = entry_index;
+ bucket->count++;
+ return true;
+}
+
+tiny_history_index_t *
+tiny_history_index_build(const tiny_store_t *store)
+{
+ tiny_history_index_t *index = calloc(1, sizeof(*index));
+ size_t position;
+ if (!index)
+ {
+ return NULL;
+ }
+ for (position = 0; position < tiny_store_entry_count(store); ++position)
+ {
+ if (!tiny_store_entry(store, position)->deleted
+ && !tiny_history_index_add(index, store, position))
+ {
+ tiny_history_index_free(index);
+ return NULL;
+ }
+ }
+ return index;
+}
+
+void
+tiny_history_index_free(tiny_history_index_t *index)
+{
+ size_t position;
+ if (!index)
+ {
+ return;
+ }
+ for (position = 0; position < index->count; ++position)
+ {
+ free(index->buckets[position].session_id);
+ free(index->buckets[position].entries);
+ }
+ free(index->buckets);
+ free(index->slots);
+ free(index);
+}
+
+bool
+tiny_history_index_session(const tiny_history_index_t *index,
+ const char *session_id,
+ const size_t **entries,
+ size_t *count)
+{
+ history_bucket_t *bucket;
+ if (!entries || !count)
+ {
+ return false;
+ }
+ *entries = NULL;
+ *count = 0;
+ bucket = find_bucket(index, session_id);
+ if (bucket)
+ {
+ *entries = bucket->entries;
+ *count = bucket->count;
+ }
+ return index && session_id;
+}
diff --git a/src/mcp/tools/common/history.h b/src/mcp/tools/common/history.h
new file mode 100644
index 0000000..ac08d99
--- /dev/null
+++ b/src/mcp/tools/common/history.h
@@ -0,0 +1,17 @@
+#ifndef TINY_CONVMEM_MCP_TOOLS_HISTORY_H
+#define TINY_CONVMEM_MCP_TOOLS_HISTORY_H
+
+#include "storage/store.h"
+
+typedef struct tiny_history_index tiny_history_index_t;
+
+tiny_history_index_t *tiny_history_index_build(const tiny_store_t *store);
+void tiny_history_index_free(tiny_history_index_t *index);
+bool
+tiny_history_index_add(tiny_history_index_t *index, const tiny_store_t *store, size_t entry_index);
+bool tiny_history_index_session(const tiny_history_index_t *index,
+ const char *session_id,
+ const size_t **entries,
+ size_t *count);
+
+#endif
diff --git a/src/mcp/tools/read_messages.c b/src/mcp/tools/read_messages.c
index 5a9e49f..1581534 100644
--- a/src/mcp/tools/read_messages.c
+++ b/src/mcp/tools/read_messages.c
@@ -3,25 +3,186 @@
#include "mcp/tools/common/common.h"
#include "mcp/tools/common/context.h"
+#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include "base/allocator.h"
+#include "storage/codec.h"
+
+#define READ_DEFAULT_LIMIT 100
+#define READ_MAX_LIMIT 1000
+#define READ_RESPONSE_BUDGET (8U * 1024U * 1024U - 64U * 1024U)
+#define CURSOR_BYTES 47U
+#define CURSOR_TEXT_MAX 64U
+
+typedef struct history_cursor
+{
+ int64_t timestamp;
+ char message_id[27];
+} history_cursor_t;
+
+static uint64_t
+session_hash(const char *value)
+{
+ uint64_t hash = UINT64_C(1469598103934665603);
+ while (*value)
+ {
+ hash ^= (unsigned char)*value++;
+ hash *= UINT64_C(1099511628211);
+ }
+ return hash;
+}
+
+static size_t
+base64url_encode(const unsigned char *input, size_t length, char *output, size_t capacity)
+{
+ static const char alphabet[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_";
+ size_t source = 0, target = 0;
+ while (source < length)
+ {
+ uint32_t value = (uint32_t)input[source++] << 16;
+ unsigned available = 1;
+ if (source < length)
+ {
+ value |= (uint32_t)input[source++] << 8;
+ available++;
+ }
+ if (source < length)
+ {
+ value |= input[source++];
+ available++;
+ }
+ if (target + available + 1 >= capacity)
+ {
+ return 0;
+ }
+ output[target++] = alphabet[(value >> 18) & 63];
+ output[target++] = alphabet[(value >> 12) & 63];
+ if (available > 1)
+ {
+ output[target++] = alphabet[(value >> 6) & 63];
+ }
+ if (available > 2)
+ {
+ output[target++] = alphabet[value & 63];
+ }
+ }
+ output[target] = '\0';
+ return target;
+}
static int
-compare_history(const void *left, const void *right)
+base64_value(unsigned char value)
+{
+ if (value >= 'A' && value <= 'Z')
+ {
+ return value - 'A';
+ }
+ if (value >= 'a' && value <= 'z')
+ {
+ return value - 'a' + 26;
+ }
+ if (value >= '0' && value <= '9')
+ {
+ return value - '0' + 52;
+ }
+ if (value == '-')
+ {
+ return 62;
+ }
+ if (value == '_')
+ {
+ return 63;
+ }
+ return -1;
+}
+
+static bool
+base64url_decode(const char *input, unsigned char output[CURSOR_BYTES])
{
- const tiny_store_entry_t *a = *(const tiny_store_entry_t *const *)left;
- const tiny_store_entry_t *b = *(const tiny_store_entry_t *const *)right;
- if (a->message.timestamp_ms < b->message.timestamp_ms)
+ size_t length = strlen(input), source = 0, target = 0;
+ uint32_t accumulator = 0;
+ unsigned bits = 0;
+ if (length != 63)
+ {
+ return false;
+ }
+ while (source < length)
+ {
+ int value = base64_value((unsigned char)input[source++]);
+ if (value < 0)
+ {
+ return false;
+ }
+ accumulator = (accumulator << 6) | (uint32_t)value;
+ bits += 6;
+ if (bits >= 8)
+ {
+ bits -= 8;
+ if (target >= CURSOR_BYTES)
+ {
+ return false;
+ }
+ output[target++] = (unsigned char)(accumulator >> bits);
+ accumulator &= bits ? (UINT32_C(1) << bits) - 1 : 0;
+ }
+ }
+ return target == CURSOR_BYTES && accumulator == 0;
+}
+
+static bool
+cursor_encode(const char *session, const tiny_message_t *message, char output[CURSOR_TEXT_MAX])
+{
+ unsigned char data[CURSOR_BYTES] = { 0 };
+ data[0] = 1;
+ tiny_codec_put_u64(data + 1, session_hash(session));
+ tiny_codec_put_u64(data + 9, (uint64_t)message->timestamp_ms);
+ memcpy(data + 17, message->message_id, 26);
+ tiny_codec_put_u32(data + 43, tiny_codec_crc32(data, 43));
+ return base64url_encode(data, sizeof(data), output, CURSOR_TEXT_MAX) == 63;
+}
+
+static bool
+cursor_decode(const char *session, const char *value, history_cursor_t *cursor)
+{
+ unsigned char data[CURSOR_BYTES];
+ if (!value || !value[0] || !base64url_decode(value, data) || data[0] != 1
+ || tiny_codec_get_u64(data + 1) != session_hash(session)
+ || tiny_codec_get_u32(data + 43) != tiny_codec_crc32(data, 43))
+ {
+ return false;
+ }
+ memcpy(cursor->message_id, data + 17, 26);
+ cursor->message_id[26] = '\0';
+ cursor->timestamp = (int64_t)tiny_codec_get_u64(data + 9);
+ return cursor->timestamp >= 0 && tiny_message_id_valid(cursor->message_id);
+}
+
+static int
+compare_cursor(const tiny_message_t *message, const history_cursor_t *cursor)
+{
+ if (message->timestamp_ms < cursor->timestamp)
{
return -1;
}
- if (a->message.timestamp_ms > b->message.timestamp_ms)
+ if (message->timestamp_ms > cursor->timestamp)
{
return 1;
}
- return strcmp(a->message.message_id, b->message.message_id);
+ return strcmp(message->message_id, cursor->message_id);
+}
+
+static size_t
+escaped_size(const char *value, size_t length)
+{
+ size_t position, total = 0;
+ for (position = 0; position < length; ++position)
+ {
+ unsigned char byte = (unsigned char)value[position];
+ total += byte < 0x20 ? 6 : (byte == '"' || byte == '\\' ? 2 : 1);
+ }
+ return total;
}
static void
@@ -29,52 +190,139 @@ call_read_messages(const mcpc_tool_t *tool, mcpc_ucbr_t *result)
{
tiny_mcp_tools_context_t *context = tiny_mcp_tools_context_current();
char *session = tiny_mcp_tool_get_string(tool, "session_id", TINY_SESSION_ID_MAX);
- const tiny_store_entry_t **items = NULL;
- size_t index, count = 0;
+ char *cursor_text = tiny_mcp_tool_get_string(tool, "cursor", CURSOR_TEXT_MAX);
+ int32_t requested_limit = READ_DEFAULT_LIMIT;
+ mcpc_errcode_t limit_status
+ = mcpc_tool_get_tpropval_i32(tool, (const char8_t *)"limit", &requested_limit);
+ const size_t *entries = NULL;
+ size_t entry_count = 0, position = 0, emitted = 0, outer_size = 256;
+ history_cursor_t cursor;
+ bool has_cursor = cursor_text != NULL, more = false;
+ const tiny_message_t *last = NULL;
tiny_json_buffer_t json;
- if (!context || !session || !session[0])
+
+ if (!context || !session || !session[0]
+ || (limit_status != MCPC_EC_0 && limit_status != MCPC_EC_ANYDATA_NOTFOUND)
+ || requested_limit < 1 || requested_limit > READ_MAX_LIMIT
+ || (has_cursor && !cursor_decode(session, cursor_text, &cursor)))
{
+ tiny_mcp_tool_add_error(result, "VALIDATION_ERROR", "invalid read_messages arguments");
free(session);
- tiny_mcp_tool_add_error(result, "VALIDATION_ERROR", "session_id is required");
+ free(cursor_text);
return;
}
- items = malloc((tiny_store_entry_count(context->store) + 1) * sizeof(*items));
- if (!items)
+ if (!tiny_mcp_tools_ensure_history_index(context)
+ || !tiny_history_index_session(context->history_index, session, &entries, &entry_count))
{
+ tiny_mcp_tool_add_error(result, "INTERNAL_ERROR", "history index allocation failed");
free(session);
- tiny_mcp_tool_add_error(result, "INTERNAL_ERROR", "out of memory");
+ free(cursor_text);
return;
}
- for (index = 0; index < tiny_store_entry_count(context->store); ++index)
+ if (has_cursor)
{
- const tiny_store_entry_t *entry = tiny_store_entry(context->store, index);
- if (!entry->deleted && strcmp(entry->message.session_id, session) == 0)
+ size_t low = 0, high = entry_count;
+ while (low < high)
{
- items[count++] = entry;
+ size_t middle = low + (high - low) / 2;
+ const tiny_store_entry_t *entry = tiny_store_entry(context->store, entries[middle]);
+ if (compare_cursor(&entry->message, &cursor) <= 0)
+ {
+ low = middle + 1;
+ }
+ else
+ {
+ high = middle;
+ }
}
+ position = low;
}
- qsort(items, count, sizeof(*items), compare_history);
+
tiny_json_init(&json);
tiny_json_raw(&json, "{\"history\":[");
- for (index = 0; index < count; ++index)
+ while (position < entry_count && emitted < (size_t)requested_limit)
{
- if (index)
+ const tiny_store_entry_t *entry = tiny_store_entry(context->store, entries[position]);
+ tiny_json_buffer_t item;
+ size_t item_size;
+ if (!entry || entry->deleted)
+ {
+ position++;
+ continue;
+ }
+ tiny_json_init(&item);
+ tiny_mcp_tool_append_message(&item, &entry->message);
+ item_size = escaped_size(item.data, item.length) + (emitted ? 2 : 0);
+ if (item.failed || (emitted && outer_size + item_size > READ_RESPONSE_BUDGET))
+ {
+ tiny_json_free(&item);
+ if (item.failed)
+ {
+ json.failed = true;
+ }
+ more = true;
+ break;
+ }
+ if (emitted)
{
tiny_json_raw(&json, ",");
}
- tiny_mcp_tool_append_message(&json, &items[index]->message);
+ tiny_json_raw_n(&json, item.data, item.length);
+ outer_size += item_size;
+ emitted++;
+ last = &entry->message;
+ position++;
+ tiny_json_free(&item);
+ }
+ if (!more)
+ {
+ size_t lookahead = position;
+ while (lookahead < entry_count)
+ {
+ const tiny_store_entry_t *entry = tiny_store_entry(context->store, entries[lookahead++]);
+ if (entry && !entry->deleted)
+ {
+ more = true;
+ break;
+ }
+ }
+ }
+ tiny_json_raw(&json, "]");
+ if (more && last)
+ {
+ char next_cursor[CURSOR_TEXT_MAX];
+ if (!cursor_encode(session, last, next_cursor))
+ {
+ json.failed = true;
+ }
+ else
+ {
+ tiny_json_raw(&json, ",\"nextCursor\":");
+ tiny_json_string(&json, next_cursor);
+ }
+ }
+ tiny_json_raw(&json, "}");
+ if (json.failed)
+ {
+ tiny_mcp_tool_add_error(result, "INTERNAL_ERROR", "response allocation failed");
+ }
+ else
+ {
+ tiny_mcp_tool_add_json(result, &json);
}
- tiny_json_raw(&json, "]}");
- tiny_mcp_tool_add_json(result, &json);
tiny_json_free(&json);
- free(items);
free(session);
+ free(cursor_text);
}
bool
tiny_mcp_register_read_messages(mcpc_server_t *server)
{
- mcpc_tool_t *tool = tiny_mcp_tool_new("read_messages", "Read a session", call_read_messages);
+ mcpc_tool_t *tool = tiny_mcp_tool_new("read_messages", "Read a session page", call_read_messages);
return tool && tiny_mcp_tool_add_property(tool, "session_id", "Session ID", MCPC_U8STR, true)
+ && tiny_mcp_tool_add_property(
+ tool, "limit", "Optional page limit from 1 to 1000; defaults to 100", MCPC_I32, false)
+ && tiny_mcp_tool_add_property(
+ tool, "cursor", "Optional opaque page cursor", MCPC_U8STR, false)
&& mcpc_server_add_tool(server, tool) == MCPC_EC_0;
}
diff --git a/src/mcp/tools/rebuild_index.c b/src/mcp/tools/rebuild_index.c
index 69f8341..7208d77 100644
--- a/src/mcp/tools/rebuild_index.c
+++ b/src/mcp/tools/rebuild_index.c
@@ -11,6 +11,7 @@ call_rebuild_index(const mcpc_tool_t *tool, mcpc_ucbr_t *result)
tiny_mcp_tools_context_t *context = tiny_mcp_tools_context_current();
tiny_rebuild_report_t report;
tiny_search_index_t *rebuilt;
+ tiny_history_index_t *history;
tiny_store_status_t status;
tiny_json_buffer_t json;
(void)tool;
@@ -25,9 +26,14 @@ call_rebuild_index(const mcpc_tool_t *tool, mcpc_ucbr_t *result)
tiny_mcp_tool_add_error(result, tiny_store_status_name(status), "index rebuild failed");
return;
}
+ tiny_history_index_free(context->history_index);
+ context->history_index = NULL;
rebuilt = tiny_mcp_tools_build_search_index(context->store);
- if (!rebuilt)
+ history = tiny_history_index_build(context->store);
+ if (!rebuilt || !history)
{
+ tiny_search_index_free(rebuilt);
+ tiny_history_index_free(history);
tiny_search_index_free(context->search_index);
context->search_index = NULL;
tiny_mcp_tool_add_error(result, "INTERNAL_ERROR", "search index allocation failed");
@@ -35,6 +41,7 @@ call_rebuild_index(const mcpc_tool_t *tool, mcpc_ucbr_t *result)
}
tiny_search_index_free(context->search_index);
context->search_index = rebuilt;
+ context->history_index = history;
tiny_json_init(&json);
tiny_json_raw(&json, "{\"status\":\"OK\",\"messages_indexed\":");
tiny_json_u64(&json, report.messages_indexed);
diff --git a/src/mcp/tools/write_message.c b/src/mcp/tools/write_message.c
index f9c078a..c4b18cc 100644
--- a/src/mcp/tools/write_message.c
+++ b/src/mcp/tools/write_message.c
@@ -185,6 +185,14 @@ call_write_message(const mcpc_tool_t *tool, mcpc_ucbr_t *result)
tiny_search_index_free(context->search_index);
context->search_index = NULL;
}
+ if (context->history_index
+ && !tiny_history_index_add(context->history_index,
+ context->store,
+ tiny_store_entry_count(context->store) - 1))
+ {
+ tiny_history_index_free(context->history_index);
+ context->history_index = NULL;
+ }
tiny_json_init(&json);
tiny_json_raw(&json, "{\"status\":\"OK\",\"message\":");
tiny_mcp_tool_append_message(&json, &written);
diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt
index 9a02ee0..0bc5d89 100644
--- a/tests/CMakeLists.txt
+++ b/tests/CMakeLists.txt
@@ -94,6 +94,7 @@ add_executable(tiny-convmem-storage-test
"${PROJECT_SOURCE_DIR}/src/domain/message.c"
"${PROJECT_SOURCE_DIR}/src/storage/codec.c"
"${PROJECT_SOURCE_DIR}/src/storage/store.c"
+ "${PROJECT_SOURCE_DIR}/src/mcp/tools/common/history.c"
)
target_include_directories(tiny-convmem-storage-test PRIVATE "${PROJECT_SOURCE_DIR}/src")
target_compile_features(tiny-convmem-storage-test PRIVATE c_std_17)
diff --git a/tests/function-coverage.md b/tests/function-coverage.md
index 8787918..8976580 100644
--- a/tests/function-coverage.md
+++ b/tests/function-coverage.md
@@ -11,6 +11,7 @@ byte/encoding logic is exposed through internal headers and tested directly.
| `sys/system.c` | test-hook helpers; UTF adapters; `native_file`; argument, path, directory, file, clock and random APIs | `boundaries`, `storage`, `smoke`: NULL/invalid handle, Unicode paths, short I/O, operation failure, cleanup |
| `domain/message.c` | `duplicate_string`, `copy_failed`, all `tiny_message_*`, role/tier parse/name, `tiny_utf8_valid`, `tiny_message_id_valid` | `domain`: enum failure, UTF-8 invalidity, ULID alphabet/length, deep copy and ownership |
| `domain/search.c` | Unicode decode/score、open-addressing lookup/rehash、transactional delta merge、candidate union | `domain`, `search-index`: ASCII/日本語、8-bit hash衝突、複数rehash、allocation failure全位置、leak balance |
+| `mcp/tools/common/history.c` | session hash map、安定順序vector、startup/write/rebuild lifecycle | `storage`, `protocol`: allocation failure掃引、100/105件pagination、restart/rebuild cursor継続 |
| `domain/search.c` | `decode_codepoints`, `gram_hash`, `find_bucket`, `add_gram`, `index_text`, all `tiny_search_*` | `domain`: ASCII case folding, Unicode grams/tags, candidates, scoring and ownership |
| `storage/store.c` | error/I/O helpers, ULID, entry growth/search, record codec, replay/snapshot/recovery, all `tiny_store_*` | `storage`, `lifecycle`: lock, append/reopen, corruption, rebuild rollback, Unicode path |
| `mcp/json.c` | `reserve`, all `tiny_json_*` | `boundaries`: empty append, growth, control escaping, signed/unsigned integer limits, OOM |
diff --git a/tests/mcp_protocol_test.py b/tests/mcp_protocol_test.py
index f739168..4d1deb2 100644
--- a/tests/mcp_protocol_test.py
+++ b/tests/mcp_protocol_test.py
@@ -15,6 +15,7 @@ from typing import Any
class Server:
def __init__(self, executable: Path, data_dir: Path) -> None:
self.stderr: list[str] = []
+ self.last_response_size = 0
self.process = subprocess.Popen(
[str(executable), "--data-dir", str(data_dir)],
stdin=subprocess.PIPE,
@@ -41,6 +42,7 @@ class Server:
raise AssertionError(
f"server closed: returncode={self.process.poll()} stderr={''.join(self.stderr)[-4000:]}"
)
+ self.last_response_size = len(response_line.encode("utf-8"))
response = json.loads(response_line)
assert response.get("jsonrpc") == "2.0", response
return response
@@ -132,6 +134,8 @@ def main() -> None:
"content",
]
assert tools["read_messages"]["inputSchema"]["required"] == ["session_id"]
+ assert tools["read_messages"]["inputSchema"]["properties"]["limit"]["type"] == "integer"
+ assert tools["read_messages"]["inputSchema"]["properties"]["cursor"]["type"] == "string"
assert tools["search_messages"]["inputSchema"]["required"] == ["query"]
assert tools["search_messages"]["inputSchema"]["properties"]["limit"]["type"] == "integer"
assert tools["delete_message"]["inputSchema"]["required"] == ["message_id"]
@@ -165,6 +169,89 @@ def main() -> None:
next_id += 1
assert len(history) == 20
+ paged_ids: list[str] = []
+ for index in range(105):
+ payload = tool_payload(
+ tool_call(
+ server,
+ next_id,
+ "write_message",
+ {"session_id": "paged", "role": "user", "content": f"page item {index}"},
+ )
+ )
+ next_id += 1
+ paged_ids.append(payload["message"]["message_id"])
+ first_page = tool_payload(
+ tool_call(server, next_id, "read_messages", {"session_id": "paged"})
+ )
+ next_id += 1
+ assert len(first_page["history"]) == 100 and first_page.get("nextCursor")
+ second_page = tool_payload(
+ tool_call(
+ server,
+ next_id,
+ "read_messages",
+ {"session_id": "paged", "cursor": first_page["nextCursor"]},
+ )
+ )
+ next_id += 1
+ assert len(second_page["history"]) == 5 and "nextCursor" not in second_page
+ assert [item["message_id"] for item in first_page["history"] + second_page["history"]] == paged_ids
+ for invalid_limit in (0, 1001):
+ assert tool_payload(
+ tool_call(
+ server,
+ next_id,
+ "read_messages",
+ {"session_id": "paged", "limit": invalid_limit},
+ ),
+ True,
+ )["status"] == "ERROR"
+ next_id += 1
+ assert tool_payload(
+ tool_call(
+ server,
+ next_id,
+ "read_messages",
+ {"session_id": "other", "cursor": first_page["nextCursor"]},
+ ),
+ True,
+ )["status"] == "ERROR"
+ next_id += 1
+
+ large_content = "\\" * (512 * 1024)
+ for index in range(6):
+ tool_payload(
+ tool_call(
+ server,
+ next_id,
+ "write_message",
+ {"session_id": "large-page", "role": "user", "content": large_content},
+ )
+ )
+ next_id += 1
+ large_page = tool_payload(
+ tool_call(
+ server,
+ next_id,
+ "read_messages",
+ {"session_id": "large-page", "limit": 100},
+ )
+ )
+ next_id += 1
+ assert 0 < len(large_page["history"]) < 6 and large_page.get("nextCursor")
+ assert server.last_response_size <= 8 * 1024 * 1024
+ assert tool_payload(
+ tool_call(
+ server,
+ next_id,
+ "read_messages",
+ {"session_id": "paged", "cursor": "broken"},
+ ),
+ True,
+ )["status"] == "ERROR"
+ next_id += 1
+
ranked = tool_payload(
tool_call(
server,
@@ -265,6 +352,33 @@ def main() -> None:
raise
else:
server.close()
+ restarted = Server(args.server.resolve(), data_dir)
+ try:
+ restarted.send(
+ request(
+ 1,
+ "initialize",
+ {
+ "protocolVersion": "2024-11-05",
+ "capabilities": {},
+ "clientInfo": {"name": "pagination-restart", "version": "1"},
+ },
+ )
+ )
+ restarted.notify(
+ {"jsonrpc": "2.0", "method": "notifications/initialized", "params": {}}
+ )
+ resumed = tool_payload(
+ tool_call(
+ restarted,
+ 2,
+ "read_messages",
+ {"session_id": "paged", "cursor": first_page["nextCursor"]},
+ )
+ )
+ assert [item["message_id"] for item in resumed["history"]] == paged_ids[100:]
+ finally:
+ restarted.close()
finally:
shutil.rmtree(data_dir, ignore_errors=True)
diff --git a/tests/storage_test.c b/tests/storage_test.c
index 3217095..0e93ad9 100644
--- a/tests/storage_test.c
+++ b/tests/storage_test.c
@@ -1,10 +1,13 @@
#include "storage/store.h"
#include "sys/system.h"
+#include "mcp/tools/common/history.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
+#include "base/allocator.h"
+
static int failures;
#define CHECK(value) \
do \
@@ -103,6 +106,31 @@ main(void)
CHECK(tiny_store_open(&store, directory) == TINY_STORE_OK);
CHECK(tiny_store_entry_count(store) == 2);
+ {
+ size_t failure_index;
+ bool built = false;
+ for (failure_index = 0; failure_index < 64; ++failure_index)
+ {
+ tiny_history_index_t *history;
+ size_t baseline = tiny_allocator_outstanding();
+ tiny_allocator_fail_after(failure_index);
+ history = tiny_history_index_build(store);
+ tiny_allocator_reset();
+ if (history)
+ {
+ const size_t *entries = NULL;
+ size_t entry_count = 0;
+ CHECK(tiny_history_index_session(history, "unit", &entries, &entry_count));
+ CHECK(entry_count == 2 && entries[0] == 0 && entries[1] == 1);
+ tiny_history_index_free(history);
+ CHECK(tiny_allocator_outstanding() == baseline);
+ built = true;
+ break;
+ }
+ CHECK(tiny_allocator_outstanding() == baseline);
+ }
+ CHECK(built);
+ }
for (operation = 0; operation < sizeof(rebuild_failures) / sizeof(rebuild_failures[0]);
++operation)
{
generated by cgit v1.2.3 (git 2.39.1) at 2026-10-19 15:14:59 +0000