summaryrefslogtreecommitdiff
path: root/scripts/python/benchmark_results.md
diff options
context:
space:
mode:
authortanaka takashi <isazaka@gmail.com>2026-06-30 19:03:25 +0900
committertanaka takashi <isazaka@gmail.com>2026-06-30 19:03:25 +0900
commit05c8e0a31a0d4192c590e833b24393208e1c751f (patch)
treef7c2c82c9327abde67bf3acf1d0258573b3bc6c2 /scripts/python/benchmark_results.md
parent4bc721c3bf7589b519c5730d093f704ce56e3f09 (diff)
fix: MCPCをvendored fork化しMCP準拠と接続安定性を改善
- Git submoduleを廃止してMCPCを直接管理 - Windowsで254リクエスト付近に発生する切断を修正 - JSON-RPC 2.0エラー応答を標準化 - tool schemaとruntimeの必須・任意引数を一致 - 1000回連続呼び出しの回帰テストを追加
Diffstat (limited to 'scripts/python/benchmark_results.md')
-rw-r--r--scripts/python/benchmark_results.md85
1 files changed, 85 insertions, 0 deletions
diff --git a/scripts/python/benchmark_results.md b/scripts/python/benchmark_results.md
new file mode 100644
index 0000000..9d512a3
--- /dev/null
+++ b/scripts/python/benchmark_results.md
@@ -0,0 +1,85 @@
+# tiny-convmem-mcp 負荷テストレポート
+- サーバ: `build\windows-vs2026\src\Release\tiny-convmem-mcp.exe`
+- ビルド: release
+- exe mtime: 2026-06-30 18:49:00
+- iterations: 100
+- warmup: 5
+- 計測日: 2026-06-30 18:49:14
+- raw JSON: `benchmark_results.json`
+
+## レイテンシサマリ
+
+| scenario | succ | fail | p50[ms] | p95[ms] | p99[ms] | mean[ms] | ops/s |
+|---|---|---|---|---|---|---|---|
+| write_message (small ~20B) | 100 | 0 | 0.397 | 1.236 | 2.052 | 0.589 | 1696.88 |
+| write_message (medium 1KiB) | 100 | 0 | 0.870 | 1.503 | 3.138 | 1.041 | 960.92 |
+| write_message (large 32KiB) | 100 | 0 | 128.242 | 133.328 | 133.560 | 128.629 | 7.77 |
+| write_message (with tags) | 100 | 0 | 0.391 | 0.700 | 1.119 | 0.443 | 2255.48 |
+| read_messages (100 msgs) | 100 | 0 | 0.339 | 0.770 | 1.385 | 0.405 | 2471.64 |
+| search_messages (global, no session) | 100 | 0 | 0.150 | 0.189 | 0.721 | 0.176 | 5675.11 |
+| search_messages (scoped to session) | 100 | 0 | 0.152 | 0.471 | 0.808 | 0.192 | 5209.04 |
+| delete_message | 100 | 0 | 0.390 | 0.712 | 0.829 | 0.445 | 2248.61 |
+| rebuild_index | 5 | 0 | 6.891 | 7.781 | 7.883 | 6.720 | 148.80 |
+| get_system_health | 100 | 0 | 0.110 | 0.178 | 0.763 | 0.132 | 7587.08 |
+| recovery (restart + count check) | 1 | 0 | 20.378 | 20.378 | 20.378 | 20.378 | 49.07 |
+| recovery small read (1 message check) | 5 | 0 | 16.598 | 16.754 | 16.765 | 16.563 | 60.37 |
+| recovery full read (100 msgs) | 5 | 0 | 16.116 | 18.455 | 18.772 | 16.717 | 59.82 |
+
+## MCP 準拠 / エラーチェック
+
+```json
+{
+ "tool_count": 6,
+ "tool_names": [
+ "write_message",
+ "read_messages",
+ "search_messages",
+ "delete_message",
+ "rebuild_index",
+ "get_system_health"
+ ],
+ "write_message_properties": [
+ "session_id",
+ "role",
+ "content",
+ "metadata_json"
+ ],
+ "write_requires": [
+ "session_id",
+ "role",
+ "content"
+ ],
+ "jsonrpc_error_response": {
+ "error_response_present": true,
+ "has_jsonrpc_field": true,
+ "has_error_field": true,
+ "error_code": -32601,
+ "raw_response": {
+ "jsonrpc": "2.0",
+ "id": 2,
+ "error": {
+ "code": -32601,
+ "message": "client not initialized"
+ }
+ }
+ }
+}
+```
+
+## 最終 health
+
+```json
+{
+ "status": "OK",
+ "format_version": 1,
+ "log_size": 920,
+ "snapshot_size": 0,
+ "active_messages": 10,
+ "deleted_messages": 0,
+ "last_sequence": 10,
+ "recovered_tail": false,
+ "snapshot_valid": false,
+ "last_error": ""
+}
+```
+