diff options
| author | tanaka takashi <isazaka@gmail.com> | 2026-06-30 19:03:25 +0900 |
|---|---|---|
| committer | tanaka takashi <isazaka@gmail.com> | 2026-06-30 19:03:25 +0900 |
| commit | 05c8e0a31a0d4192c590e833b24393208e1c751f (patch) | |
| tree | f7c2c82c9327abde67bf3acf1d0258573b3bc6c2 /scripts/python/benchmark_results.md | |
| parent | 4bc721c3bf7589b519c5730d093f704ce56e3f09 (diff) | |
fix: MCPCをvendored fork化しMCP準拠と接続安定性を改善
- Git submoduleを廃止してMCPCを直接管理
- Windowsで254リクエスト付近に発生する切断を修正
- JSON-RPC 2.0エラー応答を標準化
- tool schemaとruntimeの必須・任意引数を一致
- 1000回連続呼び出しの回帰テストを追加
Diffstat (limited to 'scripts/python/benchmark_results.md')
| -rw-r--r-- | scripts/python/benchmark_results.md | 85 |
1 files changed, 85 insertions, 0 deletions
diff --git a/scripts/python/benchmark_results.md b/scripts/python/benchmark_results.md new file mode 100644 index 0000000..9d512a3 --- /dev/null +++ b/scripts/python/benchmark_results.md @@ -0,0 +1,85 @@ +# tiny-convmem-mcp 負荷テストレポート +- サーバ: `build\windows-vs2026\src\Release\tiny-convmem-mcp.exe` +- ビルド: release +- exe mtime: 2026-06-30 18:49:00 +- iterations: 100 +- warmup: 5 +- 計測日: 2026-06-30 18:49:14 +- raw JSON: `benchmark_results.json` + +## レイテンシサマリ + +| scenario | succ | fail | p50[ms] | p95[ms] | p99[ms] | mean[ms] | ops/s | +|---|---|---|---|---|---|---|---| +| write_message (small ~20B) | 100 | 0 | 0.397 | 1.236 | 2.052 | 0.589 | 1696.88 | +| write_message (medium 1KiB) | 100 | 0 | 0.870 | 1.503 | 3.138 | 1.041 | 960.92 | +| write_message (large 32KiB) | 100 | 0 | 128.242 | 133.328 | 133.560 | 128.629 | 7.77 | +| write_message (with tags) | 100 | 0 | 0.391 | 0.700 | 1.119 | 0.443 | 2255.48 | +| read_messages (100 msgs) | 100 | 0 | 0.339 | 0.770 | 1.385 | 0.405 | 2471.64 | +| search_messages (global, no session) | 100 | 0 | 0.150 | 0.189 | 0.721 | 0.176 | 5675.11 | +| search_messages (scoped to session) | 100 | 0 | 0.152 | 0.471 | 0.808 | 0.192 | 5209.04 | +| delete_message | 100 | 0 | 0.390 | 0.712 | 0.829 | 0.445 | 2248.61 | +| rebuild_index | 5 | 0 | 6.891 | 7.781 | 7.883 | 6.720 | 148.80 | +| get_system_health | 100 | 0 | 0.110 | 0.178 | 0.763 | 0.132 | 7587.08 | +| recovery (restart + count check) | 1 | 0 | 20.378 | 20.378 | 20.378 | 20.378 | 49.07 | +| recovery small read (1 message check) | 5 | 0 | 16.598 | 16.754 | 16.765 | 16.563 | 60.37 | +| recovery full read (100 msgs) | 5 | 0 | 16.116 | 18.455 | 18.772 | 16.717 | 59.82 | + +## MCP 準拠 / エラーチェック + +```json +{ + "tool_count": 6, + "tool_names": [ + "write_message", + "read_messages", + "search_messages", + "delete_message", + "rebuild_index", + "get_system_health" + ], + "write_message_properties": [ + "session_id", + "role", + "content", + "metadata_json" + ], + "write_requires": [ + "session_id", + "role", + "content" + ], + "jsonrpc_error_response": { + "error_response_present": true, + "has_jsonrpc_field": true, + "has_error_field": true, + "error_code": -32601, + "raw_response": { + "jsonrpc": "2.0", + "id": 2, + "error": { + "code": -32601, + "message": "client not initialized" + } + } + } +} +``` + +## 最終 health + +```json +{ + "status": "OK", + "format_version": 1, + "log_size": 920, + "snapshot_size": 0, + "active_messages": 10, + "deleted_messages": 0, + "last_sequence": 10, + "recovered_tail": false, + "snapshot_valid": false, + "last_error": "" +} +``` + |
