summaryrefslogtreecommitdiff
path: root/scripts/python/benchmark_results.md
blob: 9d512a3680df27ac8ef7c7bcae7f3d9c090f3a4d (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
# tiny-convmem-mcp 負荷テストレポート
- サーバ: `build\windows-vs2026\src\Release\tiny-convmem-mcp.exe`
- ビルド: release
- exe mtime: 2026-06-30 18:49:00
- iterations: 100
- warmup: 5
- 計測日: 2026-06-30 18:49:14
- raw JSON: `benchmark_results.json`

## レイテンシサマリ

| scenario | succ | fail | p50[ms] | p95[ms] | p99[ms] | mean[ms] | ops/s |
|---|---|---|---|---|---|---|---|
| write_message (small ~20B) | 100 | 0 | 0.397 | 1.236 | 2.052 | 0.589 | 1696.88 |
| write_message (medium 1KiB) | 100 | 0 | 0.870 | 1.503 | 3.138 | 1.041 | 960.92 |
| write_message (large 32KiB) | 100 | 0 | 128.242 | 133.328 | 133.560 | 128.629 | 7.77 |
| write_message (with tags) | 100 | 0 | 0.391 | 0.700 | 1.119 | 0.443 | 2255.48 |
| read_messages (100 msgs) | 100 | 0 | 0.339 | 0.770 | 1.385 | 0.405 | 2471.64 |
| search_messages (global, no session) | 100 | 0 | 0.150 | 0.189 | 0.721 | 0.176 | 5675.11 |
| search_messages (scoped to session) | 100 | 0 | 0.152 | 0.471 | 0.808 | 0.192 | 5209.04 |
| delete_message | 100 | 0 | 0.390 | 0.712 | 0.829 | 0.445 | 2248.61 |
| rebuild_index | 5 | 0 | 6.891 | 7.781 | 7.883 | 6.720 | 148.80 |
| get_system_health | 100 | 0 | 0.110 | 0.178 | 0.763 | 0.132 | 7587.08 |
| recovery (restart + count check) | 1 | 0 | 20.378 | 20.378 | 20.378 | 20.378 | 49.07 |
| recovery small read (1 message check) | 5 | 0 | 16.598 | 16.754 | 16.765 | 16.563 | 60.37 |
| recovery full read (100 msgs) | 5 | 0 | 16.116 | 18.455 | 18.772 | 16.717 | 59.82 |

## MCP 準拠 / エラーチェック

```json
{
  "tool_count": 6,
  "tool_names": [
    "write_message",
    "read_messages",
    "search_messages",
    "delete_message",
    "rebuild_index",
    "get_system_health"
  ],
  "write_message_properties": [
    "session_id",
    "role",
    "content",
    "metadata_json"
  ],
  "write_requires": [
    "session_id",
    "role",
    "content"
  ],
  "jsonrpc_error_response": {
    "error_response_present": true,
    "has_jsonrpc_field": true,
    "has_error_field": true,
    "error_code": -32601,
    "raw_response": {
      "jsonrpc": "2.0",
      "id": 2,
      "error": {
        "code": -32601,
        "message": "client not initialized"
      }
    }
  }
}
```

## 最終 health

```json
{
  "status": "OK",
  "format_version": 1,
  "log_size": 920,
  "snapshot_size": 0,
  "active_messages": 10,
  "deleted_messages": 0,
  "last_sequence": 10,
  "recovered_tail": false,
  "snapshot_valid": false,
  "last_error": ""
}
```