diff options
Diffstat (limited to 'plans/RequirementTraceability.md')
| -rw-r--r-- | plans/RequirementTraceability.md | 130 |
1 files changed, 62 insertions, 68 deletions
diff --git a/plans/RequirementTraceability.md b/plans/RequirementTraceability.md index 9d6e93b..1983a6d 100644 --- a/plans/RequirementTraceability.md +++ b/plans/RequirementTraceability.md @@ -1,83 +1,77 @@ # Requirement Traceability -`PASS` は2026-08-09のRelease buildおよび管理者権限の最終実行で、実行可能なtest/evidenceが確認できたことを示す。最終証跡は `artifacts/evidence/final-elevated/` に保存する。 +基準は`b5ccecc`を起点とする2026-08-11の現行working treeである。2026-08-09の結果は`plans/progress/`に履歴として保持するが、現行実装のPASS根拠には使用しない。 + +状態は次の意味で使う。 + +- `PASS`: 現行sourceに実装があり、現行working treeで対応testを再実行した。 +- `PARTIAL`: 実装または自動testは確認したが、Requirementの一部、実機強制、またはOperational E2Eが未確認。 +- `FAIL`: 現行実装がRequirementを満たさない証拠がある。 +- `NOT REVERIFIED`: 過去証跡はあるが、現行working treeでは必要な権限・環境で再実行していない。 + +## Requirement mapping + +| Requirement | 主な実装 | Test / 現行証跡 | Status | +|---|---|---|---| +| OLL-01〜05 | `OllamaClient`, `OllamaSettingsFactory`, typed/extra options | `OllamaClientTests`, `ConfigurationTests` | PASS | +| OLL-06〜09 | Task temperature、固定model/prompt、Vision capability | `PromptAndRuntimeTests`, `VisionTests` | PASS | +| OLL-10〜13 | native Tool call、ToolUseGuard、retry、全phase token集計 | `InternalToolTests`, `PromptAndRuntimeTests`, `OllamaClientTests` | PASS | +| AGT-01〜06 | `AgentRuntime`, `InternalToolDispatcher`, MCP公開境界 | completion/replan/iteration/catalog/MCP stdio tests | PASS | +| PLN-01〜06 | `TaskGraph`, `PlannerService`, structural `task_update` | `Agent.Planner.Tests`, `InternalToolTests`, structured-plan retry tests | PASS | +| SES-01〜04 | `SessionStore`, conversation/graph/result保持 | session/runtime tests | PASS | +| SES-05〜06 | structured context要約、成功時のみcompact | summary success/failure runtime tests | PASS | +| FS-01〜07 | `PathPolicy`, `TaskAclManager`, `WindowsToolBackend` | policy unit testsは現行sourceで実行対象。AppContainer/ACL実効果は管理者再検証待ち | PARTIAL | +| PRC-01〜07 | `ExecutablePolicy`, `RestrictedProcessLauncher`, Job/child restriction | deterministic validationあり。実process/Jobは管理者再検証待ち | PARTIAL | +| REC-01〜05 | Ollama retry、timeout、cancel後cleanup | Core cancel lifecycle tests。Windows Job停止は管理者再検証待ち | PARTIAL | +| REC-06〜07 | `RecoveryManager`, journal、serve前Recovery | crash matrixおよび残留0は管理者再検証待ち | NOT REVERIFIED | +| SEC-INV-01〜05 | deny-by-default、Policy非変更、path/hash/descendant制約 | Core境界はPASS、OS強制部分は管理者再検証待ち | PARTIAL | +| SEC-INV-06〜08 | WFP allow/deny、credential非継承 | Windows Security/Operational E2E再検証待ち | NOT REVERIFIED | +| SEC-INV-09 | Fixed Prompt precedence | prompt tests | PASS | +| SEC-INV-10〜11 | cleanup、Recovery gate | Core cleanup lifecycleはPASS、OS resource残留0は再検証待ち | PARTIAL | +| SEC-INV-12 | stdio stdout purity | MCP stdio integration | PASS | +| NFR-01〜02 | default deny、決定的Policy/Graph/PERT | Core/Planner tests | PASS | +| NFR-03 | journal recovery | elevated crash matrix再検証待ち | NOT REVERIFIED | +| NFR-04〜06 | 公開MCP Tool 2個、依存管理、Windows限定設計 | MCP schema tests、build | PASS | +| NFR-07 | Task/Tool/Process/Token/deny相関log | `ObservabilityTests`、Audit event実装 | PASS | +| NFR-08〜09 | en/ja catalog、同一key検証、用語/設定索引 | locale/configuration tests、文書監査 | PASS | ## Acceptance Tests -| ID | Implementation / evidence | Status | +| ID | Current implementation / test | Status | |---|---|---| -| AT-01 | `McpStdioTests.InitializeListAndCallExposeOnlyTwoStrictTools` | PASS | -| AT-02 | `McpHttpTests`、E2E-02 `127.0.0.1` / `::1` | PASS | -| AT-03 | HTTP GET `/mcp`, `/sse`, `/message` assertions | PASS | -| AT-04 | `TaskRegistryTests`, MCP stdio integration | PASS | -| AT-05 | `OllamaClientTests.Chat_sends_expected_request_and_returns_usage` | PASS | -| AT-06 | `Non_vision_model_rejects_images_with_selected_locale_message_before_chat` | PASS | -| AT-07 | Windows backend実効果試験、E2E Audit | PASS | -| AT-08 | junction / symlink chain escape試験 | PASS | -| AT-09 | path/hash一致・不一致試験 | PASS | -| AT-10 | AppContainer loopback actual traffic default deny | PASS | -| AT-11 | exact host:port allowおよびwrong-port actual deny | PASS | -| AT-12 | child-process restriction 4経路、Job timeout/orphan 0 | PASS | -| AT-13 | ACL/WFP/loopback journal crash recovery試験 | PASS | -| AT-14 | `TaskGraphTests.Cycle_is_rejected_without_mutating_graph` | PASS | -| AT-15 | `TaskGraphTests.Completed_task_is_immutable` | PASS | -| AT-16 | deterministic PERT/CPM tests | PASS | -| AT-17 | runtime and graph usage accumulation tests | PASS | -| AT-18 | en/ja locale、3経路の日本語Operational E2E | PASS | -| AT-19 | `LocaleCatalogTests.AdditionalLocaleLoadsWithoutCodeChange` | PASS | -| AT-20 | malformed/unknown locale tests | PASS | +| AT-01 | MCP stdioの公開Tool 2個とstrict schema | PASS | +| AT-02 | loopback Streamable HTTP integration | PASS | +| AT-03 | GET/Legacy SSE拒否 | PASS | +| AT-04 | Task lifecycle、永続Task store、cancel連携 | PASS | +| AT-05 | Ollama request/usage unit test | PASS | +| AT-06 | 非Vision modelのlocalized拒否 | PASS | +| AT-07〜13 | Windows backend実効果、reparse、process、WFP、crash recovery | NOT REVERIFIED | +| AT-14〜17 | cycle、immutable、PERT/CPM、token集計 | PASS | +| AT-18〜20 | en/ja、追加locale、malformed/unknown locale | PASS | ## Security Tests -| ID | Implementation / evidence | Status | +| ID | Current implementation / test | Status | |---|---|---| -| ST-01 | E2E `file_read`実行、capability試験 | PASS | -| ST-02 | outside-root deny、副作用なし | PASS | -| ST-03 | backend `file_edit`実効果、ACL復旧 | PASS | -| ST-04 | delete capability独立試験 | PASS | -| ST-05 | junction escape拒否 | PASS | -| ST-06 | two-link symlink chain escape拒否 | PASS | -| ST-07 | approved canonical path/hash process成功 | PASS | -| ST-08 | hash mismatch拒否 | PASS | -| ST-09 | shell disabled拒否 | PASS | -| ST-10 | `Start-Process`、Process API、shell、rename/copyの4経路をOS child restrictionで拒否、marker/PIDなし、Audit deny | PASS | -| ST-11 | Job timeout後PID不存在、Operational E2E orphan 0 | PASS | -| ST-12 | AppContainer actual loopback default deny | PASS | -| ST-13 | IPv4/IPv6 filter生成、IPv4 exact allow actual traffic | PASS | -| ST-14 | actual wrong-port deny | PASS | -| ST-15 | public MCP exposes only two strict tools | PASS | -| ST-16 | strict tool schema、path policy deny時backend call 0 | PASS | -| ST-17 | applied ACL journalのcrash recovery、descriptor一致 | PASS | -| ST-18 | dynamic WFP engine close＋loopback exemption journal recovery、残留0 | PASS | -| ST-19 | malformed locale fatal tests | PASS | -| ST-20 | non-loopback bind rejection test | PASS | +| ST-01〜09 | capability/path/reparse/hash/shellの決定的検査あり。ACL/AppContainer実効果は再検証待ち | PARTIAL | +| ST-10〜14 | child restriction、Job timeout、WFP actual traffic | NOT REVERIFIED | +| ST-15〜16 | MCP非公開境界、Internal Tool name/schema/Policy境界 | PASS | +| ST-17〜18 | ACL/WFP/loopback crash recovery | NOT REVERIFIED | +| ST-19〜20 | malformed locale fatal、non-loopback bind拒否 | PASS | -## Operational / Release +Internal Tool testは件数を仕様としてassertしない。`Design.md`のTool catalogに記載された`file_search / file_read / file_edit / process_run / task_update / dir`の名前、schema、Policy境界、MCP非公開性を検査する。 -| Requirement | Evidence | Status | -|---|---|---| -| Release build | Release build warning 0/error 0 | PASS | -| Unit/integration | Planner 10/10, Core 47/47, skip 0 | PASS | -| win-x64 self-contained Bundle | publish directory, ZIP, manifest, checksum | PASS | -| Bundle-only elevated startup | `bundle-check.log`: administrator/BFE/WFP probe PASS | PASS | -| E2E-01 stdio | `e2e-01-stdio.log`: local Ollama、read/edit/build/test、日本語、Audit、cleanup | PASS | -| E2E-02 HTTP (`127.0.0.1`, `::1`) | `e2e-02-http-ipv4.log` / `e2e-02-http-ipv6.log` | PASS | -| clean resource state | process/ACL/WFP/loopback/temp/journal/backup残留0 | PASS | -| README-only fresh setup | ZIP fresh extraction、source参照0、README手順による最終runner | PASS | - -## MCP 2026-07-28 Migration +## Current verification record -| Requirement | Implementation / evidence | Status | +| Verification | Date / privilege | Result | |---|---|---| -| SDK 2.0.0 / stateless discover | `Directory.Packages.props`, `McpHost`, official `server-stateless` evidence | PASS (28/28 core checks, unexpected failure 0) | -| GET/DELETE 405、Legacy SSE無効、Session IDなし | `McpHttpTests` | PASS | -| stdio / strict two Tool compatibility | test-only `Agent.Mcp.TestHost`, `McpStdioTests` | PASS | -| official Tasks negotiation/lifecycle | `WithTasks`, `PersistentMcpTaskStore`, `McpHttpTests`, conformance Tasks evidence | PARTIAL: lifecycle/cancel/error/required-task PASS; MRTR fixture composition remains incompatible with alpha.11 | -| Task persistence/restart/TTL | `PersistentMcpTaskStoreTests` | PASS | -| CLI preflight / ready | `Program`, `McpHostLogObserver`, locale catalog | PASS (elevated live run required for final Gate) | -| HTTP request log/redaction | `McpHostLogObserver`, `ObservabilityTests` | PASS | -| conformance acquisition/offline runner | `Acquire-McpConformance.ps1`, `Run-McpConformance.ps1`, `Run-McpTasksConformance.ps1` | PASS | - -`0.2.0-alpha.11`のTasks extension scenarioはrelease requirementに含まれずinformationalとして実行され、正しい`CreateTaskResult`を通常`CallToolResult` schemaで検査する`wire-schema-valid`既知制約がある。これは`runner_limitations`へ分離し、application failureとは数えない。MRTR input/composition scenarioはSDK 2.0.0との組み合わせで未達のため、Tasks Extension GateはPASS扱いにしない。 +| Agent.Core.Tests | 2026-08-11 / non-elevated / Release | 78 PASS、0 FAIL、0 skip | +| Agent.Planner.Tests | 2026-08-11 / non-elevated | 10 PASS、0 FAIL、0 skip | +| Release build | 2026-08-11 / non-elevated | warning 0、error 0 | +| Agent.Windows.Tests | 2026-08-11 / elevation request経由だがWindows tokenは非管理者 | 22 PASS、19 FAIL、0 skip。19件はWFP/loopback Win32 error 5またはsymlink privilege不足で、管理者再検証が必要 | +| win-x64 self-contained Bundle | 2026-08-11 / runtime check省略 | publish/ZIP/checksum生成PASS。`agent.local.yaml`とlocal example非同梱。SHA-256 `2cdf2ef6e38d740ef87a489eb27f883c1e9be9ec9d06dbabf5895f1ddfc61c63` | +| Task Resource repository state | 2026-08-11 | `state/journals`, `state/backups`, `state/mcp-tasks`はいずれも不存在 | +| Elevated Security/Recovery/E2E/Bundle runtime check | current working tree / administrator | NOT REVERIFIED | +| MCP Tasks conformance | current working tree | NOT REVERIFIED。runner制約をapplication failureから分離し、未達をPASSにしない | -公式core runnerの個別結果では`server-stateless` 28/28、DNS rebinding、multiple POST streamがPASSした。一方、test-only conformance hostのfixture不足またはscenario間のcapability競合により、`tools-list`（description）、`json-schema-2020-12`（診断Tool）、`caching`（Prompt/Resource/Template）、`http-header-validation`（Tasks必須ToolをTasks非対応clientが呼ぶ組合せ）は未達である。production hostのexpected-failureとして隠さず、conformance総合GateはPARTIALのままとする。 +完了Gateは`Specifications.md` 20章に従う。管理者環境の再検証が終わるまで、Release全体をPASSとはしない。 |