From 55db81082f9165a3965f860cd618bac4242c56b1 Mon Sep 17 00:00:00 2001 From: tanaka takashi Date: Tue, 11 Aug 2026 22:50:23 +0900 Subject: Record final audit and conformance limits --- plans/RequirementTraceability.md | 76 +++++++++++++++++----------------------- 1 file changed, 32 insertions(+), 44 deletions(-) (limited to 'plans/RequirementTraceability.md') diff --git a/plans/RequirementTraceability.md b/plans/RequirementTraceability.md index 1983a6d..b7a84ef 100644 --- a/plans/RequirementTraceability.md +++ b/plans/RequirementTraceability.md @@ -1,13 +1,8 @@ # Requirement Traceability -基準は`b5ccecc`を起点とする2026-08-11の現行working treeである。2026-08-09の結果は`plans/progress/`に履歴として保持するが、現行実装のPASS根拠には使用しない。 +現行実装の検証対象はcommit `37386b8`（2026-08-11 JST）である。2026-08-09を含む過去の結果は`plans/progress/`に履歴として保持するが、現行PASS判定には使用しない。 -状態は次の意味で使う。 - -- `PASS`: 現行sourceに実装があり、現行working treeで対応testを再実行した。 -- `PARTIAL`: 実装または自動testは確認したが、Requirementの一部、実機強制、またはOperational E2Eが未確認。 -- `FAIL`: 現行実装がRequirementを満たさない証拠がある。 -- `NOT REVERIFIED`: 過去証跡はあるが、現行working treeでは必要な権限・環境で再実行していない。 +状態は`PASS / PARTIAL / FAIL / NOT REVERIFIED`を使う。現在、Requirement mapping、Acceptance Tests、Security Testsに未説明の`PARTIAL / FAIL / NOT REVERIFIED`はない。 ## Requirement mapping @@ -15,48 +10,40 @@ |---|---|---|---| | OLL-01〜05 | `OllamaClient`, `OllamaSettingsFactory`, typed/extra options | `OllamaClientTests`, `ConfigurationTests` | PASS | | OLL-06〜09 | Task temperature、固定model/prompt、Vision capability | `PromptAndRuntimeTests`, `VisionTests` | PASS | -| OLL-10〜13 | native Tool call、ToolUseGuard、retry、全phase token集計 | `InternalToolTests`, `PromptAndRuntimeTests`, `OllamaClientTests` | PASS | -| AGT-01〜06 | `AgentRuntime`, `InternalToolDispatcher`, MCP公開境界 | completion/replan/iteration/catalog/MCP stdio tests | PASS | -| PLN-01〜06 | `TaskGraph`, `PlannerService`, structural `task_update` | `Agent.Planner.Tests`, `InternalToolTests`, structured-plan retry tests | PASS | -| SES-01〜04 | `SessionStore`, conversation/graph/result保持 | session/runtime tests | PASS | -| SES-05〜06 | structured context要約、成功時のみcompact | summary success/failure runtime tests | PASS | -| FS-01〜07 | `PathPolicy`, `TaskAclManager`, `WindowsToolBackend` | policy unit testsは現行sourceで実行対象。AppContainer/ACL実効果は管理者再検証待ち | PARTIAL | -| PRC-01〜07 | `ExecutablePolicy`, `RestrictedProcessLauncher`, Job/child restriction | deterministic validationあり。実process/Jobは管理者再検証待ち | PARTIAL | -| REC-01〜05 | Ollama retry、timeout、cancel後cleanup | Core cancel lifecycle tests。Windows Job停止は管理者再検証待ち | PARTIAL | -| REC-06〜07 | `RecoveryManager`, journal、serve前Recovery | crash matrixおよび残留0は管理者再検証待ち | NOT REVERIFIED | -| SEC-INV-01〜05 | deny-by-default、Policy非変更、path/hash/descendant制約 | Core境界はPASS、OS強制部分は管理者再検証待ち | PARTIAL | -| SEC-INV-06〜08 | WFP allow/deny、credential非継承 | Windows Security/Operational E2E再検証待ち | NOT REVERIFIED | +| OLL-10〜13 | native Tool call、ToolUseGuard、retry、全phase token集計 | `InternalToolTests`, `PromptAndRuntimeTests`, DeepSeek Operational E2E | PASS | +| AGT-01〜06 | `AgentRuntime`, `InternalToolDispatcher`, MCP公開境界 | completion/replan/iteration/catalog tests、stdio/HTTP E2E | PASS | +| PLN-01〜06 | `TaskGraph`, `PlannerService`, structural `task_update` | Planner 10件、structured-plan retry、Task固有Tool証跡test | PASS | +| SES-01〜06 | `SessionStore`、会話/graph/result保持、structured要約 | summary成功/失敗、session/runtime tests | PASS | +| FS-01〜07 | `PathPolicy`, `TaskAclManager`, `WindowsToolBackend` | Elevated Windows Security/Recovery 42件、Operational E2E | PASS | +| PRC-01〜07 | `ExecutablePolicy`, `RestrictedProcessLauncher`, Job/child restriction | Elevated Windows suite、in-process build/self-test E2E | PASS | +| REC-01〜07 | retry、timeout、cancel cleanup、`RecoveryManager` | Core lifecycle、elevated crash/recovery matrix、残留0 | PASS | +| SEC-INV-01〜05 | deny-by-default、Policy非変更、path/hash/descendant制約 | Core境界、elevated Windows suite、E2E | PASS | +| SEC-INV-06〜08 | WFP allow/deny、credential非継承 | elevated WFP/security tests、Bundle check、E2E | PASS | | SEC-INV-09 | Fixed Prompt precedence | prompt tests | PASS | -| SEC-INV-10〜11 | cleanup、Recovery gate | Core cleanup lifecycleはPASS、OS resource残留0は再検証待ち | PARTIAL | -| SEC-INV-12 | stdio stdout purity | MCP stdio integration | PASS | -| NFR-01〜02 | default deny、決定的Policy/Graph/PERT | Core/Planner tests | PASS | -| NFR-03 | journal recovery | elevated crash matrix再検証待ち | NOT REVERIFIED | -| NFR-04〜06 | 公開MCP Tool 2個、依存管理、Windows限定設計 | MCP schema tests、build | PASS | -| NFR-07 | Task/Tool/Process/Token/deny相関log | `ObservabilityTests`、Audit event実装 | PASS | +| SEC-INV-10〜11 | cleanup、Recovery gate | Core cleanup、elevated recovery tests、全E2E後残留0 | PASS | +| SEC-INV-12 | stdio stdout purity | MCP stdio integration、Operational stdio E2E | PASS | +| NFR-01〜03 | default deny、決定的Policy/Graph/PERT、journal recovery | Core/Planner、elevated recovery tests | PASS | +| NFR-04〜06 | 公開MCP Tool 2個、依存管理、Windows限定設計 | MCP schema、Release build/Bundle | PASS | +| NFR-07 | Task/Tool/Process/Token/deny相関log | `ObservabilityTests`、Audit event、E2E log | PASS | | NFR-08〜09 | en/ja catalog、同一key検証、用語/設定索引 | locale/configuration tests、文書監査 | PASS | ## Acceptance Tests | ID | Current implementation / test | Status | |---|---|---| -| AT-01 | MCP stdioの公開Tool 2個とstrict schema | PASS | -| AT-02 | loopback Streamable HTTP integration | PASS | -| AT-03 | GET/Legacy SSE拒否 | PASS | -| AT-04 | Task lifecycle、永続Task store、cancel連携 | PASS | -| AT-05 | Ollama request/usage unit test | PASS | -| AT-06 | 非Vision modelのlocalized拒否 | PASS | -| AT-07〜13 | Windows backend実効果、reparse、process、WFP、crash recovery | NOT REVERIFIED | -| AT-14〜17 | cycle、immutable、PERT/CPM、token集計 | PASS | +| AT-01〜06 | MCP stdio/HTTP、公開schema、Task lifecycle、Ollama、Vision拒否 | PASS | +| AT-07〜13 | Windows backend、reparse、process、WFP、crash recovery | PASS | +| AT-14〜17 | cycle、immutable、PERT/CPM、全phase token集計 | PASS | | AT-18〜20 | en/ja、追加locale、malformed/unknown locale | PASS | ## Security Tests | ID | Current implementation / test | Status | |---|---|---| -| ST-01〜09 | capability/path/reparse/hash/shellの決定的検査あり。ACL/AppContainer実効果は再検証待ち | PARTIAL | -| ST-10〜14 | child restriction、Job timeout、WFP actual traffic | NOT REVERIFIED | -| ST-15〜16 | MCP非公開境界、Internal Tool name/schema/Policy境界 | PASS | -| ST-17〜18 | ACL/WFP/loopback crash recovery | NOT REVERIFIED | +| ST-01〜09 | capability/path/reparse/hash/shell、ACL/AppContainer実効果 | PASS | +| ST-10〜14 | child restriction、Job timeout、WFP actual traffic | PASS | +| ST-15〜16 | MCP非公開境界、Design Tool catalogの名前/schema/Policy境界 | PASS | +| ST-17〜18 | ACL/WFP/loopback crash recovery | PASS | | ST-19〜20 | malformed locale fatal、non-loopback bind拒否 | PASS | Internal Tool testは件数を仕様としてassertしない。`Design.md`のTool catalogに記載された`file_search / file_read / file_edit / process_run / task_update / dir`の名前、schema、Policy境界、MCP非公開性を検査する。 @@ -65,13 +52,14 @@ Internal Tool testは件数を仕様としてassertしない。`Design.md`のToo | Verification | Date / privilege | Result | |---|---|---| -| Agent.Core.Tests | 2026-08-11 / non-elevated / Release | 78 PASS、0 FAIL、0 skip | -| Agent.Planner.Tests | 2026-08-11 / non-elevated | 10 PASS、0 FAIL、0 skip | -| Release build | 2026-08-11 / non-elevated | warning 0、error 0 | -| Agent.Windows.Tests | 2026-08-11 / elevation request経由だがWindows tokenは非管理者 | 22 PASS、19 FAIL、0 skip。19件はWFP/loopback Win32 error 5またはsymlink privilege不足で、管理者再検証が必要 | -| win-x64 self-contained Bundle | 2026-08-11 / runtime check省略 | publish/ZIP/checksum生成PASS。`agent.local.yaml`とlocal example非同梱。SHA-256 `2cdf2ef6e38d740ef87a489eb27f883c1e9be9ec9d06dbabf5895f1ddfc61c63` | +| Release build | 2026-08-11 / non-elevated / Release | warning 0、error 0 | +| Agent.Core.Tests | 2026-08-11 / administrator suite | 82 PASS、0 FAIL、0 skip | +| Agent.Planner.Tests | 2026-08-11 / administrator suite | 10 PASS、0 FAIL、0 skip | +| Agent.Windows.Tests | 2026-08-11 / administrator | 42 PASS、0 FAIL、0 skip。Security/Recovery/crash matrixを含む | +| win-x64 self-contained Bundle | 2026-08-11 / publish後administrator check | directory/ZIP/checksum生成、local override非同梱、runtime check PASS。ZIP SHA-256 `8ccc538557dcb96cc95ecae7b1858de3544997b16a5675aacb05a03f539829b1` | +| Operational E2E | 2026-08-11 / administrator / `deepseek-v4-flash:cloud` | stdio、HTTP IPv4、HTTP IPv6すべてPASS。read/edit/build/self-test、ACL/WFP/temp cleanup確認 | | Task Resource repository state | 2026-08-11 | `state/journals`, `state/backups`, `state/mcp-tasks`はいずれも不存在 | -| Elevated Security/Recovery/E2E/Bundle runtime check | current working tree / administrator | NOT REVERIFIED | -| MCP Tasks conformance | current working tree | NOT REVERIFIED。runner制約をapplication failureから分離し、未達をPASSにしない | +| MCP 2026-07-28 conformance runner 0.2.0-alpha.11 | 2026-08-11 / loopback HTTP | application failure 0。適用可能checkはPASS。production fixture非公開とWindows runner終了不具合はrunner limitation 9件として非PASSのまま分離 | +| MCP Tasks conformance runner | 2026-08-11 / loopback HTTP | application failure 0。runner専用fixture不足による未実施27件をrunner limitationとして非PASSのまま分離 | -完了Gateは`Specifications.md` 20章に従う。管理者環境の再検証が終わるまで、Release全体をPASSとはしない。 +証跡は`artifacts/evidence/final-37386b8/`に生成した。conformanceのrunner limitationはRequirement PASSの根拠に使用しておらず、対応する製品機能はCore/MCP integration、Windows suite、Operational E2Eで検証した。 -- cgit v1.2.3