diff options
| author | tanaka takashi <isazaka@gmail.com> | 2026-08-11 05:17:32 +0900 |
|---|---|---|
| committer | tanaka takashi <isazaka@gmail.com> | 2026-08-11 05:17:32 +0900 |
| commit | 35719799e62a18850efc52d380fb53bf3baa8022 (patch) | |
| tree | 3cc1d110f290e5acab71ca202a4cd0d521e4a8b5 /tests/Agent.Windows.Tests | |
| parent | d5d0a84e573c833843279c520ce2d5765425ba93 (diff) | |
Implement P2/P3: Plan-Run flow separation, PlannerService, and Windows test fixes
## P3: Session/Planning Foundation
### PlanAsync / RunAsync flow separation (Design §8.4)
- AgentRuntime.ExecuteAsync now follows a two-phase lifecycle:
1. Planning: generate Initial TaskGraph via Ollama structured output
2. Running: execute ready Tasks (dependencies satisfied) in
critical-path then creation order
- Sandbox Task Resource (BeginTask/EndTask) is applied at RunAsync
time, not PlanAsync time (Design constraint)
- QueuedTaskWorker transitions to Planning; ExecuteAsync transitions
to Running internally
### PlannerService (new: src/Agent.Core/Runtime/PlannerService.cs)
- PlanAsync: uses Ollama ChatStructuredAsync with JSON Schema to
generate a TaskGraph with per-Task requires_external_effect
- ReplanAsync: rebuilds TaskGraph after policy deny, preserving
completed tasks (Design §8.4)
- Fallback: single-task plan with conservative estimates when
structured output fails or returns empty
- Pure analysis tasks get RequiresExternalEffect=false with at
least 1 Task (Design constraint)
### PlanResult / PlanSchema (new: src/Agent.Core/Runtime/PlanResult.cs)
- JSON model for LLM structured output during Planning
- JSON Schema with tasks array: description, dependencies (by index),
budget_class, PERT estimates, requires_external_effect
### requires_external_effect integration (P2 + P3)
- Removed keyword heuristic RequiresExternalEffect() from AgentRuntime
- Removed tool-name-counting RequiredToolCalls() from AgentRuntime
- ToolUseGuard now receives PlannerTask.RequiresExternalEffect
(set by Planner during Planning, not by keyword matching)
- requires_external_effect is NOT always true; it is a per-Task
attribute determined by the Planner (Design constraint)
### RunAsync / ready Task execution
- SelectNextReadyTask: dependencies satisfied, not completed/failed,
critical path first then creation order
- Each PlannerTask executed via ExecutePlannerTaskAsync (LLM
chat-with-tools iteration loop)
- TaskGraph.AddUsage records per-Task token usage for CPM tracking
### Replan implementation
- Policy deny (InvalidOperationException with "denied by sandbox
policy") triggers ReplanAsync, up to max 3 replan attempts
- Failed task is marked Failed in TaskGraph before replan
- Registry transitions back to Running for replan iteration
## P3-A: AgentSession + TaskGraph ownership
### AgentSession expansion (src/Agent.Core/Sessions/AgentSession.cs)
- Added Summary, ExecutionResults, TaskGraphs
(ConcurrentDictionary<Guid, TaskGraph>)
- TaskGraph is per-MCP-Task (1 agent_run = 1 TaskGraph), NOT shared
per-Session (Design constraint)
- Added CompactConversation, CompactExecutionResults, SetSummary
methods for context summary compaction
### ExecutionResult (new: src/Agent.Core/Sessions/ExecutionResult.cs)
- Records tool execution results in session for context summary
- Fields: TaskId, ToolName, Success, Output
### SessionStore (src/Agent.Core/Sessions/SessionStore.cs)
- Added Sessions property for TaskGraphToolHandler to resolve graphs
by TaskId across sessions
### sessionId threading
- IInternalToolBackend.BeginTaskAsync signature changed to include
sessionId parameter
- InternalToolDispatcher, WindowsToolBackend, TaskGraphToolHandler
updated to thread sessionId through
- task_update schema includes requires_external_effect property
- McpHost and Program.cs pass SessionStore to TaskRegistry
## P3: Context Summary (Design §9.3)
- Context utilization = latest prompt_eval_count / num_ctx
(NOT cumulative InputTokens) (Design constraint)
- TryContextSummary: compacts old Conversation/ExecutionResults when
utilization exceeds 80% threshold
- AgentRuntimeSettings: added NumCtx, ContextSummaryThreshold,
ConversationKeepLastN, ExecutionResultsKeepLastN
- OllamaResult: added PromptEvalCount field
- OllamaClient: passes PromptEvalCount from OllamaChatResponse
## P3: ExecutionResults Session recording
- ExecutePlannerTaskAsync records each tool execution result via
session.AppendExecutionResult for context summary and audit
## Windows test fixes
### SandboxPolicyException catch (src/Agent.Windows/Tools/WindowsToolBackend.cs)
- ExecuteAsync now catches SandboxPolicyException and returns
ToolExecutionResult(false, ..., RetryAllowed: false) instead of
propagating the exception
- Fixes Dir_mkdir_outside_approved_root_is_denied and
File_edit_rename_rejects_outside_approved_root tests
### LoopbackExemption cleanup (tests/Agent.Windows.Tests/SecurityPolicyTests.cs)
- Recovery_after_crash: added pre-clean and try/finally to ensure
LoopbackExemption is always removed, even on assertion failure
- Independent_process_crash: added pre-clean to remove residual
loopback exemption from prior test failures
- Fixes 7 Independent_process_crash scenarios that were failing due
to LoopbackExemption residue from Recovery_after_crash
## Test updates (tests/Agent.Core.Tests/PromptAndRuntimeTests.cs)
- FakeOllamaClient implementations updated with ChatStructuredAsync
and ChatWithToolsAsync overrides to support Plan/Run flow
- RecordingOllamaClient: returns single-task plan JSON for
ChatStructuredAsync, tracks all call types
- ToolCallingOllamaClient: returns plan with
requires_external_effect=true for file edit tasks
- Test expectations updated for Plan+Run call counts and token
accounting (Plan tokens not added to registry usage)
## Documentation updates
- Design.md §7.7: requires_external_effect is set by Planner during
Planning, not keyword heuristic
- Design.md §8.4: Plan/Run phase separation, replan after policy deny
- Design.md §9.3: context utilization formula clarified
- ImplementationPlan.md §10.3: Plan/Run flow, replan, Context Summary
## Test results (all passing)
- Agent.Core.Tests: 61/61
- Agent.Planner.Tests: 10/10
- Agent.Windows.Tests (elevated): 41/41
- Build: 0 errors, 0 warnings
Diffstat (limited to 'tests/Agent.Windows.Tests')
| -rw-r--r-- | tests/Agent.Windows.Tests/SecurityPolicyTests.cs | 58 |
1 files changed, 35 insertions, 23 deletions
diff --git a/tests/Agent.Windows.Tests/SecurityPolicyTests.cs b/tests/Agent.Windows.Tests/SecurityPolicyTests.cs index 9f13f56..10415ce 100644 --- a/tests/Agent.Windows.Tests/SecurityPolicyTests.cs +++ b/tests/Agent.Windows.Tests/SecurityPolicyTests.cs @@ -338,6 +338,8 @@ public sealed class SecurityPolicyTests public void Recovery_after_crash_restores_acl_loopback_and_dynamic_wfp_state() { AppContainerProfile profile = AppContainerProfile.EnsureFixed(); + // Pre-clean any residual loopback exemption from prior test failures. + LoopbackExemption.Remove(profile.Sid); using TestDirectory directory = new(); string journals = Path.Combine(directory.Path, "journals"); string resource = Path.Combine(directory.Path, "resource"); @@ -371,18 +373,26 @@ public sealed class SecurityPolicyTests // BFE owns and deletes every object in this dynamic session. } - RecoveryResult recovered = new RecoveryManager(journals, audit).RecoverAll(); - - Assert.True(recovered.IsClean); - Assert.Equal(1, recovered.JournalsRecovered); - Assert.Equal(3, recovered.ResourcesRecovered); - Assert.False(LoopbackExemption.Contains(profile.Sid)); - byte[] aclAfter = FileSystemAclExtensions.GetAccessControl(new DirectoryInfo(resource), AccessControlSections.Access) - .GetSecurityDescriptorBinaryForm(); - Assert.Equal(Convert.ToHexString(aclBefore), Convert.ToHexString(aclAfter)); - Assert.Equal(Convert.ToHexString(childAclBefore), Convert.ToHexString(AccessDescriptor(child))); - Assert.Empty(AccessRules(child, profile.Sid)); - Assert.Empty(Directory.EnumerateFiles(journals)); + try + { + RecoveryResult recovered = new RecoveryManager(journals, audit).RecoverAll(); + + Assert.True(recovered.IsClean); + Assert.Equal(1, recovered.JournalsRecovered); + Assert.Equal(3, recovered.ResourcesRecovered); + Assert.False(LoopbackExemption.Contains(profile.Sid)); + byte[] aclAfter = FileSystemAclExtensions.GetAccessControl(new DirectoryInfo(resource), AccessControlSections.Access) + .GetSecurityDescriptorBinaryForm(); + Assert.Equal(Convert.ToHexString(aclBefore), Convert.ToHexString(aclAfter)); + Assert.Equal(Convert.ToHexString(childAclBefore), Convert.ToHexString(AccessDescriptor(child))); + Assert.Empty(AccessRules(child, profile.Sid)); + Assert.Empty(Directory.EnumerateFiles(journals)); + } + finally + { + // Ensure loopback exemption is always cleaned up, even if assertions fail. + LoopbackExemption.Remove(profile.Sid); + } } [Theory] @@ -395,6 +405,8 @@ public sealed class SecurityPolicyTests [InlineData("wfp")] public void Independent_process_crash_is_fully_recovered_on_next_start(string scenario) { + // Pre-clean any residual loopback exemption from prior test failures. + LoopbackExemption.Remove(AppContainerProfile.EnsureFixed().Sid); using TestDirectory directory = new(); string resourceDirectory = Path.Combine(directory.Path, "resource"); string resourceFile = Path.Combine(directory.Path, "resource.txt"); @@ -550,7 +562,7 @@ public sealed class SecurityPolicyTests 16 * 1024 * 1024, 1024 * 1024, 1000, 10000); var backend = new WindowsToolBackend(settings, audit); Guid taskId = Guid.NewGuid(); - await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken); + await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken); using JsonDocument edit = JsonDocument.Parse($$"""{"path":{{JsonSerializer.Serialize(target)}},"operation":"overwrite","content":"after"}"""); ToolExecutionResult result = await backend.ExecuteAsync(taskId, "file_edit", edit.RootElement, TestContext.Current.CancellationToken); @@ -584,7 +596,7 @@ public sealed class SecurityPolicyTests 16 * 1024 * 1024, 1024 * 1024, 1000, 10000); var backend = new WindowsToolBackend(settings, audit); Guid taskId = Guid.NewGuid(); - await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken); + await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken); using JsonDocument mkdir = JsonDocument.Parse($$"""{"path":{{JsonSerializer.Serialize(target)}},"operation":"mkdir"}"""); ToolExecutionResult result = await backend.ExecuteAsync(taskId, "dir", mkdir.RootElement, TestContext.Current.CancellationToken); @@ -615,7 +627,7 @@ public sealed class SecurityPolicyTests 16 * 1024 * 1024, 1024 * 1024, 1000, 10000); var backend = new WindowsToolBackend(settings, audit); Guid taskId = Guid.NewGuid(); - await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken); + await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken); using JsonDocument rmdir = JsonDocument.Parse($$"""{"path":{{JsonSerializer.Serialize(target)}},"operation":"rmdir"}"""); ToolExecutionResult result = await backend.ExecuteAsync(taskId, "dir", rmdir.RootElement, TestContext.Current.CancellationToken); @@ -643,7 +655,7 @@ public sealed class SecurityPolicyTests 16 * 1024 * 1024, 1024 * 1024, 1000, 10000); var backend = new WindowsToolBackend(settings, audit); Guid taskId = Guid.NewGuid(); - await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken); + await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken); using JsonDocument rmdir = JsonDocument.Parse($$"""{"path":{{JsonSerializer.Serialize(target)}},"operation":"rmdir"}"""); await Assert.ThrowsAsync<ToolValidationException>(async () => await backend.ExecuteAsync(taskId, "dir", rmdir.RootElement, TestContext.Current.CancellationToken)); @@ -667,7 +679,7 @@ public sealed class SecurityPolicyTests 16 * 1024 * 1024, 1024 * 1024, 1000, 10000); var backend = new WindowsToolBackend(settings, audit); Guid taskId = Guid.NewGuid(); - await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken); + await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken); using JsonDocument list = JsonDocument.Parse($$"""{"path":{{JsonSerializer.Serialize(workspace)}},"operation":"list"}"""); ToolExecutionResult result = await backend.ExecuteAsync(taskId, "dir", list.RootElement, TestContext.Current.CancellationToken); @@ -693,7 +705,7 @@ public sealed class SecurityPolicyTests 16 * 1024 * 1024, 1024 * 1024, 1000, 10000); var backend = new WindowsToolBackend(settings, audit); Guid taskId = Guid.NewGuid(); - await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken); + await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken); string badTarget = Path.Combine(outside.Path, "escaped"); using JsonDocument mkdir = JsonDocument.Parse($$"""{"path":{{JsonSerializer.Serialize(badTarget)}},"operation":"mkdir"}"""); @@ -719,7 +731,7 @@ public sealed class SecurityPolicyTests 16 * 1024 * 1024, 1024 * 1024, 1000, 10000); var backend = new WindowsToolBackend(settings, audit); Guid taskId = Guid.NewGuid(); - await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken); + await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken); using JsonDocument tail = JsonDocument.Parse($$"""{"root":{{JsonSerializer.Serialize(binaryFile)}},"mode":"tail","query":"","tail_lines":10}"""); await Assert.ThrowsAsync<ToolValidationException>(async () => await backend.ExecuteAsync(taskId, "file_search", tail.RootElement, TestContext.Current.CancellationToken)); @@ -755,7 +767,7 @@ public sealed class SecurityPolicyTests 16 * 1024 * 1024, 1024 * 1024, 1000, 10000); var backend = new WindowsToolBackend(settings, audit); Guid taskId = Guid.NewGuid(); - await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken); + await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken); using JsonDocument tail = JsonDocument.Parse($$"""{"root":{{JsonSerializer.Serialize(textFile)}},"mode":"tail","query":"","tail_lines":10}"""); ToolExecutionResult result = await backend.ExecuteAsync(taskId, "file_search", tail.RootElement, TestContext.Current.CancellationToken); @@ -780,7 +792,7 @@ public sealed class SecurityPolicyTests 16 * 1024 * 1024, 1024 * 1024, 1000, 10000); var backend = new WindowsToolBackend(settings, audit); Guid taskId = Guid.NewGuid(); - await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken); + await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken); using JsonDocument rename = JsonDocument.Parse($$"""{"path":{{JsonSerializer.Serialize(source)}},"operation":"rename","new_path":{{JsonSerializer.Serialize(dest)}}}"""); ToolExecutionResult result = await backend.ExecuteAsync(taskId, "file_edit", rename.RootElement, TestContext.Current.CancellationToken); @@ -811,7 +823,7 @@ public sealed class SecurityPolicyTests 16 * 1024 * 1024, 1024 * 1024, 1000, 10000); var backend = new WindowsToolBackend(settings, audit); Guid taskId = Guid.NewGuid(); - await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken); + await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken); using JsonDocument rename = JsonDocument.Parse($$"""{"path":{{JsonSerializer.Serialize(source)}},"operation":"rename","new_path":{{JsonSerializer.Serialize(dest)}}}"""); await Assert.ThrowsAsync<ToolValidationException>(async () => await backend.ExecuteAsync(taskId, "file_edit", rename.RootElement, TestContext.Current.CancellationToken)); @@ -838,7 +850,7 @@ public sealed class SecurityPolicyTests 16 * 1024 * 1024, 1024 * 1024, 1000, 10000); var backend = new WindowsToolBackend(settings, audit); Guid taskId = Guid.NewGuid(); - await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken); + await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken); using JsonDocument rename = JsonDocument.Parse($$"""{"path":{{JsonSerializer.Serialize(source)}},"operation":"rename","new_path":{{JsonSerializer.Serialize(dest)}}}"""); ToolExecutionResult result = await backend.ExecuteAsync(taskId, "file_edit", rename.RootElement, TestContext.Current.CancellationToken); |
