summaryrefslogtreecommitdiff
path: root/tests/Agent.Windows.Tests
diff options
context:
space:
mode:
authortanaka takashi <isazaka@gmail.com>2026-08-11 05:17:32 +0900
committertanaka takashi <isazaka@gmail.com>2026-08-11 05:17:32 +0900
commit35719799e62a18850efc52d380fb53bf3baa8022 (patch)
tree3cc1d110f290e5acab71ca202a4cd0d521e4a8b5 /tests/Agent.Windows.Tests
parentd5d0a84e573c833843279c520ce2d5765425ba93 (diff)
Implement P2/P3: Plan-Run flow separation, PlannerService, and Windows test fixes
## P3: Session/Planning Foundation ### PlanAsync / RunAsync flow separation (Design §8.4) - AgentRuntime.ExecuteAsync now follows a two-phase lifecycle: 1. Planning: generate Initial TaskGraph via Ollama structured output 2. Running: execute ready Tasks (dependencies satisfied) in critical-path then creation order - Sandbox Task Resource (BeginTask/EndTask) is applied at RunAsync time, not PlanAsync time (Design constraint) - QueuedTaskWorker transitions to Planning; ExecuteAsync transitions to Running internally ### PlannerService (new: src/Agent.Core/Runtime/PlannerService.cs) - PlanAsync: uses Ollama ChatStructuredAsync with JSON Schema to generate a TaskGraph with per-Task requires_external_effect - ReplanAsync: rebuilds TaskGraph after policy deny, preserving completed tasks (Design §8.4) - Fallback: single-task plan with conservative estimates when structured output fails or returns empty - Pure analysis tasks get RequiresExternalEffect=false with at least 1 Task (Design constraint) ### PlanResult / PlanSchema (new: src/Agent.Core/Runtime/PlanResult.cs) - JSON model for LLM structured output during Planning - JSON Schema with tasks array: description, dependencies (by index), budget_class, PERT estimates, requires_external_effect ### requires_external_effect integration (P2 + P3) - Removed keyword heuristic RequiresExternalEffect() from AgentRuntime - Removed tool-name-counting RequiredToolCalls() from AgentRuntime - ToolUseGuard now receives PlannerTask.RequiresExternalEffect (set by Planner during Planning, not by keyword matching) - requires_external_effect is NOT always true; it is a per-Task attribute determined by the Planner (Design constraint) ### RunAsync / ready Task execution - SelectNextReadyTask: dependencies satisfied, not completed/failed, critical path first then creation order - Each PlannerTask executed via ExecutePlannerTaskAsync (LLM chat-with-tools iteration loop) - TaskGraph.AddUsage records per-Task token usage for CPM tracking ### Replan implementation - Policy deny (InvalidOperationException with "denied by sandbox policy") triggers ReplanAsync, up to max 3 replan attempts - Failed task is marked Failed in TaskGraph before replan - Registry transitions back to Running for replan iteration ## P3-A: AgentSession + TaskGraph ownership ### AgentSession expansion (src/Agent.Core/Sessions/AgentSession.cs) - Added Summary, ExecutionResults, TaskGraphs (ConcurrentDictionary<Guid, TaskGraph>) - TaskGraph is per-MCP-Task (1 agent_run = 1 TaskGraph), NOT shared per-Session (Design constraint) - Added CompactConversation, CompactExecutionResults, SetSummary methods for context summary compaction ### ExecutionResult (new: src/Agent.Core/Sessions/ExecutionResult.cs) - Records tool execution results in session for context summary - Fields: TaskId, ToolName, Success, Output ### SessionStore (src/Agent.Core/Sessions/SessionStore.cs) - Added Sessions property for TaskGraphToolHandler to resolve graphs by TaskId across sessions ### sessionId threading - IInternalToolBackend.BeginTaskAsync signature changed to include sessionId parameter - InternalToolDispatcher, WindowsToolBackend, TaskGraphToolHandler updated to thread sessionId through - task_update schema includes requires_external_effect property - McpHost and Program.cs pass SessionStore to TaskRegistry ## P3: Context Summary (Design §9.3) - Context utilization = latest prompt_eval_count / num_ctx (NOT cumulative InputTokens) (Design constraint) - TryContextSummary: compacts old Conversation/ExecutionResults when utilization exceeds 80% threshold - AgentRuntimeSettings: added NumCtx, ContextSummaryThreshold, ConversationKeepLastN, ExecutionResultsKeepLastN - OllamaResult: added PromptEvalCount field - OllamaClient: passes PromptEvalCount from OllamaChatResponse ## P3: ExecutionResults Session recording - ExecutePlannerTaskAsync records each tool execution result via session.AppendExecutionResult for context summary and audit ## Windows test fixes ### SandboxPolicyException catch (src/Agent.Windows/Tools/WindowsToolBackend.cs) - ExecuteAsync now catches SandboxPolicyException and returns ToolExecutionResult(false, ..., RetryAllowed: false) instead of propagating the exception - Fixes Dir_mkdir_outside_approved_root_is_denied and File_edit_rename_rejects_outside_approved_root tests ### LoopbackExemption cleanup (tests/Agent.Windows.Tests/SecurityPolicyTests.cs) - Recovery_after_crash: added pre-clean and try/finally to ensure LoopbackExemption is always removed, even on assertion failure - Independent_process_crash: added pre-clean to remove residual loopback exemption from prior test failures - Fixes 7 Independent_process_crash scenarios that were failing due to LoopbackExemption residue from Recovery_after_crash ## Test updates (tests/Agent.Core.Tests/PromptAndRuntimeTests.cs) - FakeOllamaClient implementations updated with ChatStructuredAsync and ChatWithToolsAsync overrides to support Plan/Run flow - RecordingOllamaClient: returns single-task plan JSON for ChatStructuredAsync, tracks all call types - ToolCallingOllamaClient: returns plan with requires_external_effect=true for file edit tasks - Test expectations updated for Plan+Run call counts and token accounting (Plan tokens not added to registry usage) ## Documentation updates - Design.md §7.7: requires_external_effect is set by Planner during Planning, not keyword heuristic - Design.md §8.4: Plan/Run phase separation, replan after policy deny - Design.md §9.3: context utilization formula clarified - ImplementationPlan.md §10.3: Plan/Run flow, replan, Context Summary ## Test results (all passing) - Agent.Core.Tests: 61/61 - Agent.Planner.Tests: 10/10 - Agent.Windows.Tests (elevated): 41/41 - Build: 0 errors, 0 warnings
Diffstat (limited to 'tests/Agent.Windows.Tests')
-rw-r--r--tests/Agent.Windows.Tests/SecurityPolicyTests.cs58
1 files changed, 35 insertions, 23 deletions
diff --git a/tests/Agent.Windows.Tests/SecurityPolicyTests.cs b/tests/Agent.Windows.Tests/SecurityPolicyTests.cs
index 9f13f56..10415ce 100644
--- a/tests/Agent.Windows.Tests/SecurityPolicyTests.cs
+++ b/tests/Agent.Windows.Tests/SecurityPolicyTests.cs
@@ -338,6 +338,8 @@ public sealed class SecurityPolicyTests
public void Recovery_after_crash_restores_acl_loopback_and_dynamic_wfp_state()
{
AppContainerProfile profile = AppContainerProfile.EnsureFixed();
+ // Pre-clean any residual loopback exemption from prior test failures.
+ LoopbackExemption.Remove(profile.Sid);
using TestDirectory directory = new();
string journals = Path.Combine(directory.Path, "journals");
string resource = Path.Combine(directory.Path, "resource");
@@ -371,18 +373,26 @@ public sealed class SecurityPolicyTests
// BFE owns and deletes every object in this dynamic session.
}
- RecoveryResult recovered = new RecoveryManager(journals, audit).RecoverAll();
-
- Assert.True(recovered.IsClean);
- Assert.Equal(1, recovered.JournalsRecovered);
- Assert.Equal(3, recovered.ResourcesRecovered);
- Assert.False(LoopbackExemption.Contains(profile.Sid));
- byte[] aclAfter = FileSystemAclExtensions.GetAccessControl(new DirectoryInfo(resource), AccessControlSections.Access)
- .GetSecurityDescriptorBinaryForm();
- Assert.Equal(Convert.ToHexString(aclBefore), Convert.ToHexString(aclAfter));
- Assert.Equal(Convert.ToHexString(childAclBefore), Convert.ToHexString(AccessDescriptor(child)));
- Assert.Empty(AccessRules(child, profile.Sid));
- Assert.Empty(Directory.EnumerateFiles(journals));
+ try
+ {
+ RecoveryResult recovered = new RecoveryManager(journals, audit).RecoverAll();
+
+ Assert.True(recovered.IsClean);
+ Assert.Equal(1, recovered.JournalsRecovered);
+ Assert.Equal(3, recovered.ResourcesRecovered);
+ Assert.False(LoopbackExemption.Contains(profile.Sid));
+ byte[] aclAfter = FileSystemAclExtensions.GetAccessControl(new DirectoryInfo(resource), AccessControlSections.Access)
+ .GetSecurityDescriptorBinaryForm();
+ Assert.Equal(Convert.ToHexString(aclBefore), Convert.ToHexString(aclAfter));
+ Assert.Equal(Convert.ToHexString(childAclBefore), Convert.ToHexString(AccessDescriptor(child)));
+ Assert.Empty(AccessRules(child, profile.Sid));
+ Assert.Empty(Directory.EnumerateFiles(journals));
+ }
+ finally
+ {
+ // Ensure loopback exemption is always cleaned up, even if assertions fail.
+ LoopbackExemption.Remove(profile.Sid);
+ }
}
[Theory]
@@ -395,6 +405,8 @@ public sealed class SecurityPolicyTests
[InlineData("wfp")]
public void Independent_process_crash_is_fully_recovered_on_next_start(string scenario)
{
+ // Pre-clean any residual loopback exemption from prior test failures.
+ LoopbackExemption.Remove(AppContainerProfile.EnsureFixed().Sid);
using TestDirectory directory = new();
string resourceDirectory = Path.Combine(directory.Path, "resource");
string resourceFile = Path.Combine(directory.Path, "resource.txt");
@@ -550,7 +562,7 @@ public sealed class SecurityPolicyTests
16 * 1024 * 1024, 1024 * 1024, 1000, 10000);
var backend = new WindowsToolBackend(settings, audit);
Guid taskId = Guid.NewGuid();
- await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken);
+ await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken);
using JsonDocument edit = JsonDocument.Parse($$"""{"path":{{JsonSerializer.Serialize(target)}},"operation":"overwrite","content":"after"}""");
ToolExecutionResult result = await backend.ExecuteAsync(taskId, "file_edit", edit.RootElement, TestContext.Current.CancellationToken);
@@ -584,7 +596,7 @@ public sealed class SecurityPolicyTests
16 * 1024 * 1024, 1024 * 1024, 1000, 10000);
var backend = new WindowsToolBackend(settings, audit);
Guid taskId = Guid.NewGuid();
- await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken);
+ await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken);
using JsonDocument mkdir = JsonDocument.Parse($$"""{"path":{{JsonSerializer.Serialize(target)}},"operation":"mkdir"}""");
ToolExecutionResult result = await backend.ExecuteAsync(taskId, "dir", mkdir.RootElement, TestContext.Current.CancellationToken);
@@ -615,7 +627,7 @@ public sealed class SecurityPolicyTests
16 * 1024 * 1024, 1024 * 1024, 1000, 10000);
var backend = new WindowsToolBackend(settings, audit);
Guid taskId = Guid.NewGuid();
- await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken);
+ await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken);
using JsonDocument rmdir = JsonDocument.Parse($$"""{"path":{{JsonSerializer.Serialize(target)}},"operation":"rmdir"}""");
ToolExecutionResult result = await backend.ExecuteAsync(taskId, "dir", rmdir.RootElement, TestContext.Current.CancellationToken);
@@ -643,7 +655,7 @@ public sealed class SecurityPolicyTests
16 * 1024 * 1024, 1024 * 1024, 1000, 10000);
var backend = new WindowsToolBackend(settings, audit);
Guid taskId = Guid.NewGuid();
- await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken);
+ await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken);
using JsonDocument rmdir = JsonDocument.Parse($$"""{"path":{{JsonSerializer.Serialize(target)}},"operation":"rmdir"}""");
await Assert.ThrowsAsync<ToolValidationException>(async () => await backend.ExecuteAsync(taskId, "dir", rmdir.RootElement, TestContext.Current.CancellationToken));
@@ -667,7 +679,7 @@ public sealed class SecurityPolicyTests
16 * 1024 * 1024, 1024 * 1024, 1000, 10000);
var backend = new WindowsToolBackend(settings, audit);
Guid taskId = Guid.NewGuid();
- await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken);
+ await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken);
using JsonDocument list = JsonDocument.Parse($$"""{"path":{{JsonSerializer.Serialize(workspace)}},"operation":"list"}""");
ToolExecutionResult result = await backend.ExecuteAsync(taskId, "dir", list.RootElement, TestContext.Current.CancellationToken);
@@ -693,7 +705,7 @@ public sealed class SecurityPolicyTests
16 * 1024 * 1024, 1024 * 1024, 1000, 10000);
var backend = new WindowsToolBackend(settings, audit);
Guid taskId = Guid.NewGuid();
- await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken);
+ await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken);
string badTarget = Path.Combine(outside.Path, "escaped");
using JsonDocument mkdir = JsonDocument.Parse($$"""{"path":{{JsonSerializer.Serialize(badTarget)}},"operation":"mkdir"}""");
@@ -719,7 +731,7 @@ public sealed class SecurityPolicyTests
16 * 1024 * 1024, 1024 * 1024, 1000, 10000);
var backend = new WindowsToolBackend(settings, audit);
Guid taskId = Guid.NewGuid();
- await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken);
+ await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken);
using JsonDocument tail = JsonDocument.Parse($$"""{"root":{{JsonSerializer.Serialize(binaryFile)}},"mode":"tail","query":"","tail_lines":10}""");
await Assert.ThrowsAsync<ToolValidationException>(async () => await backend.ExecuteAsync(taskId, "file_search", tail.RootElement, TestContext.Current.CancellationToken));
@@ -755,7 +767,7 @@ public sealed class SecurityPolicyTests
16 * 1024 * 1024, 1024 * 1024, 1000, 10000);
var backend = new WindowsToolBackend(settings, audit);
Guid taskId = Guid.NewGuid();
- await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken);
+ await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken);
using JsonDocument tail = JsonDocument.Parse($$"""{"root":{{JsonSerializer.Serialize(textFile)}},"mode":"tail","query":"","tail_lines":10}""");
ToolExecutionResult result = await backend.ExecuteAsync(taskId, "file_search", tail.RootElement, TestContext.Current.CancellationToken);
@@ -780,7 +792,7 @@ public sealed class SecurityPolicyTests
16 * 1024 * 1024, 1024 * 1024, 1000, 10000);
var backend = new WindowsToolBackend(settings, audit);
Guid taskId = Guid.NewGuid();
- await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken);
+ await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken);
using JsonDocument rename = JsonDocument.Parse($$"""{"path":{{JsonSerializer.Serialize(source)}},"operation":"rename","new_path":{{JsonSerializer.Serialize(dest)}}}""");
ToolExecutionResult result = await backend.ExecuteAsync(taskId, "file_edit", rename.RootElement, TestContext.Current.CancellationToken);
@@ -811,7 +823,7 @@ public sealed class SecurityPolicyTests
16 * 1024 * 1024, 1024 * 1024, 1000, 10000);
var backend = new WindowsToolBackend(settings, audit);
Guid taskId = Guid.NewGuid();
- await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken);
+ await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken);
using JsonDocument rename = JsonDocument.Parse($$"""{"path":{{JsonSerializer.Serialize(source)}},"operation":"rename","new_path":{{JsonSerializer.Serialize(dest)}}}""");
await Assert.ThrowsAsync<ToolValidationException>(async () => await backend.ExecuteAsync(taskId, "file_edit", rename.RootElement, TestContext.Current.CancellationToken));
@@ -838,7 +850,7 @@ public sealed class SecurityPolicyTests
16 * 1024 * 1024, 1024 * 1024, 1000, 10000);
var backend = new WindowsToolBackend(settings, audit);
Guid taskId = Guid.NewGuid();
- await backend.BeginTaskAsync(taskId, TestContext.Current.CancellationToken);
+ await backend.BeginTaskAsync(taskId, Guid.NewGuid(), TestContext.Current.CancellationToken);
using JsonDocument rename = JsonDocument.Parse($$"""{"path":{{JsonSerializer.Serialize(source)}},"operation":"rename","new_path":{{JsonSerializer.Serialize(dest)}}}""");
ToolExecutionResult result = await backend.ExecuteAsync(taskId, "file_edit", rename.RootElement, TestContext.Current.CancellationToken);