From 35719799e62a18850efc52d380fb53bf3baa8022 Mon Sep 17 00:00:00 2001 From: tanaka takashi Date: Tue, 11 Aug 2026 05:17:32 +0900 Subject: Implement P2/P3: Plan-Run flow separation, PlannerService, and Windows test fixes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## P3: Session/Planning Foundation ### PlanAsync / RunAsync flow separation (Design §8.4) - AgentRuntime.ExecuteAsync now follows a two-phase lifecycle: 1. Planning: generate Initial TaskGraph via Ollama structured output 2. Running: execute ready Tasks (dependencies satisfied) in critical-path then creation order - Sandbox Task Resource (BeginTask/EndTask) is applied at RunAsync time, not PlanAsync time (Design constraint) - QueuedTaskWorker transitions to Planning; ExecuteAsync transitions to Running internally ### PlannerService (new: src/Agent.Core/Runtime/PlannerService.cs) - PlanAsync: uses Ollama ChatStructuredAsync with JSON Schema to generate a TaskGraph with per-Task requires_external_effect - ReplanAsync: rebuilds TaskGraph after policy deny, preserving completed tasks (Design §8.4) - Fallback: single-task plan with conservative estimates when structured output fails or returns empty - Pure analysis tasks get RequiresExternalEffect=false with at least 1 Task (Design constraint) ### PlanResult / PlanSchema (new: src/Agent.Core/Runtime/PlanResult.cs) - JSON model for LLM structured output during Planning - JSON Schema with tasks array: description, dependencies (by index), budget_class, PERT estimates, requires_external_effect ### requires_external_effect integration (P2 + P3) - Removed keyword heuristic RequiresExternalEffect() from AgentRuntime - Removed tool-name-counting RequiredToolCalls() from AgentRuntime - ToolUseGuard now receives PlannerTask.RequiresExternalEffect (set by Planner during Planning, not by keyword matching) - requires_external_effect is NOT always true; it is a per-Task attribute determined by the Planner (Design constraint) ### RunAsync / ready Task execution - SelectNextReadyTask: dependencies satisfied, not completed/failed, critical path first then creation order - Each PlannerTask executed via ExecutePlannerTaskAsync (LLM chat-with-tools iteration loop) - TaskGraph.AddUsage records per-Task token usage for CPM tracking ### Replan implementation - Policy deny (InvalidOperationException with "denied by sandbox policy") triggers ReplanAsync, up to max 3 replan attempts - Failed task is marked Failed in TaskGraph before replan - Registry transitions back to Running for replan iteration ## P3-A: AgentSession + TaskGraph ownership ### AgentSession expansion (src/Agent.Core/Sessions/AgentSession.cs) - Added Summary, ExecutionResults, TaskGraphs (ConcurrentDictionary) - TaskGraph is per-MCP-Task (1 agent_run = 1 TaskGraph), NOT shared per-Session (Design constraint) - Added CompactConversation, CompactExecutionResults, SetSummary methods for context summary compaction ### ExecutionResult (new: src/Agent.Core/Sessions/ExecutionResult.cs) - Records tool execution results in session for context summary - Fields: TaskId, ToolName, Success, Output ### SessionStore (src/Agent.Core/Sessions/SessionStore.cs) - Added Sessions property for TaskGraphToolHandler to resolve graphs by TaskId across sessions ### sessionId threading - IInternalToolBackend.BeginTaskAsync signature changed to include sessionId parameter - InternalToolDispatcher, WindowsToolBackend, TaskGraphToolHandler updated to thread sessionId through - task_update schema includes requires_external_effect property - McpHost and Program.cs pass SessionStore to TaskRegistry ## P3: Context Summary (Design §9.3) - Context utilization = latest prompt_eval_count / num_ctx (NOT cumulative InputTokens) (Design constraint) - TryContextSummary: compacts old Conversation/ExecutionResults when utilization exceeds 80% threshold - AgentRuntimeSettings: added NumCtx, ContextSummaryThreshold, ConversationKeepLastN, ExecutionResultsKeepLastN - OllamaResult: added PromptEvalCount field - OllamaClient: passes PromptEvalCount from OllamaChatResponse ## P3: ExecutionResults Session recording - ExecutePlannerTaskAsync records each tool execution result via session.AppendExecutionResult for context summary and audit ## Windows test fixes ### SandboxPolicyException catch (src/Agent.Windows/Tools/WindowsToolBackend.cs) - ExecuteAsync now catches SandboxPolicyException and returns ToolExecutionResult(false, ..., RetryAllowed: false) instead of propagating the exception - Fixes Dir_mkdir_outside_approved_root_is_denied and File_edit_rename_rejects_outside_approved_root tests ### LoopbackExemption cleanup (tests/Agent.Windows.Tests/SecurityPolicyTests.cs) - Recovery_after_crash: added pre-clean and try/finally to ensure LoopbackExemption is always removed, even on assertion failure - Independent_process_crash: added pre-clean to remove residual loopback exemption from prior test failures - Fixes 7 Independent_process_crash scenarios that were failing due to LoopbackExemption residue from Recovery_after_crash ## Test updates (tests/Agent.Core.Tests/PromptAndRuntimeTests.cs) - FakeOllamaClient implementations updated with ChatStructuredAsync and ChatWithToolsAsync overrides to support Plan/Run flow - RecordingOllamaClient: returns single-task plan JSON for ChatStructuredAsync, tracks all call types - ToolCallingOllamaClient: returns plan with requires_external_effect=true for file edit tasks - Test expectations updated for Plan+Run call counts and token accounting (Plan tokens not added to registry usage) ## Documentation updates - Design.md §7.7: requires_external_effect is set by Planner during Planning, not keyword heuristic - Design.md §8.4: Plan/Run phase separation, replan after policy deny - Design.md §9.3: context utilization formula clarified - ImplementationPlan.md §10.3: Plan/Run flow, replan, Context Summary ## Test results (all passing) - Agent.Core.Tests: 61/61 - Agent.Planner.Tests: 10/10 - Agent.Windows.Tests (elevated): 41/41 - Build: 0 errors, 0 warnings --- src/Agent.Core/Runtime/PlannerService.cs | 307 +++++++++++++++++++++++++++++++ 1 file changed, 307 insertions(+) create mode 100644 src/Agent.Core/Runtime/PlannerService.cs (limited to 'src/Agent.Core/Runtime/PlannerService.cs') diff --git a/src/Agent.Core/Runtime/PlannerService.cs b/src/Agent.Core/Runtime/PlannerService.cs new file mode 100644 index 0000000..7a450d0 --- /dev/null +++ b/src/Agent.Core/Runtime/PlannerService.cs @@ -0,0 +1,307 @@ +using System.Text.Json; +using Agent.Core.Ollama; +using Agent.Core.Prompts; +using Agent.Core.Sessions; +using Agent.Core.Skills; +using Agent.Core.Tasks; +using Agent.Core.Tools; +using Agent.Planner; + +namespace Agent.Core.Runtime; + +/// +/// Builds an initial TaskGraph from LLM structured output (Planning phase). +/// The Planner determines requires_external_effect per-task during Planning. +/// Pure analysis tasks get a TaskGraph with at least 1 Task (RequiresExternalEffect=false). +/// +public sealed class PlannerService(IOllamaClient ollama, AgentRuntimeSettings settings) +{ + private readonly IOllamaClient _ollama = ollama; + private readonly AgentRuntimeSettings _settings = settings; + + /// + /// Generates an initial TaskGraph for the given request via Ollama structured output. + /// Returns a TaskGraph stored in the session, keyed by the MCP Task ID. + /// + public async Task PlanAsync(Guid taskId, Guid sessionId, AgentSession session, string request, double? temperature, IReadOnlyList? images, CancellationToken cancellationToken) + { + // Build a Planning prompt with skill catalog and sandbox capability summary. + IReadOnlyList selectedSkills = await SelectSkillsAsync(request, cancellationToken).ConfigureAwait(false); + + var messages = new List + { + new("system", FixedPrompt.Load()), + new("system", _settings.UserSystemPrompt), + new("system", $"Respond in locale: {_settings.Locale}."), + new("system", ComposeSkillCatalog(_settings.Skills)), + new("system", _settings.SandboxCapabilitySummary), + new("system", BuildPlanningInstruction()), + new("user", request, images) + }; + + // Use structured output to get a plan. + OllamaResult result; + try + { + result = await _ollama.ChatStructuredAsync(messages, 0.1, PlanSchema.Element, cancellationToken).ConfigureAwait(false); + } + catch + { + // Fallback: single-task plan with conservative estimates. + return CreateFallbackGraph(request); + } + + PlanResult? plan = ParsePlan(result.Content); + if (plan is null || plan.Tasks.Count == 0) + return CreateFallbackGraph(request); + + TaskGraph graph = BuildTaskGraph(plan, request); + session.TaskGraphs[taskId] = graph; + return graph; + } + + /// + /// Replans after a policy deny or task failure by adding/replacing tasks in the existing TaskGraph. + /// Completed tasks are immutable; only incomplete tasks are modified. + /// + public async Task ReplanAsync(Guid taskId, Guid sessionId, AgentSession session, string request, double? temperature, string replanReason, CancellationToken cancellationToken) + { + if (!session.TaskGraphs.TryGetValue(taskId, out TaskGraph? existing)) + return await PlanAsync(taskId, sessionId, session, request, temperature, null, cancellationToken).ConfigureAwait(false); + + var messages = new List + { + new("system", FixedPrompt.Load()), + new("system", _settings.UserSystemPrompt), + new("system", $"Respond in locale: {_settings.Locale}."), + new("system", _settings.SandboxCapabilitySummary), + new("system", BuildReplanInstruction(existing, replanReason)), + new("user", request) + }; + + OllamaResult result; + try + { + result = await _ollama.ChatStructuredAsync(messages, 0.1, PlanSchema.Element, cancellationToken).ConfigureAwait(false); + } + catch + { + return existing; + } + + PlanResult? plan = ParsePlan(result.Content); + if (plan is null || plan.Tasks.Count == 0) return existing; + + // Build a new graph, preserving completed tasks from the existing graph. + return RebuildTaskGraph(existing, plan, request); + } + + private TaskGraph BuildTaskGraph(PlanResult plan, string originalRequest) + { + var graph = new TaskGraph(32); + var idMap = new List(); + + for (int i = 0; i < plan.Tasks.Count; i++) + { + PlannedTask pt = plan.Tasks[i]; + TokenBudgetClass budgetClass = ParseBudgetClass(pt.BudgetClass); + TokenEstimate estimate = CreateEstimate(pt, budgetClass); + + // Map dependency indices to Guids. + var dependencies = new HashSet(); + foreach (int depIdx in pt.Dependencies) + { + if (depIdx >= 0 && depIdx < idMap.Count) + dependencies.Add(idMap[depIdx]); + } + + string description = string.IsNullOrWhiteSpace(pt.Description) ? originalRequest : pt.Description; + PlannerTask task = graph.AddTask(description, budgetClass, estimate, dependencies, pt.RequiresExternalEffect); + idMap.Add(task.Id); + } + + return graph; + } + + private TaskGraph RebuildTaskGraph(TaskGraph existing, PlanResult plan, string originalRequest) + { + // Collect completed task descriptions to skip them in the new graph. + var completedDescriptions = new HashSet( + existing.Tasks.Where(static t => t.State == PlannerTaskState.Completed).Select(static t => t.Description), + StringComparer.Ordinal); + + var graph = new TaskGraph(32); + var idMap = new List(); + + for (int i = 0; i < plan.Tasks.Count; i++) + { + PlannedTask pt = plan.Tasks[i]; + + // Skip tasks that are already completed in the existing graph. + if (completedDescriptions.Contains(pt.Description)) + { + // Find the completed task and add it as completed to the new graph. + PlannerTask? completedTask = existing.Tasks.FirstOrDefault(t => t.Description == pt.Description); + if (completedTask is not null) + { + TokenBudgetClass budgetClass = ParseBudgetClass(pt.BudgetClass); + TokenEstimate estimate = CreateEstimate(pt, budgetClass); + var deps = new HashSet(); + foreach (int depIdx in pt.Dependencies) + if (depIdx >= 0 && depIdx < idMap.Count) deps.Add(idMap[depIdx]); + PlannerTask added = graph.AddTask(pt.Description, budgetClass, estimate, deps, pt.RequiresExternalEffect); + graph.SetState(added.Id, PlannerTaskState.Completed); + idMap.Add(added.Id); + continue; + } + } + + TokenBudgetClass bc = ParseBudgetClass(pt.BudgetClass); + TokenEstimate est = CreateEstimate(pt, bc); + var dependencies = new HashSet(); + foreach (int depIdx in pt.Dependencies) + if (depIdx >= 0 && depIdx < idMap.Count) dependencies.Add(idMap[depIdx]); + + string description = string.IsNullOrWhiteSpace(pt.Description) ? originalRequest : pt.Description; + PlannerTask task = graph.AddTask(description, bc, est, dependencies, pt.RequiresExternalEffect); + idMap.Add(task.Id); + } + + return graph; + } + + private TaskGraph CreateFallbackGraph(string request) + { + var graph = new TaskGraph(32); + TokenEstimate estimate = new(512, 2048, 8192); + graph.AddTask(request, TokenBudgetClass.K8, estimate, requiresExternalEffect: false); + return graph; + } + + private static TokenBudgetClass ParseBudgetClass(string value) + { + try { return TokenBudgetClassExtensions.Parse(value); } + catch { return TokenBudgetClass.K4; } + } + + private static TokenEstimate CreateEstimate(PlannedTask pt, TokenBudgetClass budgetClass) + { + long o = Math.Max(1, pt.OptimisticTokens); + long m = Math.Max(o, pt.MostLikelyTokens); + long p = Math.Max(m, pt.PessimisticTokens); + long max = budgetClass.MaxTokens(); + if (p > max) p = max; + if (m > p) m = p; + if (o > m) o = m; + return new TokenEstimate(o, m, p); + } + + private static PlanResult? ParsePlan(string content) + { + try + { + return JsonSerializer.Deserialize(content, new JsonSerializerOptions + { + PropertyNameCaseInsensitive = true + }); + } + catch + { + return null; + } + } + + private async Task> SelectSkillsAsync(string request, CancellationToken cancellationToken) + { + if (_settings.Skills.Count == 0) return []; + + string catalog = string.Join("\n", _settings.Skills.Select(static s => $"- {s.Name}: {s.Description}")); + var selectionMessages = new List + { + new("system", $"Select skills relevant to the user request. Available skills:\n{catalog}\n\nReturn a JSON object with a 'selected_skills' array containing skill names. Select only relevant skills. If none are relevant, return an empty array."), + new("user", request) + }; + + string schema = """{"type":"object","properties":{"selected_skills":{"type":"array","items":{"type":"string"}}},"required":["selected_skills"]}"""; + using JsonDocument formatDoc = JsonDocument.Parse(schema); + + try + { + OllamaResult result = await _ollama.ChatStructuredAsync(selectionMessages, 0.0, formatDoc.RootElement.Clone(), cancellationToken).ConfigureAwait(false); + return ParseSelectedSkills(result.Content); + } + catch + { + return []; + } + } + + private IReadOnlyList ParseSelectedSkills(string content) + { + try + { + using JsonDocument doc = JsonDocument.Parse(content); + if (!doc.RootElement.TryGetProperty("selected_skills", out JsonElement skillsElement)) return []; + var selectedNames = new HashSet(StringComparer.Ordinal); + foreach (JsonElement item in skillsElement.EnumerateArray()) + { + string? name = item.GetString(); + if (!string.IsNullOrWhiteSpace(name)) selectedNames.Add(name); + } + var valid = new List(); + foreach (AgentSkill skill in _settings.Skills) + { + if (selectedNames.Contains(skill.Name)) + { + valid.Add(skill); + if (valid.Count >= 10) break; + } + } + return valid; + } + catch + { + return []; + } + } + + private static string ComposeSkillCatalog(IReadOnlyList skills) + { + if (skills.Count == 0) return "No skills available."; + var lines = new List { "Available skills:" }; + foreach (AgentSkill skill in skills) + lines.Add($"- {skill.Name}: {skill.Description}"); + return string.Join("\n", lines); + } + + private static string BuildPlanningInstruction() => + """ + You are a Task Planner. Decompose the user request into a TaskGraph of 1-32 tasks. + Each task must have: + - description: a clear, self-contained sub-task description + - dependencies: array of task indices (0-based) that must complete before this task + - budget_class: one of 1K, 2K, 4K, 8K, 16K, 32K, 64K, 128K (token budget for this task) + - optimistic_tokens, most_likely_tokens, pessimistic_tokens: PERT estimates (0 < O <= M <= P <= budget) + - requires_external_effect: true if this task requires a verifiable external effect (file edit, process run, etc.) + + For pure analysis or informational tasks, requires_external_effect should be false. + For tasks that modify files, run processes, or produce verifiable side effects, requires_external_effect should be true. + A single task is valid for simple requests. + """; + + private static string BuildReplanInstruction(TaskGraph existing, string reason) + { + var sb = new System.Text.StringBuilder(); + sb.AppendLine("You are a Task Planner performing replan. The previous plan encountered an issue."); + sb.AppendLine($"Replan reason: {reason}"); + sb.AppendLine("Current task states:"); + foreach (PlannerTask t in existing.Tasks) + { + sb.AppendLine($" - [{t.State}] {t.Description} (requires_external_effect={t.RequiresExternalEffect})"); + } + sb.AppendLine(); + sb.AppendLine("Completed tasks must be preserved. Modify only incomplete/failed tasks."); + sb.AppendLine("Return a complete revised plan including completed tasks."); + return sb.ToString(); + } +} \ No newline at end of file -- cgit v1.2.3