Skip to content

Post v2: Replace bool? in Command System with CommandOutcome #4749

@tig

Description

@tig

The command system currently uses bool? throughout for return values:

  • null = no handler found / not raised (continue processing)
  • false = event raised but not handled/cancelled (continue processing)
  • true = event raised and handled/cancelled (stop processing)

This is error-prone and undocumented at each call site. CommandOutcome (already defined in
Terminal.Gui/Input/CommandOutcome.cs) makes the intent explicit:

public enum CommandOutcome
{
    NotHandled,       // was null  — routing continues
    HandledStop,      // was true  — routing stops
    HandledContinue,  // was false — handled, routing may continue
}

## Scope

The migration touches every layer of the command system:

| Layer | What Changes | Count |
|-------|-------------|-------|
| **Delegate** | `CommandImplementation` return type: `bool?` → `CommandOutcome` | 1 |
| **Public API** | `InvokeCommand` / `InvokeCommands` / `InvokeCommandsBound*` return types | ~10 methods |
| **CWP Pipeline** | `RaiseActivating` / `RaiseAccepting` / `RaiseHandlingHotKey` return types | 3 methods |
| **Handler implementations** | All `AddCommand` handler bodies | 334 call sites across 46 files |
| **Default handlers** | `DefaultActivateHandler`, `DefaultAcceptHandler`, `DefaultHotKeyHandler`, `DefaultCommandNotBoundHandler` | 4 methods |
| **Return value checks** | All `is true` / `== true` / `is false` / `is null` checks on invoke results | ~65 variable sites |
| **KeyboardImpl** | Parallel command infrastructure (`InvokeCommand`, `AddCommand`, `InvokeCommandsBoundToKey`) | 3 methods + 8 handlers |
| **IKeyboard interface** | `InvokeCommandsBoundToKey` / `InvokeCommand` return types | 2 methods |
| **Tests** | `bool? ret = view.InvokeCommand(...)` and assertions | ~35 test files |
| **Conversion shims** | `CommandOutcomeExtensions.ToBool()` / `ToOutcome()` — remove or keep for edge cases | 1 file |

## Approach

### Step 1: Change the Delegate and Core API

**File: `Terminal.Gui/ViewBase/View.Command.cs`**

```csharp
// BEFORE
public delegate bool? CommandImplementation (ICommandContext? ctx);

public bool? InvokeCommand (Command command, ICommandBinding? binding) => ...
public bool? InvokeCommand (Command command, ICommandContext? ctx) => ...
public bool? InvokeCommand (Command command) => ...
public bool? InvokeCommands (Command[] commands, ICommandBinding? binding) => ...

protected bool? RaiseActivating (ICommandContext? ctx) => ...
protected bool? RaiseAccepting (ICommandContext? ctx) => ...
protected bool? RaiseHandlingHotKey (ICommandContext? ctx) => ...

protected void AddCommand (Command command, CommandImplementation impl) => ...
protected void AddCommand (Command command, Func<bool?> impl) => ...

// AFTER
public delegate CommandOutcome CommandImplementation (ICommandContext? ctx);

public CommandOutcome InvokeCommand (Command command, ICommandBinding? binding) => ...
public CommandOutcome InvokeCommand (Command command, ICommandContext? ctx) => ...
public CommandOutcome InvokeCommand (Command command) => ...
public CommandOutcome InvokeCommands (Command[] commands, ICommandBinding? binding) => ...

protected CommandOutcome RaiseActivating (ICommandContext? ctx) => ...
protected CommandOutcome RaiseAccepting (ICommandContext? ctx) => ...
protected CommandOutcome RaiseHandlingHotKey (ICommandContext? ctx) => ...

protected void AddCommand (Command command, CommandImplementation impl) => ...
protected void AddCommand (Command command, Func<CommandOutcome> impl) => ...

File: Terminal.Gui/App/Keyboard/IKeyboard.cs

// BEFORE
bool? InvokeCommandsBoundToKey (Key key);
bool? InvokeCommand (Command command, Key key, KeyBinding binding);

// AFTER
CommandOutcome InvokeCommandsBoundToKey (Key key);
CommandOutcome InvokeCommand (Command command, Key key, KeyBinding binding);

File: Terminal.Gui/App/Keyboard/KeyboardImpl.cs

Same changes as IKeyboard, plus the private AddCommand and the _commandImplementations dictionary.

Step 2: Update Return Value Semantics

Everywhere in the command system that returns or checks bool?:

Old Pattern New Pattern
return true; return CommandOutcome.HandledStop;
return false; return CommandOutcome.HandledContinue;
return null; return CommandOutcome.NotHandled;
if (result is true) if (result == CommandOutcome.HandledStop)
if (result == true) if (result == CommandOutcome.HandledStop)
if (result is false) if (result == CommandOutcome.HandledContinue)
if (result is null) if (result == CommandOutcome.NotHandled)
bool? ret = InvokeCommand(...) CommandOutcome ret = InvokeCommand(...)
return someCondition; (bool expression) return someCondition ? CommandOutcome.HandledStop : CommandOutcome.HandledContinue;

Boolean expression returns need careful mapping. The existing convention is:

  • A true boolean expression → HandledStop
  • A false boolean expression → HandledContinue

Step 3: Update Callers That Convert to bool

Several places assign InvokeCommand results to e.Handled or args.Handled:

// BEFORE
e.Handled = Parent.InvokeCommand (Command.Quit) == true;
args.Handled = InvokeCommandsBoundToMouse (args) == true;

// AFTER
e.Handled = Parent.InvokeCommand (Command.Quit) == CommandOutcome.HandledStop;
args.Handled = InvokeCommandsBoundToMouse (args) == CommandOutcome.HandledStop;

Step 4: Decide on Conversion Shims

The CommandOutcomeExtensions.ToBool() and ToOutcome() methods in CommandOutcome.cs can be:

  • Removed if no external consumers need them (since backward compat is not an issue)
  • Kept if they serve as documentation or are useful for test assertions

Recommendation: Keep but mark [Obsolete] — they document the mapping and may be useful
during review. Remove in a follow-up cleanup pass.


Files to Modify

Core Command Infrastructure

These files define the command pipeline and must change first:

File Changes
Terminal.Gui/Input/CommandOutcome.cs Optionally mark shims [Obsolete]
Terminal.Gui/ViewBase/View.Command.cs Delegate, all Invoke/Raise methods, default handlers, AddCommand overloads, TryBubbleUp, TryDispatchToTarget, DispatchDown (~1,043 lines, ~13 AddCommand sites)
Terminal.Gui/App/Keyboard/IKeyboard.cs Interface method return types (2 methods)
Terminal.Gui/App/Keyboard/KeyboardImpl.cs InvokeCommand, InvokeCommandsBoundToKey, AddCommand, handler implementations (8 handlers)
Terminal.Gui/Input/CommandBridge.cs Handler lambdas that call InvokeCommand (2 sites)
Terminal.Gui/Input/ICommandContext.cs If it has any bool? references (1 site)

View Keyboard/Mouse Integration

These files call InvokeCommand and check return values:

File Changes
Terminal.Gui/ViewBase/View.Keyboard.cs InvokeCommandsBoundToKey, InvokeCommandsBoundToHotKey return types + checks
Terminal.Gui/ViewBase/Mouse/View.Mouse.cs InvokeCommandsBoundToMouse return type + == true checks
Terminal.Gui/ViewBase/Adornment/Border.cs InvokeCommand (Command.Quit) == true check
Terminal.Gui/ViewBase/Adornment/Border.Arrangment.cs Handler implementations (7 AddCommand sites)
Terminal.Gui/App/ApplicationPopover.cs InvokeCommand (Command.Quit) is true check
Terminal.Gui/App/PopoverBaseImpl.cs Handler implementation (1 AddCommand site)
Terminal.Gui/Views/Label.cs InvokeCommand (Command.HotKey) == true check

View Handler Implementations (AddCommand Sites)

Migrate handlers in each file — sorted by call count, largest last:

File AddCommand Sites
Terminal.Gui/Views/Menu/MenuBarItem.cs 1
Terminal.Gui/Views/NumericUpDown.cs 2
Terminal.Gui/Views/Menu/PopoverMenu.cs 3
Terminal.Gui/Views/Menu/MenuBar.cs 4
Terminal.Gui/Views/Selectors/SelectorBase.cs 4
Terminal.Gui/Views/Color/ColorBar.cs 6
Terminal.Gui/Views/Color/ColorPicker.16.cs 6
Terminal.Gui/Views/GraphView/GraphView.cs 6
Terminal.Gui/Views/TextInput/DateField.cs 6
Terminal.Gui/Views/TextInput/TextValidateField.cs 6
Terminal.Gui/Views/TextInput/TimeField.cs 6
Terminal.Gui/Views/LinearRange/LinearRange.cs 8
Terminal.Gui/Views/TabView/TabView.cs 8
Terminal.Gui/Views/ComboBox.cs 12
Terminal.Gui/Views/CharMap/CharMap.cs 13
Terminal.Gui/Views/HexView.cs 17
Terminal.Gui/Views/ListView/ListView.Commands.cs 17
Terminal.Gui/Views/TreeView/TreeView.cs 19
Terminal.Gui/Views/TableView/TableView.cs 24
Terminal.Gui/Views/TextInput/TextField/TextField.Commands.cs 29
Terminal.Gui/Views/TextInput/TextView/TextView.Commands.cs 46

Examples/UICatalog

File AddCommand Sites
Examples/UICatalog/Scenarios/ContextMenus.cs 1
Examples/UICatalog/Scenarios/Editor.cs 1
Examples/UICatalog/Scenarios/EditorsAndHelpers/EditorBase.cs 1
Examples/UICatalog/Scenarios/EditorsAndHelpers/EventLog.cs 1
Examples/UICatalog/Scenarios/Mazing.cs 1
Examples/UICatalog/Scenarios/EditorsAndHelpers/ExpanderButton.cs 3
Examples/UICatalog/Scenarios/KeyBindings.cs 4
Examples/UICatalog/Scenarios/Menus.cs 4
Examples/UICatalog/Scenarios/Snake.cs 4
Examples/UICatalog/Scenarios/ViewportSettings.cs 4
Examples/UICatalog/Scenarios/EditorsAndHelpers/AllViewsView.cs 10
Examples/UICatalog/Scenarios/EditorsAndHelpers/ThemeViewer.cs 10

Tests

Tests that capture bool? from InvokeCommand or assert against it:

File Notes
Tests/UnitTestsParallelizable/ViewBase/ViewCommandTests.cs Core command tests — bool? ret = ...
Tests/UnitTestsParallelizable/ViewBase/Keyboard/KeyboardEventTests.cs InvokeCommands_Returns_Nullable_Properly test + AddCommand sites
Tests/UnitTestsParallelizable/ViewBase/Keyboard/KeyBindingsTests.cs AddCommand sites (3)
Tests/UnitTestsParallelizable/Application/Keyboard/KeyboardTests.cs bool? result = keyboard.InvokeCommandsBoundToKey(...)
Tests/UnitTestsParallelizable/Application/Keyboard/KeyboardImplThreadSafetyTests.cs AddCommand sites (2)
Tests/UnitTestsParallelizable/Views/ButtonTests.cs bool? ret = button.InvokeCommand(...)
Tests/UnitTestsParallelizable/Views/CheckBoxTests.cs bool? ret/result = ckb.InvokeCommand(...)
Tests/UnitTestsParallelizable/Views/ShortcutTests.Command.cs AddCommand sites (2)
Tests/UnitTestsParallelizable/Views/MenuItemTests.cs AddCommand sites (2)
Tests/UnitTestsParallelizable/Views/AllViewsTests.cs InvokeCommand(...) == true checks
Tests/UnitTestsParallelizable/Views/ColorPickerTests.cs bool? result = picker.InvokeCommand(...)
Tests/UnitTestsParallelizable/Application/Popover/Application.PopoverTests.cs AddCommand sites (2)
Tests/UnitTests/Application/ApplicationPopoverTests.cs AddCommand site (1)
Tests/UnitTests/Views/CheckBoxTests.cs bool? ret = ckb.InvokeCommand(...)
Plus ~20 more test files That call InvokeCommand and check/assert results

Tricky Patterns to Watch For

1. Boolean Expression Returns

Many handlers end with a boolean expression:

// View.Command.cs DefaultAcceptHandler line 314
return redirected || acceptWillBubble || ctx?.Routing == CommandRouting.BubblingUp || this is IAcceptTarget;

This needs a helper or explicit ternary:

return (redirected || acceptWillBubble || ctx?.Routing == CommandRouting.BubblingUp || this is IAcceptTarget)
    ? CommandOutcome.HandledStop
    : CommandOutcome.HandledContinue;

Consider adding an extension method:

public static CommandOutcome ToOutcome (this bool handled)
    => handled ? CommandOutcome.HandledStop : CommandOutcome.HandledContinue;

This keeps boolean-expression returns clean:

return (redirected || acceptWillBubble || ...).ToOutcome ();

2. Nullable Method Chains

// KeyboardImpl.cs line 278
AddCommand (Command.NextTabStop, () => App?.Navigation?.AdvanceFocus (...));

AdvanceFocus returns bool?. With the new system, this needs to either:

  • Change AdvanceFocus to return CommandOutcome (if it's part of the command chain)
  • Or wrap: () => (App?.Navigation?.AdvanceFocus (...)).ToOutcome ()

Check whether AdvanceFocus is only used from command handlers. If so, migrate it too.

3. Aggregation Logic in InvokeCommands

// View.Command.cs InvokeCommands (simplified)
bool? toReturn = null;
foreach (Command command in commands)
{
    bool? thisReturn = InvokeCommand (command, binding);
    if (thisReturn is { })
    {
        toReturn = thisReturn;
    }
    if (thisReturn is true) break;
}
return toReturn;

With CommandOutcome, the null-coalescence logic changes:

CommandOutcome toReturn = CommandOutcome.NotHandled;
foreach (Command command in commands)
{
    CommandOutcome thisReturn = InvokeCommand (command, binding);
    if (thisReturn != CommandOutcome.NotHandled)
    {
        toReturn = thisReturn;
    }
    if (thisReturn == CommandOutcome.HandledStop) break;
}
return toReturn;

4. The Handled Property on EventArgs

CommandEventArgs.Handled is bool, not bool?. Places that map InvokeCommand results to
args.Handled = true will use:

args.Handled = result == CommandOutcome.HandledStop;

5. Test Assertions

Tests that assert Assert.True (ret) or Assert.False (ret) need:

Assert.Equal (CommandOutcome.HandledStop, ret);
Assert.Equal (CommandOutcome.HandledContinue, ret);
Assert.Equal (CommandOutcome.NotHandled, ret);

Or with a helper: Assert.True (ret.IsHandled ()) if we add such an extension.


Recommended Migration Order

  1. Core types first — Change CommandImplementation delegate, CommandOutcome.cs (add ToOutcome(bool) helper), ICommandContext.cs
  2. View.Command.cs — All Invoke/Raise methods + default handlers (this is the heart)
  3. View.Keyboard.cs + View.Mouse.cs — InvokeBound methods + return checks
  4. KeyboardImpl + IKeyboard — Parallel command infrastructure
  5. CommandBridge — Handler lambdas
  6. Callers — Border.cs, ApplicationPopover.cs, Label.cs, PopoverBaseImpl.cs
  7. View handlers (small → large) — MenuBarItem (1) through TextView (46)
  8. Examples — All UICatalog scenarios
  9. Tests — All test files
  10. Cleanup — Remove or mark obsolete ToBool()/ToOutcome() shims

Build Verification

After each group (1-2 through 9), verify:

dotnet build --no-restore

After all groups complete:

dotnet test Tests/UnitTestsParallelizable --no-build
dotnet test Tests/UnitTests --no-build

Summary Statistics

Category Count
Core API methods changing return type ~16
Handler implementations (AddCommand sites) 334 across 46 files
Return value check sites ~65
Test files affected ~35
Total estimated sites ~450

The migration is mechanical — the three-way mapping (trueHandledStop, falseHandledContinue,
nullNotHandled) is deterministic. The only judgment calls are boolean-expression returns
(pattern #1 above) where the true/false semantics must be verified against the surrounding code.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    Status

    No status

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions