Enhanced UI Core Overview

What this covers

  1. What Enhanced UI is solving
  2. The core building blocks
  3. How interaction flows through the system
  4. The geometry model (conceptual)
  5. How rendering works
  6. How you extend the system
Download Enhanced UI

What Enhanced UI Solves

Enhanced UI provides a structured way to build interactive UIs in Unreal where widgets can:

  • Move
  • Resize
  • Respond to user input
  • Represent underlying data

The key problem it solves is:

How to separate input detection, behavior, and layout state so complex UI systems remain maintainable.

Core Building Blocks

There are three core components:

1. SmartWidget (UAxSmartWidget)

  • Detects user interaction (click, hover, drag)
  • Identifies where interaction happened (edges, corners, center)
  • Sends that information to its parent

Important constraint:

SmartWidgets do not move or resize themselves.

They are purely input

Where to look in code

  • Entry point for input:
    • NativeOnMouseButtonDown()
    • NativeOnMouseMove()
  • Handle detection:
    • DetectHandleForMouseEvent()
  • Event forwarding:
    • BroadcastMouseDownForHandle()
    • BroadcastMouseHoverForHandle()

reporters.

2. SmartCanvas (UAxSmartCanvasPanel)

  • Owns all behavior (movement, resizing, constraints)
  • Receives interaction events from SmartWidgets
  • Decides what should happen

Examples of behavior:

  • Move widget
  • Resize widget
  • Clamp within bounds
  • Apply constraints (in derived classes)

Where to look in code

  • Event entry from child:
    • HandleSmartChildMouseDown()
    • HandleSmartChildMouseUp()
  • Core interaction logic:
    • HandleMouseMove()
  • Geometry application:
    • UpdateSmartChildWidgetGeometry()

3. Slate Canvas (SAxSmartCanvas)

  • Handles rendering and low-level input bridging
  • Calls into SmartCanvas during paint and input events

This layer exists to connect UMG logic to Slate rendering.

Where to look in code

  • Paint entry:
    • OnPaint() → calls HandleOnPaint()
  • Input forwarding:
    • OnMouseButtonDown()
    • OnMouseMove()

Interaction Flow (Conceptual)

The system follows a strict flow:

				
					User interacts with Widget
        ↓
SmartWidget detects interaction zone
        ↓
Payload is created (what + where)
        ↓
SmartCanvas receives it
        ↓
SmartCanvas decides behavior
        ↓
Geometry is updated
        ↓
UI updates during paint

This separation is what allows complex systems to remain predictable.

Where to trace this in code

  1. Input starts:
    • UAxSmartWidget::NativeOnMouseButtonDown()
  2. Payload created:
    • FAxSmartWidgetPayload
  3. Forwarded:
    • MouseDownEvent.Execute(...)
  4. Received:
    • UAxSmartCanvasPanel::HandleSmartChildMouseDown()
  5. Movement:
    • UAxSmartCanvasPanel::HandleMouseMove()

Geometry Model (Conceptual)

Each widget has a SmartGeometry object.

It represents:

  • Position
  • Size
  • Constraints

There are two modes:

Normalized Mode

  • Values between 0 and 1
  • Automatically stays inside the canvas
  • Scales with parent

Pixel Mode

  • Absolute values
  • No automatic constraints
  • Used for editor-style layouts

Where to look in code

  • Struct definition:
    • FAxSmartGeometry
  • Mutation functions:
    • SetStartAndUpdateSize()
    • SetStopAndUpdateSize()
    • SetStartAndUpdateStop()
    • SetSizeAndUpdateStop()

Rendering Model

Rendering is not immediate.

Instead:

  1. Interaction updates geometry (data only)
  2. During the paint phase:
    • Geometry is converted into real UI layout
    • Widgets are positioned and sized

Where to look in code

  • Paint entry:
    • SAxSmartCanvas::OnPaint()
  • Geometry application:
    • UAxSmartCanvasPanel::HandleOnPaint()
    • UpdateSmartChildWidgetGeometry()

Data Model

Widgets can represent external data.

Example:

  • A SmartWidget can represent a MIDI note, clip, or window

The system maintains:

 
Data Object → SmartWidget
 

This allows:

  • Fast lookup
  • Efficient updates
  • No duplicate widgets

Where to look in code

  • Mapping:
    • ObjectToSmartChildWidgetMap
  • Access point:
    • GetDataObject()
  • Registration:
    • AddAndRegisterSmartWidget()

Extensibility Model

Base Behavior

SmartCanvas provides:

  • Free movement
  • Resizing
  • Basic constraints

Derived Classes

Derived classes refine behavior, not replace structure.

Examples:

  • Piano roll → snapping + quantization
  • Timeline → axis-locked movement
  • Window system → free drag + resize

Base allows everything
Derived classes restrict or reinterpret

Where to look in code

  • Override points:
    • HandleMouseMove()
    • HandleSmartChildMouseDown()
  • Example pattern:
    • Call Super::HandleMouseMove() and react to geometry change
    • OR fully override behavior

Z-Order / Interaction Layering

The system supports bringing widgets to front during interaction.

Where to look in code

  • Function:
    • BringSmartChildWidgetToFront()
  • Recommended trigger:
    • HandleSmartChildMouseDown()