Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.

Add mouse button support to the macOS shell #9054

Merged
stuartmorgan-g merged 4 commits into from
May 22, 2019
Merged

Add mouse button support to the macOS shell #9054

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1c5aa5f
Add mouse button support to the macOS shell
stuartmorgan-g May 21, 2019
fa8410e
Typo fix
stuartmorgan May 22, 2019
0daf7db
Reformat after typo fix
stuartmorgan-g May 22, 2019
04cdefd
Merge branch 'macos-mouse-buttons' of https://github.com/stuartmorgan…
stuartmorgan-g May 22, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
144 changes: 127 additions & 17 deletions shell/platform/darwin/macos/framework/Source/FLEViewController.mm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,52 @@

static const int kDefaultWindowFramebuffer = 0;

namespace {

/**
* State tracking for mouse events, to adapt between the events coming from the system and the
* events that the embedding API expects.
*/
struct MouseState {
/**
* Whether or not a kAdd event has been sent (or sent again since the last kRemove if tracking is
* enabled). Used to determine whether to send a kAdd event before sending an incoming mouse
* event, since Flutter expects pointers to be added before events are sent for them.
*/
bool flutter_state_is_added = false;

/**
* Whether or not a kDown has been sent since the last kAdd/kUp.
*/
bool flutter_state_is_down = false;

/**
* Wether or not mouseExited: was received while a button was down. Cocoa's behavior when dragging
* out of a tracked area is to send an exit, then keep sending drag events until the last button
* is released. If it was released inside the view, mouseEntered: is sent the next time the mouse
* moves. Flutter doesn't expect to receive events after a kRemove, so the kRemove for the exit
* needs to be delayed until after the last mouse button is released.
*/
Copy link
Contributor

@dkwingsmt dkwingsmt May 22, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's see if my understanding is correct:

  • If a mouse is pressed in the tracking area, then dragged out (and keeps moving), Cocoa will keep sending drag events with out-of-area positions.
  • If the mouse is then dragged back in without releasing buttons, it will send a mouseEntered while keeping sending drag events.

If the statements above are correct,

  • Should we stop sending drag events with out-of-area positions to Flutter, since they should be considered illegal data? Also perhaps to accommodate shells that can not detect out-of-area moves.
  • Should we set _mouseState.has_pending_exit to false in mouseEntered:, so that the "drag back in" action is considered a normal drag and does not trigger Reset?

Copy link
Contributor Author

@stuartmorgan-g stuartmorgan-g May 22, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a mouse is pressed in the tracking area, then dragged out (and keeps moving), Cocoa will keep sending drag events with out-of-area positions.

Yes.

If the mouse is then dragged back in without releasing buttons, it will send a mouseEntered while keeping sending drag events.

No, there's no mouseEntered; it's weirdly asymmetrical. (In that scenario, you don't get the mouseEntered until the next time you move the mouse after releasing the button.)

Should we stop sending drag events with out-of-area positions to Flutter

Let's revisit this later, since the patch isn't changing that behavior and we'll need to carefully think about what exactly the right behavior is in the context of yet-to-be-designed functionality like drag-and-drop from Flutter to other apps. (It's not as simple as just not sending events, because we don't want to drop the pointer if Flutter knows a drag is happening in that scenario, but we also don't want, say, an in-window drop target at the edge of the window to stay highlighted.)

Should we set _mouseState.has_pending_exit to false in mouseEntered:

No (per answer 2 above)

bool has_pending_exit = false;

/**
* The currently pressed buttons, as represented in FlutterPointerEvent.
*/
int64_t buttons = 0;

/**
* Resets all state to default values.
*/
void Reset() {
flutter_state_is_added = false;
flutter_state_is_down = false;
has_pending_exit = false;
buttons = 0;
}
};

} // namespace

#pragma mark - Private interface declaration.

/**
Expand All @@ -34,12 +80,9 @@ @interface FLEViewController ()
@property(nonatomic) NSTrackingArea* trackingArea;

/**
* Whether or not a kAdd event has been sent for the mouse (or sent again since
* the last kRemove was sent if tracking is enabled). Used to determine whether
* to send an Add event before sending an incoming mouse event, since Flutter
* expects a pointers to be added before events are sent for them.
* The current state of the mouse and the sent mouse events.
*/
@property(nonatomic) BOOL mouseCurrentlyAdded;
@property(nonatomic) MouseState mouseState;

/**
* Updates |trackingArea| for the current tracking settings, creating it with
Expand Down Expand Up @@ -81,6 +124,13 @@ - (void)makeResourceContextCurrent;
*/
- (void)handlePlatformMessage:(const FlutterPlatformMessage*)message;

/**
* Calls dispatchMouseEvent:phase: with a phase determined by self.mouseState.
*
* mouseState.buttons should be updated before calling this method.
*/
- (void)dispatchMouseEvent:(nonnull NSEvent*)event;

/**
* Converts |event| to a FlutterPointerEvent with the given phase, and sends it to the engine.
*/
Expand Down Expand Up @@ -447,10 +497,24 @@ - (void)handlePlatformMessage:(const FlutterPlatformMessage*)message {
}
}

- (void)dispatchMouseEvent:(nonnull NSEvent*)event {
FlutterPointerPhase phase = _mouseState.buttons == 0
? (_mouseState.flutter_state_is_down ? kUp : kHover)
: (_mouseState.flutter_state_is_down ? kMove : kDown);
[self dispatchMouseEvent:event phase:phase];
}

- (void)dispatchMouseEvent:(NSEvent*)event phase:(FlutterPointerPhase)phase {
// There are edge cases where the system will deliver enter out of order relative to other
// events (e.g., drag out and back in, release, then click; mouseDown: will be called before
// mouseEntered:). Discard those events, since the add will already have been synthesized.
if (_mouseState.flutter_state_is_added && phase == kAdd) {
return;
}

// If a pointer added event hasn't been sent, synthesize one using this event for the basic
// information.
if (!_mouseCurrentlyAdded && phase != kAdd) {
if (!_mouseState.flutter_state_is_added && phase != kAdd) {
// Only the values extracted for use in flutterEvent below matter, the rest are dummy values.
NSEvent* addEvent = [NSEvent enterExitEventWithType:NSEventTypeMouseEntered
location:event.locationInWindow
Expand All @@ -468,10 +532,13 @@ - (void)dispatchMouseEvent:(NSEvent*)event phase:(FlutterPointerPhase)phase {
NSPoint locationInBackingCoordinates = [self.view convertPointToBacking:locationInView];
FlutterPointerEvent flutterEvent = {
.struct_size = sizeof(flutterEvent),
.device_kind = kFlutterPointerDeviceKindMouse,
.phase = phase,
.x = locationInBackingCoordinates.x,
.y = -locationInBackingCoordinates.y, // convertPointToBacking makes this negative.
.timestamp = static_cast<size_t>(event.timestamp * NSEC_PER_MSEC),
// If a click triggered a synthesized kAdd, don't pass the buttons in that event.
.buttons = phase == kAdd ? 0 : _mouseState.buttons,
};

if (event.type == NSEventTypeScrollWheel) {
Expand All @@ -491,10 +558,19 @@ - (void)dispatchMouseEvent:(NSEvent*)event phase:(FlutterPointerPhase)phase {
}
FlutterEngineSendPointerEvent(_engine, &flutterEvent, 1);

if (phase == kAdd) {
_mouseCurrentlyAdded = YES;
// Update tracking of state as reported to Flutter.
if (phase == kDown) {
_mouseState.flutter_state_is_down = true;
} else if (phase == kUp) {
_mouseState.flutter_state_is_down = false;
if (_mouseState.has_pending_exit) {
[self dispatchMouseEvent:event phase:kRemove];
_mouseState.has_pending_exit = false;
}
} else if (phase == kAdd) {
_mouseState.flutter_state_is_added = true;
} else if (phase == kRemove) {
_mouseCurrentlyAdded = NO;
_mouseState.Reset();
}
}

Expand Down Expand Up @@ -622,28 +698,62 @@ - (void)keyUp:(NSEvent*)event {
}
}

- (void)mouseEntered:(NSEvent*)event {
[self dispatchMouseEvent:event phase:kAdd];
}

- (void)mouseExited:(NSEvent*)event {
if (_mouseState.buttons != 0) {
_mouseState.has_pending_exit = true;
return;
}
[self dispatchMouseEvent:event phase:kRemove];
}

- (void)mouseDown:(NSEvent*)event {
[self dispatchMouseEvent:event phase:kDown];
_mouseState.buttons |= kFlutterPointerButtonMousePrimary;
[self dispatchMouseEvent:event];
}

- (void)mouseUp:(NSEvent*)event {
[self dispatchMouseEvent:event phase:kUp];
_mouseState.buttons &= ~static_cast<uint64_t>(kFlutterPointerButtonMousePrimary);
[self dispatchMouseEvent:event];
}

- (void)mouseDragged:(NSEvent*)event {
[self dispatchMouseEvent:event phase:kMove];
[self dispatchMouseEvent:event];
}

- (void)mouseEntered:(NSEvent*)event {
[self dispatchMouseEvent:event phase:kAdd];
- (void)rightMouseDown:(NSEvent*)event {
_mouseState.buttons |= kFlutterPointerButtonMouseSecondary;
[self dispatchMouseEvent:event];
}

- (void)mouseExited:(NSEvent*)event {
[self dispatchMouseEvent:event phase:kRemove];
- (void)rightMouseUp:(NSEvent*)event {
_mouseState.buttons &= ~static_cast<uint64_t>(kFlutterPointerButtonMouseSecondary);
[self dispatchMouseEvent:event];
}

- (void)rightMouseDragged:(NSEvent*)event {
[self dispatchMouseEvent:event];
}

- (void)otherMouseDown:(NSEvent*)event {
_mouseState.buttons |= (1 << event.buttonNumber);
[self dispatchMouseEvent:event];
}

- (void)otherMouseUp:(NSEvent*)event {
_mouseState.buttons &= ~static_cast<uint64_t>(1 << event.buttonNumber);
[self dispatchMouseEvent:event];
}

- (void)otherMouseDragged:(NSEvent*)event {
[self dispatchMouseEvent:event];
}

- (void)mouseMoved:(NSEvent*)event {
[self dispatchMouseEvent:event phase:kHover];
[self dispatchMouseEvent:event];
}

- (void)scrollWheel:(NSEvent*)event {
Expand Down