[core] Mod-tap key presses not registered properly (PRESS-RELEASE in single frame)

[dmlary]

Please, before submitting a new issue verify and check:

• I tested it on latest raylib version from master branch
• I checked there is no similar issue already reported
• My code has no errors or misuse of raylib

Issue description

Very fast key presses not registering in IsKeyPressed(), but do show up in GetKeyPressed().

I've tracked this issue down to using a keyboard with QMK firmware, using mod-tap. When tapping the spacebar, I get both the KeyDown & KeyUp event in the same frame. This key pressed event isn't registered when I check IsKeyPressed(), but does show up when I call GetKeyPressed().

It exhibits most clearly when trying to play the Pang game on the raylib website: https://www.raylib.com/games/classics/loader.html?name=classic_pang

Environment

This happens both natively and for the raylib web-based demos.

• Platforms: Web & Desktop
• OS: OSX 11.7
• OpenGL:

INFO: GL: OpenGL device information:
INFO:     > Vendor:   ATI Technologies Inc.
INFO:     > Renderer: AMD Radeon Pro 5500M OpenGL Engine
INFO:     > Version:  4.1 ATI-4.6.21
INFO:     > GLSL:     4.10

Issue Screenshot

Screenshot of logging messages for code snippet below.

Top part of the log is me hitting space on laptop keyboard, bottom is me hitting space on external keyboard running QMK firmware and mod-tap on spacebar.

Code Example

minimal code logging discrepancy between IsKeyPressed() and GetKeyPressed():

                PollInputEvents();
                if (IsKeyPressed(KEY_SPACE)) {
                    log_debug("space pressed");
                }
                for (int k = GetKeyPressed(); k != 0; k = GetKeyPressed()) {
                    log_debug("pressed {}", k);
                }

[raysan5]

@dmlary Yes, that's as expected. IsKeyPressed() consumes key polled and polling happens per frame while GetKeyPressed() accumulates key presses in an internal queue. At 60 fps it shouldn't be a big issue.

In any case, for more fine-grained control of key polling mechanisms raylib allows SUPPORT_CUSTOM_FRAME_CONTROL as a config flag, so, advance users can manage the key polling themselfs as desired.

[dmlary]

To make sure this is clear, it does not work at 60fps. The keyboard itself sends both the key up and down within a single frame, every time. So the game never registers the key press.

Attempting to play the raylib demo games in this state makes most of them unplayable.

I understand this is an edge case, but anyone using raylib for their game will encounter some small percentage of users who have random keys that appear not to work.

At the very least, it’s documented now.

[raysan5]

@dmlary I can't reproduce it with my keyboard, it works as expected for me... 🤔

[dmlary]

As I said in the issue description the problem is due to me using a keyboard running QMK firmware with mod-tap enabled.

https://qmk.github.io/qmk_mkdocs/master/en/mod_tap/

This is a feature that sends quick (< 1ms delay between) key up/down events when the key is tapped, but a different key code when held.

When I was evaluating raylib for use, I was struggling to understand why so many of the demos were so unresponsive to input, even when I built them locally. This is what it came down to.

Once again, I acknowledge this is an edge case, but anyone trying to play games built on raylib with mod-tap enabled is going to have a similar experience.

Thinking through the input code, is there any reason why an up/down in a single frame doesn’t register as KeyPressed? I think this question is asking about transitioning from KeyUp priority (If it’s up by the end of the frame, it wasn’t down during), to the opposite (if it was ever down since the last frame it was down). The gameplay difference is a maximum of one frame of lag before registering that the key is no longer pressed.

This would eliminate the issue of quick key presses, but I’m not confident it wouldn’t break something else in the input handling.

[raysan5]

@dmlary Ok, thanks for the detailed explanation, I didn't know about that mod-tap feature of some keyboards.

This issue is an edge case and platform-specific (in this case keyboard-specific) so supporting it properly would require specific code for this case, increasing the complexity (and maintainability) of the sources; also I don't know how many users could have this specific requirement, in 9 years noone had reported this issue... Maybe it is a modern keyboards feature?

I'm thinking about possible simple and not too invasive solutions but the GLFW callback mechanism provided to register keys is the following:

// GLFW3 Keyboard Callback, runs on key pressed
static void KeyCallback(GLFWwindow *window, int key, int scancode, int action, int mods)
{
    // WARNING: GLFW could return GLFW_REPEAT, we need to consider it as 1
    // to work properly with our implementation (IsKeyDown/IsKeyUp checks)
    if (action == GLFW_RELEASE) CORE.Input.Keyboard.currentKeyState[key] = 0;
    else CORE.Input.Keyboard.currentKeyState[key] = 1;
}

So, if I understand it correctly, when this callback is executed the received action is already GLFW_RELEASE but a GLFW_PRESS has also previously occurred in the same frame, is that correct?

[dmlary]

With the patch inlined below, I get logs showing both GLFW_PRESS and GLFW_RELEASE happening in the same millisecond. Not every one of them, but more than half:

2022-12-14T17:08:10.281 key 32, scancode 49, action 1, mods 0
2022-12-14T17:08:10.281 key 32, scancode 49, action 0, mods 0
2022-12-14T17:08:10.710 key 32, scancode 49, action 1, mods 0
2022-12-14T17:08:10.711 key 32, scancode 49, action 0, mods 0
2022-12-14T17:08:11.177 key 32, scancode 49, action 1, mods 0
2022-12-14T17:08:11.178 key 32, scancode 49, action 0, mods 0
2022-12-14T17:08:11.647 key 32, scancode 49, action 1, mods 0
2022-12-14T17:08:11.647 key 32, scancode 49, action 0, mods 0
2022-12-14T17:08:12.172 key 32, scancode 49, action 1, mods 0
2022-12-14T17:08:12.173 key 32, scancode 49, action 0, mods 0
2022-12-14T17:08:12.627 key 32, scancode 49, action 1, mods 0
2022-12-14T17:08:12.628 key 32, scancode 49, action 0, mods 0
2022-12-14T17:08:13.026 key 32, scancode 49, action 1, mods 0
2022-12-14T17:08:13.026 key 32, scancode 49, action 0, mods 0
2022-12-14T17:08:13.499 key 32, scancode 49, action 1, mods 0
2022-12-14T17:08:13.499 key 32, scancode 49, action 0, mods 0
2022-12-14T17:08:13.970 key 32, scancode 49, action 1, mods 0
2022-12-14T17:08:13.970 key 32, scancode 49, action 0, mods 0
2022-12-14T17:08:14.426 key 32, scancode 49, action 1, mods 0
2022-12-14T17:08:14.426 key 32, scancode 49, action 0, mods 0

diff --git a/src/rcore.c b/src/rcore.c
index 7acedf6c..6995db33 100644
--- a/src/rcore.c
+++ b/src/rcore.c
@@ -5262,11 +5262,31 @@ static void WindowFocusCallback(GLFWwindow *window, int focused)
     else CORE.Window.flags |= FLAG_WINDOW_UNFOCUSED;            // The window lost focus
 }

+#include <sys/time.h>
+static void
+debug(const char *fmt, ...)
+{
+  struct timeval tv;
+  char timestamp[32];
+  va_list args;
+
+  /* write the timestamp with milliseconds */
+  gettimeofday(&tv, NULL);
+  strftime(timestamp, sizeof(timestamp), "%FT%T", localtime(&tv.tv_sec));
+  printf("%s.%03ld ", timestamp, tv.tv_usec/1000L);
+
+  va_start(args, fmt);
+  vprintf(fmt, args);
+  va_end(args);
+}
+
 // GLFW3 Keyboard Callback, runs on key pressed
 static void KeyCallback(GLFWwindow *window, int key, int scancode, int action, int mods)
 {
     if (key < 0) return;    // Security check, macOS fn key generates -1

+    debug("key %d, scancode %d, action %d, mods %d\n",
+            key, scancode, action, mods);
     // WARNING: GLFW could return GLFW_REPEAT, we need to consider it as 1
     // to work properly with our implementation (IsKeyDown/IsKeyUp checks)
     if (action == GLFW_RELEASE) CORE.Input.Keyboard.currentKeyState[key] = 0;

[raysan5]

@dmlary As per that output, both events (key-down + key-up) are processed in one single frame...

I reopen the issue for further review but at a first glance I can't see a simple solution for this specific situation, it would require a complete redesign of the implementation with additional variables... that's out of scope for raylib at the moment...

[raysan5]

@actondev Could this issue be somewhat addressed with newly available key-repeat mechanism?

@n77y commented here:

a user could without too much difficulty get around this issue by GetKeyPressed() because when a key is released it does not remove it from the queue

For me this is a good-enough solution considering this very specific use case, could you provide a simple examples?

[actondev]

Hi @raysan5 ! Yes, I mentioned in my comment as well #3245 (comment) . Note that his can happen not only with keys but with mouse buttons (as I mentioned it happens with my macbook touchpad a lot!)
From the top of my head we'd internally store a {Key/MouseButton}PressedInFrame which would get ored everytime we get a key/mouseButton press from the os/glfw & cleared at the end of the frame. What I haven't thought thoroughly though is

• what about key release? same logic as key press I assume?
• Does this mean we can remove the prevKeyState completely since we now don't implement the press logic ourselves but get the events from os? Or do we use the os key press info in a complementary way only?