Double input along with latin accents #650

dstwwy · 2019-07-18T11:55:12Z

Hello.

I'm using the suggested keyboard app in #431 on my main phone and the stock version of LineageOS keyboard on my secondary phone, and both are able to type accented (áèîõü) characters as explained in https://blog.rom1v.com/2018/03/introducing-scrcpy/#handle-text-input.

However, when inputing [accent] then [key], scrcpy isn't able to recognize that I want a accented character only, so áá turns into aáaá. Is it possible to filter the output when I type an accent, like by just skiping the first output?

Thanks for the project, it was a lifechanger ever since its first release.

rom1v · 2019-07-18T12:47:43Z

To fix #87, since v1.4, scrcpy handles letters differently than other text (411aa4f). This is the cause of this bug.

I should probably manage several "input modes" and introduce a parameter to select it, because I think there is no solution working reliably for all cases. The modes would be:

raw key events only (ignore text events)
text events (ignore raw key events which produce text) (this was th behavior until 1.3, the one you want)
mixed (text events only for non-letters) (this is the current behavior, a compromise to fix Unable to use keyboard on Unreal Engine game #87 while working relatively good for text)

dstwwy · 2019-07-18T13:51:43Z

Started testing 1.3 and got a bunch of wildly varying results, before noticing that I was on Gboard. Thanks Google.

So far it behaves as you said on 1.3, so I'll use both according to my needs. As for your solution, it seems that every options brings issues, so I don't know if it's worth pursuing instead of waiting for a fix on how adb works (dunno if even possible) or something else.

rom1v · 2019-07-18T13:54:44Z

This is not a bug in adb, just a conflict between injecting text and injecting press+release events.

dstwwy · 2019-07-18T14:00:43Z

It's not caused by how adb works (#214)? Although you explained how you are handling accents on your blog, so you might be sidestepping this issue. If so I'll be here for tests, if needed.

rom1v · 2019-07-18T19:01:47Z

It's not caused by how adb works (#214)?

No. There are 2 issues:

Android only supports ASCII for injecting text.
we need different events for generating text or for pressing keys.

If you inject à, with the workaround explained in the blog post, we can inject the text correctly, so the first issue is not a problem here.

In itself, here, the second issue should not be a problem either: it is clear that we want to inject à as text.

But related to #87, when we press a letter, sometimes we want a letter (because we are typing text), sometimes we want the press+release events (typically wasd in a game). So to support this use case, all text events with letters (without accents) are ignored and raw events for letters are forwarded instead.

As a consequence, now when you press à, it generates both the "key a pressed", "key a released", and "text à".

In other words, even if issue 1 did not exist, you would still have the problem.

If so I'll be here for tests, if needed.

👍

dstwwy · 2019-07-19T00:09:27Z

Ah, a key up event, the game example was very clear. Now I see the problem.

dstwwy · 2019-08-13T00:55:04Z

Could you add your parameter suggestion on a new release, while there's no definitive solution? Doesn't look like it should break anything since the legacy code would stay inactive unless selected.

rom1v · 2019-08-13T08:47:02Z

Yes, I planned to add an option --text-events={all, non-alpha, none} before the v1.10 release, but I didn't had time to implement it. So I'll do it later :)

fefo-dev · 2019-11-05T16:45:35Z

Hi @rom1v , do you still plan to add the toggle on your next release? So far I've been using both 1.3 and 1.9 (.10 has the console issue), and I'm looking forward to it.

rom1v · 2019-11-05T17:38:48Z

do you still plan to add the toggle on your next release?

Yes.

For 1.11, in addition to what is already on dev, I would like to fix:

Touch input offset on secondary monitor #15 (PR fix for resizing HIDPI MacOS windows #848)
scrcpy-noconsole.exe shows a console #691 (no-console issue)
this one

Expose an option to configure how key/text events are forwarded to the Android device. Fixes <#650>

rom1v · 2019-11-06T22:30:56Z

I implemented it in textevents.

scrcpy-textevents.zip
SHA256: 0086a66ca8eac3faef21082becd696b33cd4c0e803c8acd6ba1d8ec068d8e1a9

(extract this in your scrcpy v1.10 folder to override)

Now, the text works like in v1.3:

scrcpy --prefer-text-events=always  # this is the default

To get the behavior of v1.4→v1.10:

scrcpy --prefer-text-events=non-alpha

And there is another mode, which never sends text events (probably useful only for US keyboard on both the computer and the device).

scrcpy --prefer-text-events=never

rom1v · 2019-11-07T06:59:40Z

I think I will simplify, and only keep the behavior of always and non-alpha, so that the option may become --prefer-text-events or even --prefer-text without parameter.

fefo-dev · 2019-11-07T16:14:13Z

Working as intended here on default, thanks. Dunno how optimal this solution works for you, but I can't see a better way to do it outside of a toggle on an UI that doesn't exist now.

If you got a no console build I wouldn't mind, hehe. Bundling it on the main app is a bigger headache that doing both?

rom1v · 2019-11-07T16:37:40Z

If you got a no console build I wouldn't mind

#691 is not fixed yet, so the scrcpy-noconsole.exe is still broken.

I will probably keep the v1.10 behavior by default (and --prefer-text will enable the option).

Expose an option to configure how key/text events are forwarded to the Android device. Enabling the option avoids issues when combining multiple keys to enter special characters, but breaks the expected behavior of alpha keys in games (typically WASD). Fixes <#650>

rom1v · 2019-11-07T18:03:30Z

I think I will simplify, and only keep the behavior of always and non-alpha, so that the option may become --prefer-text-events or even --prefer-text without parameter.

Done: c916af0 (textevents.2)

rickx · 2019-11-15T10:21:31Z

Testing textevents.2 but i only get the stream and 1 chance for 1 (mouse) input. After that it does not pass any input at all. Win64 version.

rom1v · 2019-11-15T10:27:04Z

@rickx Did you also recompile the server part?

You should get an error in the ouput of adb logcat when it start to fail, please check.

Could you test dev branch (the commit is now merged into dev).

rickx · 2019-11-15T10:28:20Z

hmm nope, just copied the server. I should compile it, right?

That's what I get:
11-15 11:30:44.469 593 755 E InputDispatcher: Key event has invalid action code 0x3
11-15 11:30:44.470 593 755 W InputManager: Input event injection from pid 26968 failed.

(twice).
Then it stops.

rickx · 2019-11-15T11:58:43Z

On dev now. Server compilation gives me error so I'm stuck.

ninja -C "build-server"
ninja: Entering directory `build-server'
[0/1] Generating scrcpy-server with a custom command.

> Task :server:compileReleaseAidl FAILED
/usr/lib/android-sdk/build-tools/28.0.3/aidl: 1: /usr/lib/android-sdk/build-tools/28.0.3/aidl: Syntax error: redirection unexpected


FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':server:compileReleaseAidl'.
> com.android.ide.common.workers.WorkerExecutorException: 1 exception was raised by workers:
  java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: com.android.ide.common.process.ProcessException: Error while executing process /usr/lib/android-sdk/build-tools/28.0.3/aidl with arguments {-p/usr/lib/android-sdk/platforms/android-29/framework.aidl -o/home/federico/scrcpy_dev/scrcpy/server/build/generated/aidl_source_output_dir/release/compileReleaseAidl/out -I/home/federico/scrcpy_dev/scrcpy/server/src/release/aidl -I/home/federico/scrcpy_dev/scrcpy/server/src/main/aidl -d/tmp/aidl215713742388453911.d /home/federico/scrcpy_dev/scrcpy/server/src/main/aidl/android/view/IRotationWatcher.aidl}

rom1v · 2019-11-15T12:03:07Z

hmm nope, just copied the server. I should compile it, right?

Yes, that's the problem.

11-15 11:30:44.469 593 755 E InputDispatcher: Key event has invalid action code 0x3

This is due to a protocol issue between the client and the server ("they don't talk the same language").

rom1v · 2019-11-15T12:12:07Z

How did you install your Android SDK?

/usr/lib/android-sdk/build-tools/28.0.3/aidl: 1: /usr/lib/android-sdk/build-tools/28.0.3/aidl: Syntax error: redirection unexpected

It seems related to SDK build tools 28.0.3: #22 (comment)

Could you upgrade to 29.*?

rickx · 2019-11-18T09:14:59Z

I found out it's not possible because I was on a x386 system. Restarted with buster 64 bit. After some trouble with java I was successful, everything built fine.

rickx · 2019-11-18T10:07:21Z

Still issues with input. I'd like to have the debug output so I added those log entries in scrcpy.c. But I get no output in cmd window (w10). Do I need to set a debug flag or something?

rom1v · 2019-11-18T10:36:09Z

Still issues with input.

Which issues? The same?

But I get no output in cmd window (w10).

How did you build it?

Do I need to set a debug flag or something?

By default, if you don't pass --buildtype=release to the meson command, it's in debug.

rickx · 2019-11-20T14:13:02Z

Still issues with input.

Which issues? The same?

Well, it's something weird. It's most probably not scrcpy's fault. I have an android app written in qt. This app has some input fields which "give a twist" to the input, when the input is done through scrcpy: usually they take the first letter and the last and switch them. So e.g. HELLO becomes ELLOH.
This only happens when the input field looses focus, it looks good before that. If I use the onscreen keyboard which pops up (android's), the twist does not happen. So, to debug this, I wanted to activate the logging of what is sent from pc to device.

But I get no output in cmd window (w10).

How did you build it?

Just make -f Makefile.CrossWindows

Do I need to set a debug flag or something?

By default, if you don't pass --buildtype=release to the meson command, it's in debug.

ok, that's probably the thing. Thanks. Does that mean I just remove the buildtype param? Or replace release with something else?

rom1v · 2019-11-20T14:57:06Z

I have an android app written in qt. This app has some input fields which "give a twist" to the input, when the input is done through scrcpy: usually they take the first letter and the last and switch them. So e.g. HELLO becomes ELLOH.

Can you reproduce it by injecting the text events manually (without scrcpy):

adb shell input text H
adb shell input text E
adb shell input text L
adb shell input text L
adb shell input text O

Does that mean I just remove the buildtype param? Or replace release with something else?

Either remove or replace by --buildtype=debug (you need to remove your builddir beforehand; alternatively, you can change its configuation using meson configure your_builddir --buildtype=debug).

alexdevos · 2020-03-01T12:41:37Z

Hello @rom1v, I'm running scrcpy version 1.12 (installed via snap) and I'm unable to have character as é, à or è when I type those character on my laptop keyboard : nothing appears on my Android phone. I tried scrcpy --prefer-text but I have same problem.
When I try adb shell input text E in a command line, I get the E on the phone, but with adb shell input text é nothing appears.
I also read #431 but it didn't help me. What can I do/test ?

By the way, thank you for this great tool 😃 !

rom1v · 2020-03-01T13:26:09Z

What is your keyboard on Android?

alexdevos · 2020-03-01T14:28:57Z

What is your keyboard on Android?

I have AnySoftKeyboard French v. 4.0.925.2525 (on AnySoftKeyboard v. 1.10.1096.6266) but I also tried with the LineageOS 16 stock keyboard and it doesn't work (I tried on two different phones with LOS 16 : Samsung Galaxy S5 and Samsung Galaxy S5+).

edit : I have also the same problem with Samsung Galaxy S3 and LineageOS 14.1 stock keyboard

KaKi87 · 2020-08-01T12:52:45Z

Hello,

While using --prefer-text, there are still some special characters that I can't input : ², ¨, £, ¤, µ, §

Thanks

rom1v · 2020-08-01T12:54:58Z

@KaKi87 https://github.com/Genymobile/scrcpy/blob/master/FAQ.md#special-characters-do-not-work

KaKi87 · 2020-08-01T13:03:45Z

Injecting text input is limited to ASCII characters.

² is ASCII 253
¨ is ASCII 249
£ is ASCII 156
¤ is ASCII 207
µ is ASCII 230
§ is ASCII 245

rom1v · 2020-08-01T13:25:49Z

ASCII is on 7 bits, so it contains only values up to 127.

I don't know what are the values you provide, since they are not latin1/cp1252/iso8859-1 either.

KaKi87 · 2020-08-01T13:31:45Z

I see, so it only supports the non-extended ASCII table.

https://theasciicode.com.ar/

Would it be possible to implement support for the rest ?

rom1v · 2020-08-01T14:02:08Z

This is a limitation on Android on key injection. There are alternatives (like installing a separate app containing IME), see #37 and related.

Kajziba · 2023-03-19T14:46:05Z

Ah, a key up event, the game example was very clear. Now I see the problem.

This was referenced Sep 4, 2019

Accents error #781

Closed

SendKeys using app, only numbers received in phone #784

Open

rom1v mentioned this issue Oct 8, 2019

accent mark #849

Closed

rom1v added a commit that referenced this issue Nov 6, 2019

Add --prefer-text-events option

44791d6

Expose an option to configure how key/text events are forwarded to the Android device. Fixes <#650>

rom1v mentioned this issue Nov 6, 2019

Black Desert Mobile Left Click Issue #887

Closed

rom1v mentioned this issue Nov 8, 2019

[WIP]Support inject unicoe text #903

Closed

3 tasks

fefo-dev mentioned this issue Nov 19, 2019

scrcpy-noconsole.exe shows a console #691

Closed

rom1v closed this as completed Nov 20, 2019

rom1v mentioned this issue Jan 8, 2020

Some characters are not written correctly #1067

Closed

alexdevos mentioned this issue Mar 1, 2020

Cannot write special characters with the computer's keyboard #974

Closed

rom1v mentioned this issue Sep 21, 2020

Accents sending double letters #1779

Closed

RafaelLinux mentioned this issue Aug 19, 2022

Spelling marks showed as two distinct characters ("aáaâaä") #3450

Closed

2 tasks

Kajziba mentioned this issue Mar 19, 2023

> Injecting text input is limited to ASCII characters. Genymobile/copydeps#13

Open

seeu100 mentioned this issue Jun 1, 2024

scrcpy seeu100/blog#77

Open

Double input along with latin accents #650

Double input along with latin accents #650

Comments

dstwwy commented Jul 18, 2019

rom1v commented Jul 18, 2019 • edited Loading

dstwwy commented Jul 18, 2019

rom1v commented Jul 18, 2019

dstwwy commented Jul 18, 2019

rom1v commented Jul 18, 2019 • edited Loading

dstwwy commented Jul 19, 2019

dstwwy commented Aug 13, 2019

rom1v commented Aug 13, 2019

fefo-dev commented Nov 5, 2019

rom1v commented Nov 5, 2019

rom1v commented Nov 6, 2019 • edited Loading

rom1v commented Nov 7, 2019

fefo-dev commented Nov 7, 2019

rom1v commented Nov 7, 2019

rom1v commented Nov 7, 2019

rickx commented Nov 15, 2019 • edited Loading

rom1v commented Nov 15, 2019

rickx commented Nov 15, 2019 • edited Loading

rickx commented Nov 15, 2019

rom1v commented Nov 15, 2019

rom1v commented Nov 15, 2019

rickx commented Nov 18, 2019

rickx commented Nov 18, 2019 • edited Loading

rom1v commented Nov 18, 2019

rickx commented Nov 20, 2019

rom1v commented Nov 20, 2019

alexdevos commented Mar 1, 2020 • edited Loading

rom1v commented Mar 1, 2020

alexdevos commented Mar 1, 2020 • edited Loading

KaKi87 commented Aug 1, 2020

rom1v commented Aug 1, 2020

KaKi87 commented Aug 1, 2020

rom1v commented Aug 1, 2020

KaKi87 commented Aug 1, 2020

rom1v commented Aug 1, 2020

Kajziba commented Mar 19, 2023

rom1v commented Jul 18, 2019 •

edited

Loading

rom1v commented Jul 18, 2019 •

edited

Loading

rom1v commented Nov 6, 2019 •

edited

Loading

rickx commented Nov 15, 2019 •

edited

Loading

rickx commented Nov 15, 2019 •

edited

Loading

rickx commented Nov 18, 2019 •

edited

Loading

alexdevos commented Mar 1, 2020 •

edited

Loading

alexdevos commented Mar 1, 2020 •

edited

Loading