Publisher command: evaluate user expressions, support joysticks

.appveyor.yml

@@ -46,8 +46,11 @@ for:
         - job_group: tests
           APPVEYOR_BUILD_WORKER_IMAGE: Ubuntu2004
     install:
+      - 'sudo apt-get --ignore-missing update || true'
       - 'export extras_pkg="linux-*-extra-$(uname -r)"'
       - 'sudo apt-get install -y $extras_pkg ncat'
+      - 'sudo apt-get install -y libsdl2-2.0-0'   # For PySDL2. On Windows/macOS the binaries are pulled from PyPI.
+      - 'sudo apt-get install -y libasound2-dev'  # For RtMidi.
       - git submodule update --init --recursive
       - python -m pip install --upgrade pip setuptools nox
     test_script:

CONTRIBUTING.md

@@ -51,6 +51,26 @@ Alternatively, you can just compile DSDL manually directly in the project root.
 Configure the IDE to run Black on save.
 See the Black documentation for integration instructions.
 
+### Capturing video for documentation
+
+Capture desktop region:
+
+```bash
+ffmpeg -video_size 1920x1500 -framerate 10 -f x11grab -i :0.0+3840,117 output.mp4 -y
+```
+
+Convert captured video to GIF:
+
+```bash
+ffmpeg -i output.mp4 output.gif
+```
+
+Stream webcam via MJPEG using VLC (open the stream using web browser or VLC):
+
+```bash
+cvlc v4l2:///dev/video0 :chroma=mjpg :live-caching=10 --sout '#transcode{vcodec=mjpg}:std{access=http{mime=multipart/x-mixed-replace;boundary=-7b3cc56e5f51db803f790dad720ed50a},mux=mpjpeg,dst=0.0.0.0:8080}' --network-caching=0
+```
+
 ## Releasing
 
 The tool is versioned by following [Semantic Versioning](https://semver.org).

README.md

@@ -28,6 +28,11 @@ Afterward do endeavor to read the docs: **`yakut --help`**
 
 Check for new versions every now and then: **`pip install --upgrade yakut`**
 
+### GNU/Linux
+
+In order to use MIDI controllers you may need to manually install SDL2.
+In most distros the package name begins with `libsdl2`.
+
 ### Common issues
 
 If you are experiencing illegal instruction faults on aarch64, upgrade NumPy and Cython: `pip install -U numpy cython`.
@@ -53,6 +58,8 @@ as long as the resulting abbreviation is unambiguous.
 
 There is a dedicated `--help` option for every subcommand.
 
+Yakut may also be invoked via its alias **`y`** as long as this name does not conflict with another installed program.
+
 ## Compiling DSDL
 
 Suppose we have our custom DSDL namespace that we want to use.
@@ -167,23 +174,11 @@ consider configuring it as the default via environment variables as shown earlie
 
 Next there are practical examples (configuring the transport is left as an exercise to the reader).
 
-### Publishing messages
-
-Publishing two messages synchronously twice (four messages total);
-notice how we specify the subject-ID before the data type name:
-
-```bash
-export UAVCAN__UDP__IFACE=127.63.0.0
-export UAVCAN__NODE__ID=42
-yakut pub 33:uavcan.si.unit.angle.Scalar.1.0 'radian: 2.31' uavcan.diagnostic.Record.1.1 'text: "2.31 rad"' -N2
-```
-
-We did not specify the subject-ID for the second subject, so Yakut defaulted to the fixed subject-ID.
-
 ### Subscribing to subjects
 
-Subscribe to subject 33 of type `uavcan.si.unit.angle.Scalar.1.0`
-to receive messages published by the above command:
+Subscribe to subject 33 of type `uavcan.si.unit.angle.Scalar.1.0` as shown below;
+notice how we specify the subject-ID before the data type name.
+You will see output if there is a publisher on this subject (more on this in the next section).
 
 ```bash
 $ export UAVCAN__UDP__IFACE=127.63.0.0
@@ -207,6 +202,58 @@ $ yakut sub 33:uavcan.si.unit.angle.Scalar.1.0
   radian: 2.309999942779541
 ```
 
+### Publishing messages
+
+Publishing two messages synchronously twice (four messages total):
+
+```bash
+export UAVCAN__UDP__IFACE=127.63.0.0
+export UAVCAN__NODE__ID=42
+yakut pub -N2 33:uavcan.si.unit.angle.Scalar.1.0 'radian: 2.31' \
+                 uavcan.diagnostic.Record.1.1    'text: "2.31 rad"'
+```
+
+We did not specify the subject-ID for the second subject, so Yakut defaulted to the fixed subject-ID.
+
+The above example publishes constant values which is rarely useful.
+You can define arbitrary Python expressions that are evaluated by Yakut before publication.
+Such expressions are entered as strings marked with [YAML tag](https://yaml.org/spec/1.2/spec.html#id2761292) `!$`.
+There may be an arbitrary number of such expressions in a YAML document,
+and their results may be arbitrary as long as the final structure can initialize the specified message.
+The following example will publish a sinewave with frequency 1 Hz, amplitude 10 meters:
+
+```bash
+yakut pub -T 0.01 1234:uavcan.si.unit.length.Scalar.1.0 '{meter: !$ "sin(t * pi * 2) * 10"}'
+```
+
+Notice that we make use of variables like `t` or standard functions like `sin` in the expression.
+You will see the full list of available symbols and functions if you run `yakut pub --help`.
+
+One particularly important capability of this command is the ability to read data from connected
+joysticks or MIDI controllers.
+It allows the user to control UAVCAN processes or equipment in real time, simulate sensor feeds, etc.
+Function `A(x,y)` returns the normalized value of channel `y` from connected controller `x`
+(for full details see `yakut pub --help`);
+likewise, there is `B(x,y)` for push buttons and `T(x,y)` for toggle switches.
+The next example shows how to publish 3D angular velocity setpoint, thrust setpoint, and the arming switch state,
+allowing the user to control these parameters interactively:
+
+```bash
+yakut pub -T 0.1 \
+    5:uavcan.si.unit.angular_velocity.Vector3.1.0 'radian_per_second: !$ "[A(1,0)*10, A(1,1)*10, (A(1,2)-A(1,5))*5]"' \
+    6:uavcan.si.unit.power.Scalar.1.0 'watt: !$ A(2,10)*1e3' \
+    7:uavcan.primitive.scalar.Bit.1.0 'value: !$ T(1,5)'
+```
+
+The list of available controllers and how their axes are mapped can be seen using `yakut joystick`,
+as shown in the video:
+
+[![yakut joystick](https://img.youtube.com/vi/YPr98KM1RFM/maxresdefault.jpg)](https://www.youtube.com/watch?v=YPr98KM1RFM)
+
+Here is an example where a MIDI controller is used to interactively change the frequency and amplitude of a sinewave:
+
+[![yakut publish](https://img.youtube.com/vi/DSsI882ZYh0/maxresdefault.jpg)](https://www.youtube.com/watch?v=DSsI882ZYh0)
+
 ### Invoking RPC-services
 
 Given custom data types:

setup.cfg

@@ -57,6 +57,12 @@ install_requires =
     click           ~= 7.1
     psutil          ~= 5.8
     scipy           ~= 1.6
+    # SDL2 bindings. For Windows and macOS there are pre-built SDL2 binaries provided.
+    pysdl2          <  2.0
+    pysdl2-dll      ~= 2.0; sys_platform == "win32" or sys_platform == "darwin"
+    # MIDI controller support.
+    mido            ~= 1.2
+    python-rtmidi   ~= 1.4
 
 [options.packages.find]
 # https://setuptools.readthedocs.io/en/latest/setuptools.html#find-namespace-packages
@@ -67,6 +73,7 @@ include =
 [options.entry_points]
 console_scripts =
     yakut = yakut:main
+    y     = yakut:main
 
 [options.package_data]
 * =
@@ -169,7 +176,9 @@ disable=
     import-error,
     misplaced-comparison-constant,
     unsubscriptable-object,
-    too-many-statements
+    too-many-statements,
+    too-many-instance-attributes,
+    eval-used
 
 [pylint.REPORTS]
 output-format=colorized

tests/cmd/interaction.py

@@ -83,7 +83,7 @@ def _unittest_pub_sub_regular(transport_factory: TransportFactory, compiled_dsdl
         "--timeout=5",
         timeout=10.0,
     )
-    parsed = yakut.yaml.YAMLLoader().load(stdout)
+    parsed = yakut.yaml.Loader().load(stdout)
     assert parsed[430]["protocol_version"] == {
         "major": pyuavcan.UAVCAN_SPECIFICATION_VERSION[0],
         "minor": pyuavcan.UAVCAN_SPECIFICATION_VERSION[1],

tests/cmd/joystick.py

@@ -0,0 +1,27 @@
+# Copyright (c) 2021 UAVCAN Consortium
+# This software is distributed under the terms of the MIT License.
+# Author: Pavel Kirienko <[email protected]>
+
+import asyncio
+import pytest
+from tests.subprocess import Subprocess
+
+
+# noinspection SpellCheckingInspection
+@pytest.mark.asyncio
+async def _unittest_joystick() -> None:
+    asyncio.get_running_loop().slow_callback_duration = 10.0
+
+    # This command interacts directly with the hardware.
+    # We can't test it end-to-end in a virtualized environment without being able to emulate the connected hardware.
+    # For now, we just check if it runs at all, which is not very helpful but is better than nothing.
+    # Eventually we should find a way to emulate connected joysticks and MIDI controllers.
+    proc = Subprocess.cli("joy", stdout=open("stdout", "wb"), stderr=open("stderr", "wb"))
+    assert proc.alive
+    await asyncio.sleep(5)
+    assert proc.alive, open("stderr", "r").read()
+    proc.wait(10.0, interrupt=True)
+
+    # The null controller shall always be available.
+    with open("stdout", "r") as f:
+        assert "null" in f.read()

tests/cmd/publish.py → tests/cmd/publish/basic.py

@@ -34,8 +34,6 @@ def _unittest_publish(compiled_dsdl: typing.Any) -> None:
         "pub",
         "4444:uavcan.si.unit.force.Scalar.1.0",
         "{}",
-        "--count",
-        "0",
         timeout=5.0,
         ensure_success=False,
         environment_variables=env,

tests/cmd/publish/expression.py

@@ -0,0 +1,93 @@
+# Copyright (c) 2021 UAVCAN Consortium
+# This software is distributed under the terms of the MIT License.
+# Author: Pavel Kirienko <[email protected]>
+
+from __future__ import annotations
+import time
+import json
+import typing
+from pytest import approx
+from tests.dsdl import OUTPUT_DIR
+from tests.subprocess import execute_cli, Subprocess
+
+
+def _unittest_publish_expression_a(compiled_dsdl: typing.Any, serial_broker: str) -> None:
+    _ = compiled_dsdl
+    env = {
+        "YAKUT_PATH": str(OUTPUT_DIR),
+        "UAVCAN__SERIAL__IFACE": serial_broker,
+        "UAVCAN__NODE__ID": "1234",
+    }
+
+    proc_sub = Subprocess.cli(
+        "--format=json",
+        "sub",
+        "7654:uavcan.primitive.array.Real64.1.0",
+        environment_variables=env,
+    )
+
+    wall_time_when_started = time.time()
+    execute_cli(
+        "-vv",
+        "pub",
+        "7654:uavcan.primitive.array.Real64.1.0",
+        "value: [!$ 1 + n + t * 0.1, !$ 'cos(A(0, 12))', !$ 'B(0, 34)', !$ 'B(0, 56)', 123456, !$ 123456, !$ time()]",
+        #           {1.0, 2.1}           1.0                 0              0          123456     123456     (time)
+        "--count=2",
+        timeout=10.0,
+        environment_variables=env,
+    )
+
+    _, stdout, _ = proc_sub.wait(10.0, interrupt=True)
+    print(stdout)
+    msgs = list(map(json.loads, stdout.splitlines()))
+    print(msgs)
+    assert msgs[0]["7654"]["value"] == [
+        approx(1.0),
+        approx(1.0),
+        approx(0),
+        approx(0),
+        approx(123456),
+        approx(123456),
+        approx(wall_time_when_started, abs=10.0),
+    ]
+    assert msgs[1]["7654"]["value"] == [
+        approx(2.1),
+        approx(1.0),
+        approx(0),
+        approx(0),
+        approx(123456),
+        approx(123456),
+        approx(wall_time_when_started + 1.0, abs=10.0),
+    ]
+
+
+def _unittest_publish_expression_b(compiled_dsdl: typing.Any, serial_broker: str) -> None:
+    _ = compiled_dsdl
+    env = {
+        "YAKUT_PATH": str(OUTPUT_DIR),
+        "UAVCAN__SERIAL__IFACE": serial_broker,
+        "UAVCAN__NODE__ID": "1234",
+    }
+
+    proc_sub = Subprocess.cli(
+        "--format=json",
+        "sub",
+        "7654:uavcan.primitive.String.1.0",
+        environment_variables=env,
+    )
+
+    execute_cli(
+        "pub",
+        "7654:uavcan.primitive.String.1.0",
+        "value: !$ str(pyuavcan.dsdl.get_model(dtype))",
+        "--count=1",
+        timeout=10.0,
+        environment_variables=env,
+    )
+
+    _, stdout, _ = proc_sub.wait(10.0, interrupt=True)
+    print(stdout)
+    msg = json.loads(stdout)
+    print(msg)
+    assert msg["7654"]["value"] == "uavcan.primitive.String.1.0"

yakut/VERSION

@@ -1 +1 @@
-0.4.1
+0.5.0