AmebaPro with KVS WebRTC and Producer

This project is going to demonstrate how to use KVS Producer and WebRTC on AmebaPro

Amazon FreeRTOS-LTS libraries are supported on AmebaPro now, it can be used to communicate with AWS IoT core easily! Version 202012.00

You can review the ChangeLog before using the project.

Brief Introduction to Amazon KVS and FreeRTOS-LTS

📘 Amazon Kinesis Video Streams
  include two main services, which are Producer and WebRTC

📘 KVS Producer
  stream live video from devices to the AWS Cloud, or build applications for batch-oriented video analytics.

📘 KVS WebRTC
  enable real-time communication (RTC) across browsers and mobile applications via simple APIs.

📘 FreeRTOS-LTS Libraries
  provides feature stability, and security patches and critical bug fixes for two years from release date.

Applications on AmebaPro

💡 AI Face Detection

  AmebaPro(Producer) + Rekognition(AWS AI) = video cloud storage and content analytics ☁️

1. Stream live video from AmebaPro to KVS 2. AI face detection by using AWS Rekognition API   👉

💡 Real-time communucation

  AmebaPro(WebRTC master) + Viewer(WebRTC Client) = real-time communication, 1-way video & 2-way audio

1. AmebaPro start a signaling channel, as a master 2. Browser run as a viewer (client) 3. Start a P2P connection   👉

💡 Object Detection on Cloud

  AmebaPro(image uploader) + Rekognition(AWS AI) = object detection and classification ☁️

1. Take a snapshot by camera sensor 2. Upload image from AmebaPro to S3 3. Trigger Lambda function to do object detection   👉   Notes: Based on FreeRTOS-v202012-LTS framework

💡 FreeRTOS-LTS-v202012.00 Libraries

  AmebaPro can connect to AWS IoT with the long term support libraries maintained by Amazon.
  FreeRTOS demos: coreHTTP, coreMQTT, Over-the-air updates(OTA), AWS IoT Device Shadow...

💡 Edge AI - Human Detection on AmebaPro

  AmebaPro can do human detection by itself! (inference on edge device).

💡 Edge AI + Cloud AI - Human Detection + Face Detection

  AmebaPro do human detection on edge and upload video to KVS to detect face by using Amazon Rekognition.

street_human_detection_VLC2.mp4

1. Do the human detection on AmebaPro 2. Stream live video from AmebaPro to KVS 3. AI face detection by using AWS Rekognition API   👉

Demo code

✏️ KVS WebRTC    project/component/common/example/kvs_webrtc

✏️ KVS Producer    project/component/common/example/kvs_producer

✏️ S3 file upload    project/component/common/application/amazon/JPEG_snapshot_s3_upload_example

✏️ FreeRTOS-LTS    project/component/common/application/amazon/amazon-freertos-202012.00/demos

✏️ Edge AI - Human detection    project/realtek_amebapro_v0_example/src/obj_detect

IDE/toolchain Requirement

🔨 IAR Embedded Workbench - IAR Systems
  Please use IAR version 8.3

🔨 GCC toolchain
  Linux: asdk-6.4.1-linux-newlib-build-3026-x86_64
  Cygwin: asdk-6.4.1-cygwin-newlib-build-2778-i686

Clone Project

To check out this repository:

git clone --recurse-submodules https://github.com/HungTseLee/KVS_WebRTC_on_AmebaPro.git

If you already have a checkout, run the following command to sync submodules recursively:

git submodule update --init --recursive

If there is GCC makefile error like: "No rule to make target …", it may mean that some codes have not been downloaded correctly.
Please run the above command again to download the missing codes.

If there is compile error with shell script in "component/soc/realtek/8195b/misc/gcc_utility/", you may need to run following command

dos2unix component/soc/realtek/8195b/misc/gcc_utility/*

ChangeLog

• (Jul 12, 2021) Release version v2.0 for using KVS and LTS library on AmebaPro.
• (Jul 09, 2021) Support Amazon FreeRTOS-LTS library and its demos. version 202012.00.
  The branch: AmebaPro-202012.00-LTS-dev has been merged into main.
• (May 12, 2021) Release version v1.0 for using KVS on AmebaPro.