Version 0.6:

- Update VHT script to pick the right Arm Virtual Hardware on all regions.
- Update instructions to use the VHT script.

Signed-off-by: Vincent Coubard <vincent.coubard@arm.com>

LICENCE.md

@@ -3,13 +3,13 @@ as can be found in: LICENSE-apache-2.0.txt
 
 Folders containing files under different permissive license than Apache 2.0 are listed below. Each folder should contain its own README file with license specified for its files. The original license text is included in those source files.
 
-- [lib/amazon_freertos](lib/amazon_freertos) - MIT
-- [lib/mbedcrypto](lib/mbedcrypto) - Apache-2.0
-- [lib/mcuboot](lib/mcuboot) - Apache-2.0
-- [lib/ml-embedded-evaluation-kit](lib/ml-embedded-evaluation-kit) - Apache-2.0
-- [lib/tf-m](lib/tf-m) - BSD-3-Clause
-- [lib/tfm_test](lib/tfm_test) - BSD-3-Clause
-- [lib/VHT](source/lib/VHT) - Apache-2.0
+- [lib/amazon_freertos](https://github.com/aws/amazon-freertos) - MIT
+- [lib/mbedcrypto](https://github.com/ARMmbed/mbedtls) - Apache-2.0
+- [lib/mcuboot](https://github.com/mcu-tools/mcuboot) - Apache-2.0
+- [lib/ml-embedded-evaluation-kit](https://review.mlplatform.org/plugins/gitiles/ml/ethos-u/ml-embedded-evaluation-kit) - Apache-2.0
+- [lib/tf-m](https://git.trustedfirmware.org/TF-M/trusted-firmware-m.git) - BSD-3-Clause
+- [lib/tfm_test](https://git.trustedfirmware.org/TF-M/tf-m-tests.git) - BSD-3-Clause
+- [lib/VHT](https://github.com/ARM-software/VHT) - Apache-2.0
 - [tinycbor](kws/ota/ota_for_aws/source/dependency/3rdparty/tinycbor) - MIT License
 - [coreJSON](kws/ota/ota_for_aws/source/dependency/coreJSON) - MIT License
 - [tfm test service](bsp/test_services) - BSD-3-Clause

README.md

@@ -34,10 +34,16 @@ To utilize the Arm Virtual Hardware, you will need to create an [AWS Account](ht
 #### Launching the instance in EC2 [(AWS on getting started)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html)
 1. Go to [EC2](https://console.aws.amazon.com/ec2/v2/) in the AWS Web Console.
 1. Select **Launch Instances** which will take you to a wizard for launching the instance.
-  1. **Step 1: Choose an [Amazon Machine Image (AMI)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html)** - In the Search box, type `Arm Virtual Hardware` then find the item called "Arm Virtual Hardware" that is by Arm, and press Select for that item. This image contains all the software necessary to build and run the Arm IoT Total Solutions.
-  This will raise a subscription page/pop-up titled, **Arm Virtual Hardware**. You will note that the subscription is free from Arm, but AWS does charge for the costs of the instances themselves according to the pricing chart provided. You must select Continue if you want to move forward.
-  1. **Step 2: Choose an Instance Type** - Select one of the instance types from the list. We recommend the **c5.large**. Keep in mind that there are charges that accrue while the instance is running.
-  From here you may select **Review and Launch** to move directly to the launch page or select **Next: Configure Instance Details** if you need to set any custom settings for this instance.
+  
+     1. **Step 1: Choose an [Amazon Machine Image (AMI)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html)** - In the Search box, type `Arm Virtual Hardware` then find the item called "Arm Virtual Hardware" that is by Arm, and press Select for that item. This image contains all the software necessary to build and run the Arm IoT Total Solutions.
+        This will raise a subscription page/pop-up titled, **Arm Virtual Hardware**. You will note that the subscription is free from Arm, but AWS does charge for the costs of the instances themselves according to the pricing chart provided. 
+  
+        > Arm Virtual Hardware for Corstone-300 is available as a public beta on AWS Marketplace. To help you get started, AWS are offering more than 100 hours of free AWS EC2 CPU credits for the first 1,000 qualified users. Click here to find out more: https://www.arm.com/company/contact-us/virtual-hardware.
+  
+        You must select Continue if you want to move forward.
+     
+     1. **Step 2: Choose an Instance Type** - Select one of the instance types from the list. We recommend the **c5.large**. Keep in mind that there are charges that accrue while the instance is running.
+        From here you may select **Review and Launch** to move directly to the launch page or select **Next: Configure Instance Details** if you need to set any custom settings for this instance.
 
 #### Selecting the instance
 Once you complete the wizard by initiating the instance launch you will see a page that allows you to navigate directly to the new instance. You may click this link or go back to your list of instances and find the instance through that method.
@@ -49,6 +55,42 @@ Whichever way you choose find your new instance and select its instance ID to op
 1. Ensure the User name field is set to `ubuntu`.
 1. Select the **Connect** button to open the session. This will put you in a browser window where you will have an SSH terminal window ready for your input.
 
+### Launch Using a local terminal
+
+1. Install [AWS CLI 2](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) on your machine. 
+2. [Configure](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html) the access key, secret key and region that AWS CLI will use. If your organization uses AWS Single Sign-On, the [configuration process](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sso.html) is slightly different. Make sure the region selected matches the region of the SSO service.
+3. Create a new key pair. 
+
+```sh
+aws ec2 create-key-pair --key-name MyKeyPair
+```
+
+4. When AWS CLI display the new key pair. Save the key material in a `.pem` file. The file permission must be set to `400`. 
+
+```sh
+chmod 400 MyKeyPair.pem
+```
+
+5. Launch a new instance with the key pair created. The key pair can be reused to create new instances.
+
+```sh
+./scripts/vht_cli.py -k MyKeyPair start
+```
+
+#### Connecting to the instance:
+
+1. Get the IP of the instance started
+
+```sh
+./scripts/vht_cli.py -k MyKeyPair status
+```
+
+2. Connect to the instance using SSH and the private key saved localy. 
+
+```sh
+ssh -i "MyKeyPair.pem" ubuntu@<instance ip address>
+```
+
 ## Build and execute the application
 
 To update the application, a set of scripts is included to setup the environment, 
@@ -132,7 +174,7 @@ rm -rf ATS-Keyword
  To utilize the Arm Virtual Hardware, you will need to create an AWS Account if you don’t already have one.
 
 ```sh
-./scripts/avh_cli.py -k <key pair name> start
+./scripts/vht_cli.py -k <key pair name> start
 ```
 
 3. Launch GitHub Self-Hosted Runner
@@ -328,9 +370,10 @@ Integrating a new model means integrating its source code and requires update of
 | Repository                                                                                                    | Description                                                                                                                                                                                                                        |
 |---------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | [Arm AI Ecosystem Catalog](https://www.arm.com/why-arm/partner-ecosystem/ai-ecosystem-catalog)                | Connects you to the right partners, enabling you to build the next generation of AI solutions                                                                                                                                      |
+| [Arm CMSIS Build](https://arm-software.github.io/CMSIS_5/Build/html/index.html)                               | Documentation for the build system used by applications in this repository.                                                                                                                                                        |
+| [Arm IoT Ecosystem Catalog](https://www.arm.com/why-arm/partner-ecosystem/iot-ecosystem-catalog)              | Explore Arm IoT Ecosystem partners who can help transform an idea into a secure, market-leading device.                                                                                                                            |
 | [Arm ML Model Zoo](https://github.com/ARM-software/ML-zoo)                                                    | A collection of machine learning models optimized for Arm IP.                                                                                                                                                                      |
 | [Arm Virtual Hardware Documentation](https://mdk-packs.github.io/VHT-TFLmicrospeech/overview/html/index.html) | Documentation for [Arm Virtual Hardware](https://www.arm.com/products/development-tools/simulation/virtual-hardware)                                                                                                               |
-| [Arm CMSIS Build](https://arm-software.github.io/CMSIS_5/Build/html/index.html)                               | Documentation for the build system used by applications in this repository.                                                                                                                                                        |
 | [AWS FreeRTOS](https://docs.aws.amazon.com/freertos/)                                                         | Documentation for AWS FreeRTOS.                                                                                                                                                                                                    |
 | [AWS IoT](https://docs.aws.amazon.com/iot/latest/developerguide/what-is-aws-iot.html)                         | Documentation for AWS IoT.                                                                                                                                                                                                         |
 | [Trusted Firmware-M](https://tf-m-user-guide.trustedfirmware.org/)                                            | Documentation for Trusted Firmware-M                                                                                                                                                                                               |

scripts/avh_cli.py → scripts/vht_cli.py

-#!/usr/bin/env python3
+#!/usr/bin/env python
 
 '''
 Copyright 2021 Arm Ltd
@@ -16,6 +16,7 @@ See the License for the specific language governing permissions and
 limitations under the License.
 '''
 
+
 import argparse
 import sys
 import os
@@ -25,10 +26,8 @@ import logging
 import platform
 
 
-# AWS AVH-TEST connection details
-# TODO will eventually change to public AMI
-# version 0.3.0
-ImgId="ami-030e410551d9b5fa5"
+# Arm Virtual Hardware AMI version
+avh_version="1.0.0"
 
 InstType="t3a.medium"
 
@@ -52,19 +51,41 @@ else:
     nc="\033[0m"
 
 
+def get_avh_imgid(region):
+    global avh_version
+
+    cmd_im = "aws ec2 describe-images --region {} --filters Name=name,Values='ArmVirtualHardware-{}*' Name=owner-alias,Values='aws-marketplace' --output json".format(
+    	    region, avh_version)
+
+    try:
+        out_im = subprocess.check_output(cmd_im, stderr=subprocess.STDOUT, shell=True).decode("utf-8")
+    except subprocess.CalledProcessError as e:
+        logging.error(red + "Could not get Arm Virtual Hardware Instance ID for region {}.".format(region) + nc)
+        sys.exit(1)
+
+    outd_im = json.loads(out_im)
+    for instance in outd_im['Images']:
+        ImgId = instance['ImageId']
+        logging.info("Image ID is {}".format(ImgId))
+
+    return ImgId
+
+
 def start_avh(profile, key, region, initfile):
-    global ImgId, InstType
+    global InstType
+
+    print("Starting AVH instance...")
 
-    if status_avh(profile, region, False)[0] > 0 :
+    [n, arr, ImgId] = status_avh(profile, region, False)
+
+    if n > 0 :
         print(orange + "One or more AVH instances are already running." + nc)
         confirm = input("Please confirm to launch a new instance [Y/n]: ")
         if confirm != "Y":
             print("Cancelling... No instance will be started.")
             sys.exit(1)
 
-    print("Starting AVH instance...")
-
-    cmd = "aws ec2 run-instances --profile {} --image-id {} --instance-type {} --key-name {} --region {}".format(
+    cmd = "aws ec2 run-instances --profile {} --image-id {} --instance-type {} --key-name {} --region {} --output json".format(
             profile, ImgId, InstType, key, region)
 
     if initfile != None :
@@ -73,9 +94,32 @@ def start_avh(profile, key, region, initfile):
     logging.debug("Command is: {}".format(cmd))
 
     try:
-        out = subprocess.check_output(cmd, shell=True).decode("utf-8")
-    except subprocess.CalledProcessError:
-        logging.error(red + "Running the start command failed." + nc)
+        out = subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=True).decode("utf-8")
+    except subprocess.CalledProcessError as e:
+        print(e.output)
+        errtype = str(e.output).split(")")[0].split("(")[1]
+        if errtype == "VPCIdNotSpecified":
+            logging.info(orange + "Running the start command failed: default VPC not found. Creating VPC..." + nc)
+
+            cmd_vpc = "aws ec2 create-default-vpc --profile {} --region {}".format(
+	    	profile, region)
+
+            try:
+                out_vpc = subprocess.check_output(cmd_vpc, shell=True).decode("utf-8")
+            except subprocess.CalledProcessError:
+                logging.error(red + "VPC creation failed." + nc)
+                sys.exit(1)
+
+            logging.info("VPC created. Trying to start instance again...")
+
+            try:
+                out = subprocess.check_output(cmd, shell=True).decode("utf-8")
+            except subprocess.CalledProcessError as e:
+                logging.error(red + "Running the start command failed." + nc)
+            sys.exit(1)
+        else:
+            logging.error(red + "Running the start command failed." + nc)
+            sys.exit(1)
 
     outd = json.loads(out)
     for instance in outd['Instances']:
@@ -84,7 +128,8 @@ def start_avh(profile, key, region, initfile):
     return
 
 def status_avh(profile, region, printv):
-    global ImgId
+    # Get ImageID from region
+    ImgId = get_avh_imgid(region)
 
     n_instances = 0
     arr_inst = []
@@ -120,11 +165,11 @@ def status_avh(profile, region, printv):
             else :
                 print("| {}\t| {}\t\t| {}\t|".format(i[0], i[1], i[2]))
 
-    return n_instances, arr_inst
+    return n_instances, arr_inst, ImgId
 
 
 def stop_avh(profile, region):
-    [n, arr] = status_avh(profile, region, False)
+    [n, arr, ImgId] = status_avh(profile, region, False)
 
     if n == 0 :
         logging.error(red + "No AVH instance is running." + nc)
@@ -137,7 +182,7 @@ def stop_avh(profile, region):
             if i[2] == 'running' :
                 print("Running instance ID {} found (public IP {})".format(i[0], i[1]))
 
-        instanceid = input("The instance will be terminated and all data will be lost. Please enter the instance ID to stop it (leave empty to cancel): ")
+        instanceid = input(orange + "The instance will be terminated and all data will be lost. " + nc + "Please enter the instance ID to stop it (leave empty to cancel): ")
 
         if(instanceid == ""):
             print("Cancelling... No instance will be stopped.")
@@ -176,7 +221,7 @@ if __name__ == "__main__":
     # AWS CLI version check
     awscli_version_m = float(awscli_version_f.split(".")[0] + "." + awscli_version_f.split(".")[1])
     if awscli_version_m < 2.2 :
-        logging.warning(red + "AWS CLI version is: {}. Minimum required is 2.2".format(awscli_version_m) + nc)
+        logging.warning(orange + "AWS CLI version is: {}. Minimum required is 2.2".format(awscli_version_m) + nc)
 
     # Profile check
     try:
@@ -195,12 +240,12 @@ if __name__ == "__main__":
     if args.command.lower() == "start" :
         # Key check
         if args.key == None:
-            args.key = "avh_user"
-            if os.path.isfile(os.path.join(os.path.expanduser('~'),'.ssh/avh_user.pem')) == False :
-                logging.info(red + "No key specified and avh_user key not found, creating 'avh_user' key..." + nc)
+            args.key = "avh_user_{}".format(args.region)
+            if os.path.isfile(os.path.join(os.path.expanduser('~'),'.ssh/avh_user_{}.pem'.format(args.region))) == False :
+                logging.info(orange + "No key specified and avh_user_{} key not found, creating 'avh_user_{}' key...".format(args.region, args.region) + nc)
 
-                cmd = "aws ec2 create-key-pair --key-name avh_user --profile {} --region {}".format(
-                            args.profile, args.region)
+                cmd = "aws ec2 create-key-pair --key-name avh_user_{} --profile {} --region {} --output json".format(
+                            args.region, args.profile, args.region)
                 try:
                     out = subprocess.check_output(cmd, shell=True).decode("utf-8")
                 except subprocess.CalledProcessError:
@@ -208,7 +253,7 @@ if __name__ == "__main__":
                     sys.exit(1)
 
                 outd = json.loads(out)
-                key_file = open(os.path.join(os.path.expanduser('~'),'.ssh/avh_user.pem'), 'x')
+                key_file = open(os.path.join(os.path.expanduser('~'),'.ssh/avh_user_{}.pem'.format(args.region)), 'x')
                 try:
                     key_file.write(outd["KeyMaterial"])
                 except:
@@ -219,11 +264,12 @@ if __name__ == "__main__":
 
                 # Set permissions
                 if platform.system() == "Linux" :
-                    os.system('chmod 600 ~/.ssh/avh_user.pem')
+                    os.system('chmod 600 ~/.ssh/avh_user_{}.pem'.format(args.region))
 
-                logging.info(orange + "Key has been saved as {}.ssh".format(os.path.expanduser('~') + os.path.sep) + os.path.sep + "avh_user.pem." + nc)
+                logging.info(orange + "Key has been saved as {}.ssh".format(os.path.expanduser('~') + os.path.sep) 
+		    + os.path.sep + "avh_user_{}.pem.".format(args.region) + nc)
             else:
-                logging.info(orange + "Using 'avh_user' key in {}.ssh ...".format(os.path.expanduser('~') + os.path.sep) + nc)
+                logging.info(orange + "Using 'avh_user_{}' key in {}.ssh ...".format(args.region, os.path.expanduser('~') + os.path.sep) + nc)
 
         start_avh(args.profile, args.key, args.region, args.cloudinit)