Update Instructions for Telesurgery Workflow (#167)

### Description
- Offer Dockerized environments for running Telesurgery apps in both
simulated and real-world settings, complete with step-by-step guides for
each.

---------

Signed-off-by: Victor Chang <[email protected]>
Signed-off-by: Sachidanand Alle <[email protected]>
Co-authored-by: Sachidanand Alle <[email protected]>
Co-authored-by: Mingxin Zheng <[email protected]>

Author: 3 people

.github/markdown-link-check.json

@@ -9,6 +9,9 @@
         },
         {
             "pattern": "^#%EF%B8%8F.*"
+        },
+        {
+            "pattern": "https://developer.nvidia.com/cuda-downloads"
         }
     ],
     "timeout": "20s",

workflows/robotic_surgery/README.md

@@ -23,7 +23,7 @@
 - 16GB RAM minimum
 
 ### Software Requirements
-- NVIDIA Driver Version >= 555
+- [NVIDIA Driver Version >= 555](https://www.nvidia.com/en-us/drivers/)
 - Python 3.10
 
 ## Quick Start

workflows/robotic_ultrasound/README.md

@@ -21,10 +21,10 @@
 - 50GB of disk space
 
 ### Software Requirements
-- NVIDIA Driver Version >= 555
-- CUDA Version >= 12.6
+- [NVIDIA Driver Version >= 555](https://www.nvidia.com/en-us/drivers/)
+- [CUDA Version >= 12.6]((https://developer.nvidia.com/cuda-downloads))
 - Python 3.10
-- RTI DDS License
+- [RTI DDS License](https://www.rti.com/free-trial)
 
 ## Quick Start
 

workflows/telesurgery/README.md

@@ -2,161 +2,193 @@
 
 ![Telesurgery Workflow](../../docs/source/telesurgery_workflow.jpg)
 
-## Table of Contents
-- [System Requirements](#system-requirements)
-- [Quick Start](#quick-start)
-- [Running the Workflow](#running-the-workflow)
-- [Licensing](#licensing)
 
-## System Requirements
+- [Telesurgery Workflow](#telesurgery-workflow)
+  - [Prerequisites](#prerequisites)
+    - [System Requirements](#system-requirements)
+    - [Common Setup](#common-setup)
+  - [Running the System](#running-the-system)
+    - [Real World Environment](#real-world-environment)
+    - [Simulation Environment](#simulation-environment)
+  - [Advanced Configuration](#advanced-configuration)
+    - [NTP Server Setup](#ntp-server-setup)
+    - [NVIDIA Video Codec Configuration](#advanced-nvidia-video-codec-configuration)
+  - [Troubleshooting](#troubleshooting)
+    - [Common Issues](#common-issues)
+  - [Licensing](#licensing)
 
-### Hardware Requirements
-- Ubuntu 22.04
-- NVIDIA GPU with compute capability 8.6 and 32GB of memory
-   - GPUs without RT Cores, such as A100 and H100, are not supported
-- 50GB of disk space
 
-### Software Requirements
-- NVIDIA Driver Version >= 555
-- CUDA Version >= 12.6
-- Python 3.10
-- RTI DDS License
+## Prerequisites
 
-## Quick Start
+### System Requirements
 
-### x86 & AARCH64 (IGX) Setup
+#### Hardware Requirements
+- Ubuntu >= 22.04
+- NVIDIA GPU with compute capability 8.6 and 8GB of memory
+   - GPUs without RT Cores, such as A100 and H100, are not supported
+- 50GB of disk space
+- XBOX Controller or Haply Inverse 3.
 
-1. **Set up a Docker environment with CUDA enabled (IGX only):**
-   ```bash
-   cd <path-to-i4h-workflows>
-   xhost +
-   workflows/telesurgery/docker/setup.sh run
 
-   # Inside Docker
-   workflows/telesurgery/docker/setup.sh init
-   ```
+#### Software Requirements
+- [NVIDIA Driver Version >= 570](https://developer.nvidia.com/cuda-downloads)
+- [CUDA Version >= 12.8](https://developer.nvidia.com/cuda-downloads)
+- Python 3.10
+- [RTI DDS License](https://www.rti.com/free-trial)
+- [Docker](https://docs.docker.com/engine/install/) 28.0.4+
+- [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) 1.17.5+
 
-2. **Set up the x86 environment with CUDA enabled:**
-   ```bash
-   cd <path-to-i4h-workflows>
-   xhost +
-   workflows/telesurgery/docker/setup.sh init
-   ```
+### Common Setup
 
-3. **Create and activate a [conda](https://www.anaconda.com/docs/getting-started/miniconda/install#quickstart-install-instructions) environment:**
-   ```bash
-   source ~/miniconda3/bin/activate
-   conda create -n telesurgery python=3.10 -y
-   conda activate telesurgery
-   ```
+#### 1. RTI DDS License Setup
+```bash
+export RTI_LICENSE_FILE=<full-path-to-rti-license-file>
+# for example
+export RTI_LICENSE_FILE=/home/username/rti/rti_license.dat
+```
 
-4. **Run the setup script:**
-   ```bash
-   cd <path-to-i4h-workflows>
-   bash tools/env_setup_telesurgery.sh
-   ```
+> [!Note]
+> RTI DDS is the common communication package for all scripts. Please refer to [DDS website](https://www.rti.com/products) for registration. You will need to obtain a license file and set the `RTI_LICENSE_FILE` environment variable to its path.
 
-> Make sure your public key is added to the github account if the git authentication fails.
+#### 2. Environment Configuration
+When running the Patient and the Surgeon applications on separate systems, export the following environment variables:
 
-### Obtain RTI DDS License
+```bash
+export PATIENT_IP="<IP Address of the system running the Patient application>"
+export SURGEON_IP="<IP Address of the system running the Surgeon application>"
 
-RTI DDS is the communication package used by all scripts. Please refer to the [DDS website](https://www.rti.com/products) for registration. You will need to obtain a license file and set the `RTI_LICENSE_FILE` environment variable to its path.
+# Export the following for NTP Server (Optional)
+export NTP_SERVER_HOST="<IP Address of the NTP Server>"
+export NTP_SERVER_PORT="123"
+```
 
-### NTP Server (Optional)
+> [!Note]
+> For NTP settings and variables, refer to the [NTP (Network Time Protocol) Server](#ntp-server-setup) section for additional details.
 
-An NTP (Network Time Protocol) server provides accurate time information to clients over a computer network. NTP is designed to synchronize the clocks of computers to a reference time source, ensuring all devices on the network maintain the same time.
+## Running the System
+
+### Real World Environment
 
+#### 1. Build Environment
 ```bash
-# Run your own NTP server in the background
-docker run -d --name ntp-server --restart=always -p 123:123/udp cturra/ntp
+cd <path-to-i4h-workflows>
+workflows/telesurgery/docker/real.sh build
+```
 
-# Check if it's running
-docker logs ntp-server
+#### 2. Running Applications
 
-# fix server ip in env.sh for NTP Server
-export NTP_SERVER_HOST=<NTP server address>
+##### Patient Application
+```bash
+# Start the Docker Container
+workflows/telesurgery/docker/real.sh run
 
-# To stop the server
-# docker stop ntp-server && docker rm ntp-server
-```
+# Using RealSense Camera with NVIDIA H.264 Encoder
+python patient/physical/camera.py --camera realsense --name room --width 1280 --height 720
 
-### Environment Variables
+# Using CV2 Camera with NVIDIA H.264 Encoder
+python patient/physical/camera.py --camera cv2 --name robot --width 1920 --height 1080
 
-Before running any scripts, set up the following environment variables:
+# Using RealSense Camera with NVJPEG Encoder
+python patient/physical/camera.py --camera realsense --name room --width 1280 --height 720 --encoder nvjpeg
 
-1. **PYTHONPATH**: Set this to point to the **scripts** directory:
-   ```bash
-   export PYTHONPATH=<path-to-i4h-workflows>/workflows/telesurgery/scripts
-   ```
-   This ensures Python can find the modules under the [`scripts`](./scripts) directory.
+# Using CV2 Camera with NVJPEG Encoder
+python patient/physical/camera.py --camera cv2 --name robot --width 1920 --height 1080 --encoder nvjpeg
+```
 
-2. **RTI_LICENSE_FILE**: Set this to point to your RTI DDS license file:
-   ```bash
-   export RTI_LICENSE_FILE=<path-to-rti-license-file>
-   ```
-   This is required for the DDS communication package to function properly.
+##### Surgeon Application
+```bash
+# Start the Docker Container
+workflows/telesurgery/docker/real.sh run
 
-3. **NDDS_DISCOVERY_PEERS**: Set this to the IP address receiving camera data:
-   ```bash
-   export NDDS_DISCOVERY_PEERS="surgeon IP address"
-   ```
-More recommended variables can be found in [env.sh](./scripts/env.sh).
+# Start the Surgeon Application with NVIDIA H.264 Decoder
+python surgeon/camera.py --name [robot|room] --width 1280 --height 720 2> /dev/null
 
-## Running the Workflow
+# Run the Surgeon Application with NVJPEG Decoder
+python surgeon/camera.py --name [robot|room] --width 1280 --height 720 --decoder nvjpeg
+```
 
+##### Gamepad Controller Application
 ```bash
-cd <path-to-i4h-workflows>/workflows/telesurgery/scripts
-source env.sh  # Make sure all env variables are correctly set in env.sh
+# Start the Docker Container
+workflows/telesurgery/docker/real.sh run
 
-export PATIENT_IP=<patient IP address>
-export SURGEON_IP=<surgeon IP address>
+# Run the Gamepad Controller Application
+python surgeon/gamepad.py --api_host ${PATIENT_IP} --api_port 8081
 ```
-> Make sure the MIRA API Server is up and running (port: 8081) in the case of a physical world setup.
 
-### [Option 1] Patient in Physical World _(x86 / aarch64)_
-
-When running on IGX (aarch64), ensure you are in the Docker environment set up previously.
+### Simulation Environment
 
+#### 1. Build Environment
 ```bash
-# Stream camera output
-python patient/physical/camera.py --camera realsense --name room --width 1280 --height 720
-python patient/physical/camera.py --camera cv2 --name robot --width 1920 --height 1080
+cd <path-to-i4h-workflows>
+workflows/telesurgery/docker/sim.sh build
 ```
 
-### [Option 2] Patient in Simulation World _(x86)_
+#### 2. Running Applications
 
+##### Patient Application
 ```bash
-# Download the assets
-i4h-asset-retrieve
+# Start the Docker Container
+workflows/telesurgery/docker/sim.sh run
 
-python patient/simulation/main.py [--encoder nvc]
+# Start the Patient Application with NVIDIA H.264 Encoder
+python patient/simulation/main.py
+
+# Start the Patient Application with NVJPEG Encoder
+python patient/simulation/main.py --encoder nvjpeg
 ```
 
-### Surgeon Connecting to Patient _(x86 / aarch64)_
+##### Surgeon Application
+```bash
+# Start the Docker Container
+workflows/telesurgery/docker/sim.sh run
 
-# capture robot camera stream
-NDDS_DISCOVERY_PEERS=${PATIENT_IP} python surgeon/camera.py --name robot --width 1280 --height 720 [--decoder nvc]
+# Start the Surgeon Application with NVIDIA H.264 Decoder
+python surgeon/camera.py --name robot --width 1280 --height 720 2> /dev/null
 
-# capture room camera stream (optional)
-NDDS_DISCOVERY_PEERS=${PATIENT_IP} python surgeon/camera.py --name room --width 1280 --height 720 [--decoder nvc]
+# Run the Surgeon Application with NVJPEG Decoder
+python surgeon/camera.py --name robot --width 1280 --height 720 --decoder nvjpeg
+```
+
+##### Gamepad Controller Application
+```bash
+# Start the Docker Container
+workflows/telesurgery/docker/sim.sh run
 
-# Connect to gamepad controller and send commands to API Server
+# Run the Gamepad Controller Application
 python surgeon/gamepad.py --api_host ${PATIENT_IP} --api_port 8081
 ```
 
-### Using H.264/HEVC Encoder/Decoder from NVIDIA Video Codec
+## Advanced Configuration
+
+### NTP Server Setup
+An NTP (Network Time Protocol) server provides accurate time information to clients over a computer network. NTP is designed to synchronize the clocks of computers to a reference time source, ensuring all devices on the network maintain the same time.
+
+```bash
+# Run your own NTP server in the background
+docker run -d --name ntp-server --restart=always -p 123:123/udp cturra/ntp
 
-Camera data can be streamed using either the H.264 or HEVC (H.265) codecs. To enable this for the Patient and Surgeon applications, use the `--encoder nvc` or `--decoder nvc` argument, respectively.
+# Check if it's running
+docker logs ntp-server
 
-Encoding parameters can be customized in the Patient application using the `--encoder_params` argument, as shown below:
+# fix server ip in env.sh for NTP Server
+export NTP_SERVER_HOST=<NTP server address>
+
+# To stop the server
+docker stop ntp-server && docker rm ntp-server
+```
+
+### Advanced NVIDIA Video Codec Configuration
+
+The applications streams H.264 by default using NVIDIA Video Codec. Additional encoding parameters can be customized in the Patient application using the `--encoder_params` argument:
 
 ```bash
 python patient/simulation/main.py --encoder nvc --encoder_params patient/nvc_encoder_params.json
 ```
 
-#### Sample Encoding Parameters for the NVIDIA Video Codec
+#### Sample Encoding Parameters
 
-Here’s an example of encoding parameters in JSON format:
+Here's an example of encoding parameters in JSON format:
 
 ```json
 {
@@ -169,16 +201,55 @@ Here’s an example of encoding parameters in JSON format:
 }
 ```
 
-> [!NOTE]
-> H.264 or HEVC (H.265) codecs are available on x86 platform only.
+### Advanced NVJPEG Configuration
+
+Adjust the quality of encoded frames using the NVJPEG encoder by editing the [nvjpeg_encoder_params.json](./scripts/patient/nvjpeg_encoder_params.json) file. Simply change the quality parameter to a value between 1 and 100:
 
-### Important Notes
-1. You may need to run multiple scripts simultaneously in different terminals or run in background (in case of docker)
-2. A typical setup requires multiple terminals running:
-   - Patient: Camera1, Camera2, Controller, etc.
-   - Surgeon: Camera1, Camera2, Controller, etc.
+```json
+{
+    "quality": 90
+}
+```
 
-If you encounter issues not covered in the notes above, please check the documentation for each component or open a new issue on GitHub.
+## Troubleshooting
+
+### Common Issues
+
+#### Docker Build Error
+Q: I get the following error when building the Docker image:
+```bash
+ERROR: invalid empty ssh agent socket: make sure SSH_AUTH_SOCK is set
+```
+
+A: Start the ssh-agent
+```bash
+eval "$(ssh-agent -s)" && ssh-add
+```
+#### Unable to launch the applications when using NVIDIA Video Codec
+
+Q: I'm getting an error when I start the application with the NVIDIA Video Codec.
+
+```BASH
+[error] [nv_video_encoder.cpp:101] Failed to create encoder: LoadNvEncApi : Current Driver Version does not support this NvEncodeAPI version, please upgrade driver at /workspace/holohub/build/nvidia_video_codec/_deps/nvc_sdk/NvEncoder/NvEncoder.cpp:82
+```
+
+**A:** NVIDIA Video Codec requires CUDA version 12 (driver version 570.0) or later. Check out the [NVIDIA Video Codec System Requirements](https://developer.nvidia.com/nvidia-video-codec-sdk/download) section for more details. **
+
+
+#### Update CUDA Driver on IGX
+```bash
+# ssh to igx-host to run the following commands
+sudo systemctl isolate multi-user
+
+sudo apt purge nvidia-kernel-*
+sudo add-apt-repository ppa:graphics-drivers/ppa
+sudo apt update
+
+sudo apt-get -y install linux-headers-nvidia-tegra aptitude
+sudo aptitude install nvidia-driver-570-open # Resolve any conflicts
+
+# hard reboot igx (soft reboot may not work)
+```
 
 ## Licensing