OBSW for the EIVE project
Go to file
Robin Mueller 85d0ffed0d
Some checks failed
EIVE/eive-obsw/pipeline/head There was a failure building this commit
implement transitions
2024-04-16 17:40:13 +02:00
.idea cmake.xml 2023-07-25 16:32:22 +02:00
.run add Q7S run config 2023-02-17 17:32:15 +01:00
archive Added new payload PST 2023-11-14 13:25:53 +01:00
arduino@3ea528dc5f arduino update 2020-10-13 20:56:02 +02:00
automation CI/CD: Use 8 jobs 2023-04-17 11:55:08 +02:00
bsp_egse renaming of fsfw_hal components 2022-11-10 18:07:59 +01:00
bsp_hosted that should do the job 2024-04-11 10:49:29 +02:00
bsp_linux_board now hosted should compile again 2022-12-14 09:53:38 +01:00
bsp_q7s need to fix some linker errors.. 2024-04-16 15:33:42 +02:00
bsp_te0720_1cfa renaming of fsfw_hal components 2022-11-10 18:07:59 +01:00
cmake continue PLOC SUPV 2023-11-13 15:33:11 +01:00
common need to fix some linker errors.. 2024-04-16 15:33:42 +02:00
doc added logo 2021-07-26 11:04:56 +02:00
dummies cleanup 2024-03-18 17:21:08 +01:00
fsfw@fb2e480705 bump fsfw 2024-04-16 13:58:20 +02:00
generators reran gens 2024-04-10 11:53:21 +02:00
hooks added hook to automatically update submodules after checkout 2023-02-17 12:32:29 +01:00
linux implement transitions 2024-04-16 17:40:13 +02:00
misc update .cproject file 2023-06-25 01:01:21 +02:00
mission compiles again 2024-04-16 17:33:11 +02:00
scripts update intall-obsw-yocto.sh 2023-04-12 13:40:21 +02:00
test avoid exceptions 2023-03-08 14:50:25 +01:00
thirdparty bumped sagittactl and tmtc 2024-02-29 12:40:17 +01:00
tmtc@a8d0143b1e continue the refactoring 2024-04-12 18:30:18 +02:00
unittest that should do the job 2024-04-11 10:49:29 +02:00
watchdog avoid exceptions 2023-03-08 14:50:25 +01:00
.clang-format added clang script and forward declarations 2022-01-17 15:57:52 +01:00
.dockerignore added dockerignore 2021-05-24 15:47:48 +02:00
.gitignore continue PLOC SUPV 2023-11-13 15:33:11 +01:00
.gitmodules re-add str submodule 2023-04-13 17:58:36 +02:00
CHANGELOG.md changelog 2024-04-10 11:57:17 +02:00
clone-submodules-no-privlibs.sh added hook to automatically update submodules after checkout 2023-02-17 12:32:29 +01:00
CMakeLists.txt bump version 2024-04-10 11:56:19 +02:00
docker-compose.yml trying older version 2021-06-08 15:53:38 +02:00
Justfile justfile update 2022-01-10 14:02:58 +01:00
LICENSE added LICENSE and NOTICE file 2021-08-20 10:01:37 +02:00
NOTICE added LICENSE and NOTICE file 2021-08-20 10:01:37 +02:00
q7s-env-em.sh this should be robust 2023-05-11 19:33:06 +02:00
q7s-env.sh this should be robust 2023-05-11 19:33:06 +02:00
README.md add documentation of initial delay 2023-06-29 15:06:25 +02:00
release-checklist.md prep v4.0.0 final 2023-06-22 17:34:56 +02:00

EIVE On-Board Software

Index

  1. General
  2. Prerequisites
  3. Building the Software
  4. Useful and Common Host Commands
  5. Setting up Prerequisites
  6. Remote Debugging
  7. Remote Reset
  8. TMTC testing
  9. Direct Debugging
  10. Transfering Files to the Q7S
  11. Q7S OBC
  12. Static Code Analysis
  13. Eclipse
  14. CLion
  15. Running the OBSW on a Raspberry Pi
  16. Running OBSW on EGSE
  17. Manually preparing sysroots to compile gpsd
  18. FSFW
  19. Coding Style

General information

Target systems:

  • OBC with Linux OS
  • Host System
    • Generic software components which are not dependant on hardware can also be run on a host system. All host code is contained in the bsp_hosted folder
    • Tested for Linux (Ubuntu 20.04) and Windows 10
  • Raspberry Pi
    • EIVE OBC can be built for Raspberry Pi as well (either directly on Raspberry Pi or by installing a cross compiler)

The steps in the primary README are related to the main OBC target Q7S. The CMake build system can be used to generate build systems as well (see helper scripts in cmake/scripts:

  • Linux Raspberry Pi: See special section below. Uses the bsp_linux_board folder
  • Linux Trenz TE7020_1CFA: Uses the bsp_te0720_1cfa folder
  • Linux Host: Uses the bsp_hosted BSP folder and the CMake Unix Makefiles generator.
  • Windows Host: Uses the bsp_hosted BSP folder, the CMake MinGW Makefiles generator and MSYS2.

Prerequisites

There is a separate prerequisites which specifies how to set up all prerequisites.

Building the OBSW and flashing it on the Q7S

  1. ARM cross-compiler installed, either as part of Vivado 2018.2 installation or as a separate download. The Xiphos SDK also installs a cross-compiler, but its version is currently too old to compile the OBSW (7.3.0).
  2. Q7S sysroot on local development machine. It is installed by the Xiphos SDK
  3. Recommended: Eclipse or Vivado 2018.2 SDK for OBSW development
  4. TCF agent running on Q7S

Hardware Design

  1. Vivado 2018.2 for programmable logic design

Building the software

CMake

When using Windows, run theses steps in MSYS2.

  1. Clone the repository with

    git clone https://egit.irs.uni-stuttgart.de/eive/eive-obsw.git
    
  2. Update all the submodules

    git submodule update --init
    
  3. Create two system variables to pass the system root path and the cross-compiler path to the build system. You only need to do this once when setting up the build system. Example for Unix:

    export CROSS_COMPILE_BIN_PATH=<absolutePathToCrossCompilerBinPath>
    export ZYNQ_7020_SYSROOT=<absolutePathToSysroot>
    
  4. Ensure that the cross-compiler is working with ${CROSS_COMPILE_BIN_PATH}/arm-linux-gnueabihf-gcc --version and that the sysroot environmental variables have been set like specified in the root filesystem chapter.

  5. Run the CMake configuration to create the build system in a build-Debug-Q7S folder. Add -G "MinGW Makefiles in MinGW64 on Windows.

    mkdir cmake-build-debug-q7s && cd cmake-build-debug-q7s
    cmake -DTGT_BSP="arm/q7s" -DCMAKE_BUILD_TYPE=Debug ..
    cmake --build . -j
    

    Please note that you can also use provided shell scripts to perform these commands.

    cp scripts/q7s-env.sh ..
    cp scripts/q7s-env-em.sh ..
    

    Adapt these scripts for your needs by editing the CROSS_COMPILE_BIN_PATH and ZYNQ_7020_SYSROOT. After that, you can run the following commands to set up the FM build

    cd ..
    ./q7s-env.sh
    q7s-make-debug.sh
    

    You can build the EM setup by running

    export EIVE_Q7S_EM=1
    

    or by running the q7s-env-em.sh script instead before setting up the build configuration.

    The shell scripts will invoke a Python script which in turn invokes CMake with the correct arguments to configure CMake for Q7S cross-compilation. You can look into the command output to see which commands were run exactly.

    There are also different values for -DTGT_BSP to build for the Raspberry Pi or the Beagle Bone Black: arm/raspberrypi and arm/beagleboneblack.

  6. Build the software with

    cd cmake-build-debug-q7s
    cmake --build . -j
    

Preparing and executing an OBSW update

A OBSW update consists of a xz compressed file eive-sw-update.tar.xz which contains the following two files:

  1. Stripped OBSW binary eive-obsw-stripped
  2. OBSW version text file with the name obsw_version.txt

These files can be created manually:

  1. Build the release image inside cmake-build-release-q7s

  2. Switch into the build directory

  3. Run the following command to create the version file

    git describe --tags --always --exclude docker_* > obsw_version.txt
    

    You can also use the create-version-file.sh helper shell script located in the scripts folder to do this.

  4. Set the Q7S user as the file owner for both files

    sudo chown root:root eive-obsw-stripped
    sudo chown root:root obsw_version.txt
    
  5. Run the following command to create the compressed archive

    tar -cJvf eive-sw-update.tar.xz eive-obsw-stripped obsw_version.txt
    

You can also use the helper script create-sw-update.sh inside the build folder after sourcing the q7s-env.sh helper script to perform all steps including a rebuild.

After creating these files, they need to be transferred onto the Q7S to either the /mnt/sd0/bin or /mnt/sd1/bin folder if the OBSW update is performed from the SD card. It can also be transferred to the /tmp folder to perform the update from a temporary directory, which does not rely on any of the SD cards being on and mounted. However, all files in the temporary directory will be deleted if the Linux OS is rebooted for any reason.

After both files are in place (this is checked by the OBSW), the example command sequence is used by the OBSW to write the OBSW update to the QSPI chip 0 and slot 0 using SD card 0:

tar -xJvf eive-update.tar.xz
xsc_mount_copy 0 0
cp eive-obsw-stripped /tmp/mntupdate-xdi-qspi0-nom-rootfs/usr/bin/eive-obsw
cp obsw_update.txt /tmp/mntupdate-xdi-qspi0-nom-rootfs/usr/share/obsw_update.txt
writeprotect 0 0 1

Some context information about the used commands:

  1. It mounts the target chip and copy combination into the /tmp folder using the xsc_mount_copy <chip> <copy> utility. This also unlocks the writeprotection for the chip. The mount point name inside /tmp depends on which chip and copy is used

    • Chip 0 Copy 0: /tmp/mntupdate-xdi-qspi0-nom-rootfs
    • Chip 0 Copy 1: /tmp/mntupdate-xdi-qspi0-gold-rootfs
    • Slot 1 Copy 0: /tmp/mntupdate-xdi-qspi1-nom-rootfs
    • Slot 1 Copy 1: /tmp/mntupdate-xdi-qspi1-gold-rootfs
  2. Writing the file with a regular cp <source> <target> command

  3. Enabling the writeprotection using the writeprotect <chip> <copy> 1 utility.

Build for the Q7S target root filesystem with yocto

The EIVE root filesystem will contain the EIVE OBSW and the Watchdog component. It is currently generated with yocto, but the tool can not compile the primary OBSW due to toolchain version incompatibility. Therefore, the OBSW components are currently compiled using the toolchain specified in this README (e.g. installed by Vivado).

However, it is still possible to install the two components using yocto. A few helper files were provided to make this process easier. The following steps can be used to install the OBSW components and a version file to the yocto sources for the generation of the complete EIVE root file system image. The steps here are shown for Ubuntu, you can use the according Windows helper scripts as well.

  1. Copy the q7s-env.sh script to the same layer as the eive-obsw.

    cp scripts/q7s-env.sh ..
    cd ..
    ./q7s-env.sh
    q7s-make-release.sh
    
  2. Compile the OBSW components in release mode

    cd cmake-build-release-q7s
    cmake --build . -j
    
  3. Make sure the q7s-yocto repository or the q7s-package repository and its q7s-yocto submodule were cloned in the same directory layer as the eive-obsw.

  4. Run the install script to install the files into q7s-yocto.

    install-obsw-yocto.sh
    
  5. Navigate into the q7s-yocto repo and review the changes. You can then add and push those changes.

  6. You can now rebuild the root filesystem with the updated OBSW using yocto. This probably needs to be done on another machine or in a VM. The q7s-yocto repository contains details on how to best do this.

Building in Xilinx SDK 2018.2

  1. Open Xilinx SDK 2018.2
  2. Import project
    • File → Import → C/C++ → Existing Code as Makefile Project
  3. Set build command. Replace <target> with either debug or release.
    • When on Linux right click project → Properties → C/C++ Build → Set build command to make <target> -j
      • -j causes the compiler to use all available cores
      • The target is used to either compile the debug or the optimized release build.
    • On windows create a make target additionally (Windows → Show View → Make Target)
      • Right click eive_obsw → New
      • Target name: all
      • Uncheck "Same as the target name"
      • Uncheck "Use builder settings"
      • As build command type: cmake --build .
      • In the Behaviour tab, you can enable build acceleration
  4. Run build command by double clicking the created target or by right clicking the project folder and selecting Build Project.

Useful and Common Commands

Build generation

Replace Debug with Release for release build. Add -G "MinGW Makefiles or -G "Ninja" on Windows or when ninja should be used. You can build with cmake --build . -j after build generation. You can finds scripts in cmake/scripts to perform the build commands automatically.

Q7S OBSW

The EIVE OBSW is the default target if no target is specified.

Debug

mkdir cmake-build-debug-q7s && cd cmake-build-debug-q7s
cmake -DTGT_BSP=arm/q7s -DCMAKE_BUILD_TYPE=Debug ..
cmake --build . -j

Release

mkdir cmake-build-release-q7s && cd cmake-build-release-q7s
cmake -DTGT_BSP=arm/q7s -DCMAKE_BUILD_TYPE=Release ..
cmake --build . -j

Hosted OBSW

You can also use the FSFW OSAL host to build on Windows or for generic OSes. You can use the clone-submodules-no-privlibs.sh script to only clone the required (non-private) submodules required to build the hosted OBSW.

mkdir cmake-build-debug && cd cmake-build-debug
cmake -DFSFW_OSAL=host -DCMAKE_BUILD_TYPE=Debug ..
cmake --build . -j

You can also use the linux OSAL:

mkdir cmake-build-debug && cd cmake-build-debug
cmake -DFSFW_OSAL=linux -DCMAKE_BUILD_TYPE=Debug ..
cmake --build . -j

Please note that some additional Linux setup might be necessary. You can find more information in the Linux section of the FSFW example

Q7S Watchdog

The watchdog will be built along side the primary OBSW binary.

Unittests

To build the unittests, the corresponding target must be specified in the build command. The configure steps do not need to be repeated if the folder has already been configured.

mkdir cmake-build-debug && cd cmake-build-debug
cmake ..
cmake --build . --target eive-unittests -j

Connect to EIVE flatsat

DNS

ssh eive@flatsat.eive.absatvirt.lw

IPv6

ssh eive@2001:7c0:2018:1099:babe:0:e1fe:f1a5

IPv4

ssh eive@192.168.199.227

Connecting to the serial console or ssh console

A serial console session is up permanently in a tmux session

Serial console

You can check whether the sessions exist with tmux ls. This is the command to connect to the serial interface of the FM using the RS422 interface of the flight preparation panel:

tmux a -t q7s-fm-fpp

If the session does not exist, you can create it like this

tmux new -s q7s-fm-fpp -t /bin/bash
launch-q7s-fpp

Other useful tmux commands:

  • Enable scroll mode: You can press ctrl + b and then [ (AltGr + 8) to enable scroll mode. You can quit scroll mode with q.
  • Kill a tmux session: run ctrl + b and then k.
  • Detach from a tmux session: run ctrl + b and then d
  • Pipe last 3000 lines into file for copying or analysis: 1. ctrl + b and : 2. capture-pane -S -3000 + enter 3. save-buffer /tmp/tmux-output.txt + enter

SSH console

You can use the following command to connect to the Q7S with ssh:

q7s-fm-ssh

Set up all port forwarding at once

Port forwarding is necessary for remote-debugging using the tcf-agent, copying files with scp & q7s-cp.py and sending TMTC commands. You can specify the -L option multiple times to set up all port forwarding at once.

ssh -L 1534:192.168.155.55:1534 \
   -L 1535:192.168.155.55:22 \
   -L 1536:192.168.155.55:7301 \
   -L 1537:127.0.0.1:7100 \
   -L 1538:192.168.133.10:1534 \
   -L 1539:192.168.133.10:22 \
   -L 1540:192.168.133.10:7301 \
   eive@2001:7c0:2018:1099:babe:0:e1fe:f1a5 \
   -t 'CONSOLE_PREFIX="[Q7S Tunnel]" /bin/bash'

There is also a shell script called q7s-port.sh which can be used to achieve the same.

Setting up prerequisites

Getting system root for Linux cross-compilation

Cross-compiling any program for an embedded Linux board generally required parts of the target root file system on the development/host computer. For the Q7S, you can install the cross-compilation root file system by simply installing the SDK. You can find the most recent SDK here.

If you are compiling for the Q7S or the TE7020, the ZYNQ_7020_SYSROOT environment variable must be set to the location of the SDK compile sysroot. Here is an example on how to do this in Ubuntu, assuming the SDK was installed in the default location

export ZYNQ_7020_SYSROOT="/opt/xiphos/sdk/ark/sysroots/cortexa9hf-neon-xiphos-linux-gnueabi"

If you are comiling for the Raspberry Pi, you have to set the LINUX_ROOTFS environmental variable instead. You can find a base root filesystem for the Raspberry Pi here.

Installing Vivado and the Xilinx development tools

It's also possible to perform debugging with a normal Eclipse installation by installing the TCF plugin and downloading the cross-compiler as specified in the section below. However, if you want to generate the *.xdi files necessary to update the firmware, you need to installed Vivado with the SDK core tools.

  • Install Vivado 2018.2 and Xilinx SDK. Install the Vivado Design Suite - HLx Editions - 2018.2 Full Product Installation instead of the updates. It is recommended to use the installer.

  • Install settings. In the Devices selection, it is sufficient to pick SoC → Zynq-7000:




Installing on Linux - Device List Issue

When installing on Ubuntu, the installer might get stuck at the Generating installed device list step. When this happens, you can kill the installation process (might be necessara to kill a process twice) and generate this list manually with the following commands, according to this forum entry.

  1. Install the following library

    sudo apt install libncurses5
    
  2. Execute the following command

    sudo <installRoot>/Vivado/2018.2/bin/vivado -nolog -nojournal -mode batch -source 
    <installRoot>/.xinstall/Vivado_2018.2/scripts/xlpartinfo.tcl -tclargs 
    <installRoot>/Vivado/2018.2/data/parts/installed_devices.txt
    

For Linux, you can also download a more recent version of the Linaro 8.3.0 cross-compiler from here

Compatibility issues with wayland on more recent Linux distributions

If Vivado crashes and you find following lines in the hs_err_pid* files:

#
# An unexpected error has occurred (11)
#
Stack:
/opt/Xilinx/Vivado/2017.4/tps/lnx64/jre/lib/amd64/server/libjvm.so(+0x923da9) [0x7f666cf5eda9]
/opt/Xilinx/Vivado/2017.4/tps/lnx64/jre/lib/amd64/server/libjvm.so(JVM_handle_linux_signal+0xb6) [0x7f666cf653f6]
/opt/Xilinx/Vivado/2017.4/tps/lnx64/jre/lib/amd64/server/libjvm.so(+0x9209d3) [0x7f666cf5b9d3]
/lib/x86_64-linux-gnu/libc.so.6(+0x35fc0) [0x7f66a993efc0]
/opt/Xilinx/Vivado/2017.4/tps/lnx64/jre/lib/amd64/libawt_xawt.so(+0x42028) [0x7f664e24d028]
...

You can solve this by logging in with xorg like specified here.

Using docnav on more recent Linux versions

If you want to use docnav for navigating Xilinx documentation, it is recommended to install it as a standalone version from here. This is because the docnav installed as part of version 2018.2 requires libpng12, which is not part of more recent disitributions anymore.

Installing toolchain without Vivado

You can download the toolchains for Windows and Linux from the EIVE cloud.

Installing CMake and MSYS2 on Windows

  1. Install MSYS2 and CMake first.

  2. Open the MinGW64 console. It is recommended to set up aliases in .bashrc to navigate to the software repository quickly

  3. Run the following commands in MinGW64

    pacman -Syu
    

    It is recommended to install the full base development toolchain

    pacman -S base-devel
    

    It is also possible to only install required packages

    pacman -S mingw-w64-x86_64-cmake mingw-w64-x86_64-make mingw-w64-x86_64-gcc mingw-w64-x86_64-gdb python3
    

Installing CMake on Linux

  1. Run the following command

    sudo apt-get install cmake
    

Updating system root for CI

If the system root is updated, it needs to be manually updated on the buggy file server. If access on buggy.irs.uni-stuttgart.de is possible with ssh and the rootfs in the cloud was updated as well, you can update the rootfs like this:

cd /var/www/eive/tools
wget https://eive-cloud.irs.uni-stuttgart.de/index.php/s/SyXpdBBQX32xPgE/download/cortexa9hf-neon-xiphos-linux-gnueabi.tar.gz

Setting up UNIX environment for real-time functionalities

Please note that on most UNIX environments (e.g. Ubuntu), the real time functionalities used by the UNIX pthread module are restricted, which will lead to permission errors when creating these tasks and configuring real-time properites like scheduling priorities.

To solve this issues, try following steps:

  1. Edit the /etc/security/limits.conf file and add following lines at the end:
<username>   hard   rtprio  99
<username>   soft   rtprio  99

The soft limit can also be set in the console with ulimit -Sr if the hard limit has been increased, but it is recommended to add it to the file as well for convenience. If adding the second line is not desired for security reasons, the soft limit needs to be set for each session. If using an IDE like eclipse in that case, the IDE needs to be started from the console after setting the soft limit higher there. After adding the two lines to the file, the computer needs to be restarted.

It is also recommended to perform the following change so that the unlockRealtime script does not need to be run anymore each time. The following steps raise the maximum allowed message queue length to a higher number permanently, which is required for some framework components. The recommended values for the new message length is 130.

  1. Edit the /etc/sysctl.conf file

    sudo nano /etc/sysctl.conf
    

    Append at end:

    fs/mqueue/msg_max = <newMsgMaxLen>
    

    Apply changes with:

    sudo sysctl -p
    

    A possible solution which only persists for the current session is

    echo <newMsgMax> | sudo tee /proc/sys/fs/mqueue/msg_max
    

TCF-Agent

Most of the steps specified here were already automated

  1. On reboot, some steps have to be taken on the Q7S. Set static IP address and netmask

    ifconfig eth0 192.168.133.10
    ifconfig eth0 netmask 255.255.255.0
    
  2. tcfagent application should run automatically but this can be checked with

    systemctl status tcfagent
    
  3. If the agent is not running, check whether agent is located inside usr/bin. You can run it manually there. To perform auto-start on boot, have a look at the start-up application section.

Remote Debugging

Open SSH connection to flatsat PC:

ssh eive@flatsat.eive.absatvirt.lw

or

ssh eive@2001:7c0:2018:1099:babe:0:e1fe:f1a5

or

ssh eive@192.168.199.227

If the static IP address of the Q7S has already been set, you can access it with ssh

ssh root@192.168.133.10

If this has not been done yet, you can access the serial console of the Q7S like this

picocom -b 115200 /dev/q7sSerial

The flatsat has the aliases and shell scripts q7s_ssh and q7s_serial for this task as well. If the serial port is blocked for some reason, you can kill the process using it with q7s_kill.

You can use AltGr + X to exit the picocom session.

To debug an application, first make sure a static IP address is assigned to the Q7S. Run ifconfig on the Q7S serial console.

ifconfig

Set IP address and netmask with

ifconfig eth0 192.168.133.10
ifconfig eth0 netmask 255.255.255.0

To launch application from Xilinx SDK setup port fowarding on the development machine (not on the flatsat!)

ssh -L 1534:192.168.133.10:1534 eive@2001:7c0:2018:1099:babe:0:e1fe:f1a5 -t bash

This forwards any requests to localhost:1534 to the port 1534 of the Q7S with the IP address 192.168.133.10. This needs to be done every time, so it is recommended to create an alias or shell script to do this quickly.

Note: When now setting up a debug session in the Xilinx SDK or Eclipse, the host must be set to localhost instead of the IP address of the Q7S.

Remote Reset

  1. Launch xilinx hardware server on flatsat with alias
launch-hwserver-xilinx
  1. On host PC start xsc
  2. In xsct console type the follwing command to connect to the hardware server (replace </flatsat-pc-ip-address/> with the IP address of the flatsat PC. Can be found out with ifconfig)
connect -url tcp:</flatsat-pc-ip-address/>:3121
  1. The following command will list all available devices
targets
  1. Connect to the APU of the Q7S
target </APU-number/>
  1. Perform reset
rst

TMTC testing

The OBSW supports sending PUS TM packets via TCP or the PDEC IP Core which transmits the data as CADU frames. To make the CADU frames receivabel by the TMTC porgram, a python script is running as systemd service named tmtc_bridge on the flatsat PC which forwards TCP commands to the TCP server of the OBC and reads CADU frames from a serial interface.

You can check whether the service is running the following command on the flatsat PC

systemctl status tmtc_bridge

The PUS packets transported with the CADU frames are extracted and forwared to the TMTC program's TCP client. The code of the TMTC bridge can be found here. To connect the TMTC program to the TMTC-bridge a port forwarding from a host must be set up with the following command:

ssh -L 1537:127.0.0.1:7100 eive@2001:7c0:2018:1099:babe:0:e1fe:f1a5 -t bash

You can print the output of the systemd service with

journalctl -u tmtc_bridge

This can be helpful to determine whether any TCs arrive or TMs are coming back.

Note: The encoding of the TM packets and conversion of CADU frames takes some time. Thus the replies are received with a larger delay compared to a direct TCP connection.

Direct Debugging

  1. Assign static IP address to Q7S

    • Open serial console of Q7S (Accessible via the micro-USB of the PIM, see also Q7S user manual chapter 10.3)
    • Baudrate 115200
    • Login to Q7S:
      • user: root
      • pw: root
  2. Connect Q7S to workstation via ethernet

  3. Make sure the netmask of the ehternet interface of the workstation matches the netmask of the Q7S

    • When IP address is set to 192.168.133.10 and the netmask is 255.255.255.0, an example IP address for the workstation is 192.168.133.2
  4. Make sure th tcf-agent is running by checking systemctl status tcf-agent

  5. In Xilinx SDK 2018.2 right click on project → Debug As → Debug Configurations

  6. Right click Xilinx C/C++ applicaton (System Debugger) → New →

  7. Set Debug Type to Linux Application Debug and Connectin to Linux Agent

  8. Click New

  9. Give connection a name

  10. Set Host to static IP address of Q7S. e.g. 192.168.133.10

  11. Test connection (This ensures the TCF Agent is running on the Q7S)

  12. Select Application tab

  • Project Name: eive_obsw
  • Local File Path: Path to OBSW application image with debug symbols (non-stripped)
  • Remote File Path: /tmp/<OBSW NAME>

Transfering Files to the Q7S

To transfer files from the local machine to the Q7S, use port forwarding

ssh -L 1535:192.168.133.10:22 eive@2001:7c0:2018:1099:babe:0:e1fe:f1a5

An example file can be copied like this

scp -P 1535 example root@localhost:/tmp

Copying a file from Q7S to flatsat PC

scp -P 22 root@192.168.133.10:/tmp/kernel-config /tmp

From a windows machine files can be copied with putty tools (note: use IPv4 address)

pscp -scp -P 22 eive@192.168.199.227:</directory-to-example-file/>/example-file </windows-machine-path/>

A helper script named q7s-cp.py can be used together with the q7s-port.sh script to make this process easier.

Q7S OBC

Launching an application at start-up - deprecated

This way to enable auto-startup is deprecated. It is instead recommended to tweak the yocto recipes file for the related systemd service to enable auto-startup with SYSTEMD_AUTO_ENABLE.

You can also do the steps performed here on a host computer inside the q7s-rootfs directory of the Q7S base repository. This might be more convenient while also allowing to update all images at once with the finished rootfs.xdi.

Load the root partiton from the flash memory (there are to nor-flash memories and each flash holds two xdi images). Note: It is not possible to modify the currently loaded root partition, e.g. creating directories. To do this, the parition needs to be mounted.

  1. Disable write protection of the desired root partition

    writeprotect 0 0 0 # unlocks nominal image on nor-flash 0  
    
  2. Mount the root partition

    xsc_mount_copy 0 0 # Mounts the nominal image from nor-flash 0
    

    The mounted partition will be located inside the /tmp folder

  3. Copy the executable to /usr/bin

  4. Make sure the permissions to execute the application are set

    chmod +x application
    
  5. Create systemd service in /etc/systemd/system. The following shows an example service.

    cat > example.service
    [Unit]
    Description=Example Service
    StartLimitIntervalSec=0
    
    [Service] 
    Type=simple
    Restart=always
    RestartSec=1
    User=root
    ExecStart=/usr/bin/application
    
    [Install]
    WantedBy=multi-user.target
    
  6. Enable the service. This is normally done with systemctl enable <service> which would create a symlink in the multi-user.target.wants directory. However, this is not possible when the service is created for a mounted root partition. It is also not possible during run time because symlinks can't be created in a read-only filesystem. Therefore, relative symlinks are used like this:

    cd etc/systemd/system/multi-user.target.wants/
    ln -s  ../example.service example.service
    

    You can check the symlinnks with ls -l

  7. The modified root partition is written back when the partion is locked again.

    writeprotect 0 0 1 
    
  8. Now verify the application start by booting from the modified image

    xsc_boot_copy 0 0
    
  9. After booting verify if the service is running

    systemctl status example
    

Current user systemd services

The following custom systemd services are currently running on the Q7S and can be found in the /etc/systemd/system folder. You can query that status of a service by running systemctl status <serviceName>.

eive-watchdog

The watchdog will create a pipe at /tmp/watchdog-pipe which can be used both by the watchdog and the EIVE OBSW. The watchdog will only read from this pipe while the OBSW will only write to this pipe. The watchdog checks for basic ASCII commands as a first basic feature set. The most important functionality is that the watchdog cant detect if a timeout has happened. This can happen beause the OBSW is hanging (or at least the CoreController thread) or there is simply now OBSW running on the system. It does to by checking whether the FIFO is regulary written to, which means the EIVE OBSW is alive.

If the EIVE OBSW is alive, a special file called /tmp/obsw-running will be created. This file can be used by any other software component to query whether the EIVE OBSW is running. The EIVE OBSW itself can be configured to check whether this file exists, which prevents two EIVE OBSW instances running on the Q7S at once.

If a timeout occurs, this special file will be deleted as well. The watchdog and its configuration will be directly integrated into this repostory, which makes adaptions easy.

tcf-agent

This starts the /usr/bin/tcf-agent program to allows remote debugging

eive-early-config

This is a configuration script which runs early after local-fs.target and sysinit.target Currently only pipes the output of xsc_boot_copy into the file /tmp/curr_copy.txt which can be used by other software components to read the current chip and copy.

eive-post-ntpd-config

This is a configuration scripts which runs after the Network Time Protocol has run. This script currently sets the static IP address 192.168.133.10 and starts the can interface.

Initial boot delay

You can create a file named boot_delays_secs.txt inside the home folder to delay the OBSW boot for 6 seconds if the file is empty of for the number of seconds specified inside the file. This can be helpful if something inside the OBSW leads to an immediate crash of the OBC.

PCDU

Connect to serial console of P60 Dock

picocom -b 500000 /dev/ttyUSBx

General information

cmp ident

List parameter table: x values: 1,2 or 4

param list x

Table 4 lists HK parameters Changing parameters First switch to table where parameter shall be changed (here table id is 1)

p60-dock # param mem 1
p60-dock # param set out_en[0] 1
p60-dock # param get out_en[0]
GET out_en[0] = 1

Core commands

Display currently running image:

xsc_boot_copy

Rebooting currently running image:

xsc_boot_copy -r

Setting time on Q7S

Setting date and time (only timezone UTC available)

timedatectl set-time 'YYYY-MM-DD HH:MM:SS'

Setting UNIX time

date +%s -s @1626337522

This only sets the system time and does not updating the time of the real time clock. To harmonize the system time with the real time clock run

hwclock -w

Reading the real time clock

hwclock --show

pa3tool Host Tool

The pa3tool is a host tool to interface with the ProASIC3 on the Q7S board. It was installed on the clean room PC but it can also be found on the Traq platform.

For more information, see Q7S datasheet.

Creating files with cat and echo

The only folder which can be written in the root filesystem is the tmp folder.

You can create a simple file with initial content with echo

echo "Hallo Welt" > /tmp/test.txt
cat /tmp/test.txt

For more useful combinations, see this link.

Using the scratch buffer of the ProASIC3

The ProASIC3 has a 1024 byte scratch buffer. The values in this scratch buffer will survive a reboot, so this buffer can be used as an alternative to the SD cards to exchange information between images or to store mission critical information.

You can use xsc_scratch --help for more information.

Write to scratch buffer:

xsc_scratch write TEST "1"

Read from scratch buffer:

xsc_scratch read TEST

Read all keys:

xsc_scratch read

Get fill count:

xsc_scratch read | wc -c

Custom device names in Linux with the udev module

You can assign custom device names using the Linux udev system. This works by specifying a rules file inside the /etc/udev/rules.d folder which creates a SYMLINK if certain device properties are true.

Each rule is a new line inside a rules file. For example, the rule

SUBSYSTEM=="tty", ATTRS{interface}=="Dual RS232-HS", ATTRS{bInterfaceNumber}=="01", SYMLINK+="ploc_supv

Will create a symlink /dev/ploc_supv if a connected USB device has the same interface and bInterfaceNumber properties as shown above.

You can list the properties for a given connected device using udevadm. For example, you can do this for a connected example device /dev/ttyUSB0 by using

udevadm info -a /dev/ttyUSB0

Using system when debugging

Please note that when using a system call in C++/C code and debugging, a new thread will be spawned which will appear on the left in Eclipse or Xilinx SDK as a sh program. The debugger might attach to this child process automatically, depending on debugger configuration, and the process needs to be selected and continued/started manually. You can enable or disable this behaviour by selecting or deselecting the Attach Process Children option in the Remote Application Configuration for the TCF plugin like shown in the following picture


Libgpiod

Detect all gpio device files:

gpiodetect

Get info about a specific gpio group:

gpioinfo <name of gpio group>

The following sets the gpio 18 from gpio group gpiochip7 to high level.

gpioset gpiochip7 18=1

Setting the gpio to low.

gpioset gpiochip7 18=0

Show options for setting gpios.

gpioset -h

To get the state of a gpio:

gpioget <gpiogroup> <offset>

Example to get state: gpioget gpiochip7 14

Both the MIOs and EMIOs can be accessed via the zynq_gpio instance which comprises 118 pins (54 MIOs and 64 EMIOs).

Xilinx UARTLIE

Get info about ttyUL* devices

cat /proc/tty/driver

I2C

Getting information about some I2C device

ls /sys/class/i2c-dev/i2c-0/device/device/driver

This shows the memory mapping of /dev/i2c-0.

You can use the i2cdetect utility to scan for I2C devices. For example, to do this for bus 0 (/dev/i2c-0), you can use

i2cdetect -r -y 0

CAN

ip link set can0 down
ip link set can0 type can loopback off
ip link set can0 up type can bitrate 1000000

Following command sends 8 bytes to device with id 99 (for petalinux)

cansend can0 -i99 99 88 77 11 33 11 22 99

For Q7S use this:

cansend can0 5A1#11.22.33.44.55.66.77.88

Turn loopback mode on:

ip link set can0 type can bitrate 1000000 loopback on 

Reading data from CAN:

candump can0

Dump content of file in hex

cat file.bin | hexdump -C

All content will be printed with

cat file.bin | hexdump -v

To print only the first X bytes of a file

cat file.bin | hexdump -v -n X 

Preparation of a fresh rootfs and SD card

See q7s-package repository README

Running cppcheck on the Software

Static code analysis can be useful to find bugs. cppcheck can be used for this purpose. On Windows you can use MinGW64 to do this.

pacman -S mingw-w64-x86_64-cppcheck

On Ubuntu, install with

sudo apt-get install cppcheck

You can use the Eclipse integration or you can perform the scanning manually from the command line. CMake will be used for this.

Run the CMake build generation commands specified above but supply -DCMAKE_EXPORT_COMPILE_COMMANDS=ON to the build generation. Invoking the build command will generate a compile_commands.json file which can be used by cppcheck.

cppcheck --project=compile_commands.json --xml 2> report.xml

Finally, you can convert the generated .xml file to HTML with the following command

cppcheck-htmlreport --file=report.xml --report-dir=cppcheck --source-dir=..

CLion

CLion is the recommended IDE for the development of the hosted version of EIVE. You can also set up CLion for cross-compilation of the primary OBSW.

There is a shared .idea/cmake.xml file to get started with this. To make cross-compilation work, two special environment variables need to be set:

  • ZYNQ_7020_ROOTFS pointing to the root filesystem
  • CROSS_COMPILE pointing to the the full path of the cross-compiler without the specific tool suffix. For example, if the the cross-compiler tools are located at /opt/q7s-gcc/gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf/bin, this variable would be set to /opt/q7s-gcc/gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf/bin/arm-linux-gnueabihf

Eclipse

When using Eclipse, there are two special build variables in the project properties → C/C++ Build → Build Variables called Q7S_SYSROOT or RPI_SYSROOT. You can set the sysroot path in those variables to get any additional includes like gpiod.h in the Eclipse indexer.

Setting up default Eclipse for Q7S projects - TCF agent

The TCF agent can be used to perform remote debugging on the Q7S.

  1. Copy the .cproject file and the .project file inside the misc/eclipse folder into the repo root

    cd eive-obsw
    cp misc/eclipse/.cproject .
    cp misc/eclipse/.project .
    
  2. Open the repo in Eclipse as a folder.

  3. Install the TCF agent plugin in Eclipse from the releases. Go to Help → Install New Software and use the download page, for example https://download.eclipse.org/tools/tcf/releases/1.7/1.7.0/ to search for the plugin and install it. You can find the newest version here

  4. Go to Window → Perspective → Open Perspective and open the Target Explorer Perspective. Here, the Q7S should show up if the local port forwarding was set up as explained previously. Please note that you have to connect to localhost and port 1534 with port forwaring set up.

  5. A launch configuration was provided, but it might be necessary to adapt it for your own needs. Alternatively:

  • Create a new TCF Remote Application by pressing the cogs button at the top or going to Run → Debug Configurations → Remote Application and creating a new one there.

  • Set up the correct image in the main tab (it might be necessary to send the image to the Q7S manually once) and file transfer properties

  • It is also recommended to link the correct Eclipse project.

After that, comfortable remote debugging should be possible with the Debug button.

A build configuration and a shell helper script has been provided to set up the path variables and build the Q7S binary on Windows, but a launch configuration needs to be newly created because the IP address and path settings differ from machine to machine.

Running the EIVE OBSW on a Raspberry Pi

Special section for running the EIVE OBSW on the Raspberry Pi. The Raspberry Pi build uses the bsp_rpi BSP folder, and a very similar cross-compiler.

For running the software on a Raspberry Pi, it is recommended to follow the steps specified in the fsfw example and using the TCF agent to have a similar set-up process also required for the Q7S. You should run the following command first on your Raspberry Pi

sudo apt-get install gpiod libgpiod-dev

to install the required GPIO libraries before cloning the system root folder.

Running OBSW on EGSE

The EGSE is a test system from arcsec build arround a raspberry pi 4 to test the star tracker. The IP address of the EGSE (raspberry pi) is 192.168.18.31. An ssh session can be opened with

ssh pi@192.168.18.31

Password: raspberry

To run the obsw perform the following steps:

  1. Build the cmake EGSE Configuration
    • the sysroots for the EGSE can be found here
    • toolchain for linux host can be downloaded from here
    • toolchain for windows host from here (the raspios-buster-armhf toolchain is the right one for the EGSE)
  2. Disable the ser2net systemd service on the EGSE
$ sudo systemctl stop ser2net.service
  1. Power on the star tracker by running
$ ~/powerctrl/enable0.sh`
  1. Run portforwarding script for tmtc tcp connection and tcf agent on host PC
$ ./scripts/egse-port.sh
  1. The star tracker can be powered off by running
$ ~/powerctrl/disable0.sh

Manually preparing sysroots to compile gpsd

Copy all header files from here to the /usr/include directory and all static libraries to /usr/lib.

Flight Software Framework (FSFW)

An EIVE fork of the FSFW is submodules into this repository. To add the master upstream branch and merge changes and updates from it into the fork, run the following command in the fsfw folder first:

git remote add upstream https://egit.irs.uni-stuttgart.de/fsfw/fsfw.git
git remote update --prune

After that, an update can be merged by running

git merge upstream/master

Alternatively, changes from other upstreams (forks) and branches can be merged like that in the same way.

Coding Style

  • the formatting is based on the clang-format tools

Setting up auto-formatter with clang-format in Xilinx SDK

  1. Help → Install New Software → Add
  2. In location insert the link http://www.cppstyle.com/luna
  3. The software package CppStyle should now be available for installation
  4. On windows download the clang-formatting tools from https://llvm.org/builds/. On linux clang-format can be installed with the package manager.
  5. Navigate to Preferences → C/C++ → CppStyle
  6. Insert the path to the clang-format executable
  7. Under C/C++ → Code Style → Formatter, change the formatter to CppStyle (clang-format)
  8. Code can now be formatted with the clang tool by using the key combination Ctrl + Shift + f

Setting up auto-fromatter with clang-format in eclipse

  1. Help → Eclipse market place → Search for "Cppstyle" and install
  2. On windows download the clang-formatting tools from https://llvm.org/builds/. On linux clang-format can be installed with the package manager.
  3. Navigate to Preferences → C/C++ → CppStyle
  4. Insert the path to the clang-format executable
  5. Under C/C++ → Code Style → Formatter, change the formatter to CppStyle (clang-format)
  6. Code can now be formatted with the clang tool by using the key combination Ctrl + Shift + f