Sunday 14 August 2016

Final Report on Porting Rock to RTEMS

GSOC : Final Report
This post would serve as my final report on my project as well as it would act as a reference to anyone who wishes to continue with my project. 

Introduction and Project idea
Rock(http://rock-robotics.org/stable/) is an open source software framework for development of robotic systems. Rock along with RTEMS has been used in ESA for quite a long time and that is why there had been several porting attempts of Rock. However due to some or the other reason none of the attempts could be merged before.And though scripts are present without maintenance in a year or so they lose their value.

This GSoC project aimed at gathering all the scripts updating them and finally preparing a usable port of Rock for RTEMS. Other specifics of the project can be obtained from my proposal

Status of Work Done
As per the  originally planned approach from the proposal,the work is done up to  around 80% of Phase-1 and around 10% of Phase-3.The details are as follows (please refer my proposal for a clearer idea):

(a) Phase-1 work
The phase 1 work was divided into two parts:

       (i) Updating the Porting scripts of all the external                                                                                         dependencies.The work was done through my repository. All the scripts were tested to build               on Linux Mint 17.2. They install fine and work with the RTT fork. However specific                           applications to test them on QEMU is still a work in progress. 

      (ii) Updating the Porting scripts of the Rock core dependencies.For this the RTT was updated but             subsequent tests on the changes made were failing.The work ahead should not begin without               testing the RTT.The commits were made on a RTT fork.

Work left: Typelib, Orogen, Rock base Packages could not be ported as I was stuck with RTT.Example applications for this is also a work in progress(see this).Then an Autoproj script for automating the entire port had to be written which could not be done. After that Rock examples and packages useful to RTEMS should have been selected and ported.However this could not be done at all.

(b)Phase-2 work

Work left: The phase-2 part of porting examples and packages could not be done at all.

(c) Phase-3 work
The phase-3 work of the project was preparing a RSB recipe for the port.The entire port was not over,so as soon as a part was over its build recipe was provided. I  started with a libxml recipe. The work was done through my  RSB fork.

Work left: The total port had to be automated by RTEMS Source Builder. All dependencies external as well as internal dependencies (RTT,Typelib,Orogen) had to be provided RSB recipes for building which is left.

The workflow
The workflow in this project is important because Rock is an amalgamation of a large number of software frameworks.For porting Rock to RTEMS, RTEMS support needs to be added to all these softwares. When it comes to Rock, the core dependencies should be ported first rather than the base dependencies.A detailed explanation of this follows (my initial workflow as mentioned in the proposal needed to be changed with time.)

STEP-1: Update all the external dependencies of Rock. i.e. 1) Boost (v1.54 recommended)
                                                                                                2) Eigen
                                                                                                3) Libxml
                                                                                                4) OmniORB

STEP-2: Make RSB recipes for them as soon as they were successfully built and tested.

STEP-3:  Start porting the Rock core dependencies followed by the Rock Base dependencies.

Explanation:
STEP-1: 

1)Boost(v1.54) is an open source peer-reviewed library(http://www.boost.org/).Boost is essentially used for getting the serialization and the threading libraries for RTT.It is highly recommended that v 1.54 be used so that we don't get compatibility errors.Only Boost v1.54 contains the specific templates that the whole of RTT is designed for. Changing the codes in so many places will be really cumbersome. But check the latest versions once more to make sure. For cross-compiling for RTEMS what you have to do is just enable static libraries that is done through the flags(see Phase 1 commits).

2)Eigen is a library for linear algebra ,matrix and vector operations.It is esssential for the Rock Base packages.The patches were just to make sure latest versions get installed.

3)Libxml is an XML C parser that aids in the serialization process. Libxml is however not used anymore. Its successor libxml2 is used. For the changes in cross compilation see Phase 1 commits.It was however mentioned that in later stages of the port libxml2 needs to be replaced  by castxml2 or llvm front end.However I could not reach that stage so I am avoiding a comment on it.

4)OmniORB is an Object Request Broker and it helps in invoking operations on an object on an object on a computer across a network, on different Operating System, On diffrent CPU systems or even implemented with diffrent programming languages.The tests with the OmniORB for the patches worked well.For testing the instructions in the OmniORB Documentation canbe followed using QEMU to run RTEMS.For understanding its functionality.It takes a little effort to understand CORBA.Please refer previous posts on this blog.

This part of my project is complete and following are the instructions to build the external Rock dependencies.

1> Prepare the following directory structure Port/x86/ and then in that /Build and /Install.Then in the / /Build directory prepare /Build/pkgs.

       

2> Go to Port/x86/Build and clone my repository

      $git clone https://github.com/Sambeet161616/Rock-Port_ana.git

3> Now go to the cloned repository and change the user name as per your system in the script env.sh.
     
      $ cd Rock-Port_ana/
      $  gedit env.sh

       Then in  gedit edit the underlined in the script env.sh as per your system

      export BUILD_PREFIX=/home/sambeet/Port/x86/Build
      export INSTALL_PREFIX=/home/sambeet/Port/x86/Install

4> Now give the following command  to set the environment variables
   
     $ source env.sh

5> Now run each of the scripts to build the dependencies
    $ bash libxml.sh 
     
     Replace libxml.sh with boost_cross_compiler.sh, eigen_cross_compiler.sh,                                      xerces_c_cross_compiler.sh, prepare_RTEMS_mod.sh.

Note: Please do ensure proper internet connection while running the scripts.

6> After the scripts are run successfully, we can see the boost, eigen, libxml, omniORB, rtems and xerces folders in the Port/x86/Install folder.


STEP 2:

To make RSB recipes for the above builds, RSB documentation provides easy to understand instructions. The work on this is on going  on my repository.

STEP 3:

The Rock core dependencies needs to be ported in a particular order i.e rtt,typelib,then orogen followed by base packages (base/orogen,base/cmake,base/types).


To add RTEMS support to RTT and test it is where I am stuck now, so I would like to write to give a rather good explanation of the problems I encountered.

Rock software package fundamentally is the usage of Orocos Real Time Toolkit Libraries to build and control real time components in a very convenient and user friendly way. Rock uses Orogen,the code generator to generate the components and a ruby-scripting for controlling the components.(N.B. This is not the only way to run Orocos components.There are two other ways to complete this: (a)Using the Orocos Component Library http://www.orocos.org/stable/documentation/ocl/v2.x/doc-xml/orocos-deployment.html (b)Manually write and compile the component using Cmake. Have a look at this) Very important in this regard is the RTT Cheat Sheet

Adding RTEMS support to RTT.The detailed approach to adding RTEMS support is mentioned in Orocos Component Builder's Manual. Apart from adapting the fosi.h file for RTEMS which is very similar to GNU Linux since RTEMS supports POSIX standard, we should ensure a static build of RTT libraries as the Dynamic Loader is a relatively new concept for RTEMS as compared to static. My patches on RTT build fine.But I still could not build a RTT component and all tests regarding this have been failing. For testing the RTT on could utilise the manually written component. And the following command from the terminal could be given (could give errors especially linking errors with Boost,need work!)

sambeet@Holmes ~/NewRockPort/x86/Build/rock/rtt $ sudo /home/sambeet/NewRockPort/x86/Install/rtems/4.11.0-rc3/bin/i386-rtems4.11-g++ -B /home/sambeet/NewRockPort/x86/Install/rtems/4.11.0-rc3/i386-rtems4.11/pc486/lib/ -specs bsp_specs -qrtems HelloWorld.cpp /usr/local/lib/liborocos-rtt-rtems.a /usr/local/lib/orocos/rtems/types/librtt-typekit-rtems.a -I ./ -I /usr/local/boost_1_54_0/include/ -L /usr/local/boost_1_54_0/lib/ -I /home/sambeet/NewRockPort/x86/Install/rtems/4.11.0-rc3/i386-rtems4.11/pc486/lib/include/ -L /home/sambeet/NewRockPort/x86/Install/rtems/4.11.0-rc3/i386-rtems4.11/pc486/lib/ -static -DOROCOS_TARGET=rtems /usr/local/boost_1_54_0/lib/libboost_filesystem.a /usr/local/boost_1_54_0/lib/libboost_serialization.a 2>&1 | tee testout.txt

However the tests are still not working i.e I cannot still guarantee that RTT is ported successfully.The important issues in this regard:

1) Make sure that Boost 1.54 is used as mentioned before.

2) Before attempting a static build on RTEMS , one should attempt a static build on GNU Linux.

3) For the testing do not use OCL (Orocos Components Library) Deployer as Rock does not use it. Remove any references to OCL.

I would also like to share my correspondences with my Rock co-mentors to give a better idea to anybody who would like to go into the details of the errors I faced.Here is the link

That will be all the work that is done on this project. I hope this blog post can be enough for anyone who wants to start on this project.I will summarise by providing a Getting Started on the porting Rock to RTEMS:

1) Before starting complete the RTEMS Hello World Examples.

2) Complete the Rock Basic tutorials to get a basic idea about Rock.

3) Next try to build a manual(handwritten) component as well as an Orogen component by studying RTT Manuals.

4) Go through this blog post to get a basic idea of what is the general workflow and work according to progress of the project (this is the current status, I will be updating this blog as I progress).

Please note, there are two other porting Rock to RTEMS. One is the original ESA project whose scripts I have  provided here.I have started my work on this. There is however another project by Matthias Goldhoorn.However many of the codes are in private repositories,and moreover Matthias Goldhoorn is not a part of Rock anymore since we spoke last.I did not understand the changes in buildconf.I welcome others to give a try by this approach.

5)Now you can start the port preferably from external dependencies.Try replicating Phase-1 commits and then proceed to RTT port (or the latest work).

Thank you for patient reading :)














1 comment:

  1. Nice blog and absolutely outstanding. You can do something much better but i still say this perfect.Keep trying for the best.
    builderall tools

    ReplyDelete