Intelligence.
Engineered.

Development of an autonomous lawn-mowing AGV using the SiMa.ai Modalix board.

Project technology stack and contributions:

• Machine learning acceleration for LightGlue and SuperPoint.
• RTAB-Map for simultaneous localization and mapping (SLAM).
• Nav2 for autonomous navigation.
• ROS 2 software stack.
• Sensor stack development, including device drivers and ROS 2 packages.
• EKF sensor fusion combining IMU, wheel odometry, and visual odometry.
• Behaviour Tree integration.
• Image processing on the EV74 DSP.

Contribution to the Project:

•  Simultaneous Localization and Planning ( SLAM )  and 3D Map research and development.
  
• Prototype development for new features ( Hardware / Software ).

Contribution to the Project:

• Development of firmware for micro crazily drones 
• Development of ROS based controller software based on Behaviour Tree Modelling

Development of Software for proprietary Autopilot software for Drones based on PX4 software stack

Contribution to the Project:

• NUTTX  RTOS ecosystem and integration of proprietary algorithms.
• Device Driver development and integration of hardware. 

Pick and place is one of the upcoming domains that is pushing for automation. The vision-based solution was developed using 2D and 3D cameras for specific customer requirements.

Contribution to the Project:

• Initial Software architecture design and prototype development.
• Validation of vision algorithms as recommended by the vision team and testing again Project Requirement Specification.
• Discussion with development and solutions to any limitations/bugs as discovered.
• Final Code analysis/review of the development team.

The satellite docking validation system is a project required by an esteem client which wanted to validate algorithms/sensors for satellite docking in space. This was achieved by two ABB IRB6700 robots mounted on 10 meter long conveyor belt. Both robots move towards each other, with motion directed by data received in real time from sensor on each of the robot end actuator and centralized algorithm for docking.

Contribution in the project:

• Externally Guided Motion (EGM) programming for both IRB6700 and conveyor.
• ABB Robot Studio based simulation setup design to test system containing the IRB6700 and conveyor motion.
• Platform for interfacing client satellite docking algorithm with real time motion control using EGM.
• Coordinate transformation logic (between ABB, sensor and client algorithm) along with robot and conveyor motion management based on algorithm motion commands.

Cognitive intelligence for a robot with understanding of its environment was the focus of the Research project. User interacts with the robot using speech commands which based on the understanding of the environment (objects, actions etc) robot either performs or replies with question. This initial prototype was developed in 5 months with code using NLP, ML and Graph Libraries in Python.

Contribution in the projec:

• Requirement gathering, Analysis and Software Design of the cognitive system.
• Design and development of prototype model in Python.
• Integration of cognitive system with ABB YuMi robot.

The Next Generation Robot Platform Software is one of the most important project which ABB Robotics is executing. This involves discarding legacy Robot Software Platform and creating new Software Platform from scratch which being more Open Source Compliant. It incorporates modular architecture, network security and allows 3rd Party software interfacing. The software is getting developed on Python and C/C++. The project also involves migrating from VxWorks to Linux OS.

My contribution included but not limited to:

• Design of Robot Software architecture for Software Management.
• Development of Software module for Robot in Python which handles the Software Installation and Management including security.
• Development of PC interface to Industrial robot in C# and Python.
• Linux platform support to other teams in migrating software from VxWork to Linux.
• Boot time Optimization in new Robot System Software using better System configuration and migrating to UBIFS file system from JFFS. This activity reduced the Boot Time from 24 Sec to 18 Sec.
• Developed the mechanism and software for updating Linux RFS while in operation. New RAMFS file system is loaded to memory and Linux OS is directed to utilize RAMFS. Once the switch is over, flash is formatted and new updated RFS is programmed.

This sponsored research project by Renault-Nissan focused on developing autonomous overtaking maneuver for vehicles. The project demonstrated the effectiveness of developed algorithm on TORCS and UC/Win simulators. The developed algorithm utilized non-linear time scaling and velocity cone based collision detection. Publication based on the work was accepted in IVS 2015.

Project Included:

• Multiple candidate trajectories are generated once 
• The algorithm could predict collision with multiple vehicles in parallel lanes using velocity cone approach while deciding overtaking trajectory.
• Collision free overtaking trajectory with least change in velocity profile is selected among multiple candidate trajectories.
• Non-linear time scaling is utilized for generating trajectory for vehicle.
• Developing C++ interface between developed algorithm and TORCS / UC-Win simulators.

AR Drone 2.0 is a commercially available Quad-copter which can be controlled using library provided by Parrot or using AT commands and uses WIFI to transfer the commands to Quad-copter.

In this Experiment Asus Xtion Pro which is OpenNI compliant device is used with NITE middle ware to detect the motion and hand gestures of the user. Once detected ,these are converted to appropriate motion command for the quadcopter.

Project components :

• Using Asus Xtion Pro and NITE middle-ware solution was developed to detect various hand gestures like UP, Down, WAVE etc.
• AR Drone 2.0 library was used along with remote control to calculate and calibrate the Quad-copter motion movements. Calibrated values were then converted passed to user program, allowing precise movement of the quadcopter.
• AR Drone 2.0 control logic was then linked to the hand gestures detected by Asus Xtion Pro, hence allowing drone to be controlled using hand gestures.

In September 2008, an Open Source software platform primarily for smart mobile phones called Android was launched by the Open Handset alliance led by Google. Android is a complete embedded software stack, comprising of a modified Linux based kernel, middleware, application framework and applications. To run Android on their platform, Original Equipment Manufacturers (OEMs) need to make hardware specific changes such as adding the Bootloader and writing kernel drivers for their devices and interfacing the Android stack with hardware platform drivers. STMicro took a strategic decision to move Android from Mobile platform to other Consumer Electric platforms such as SetTop Box and Digital TV.

Responsibilities in the project:

• Configuring and porting Linux kernel to support Android Framework including drivers. The Android was ported to SH4 (CPU) proprietary platform of STMicro.
• Interfacing STMicro proprietary media framework HAVANA with Android framework . Android framework was also extended for features supported by Media framework of STMicro.
• IR Remote(LIRC) interface support was added in Android to allow navigation in absence of touch control in set-top box.
• Porting Linux debugging and performance monitoring tools to Android eg : Strace , GDB , Valgrind etc.

ST Micro decided to port Android to Set-top Box, where we faced a simple problem "How to navigate inside Android without touch screen ?"

As the Set-Top Box was connected to TV set and only interface available for interaction was a normal IR remote . Therefore it was decided to integrate IR set-box remote with Android subsystem allowing seamless experience to the user.

Project milestones :

• Native Keyboard interface in Android was activated.
• Support for LIRC was integrated inside Android Linux Kernel.
• Small keyboard kernel driver was created, which would emulate keyboard commands when IR remote keys data is transferred from user space.
• Application was written in "C" and embedded in Android framework which would catch the IR commands received from kernel driver and convert them to Keyboard commands.
• Android in-build keyboard support module detects different key-press events from kernel drivers. Keyboard commands from user space are transferred to keyboard kernel driver. The kernel driver then emulates corresponding keyboard key press events, allowing Android to respond to IR remote.

System upgrade is a tedious task which requires from formatting flash partitions, downloading new image to burning new image into the new system. Mostly in embedded world JTAG is prominently used for accessing ,configuring and formatting the system flash.

Due to high cost of JTAG, it was decided to replace the system upgrader using JTAG with serial port (UART) and USB based updating process.

Project Responsibilities:

• Development of new Micro Boot loader with XModem support over UART.
• Developing new version of standard Boot loader with support for upgradation.
• Developing module in Boot loader to read PFF (Philips File Format) from USB and upgrading the Nand Flash.
• Design Bootlaoder state machine with GUI feedback for upgradation process.

Embedded systems require high level of security and authentication in terms of software that runs on the system . Secure Boot is term normally used in systems where at boot time software modules are verified for authenticity before being launched.This is achieved by adding cryptic signature over each software module image and being verified before executing.

Points :

• Our embedded system, Digital signature based software authentication at boot-up was implemented using AES and SHA1. This requires each software component to verify the digital signature of module it is initiating at startup.
• The Digital Signature based Secure Boot was implemented in software which being computationally intensive increased the overall boot time of the system.
• Solution for reducing increased boot time was shifting the computationally intensive operations to GPU processing unit, which had better hardware to handle such task and CPU acting just a coordinator for data flow.
• The GPU and CPU were able to interact using hardware based Semaphore and thus providing faster Boot time with Secure boot.

Embedded Systems these days rely on Nand Flash as main storage medium due to its small size , high I/O speed and low power consumption .Nand Flash is mostly available in SLC and MLC configuration where SLC offers more stable operations and MLC data density and cheaper cost.

In our Embedded system we decided to move from SLC Nand Flash to MLC Flash based SD card as main storage. This shift required low level software modification including linux kernel, drivers support and user-space modification.

My responsibilties were, but not limited to:

• Modifying and configuring boot loader to support SD card
• Configuring Linux kernel and adding driver support for SD card.
• Integrated new module in BootLoader to support flashing of SD card with new partition information (MBR) and software binaries in partitions available in PFF(Philips File Format).
• Developed tools for PC to generate propriety format PFF which contains all the binaries and information which is used by Boot loader to write the SD card with the Software and partition information.
• Shifting to new File System (EXT4) from bare Nand Flash based (JFFS2) file system.
• Performance measurements of I/O with new file system .
• Boot time measurements and optimization with new system.

The Embedded system was designed to operate with Nand Flash but when requirement came to shift the system to SD card Interface as the main Storage medium, Pre-Study work and prototyping was required.

Important points :

• Micro Boot loader development to support SD card Interface.
• Linux configuration and SD card driver development .
• Study of new Partition table management.
• Modification of Upgradation mechanism of the system to support SD card.
• Performance and Boot time Bench Marking and Optimization.

My contribution:

• Linux configuration and RFS modification to support the new interface.
• Study of Partition table management and implementation proposal in our system.
• Development of module in boot loader to support new partition table structure (MBR and EBR).
• Modification of PFF (Philips File Format) structure to support SD card images.
• Development to tools to generate MBR and EBR tables on the run to be places in the SD card while System Flash updating.
• Development of Micro Boot Loader and integrating with Micro Controller .
• Development of Boot loader StateMachine design for upgradation with GUI feedback to user.

Fast Boot in Embedded systems is essential for consumer devices in Mobile, automotive and entertainment markets.

This project "System level" optimization was started with a view of reducing the Boot-Up of current by 50% by optimizing the system boot structure.

Project components :

• Optimizing the Boot loader configuration to remove any unwanted modules and reducing boot-up time.
• Next in line was the Linux Kernel configuration where it was concentrated on various services started at boot and segregating them into"What needs to be started Now"and "What can be delayed".
• By changing structure of drivers attached to Linux kernel and delaying there loading we were able to reduce the size of kernel image by 10%.
• Hardware was configured at start up differently to support faster Boot up.

Embedded Systems are shifting towards Linux as the main Operating System due to Free and Open Source distribution . Linux supports large variety architectures and flexible enough to be used from Micro-processor based systems to Servers. Linux can support variety of hardware devices and configurations thus require drivers support to be added for accessing new hardware devices.

My project responsibilities were:

• Development and porting of I2C driver from one board to another.
• Adding support for new I2C buses in the driver with Bug Fixing.
• Maintaining Nand Flash Driver

This Robotic project idea was conceived in my B.Tech college 3rd year, 2006. We created a Speech controlled robot, which would detect user speech patters, recognize them with pre-trained data and generate appropriate action. This used silicon based voice recognition chip ( HM2007 ) interfaced with 8051 micro-controller.

My role in project :

• Designing the digital circuit required for the voice recognition chip, its support peripheral chips.
• Designing, fabricating and testing the PCB at our PCB fabrication lab in college.
• Designing and testing of motor driving circuit and interface circuit with 8051 micro-controller.
• Designing of power supply for the circuit (AC to DC) .
• Programming the 8051c in assembly language to operate the servo motors of robot and receive inputs from HM2007 and control the robot car in predefined manner.

How the System worked :

• The HM2007 speech recognition chip was programmed to recognize the user speech ( Training mode) commands by button mic connect to it.
• The HM2007 was then taken to recognition mode using special digital keyboard connected to it.
• Speech commands while in training mode are saved in NVRAM available on the circuit board connected to the HM2007. These commands are then tagged with unique numbers and provided as output for recognized speech.
• The output of the speech recognition chip HM2007, which was programmed to receive speech inputs and generate corresponding unique speech detection tags. HM2007 output ports were connected to I/O ports of 8051 microcontroller. Micro controller was programmed to perform predefined operations once unique speech tags are received from HM2007.