ISRO| Out of This World!

Project Details

Awards & Nominations

ISRO has received the following awards and nominations. Way to go!

Global Nominee

The Challenge | Out of This World!

Create an app to pilot an unmanned aerial system (UAS), such as a NASA space drone, utilizing the 6-axis gyro sensor within a smartphone or tablet. The piloting app can be combined with multiple sensors for flight precision and the best maneuverable flight techniques for off-Earth planetary drones.

Towards Perfection

Increasing the control over your Unmanned Aerial Vehicle (UAV) by providing deep levels of manual control in extraordinary circumstances.

ISRO

About us!


Pitch deck: https://docs.google.com/presentation/d/1Luyp92FIXVhhQb66Fpqtm7bC3Gfp16Nx4bSJE-bRUyU/edit?usp=sharing

Motivation

The recent failure of Chandrayaan 2, a lunar exploration mission developed by the Indian Space Research Organisation, kept our team wondering “why couldn’t a better solution be developed?”. Last week we heard about NASA SpaceApps Challenge from our university. Looking through the challenges, we found “Out of this world”. A challenge that asks us to design a manual control system for an unmanned aerial vehicle (UAV) under extraordinary circumstances like technical glitches. We right away knew this is the challenge we will be working on!

Learning from the recent example, we realized the gravity of the problem. Loss of control leads to the end of your mission, huge monetary losses and most importantly it contributes to the accumulation of space debris, which may lead to Kessler syndrome. Implementing our solution will help to organizations like NASA save multi-million- or multi-billion-dollar worth equipment and give their mission a new life.

Solution

We, team ISRO, provide greater control over your UAV. During a technical glitch, we give manual control of the UAV to the pilot using his/her smartphone, with the help of already developed transmission technologies like Wifi becoming more accessible and networks being established in space.

But you can’t expect the pilot to take manual control for the entire mission. This is where our code injecting system comes handy. Our innovative code injecting system with which a user can send pieces of code from our app to the UAV without needing it to restart or recompile helping to improve the functionality of the UAV throughout the mission.

Our smart app feature made using artificial intelligence and machine learning model suggests the pilot with the best inputs to be taken to reach the initial path again. It also has an autopilot mode, where our smart app takes the manual control of the UAV. The modes can easily be switched depending upon the situation. This reduces the possibility of human errors to a great extent.

However, the two major problems we still face is the time delay(owing to signal latency) between the input command and its execution, and the obstacles that the UAV may encounter on the new path. Let us suppose our UAV is 380,000 km away from earth, there is a time delay of 2 seconds (1.24 seconds in theory). Our kinematics formulas take the three- dimensional picture into consideration and estimate the position of the UAV 4 seconds later. This future position is shown on the app so that the pilot is always on par with the UAV. Yet another problem is the obstacles the UAV may face on its new path. To solve this we have the proximity sensors which give real-time input to our app. Our app visualizes its size and shape on the screen, facilitating maneuvering of the UAV.

Lastly, the security of our app is of great concern to us as well as to our users namely NASA, Black Swift Technologies, etc.

Features

1)Code Injection

2)Predicted path

3)Smart App

4)Obstacle detection and visualization

5)Security System

Code Injection: Our app provides the pilot with complete flexibility to amend the source code and change the functionality of code running the autonomous UAV. For instance, we encountered some issues with the pre-fixed path the UAV has to follow to reach its destination and we would like to update the path.

Furthermore, this feature comes in handy during extraordinary conditions like technical glitches, or abnormal behavior of UAV. In such situations, having the ability to update the code helps the UAV to get back to its normal functionality giving the pilot superior control over the UAV.

You can view our prototype at https://youtu.be/oE0ZFWDUZxQ

Predicted path: During technical glitches, it’s very likely for the UAV to deviate from its desired path. Taking into consideration initial parameters of UAV like velocity, position, acceleration, and inclination in 3D, we can predict the path that the UAV would follow and is reflected in the app which expedites the user understanding of UAV's situation. Based on the predicted path and taking into account the time delay, the position of the UAV reflected in the app would be the predicted position of the UAV. For instance, time delay is 2 seconds, if the current position of the UAV is position “X” and would be at position “Y” 4 seconds later, the app shows the position of UAV at position “Y” as it takes 2 seconds to receive the signal from the UAV and another 2 seconds to send necessary commands. Thus, our app shows the future position of the UAV, keeping the pilot always on par with the UAV.

This feature, after the deviation, takes input from NASA distribution of space debris data and gives the best path to reach the initial path. The intelligent system chooses the path with the least or no debris to reach the initial desired path.

Smart App: We can get the predicted path using the previous feature. However, it's the pilot who would be taking the calls to maneuver the UAV. We know that during these kinds of situations, it’s very stressful for the pilot and can lead to possible human errors. Our smart app feature based on Artificial Intelligence and Machine Learning models comes handy in such situations and suggests instructions to input to the pilot. It also has an autopilot mode where our intelligent app takes manual control of the UAV. The modes can be easily switched making human errors almost zero.

Obstacle detection and visualization : During unfortunate yet unavoidable circumstances, we take manual control of the UAV in order to maneuver back to its initial path. However, the new path could be dangerous with obstacles like space debris which might lead to a collision. So, it’s important to detect and reflect the obstacle on the app for easy maneuvering.

The proximity sensors inbuilt in the UAV are used to detect the obstacles around the UAV and they are reflected in real-time(taking into consideration the signal latency) on the app which facilitates the pilot to dodge the obstacle successfully.

You can view our prototype at https://youtu.be/z3LyKYZ2JC4

Security system: UAVs are vulnerable to hacking which might jeopardize the mission. Thus, having a good security system is vital for the success of the mission. An employee would log in with his/her credentials, and based on the level of authority and position, he/she would be allowed to maneuver or control the code injection for certain parts of the UAV. We plan to use a Merkle tree, one of the components of a blockchain, in the form of an immutable database so we have a tamper-proof history of modifications to the path of the UAV and the code injected into it.

Working of the App

The layout of the app is simple. As you can see in the attached videos, the path of the UAV is clearly displayed along with the real-time position of the UAV. There would be 2 joysticks at the bottom corners of the app where one of them is used for controlling the 2-dimensional motion of the UAV and the other is used to control the altitude. You would be wondering why we are using joysticks instead of gyro sensors of the mobile phone. These are the main reasons behind our approach:

1)Universalizability: The latest smartphones are built with very sensitive gyro sensors. However, not all the previous models of smartphones are built with high sensitivity gyro sensors. Thus, we decided to use the joystick as a tool for maneuvering the UAV which caters to most of the smartphones available in the market.

2)Dexterity: We understand different people have various levels of dexterity which could be a possible issue in manually maneuvering the UAV. However, the usage of joystick eliminates this problem.

Moreover,to cater to our target market the fact that our users mission last for long duration's, we choose the joystick. Suppose there have been many technical glitches in the UAV which would take quite some time to solve using our code injecting system. Controlling the UAV with a gyro sensor for all this time may lead to wrist and joint pains in the pilots hand. Comparatively, using a joystick is easy as well as error free.

Business Model

Being in a very niche but important market our target market is organizations like NASA, Black Swift Technologies, and Aerobotics. Our marketing strategies are straightforward and directly appealing to our target market. Furthermore, our revenue model is simple where we charge our customers from mission to mission.

Future Plans

We are currently developing the AI and machine learning model that suggests input to the pilot and working on the Merkle tree which enhances the security of our app. We hope to finish our work on the above-mentioned features by this Christmas.

We wish to finish our app by March 2020 and start testing it for a period of two months. We will be simultaneously starting to market our app during the beta phase. Our team, driven by interest and determination catalyzed with skills, will be launching the app in the market by May 2020.

Acknowledgment

We would like to thank NASA for providing us with myriad innovative challenges, a fun-filled hackathon and providing a great learning opportunity for everyone around the world. We would also like to thank NASA for its informative resources which helped us throughout our project. We used the NASA dataset about gyro sensors to compare it with the use of a joystick and used other resources to further develop our idea by understanding the problem better. We also used the NASA Space debris data (available for the challenge “Orbital Scrap Metal-The Video Game”) for our intelligent app to choose the path with least or no space debris.

We would also like to thank the Singapore organizers for a wonderful learning experience and for providing us with tasty food during the hackathon.

Slides & Videos

Slides: https://docs.google.com/presentation/d/1Luyp92FIXVhhQb66Fpqtm7bC3Gfp16Nx4bSJE-bRUyU/edit?usp=sharing

Code injecting video: https://youtu.be/oE0ZFWDUZxQ

Obstacle detection and visualization: https://youtu.be/z3LyKYZ2JC4

Prototype Code:

Link to code of manual control and obstacle detection: https://github.com/RohanGautam/SpaceApps-Drone-Control

Link to code of injecting code system: https://github.com/RohanGautam/SpaceApps-Drone-Inject

References

Appendix