Human following robot swarm

3 minute read

In this post, I will gather all information during the development of the Human Following robot swarm project.

Aim of the project

The aim of the project is to develop a middleware that would allow a small swarm of robots to follow a particular person or group of people. There are plenty of applications for this general framwork, and one of them is carrying different loads of baggage in the airport conditions.

In order to understand the overall problem, let’s first define the main concepts here:

  1. What is human following and how is it done?
  2. What are the robot swarms?

Human tracking and following

For a robot to follow a human, it first needs to detect and track that person. There is a wide range of approaches to solve this problem. The most common one is to use simple RGB cameras and use deep learning detection algorithms. YOLO algorithm is well suited for the problem of object/human detection. Moreover, that approach is best for tracking a particular person in a crowded environment as the recognition algorithms can be also easily integrated. The only limitation of this approach is that it is not able to define the distance to the person detected and clearly define its pose in the world. In order to overcome that issue, the usage of either RGBD cameras or fusion of data from a camera with a LiDAR sensor can be used. Several works adressing that problem are already available.

Robot swarm

As I don’t have any experience with developing robot swarms, I will dive more into the literature and outline my findings here. For now I have just a general understanding of Social Force Model, but I did not yet made any modellings. Moreover, I will dive into Artificial Potential Method and see if it could also work for me.

Implementation details

As it is always a good idea to start developing and testing algorithms in simulation, I will start using Gazebo for simulation and ROS for communication.

Human modelling

I found a nice pedsim_ros ROS package which is a great tool for the simulation of realistic human crowds in simulation. The package is based on a Social Force Model which allows to simulate realistic behaviors for crowds by avoiding obstacles while following their desired trajectories. The package is mostly integrated into RViZ and has lots of adjustable parameters for various simulation scenarios. This also allows it to simulate individual persons and groups of people following different trajectories.

Recently, there was a major update that allowed to integrate people messages simulated in RViZ to be also visualized in a Gazebo simulator. This allows building more complex environments that could take physics interactions into account.

I have created a simple environment with three people walking around where two of them are grouped together. Screenshots below show the RViZ and Gazebo top views.

However, the only simulation model that is officially supported is the shown gray bounding boxes. There is also a possibility to substitute it with a static human model, but it’s still not realistic enough. Moreover, the package has an additional drawback that it requires a considerable amount of both computational resources and interfacing efforts to work properly in Gazebo. Nevertheless, it is a great tool that could be used in a later stages of project, when testing of following algorithms in pedestrian dense environments will be required.

For now it would be best to use a human actor model designed in Gazebo.

Now I will spend time designing a world model with this actor model which will used as a “target” person.

To be updated.

Updated: