Semantic Robots

Semantic Robots is a strategic partnership between the Centre for Applied Autonomous Sensor Systems, which MPI is a part of, and several industrial partners, all leaders in their respective fields. This is a 6-year research profile funded by The Knowledge Foundation for a total of 36MSEK, matched by equal contribution from the industry. The main objective of this effort is to further develop our key research areas through close cooperation with leading industrial players.

MPI is a principle investigator of one of three research directions in this project, with the main goal to create methods and tools for discovering semantic knowledge from aerial imagery. The project is comprised of three major parts: image segmentation and classification, discovering semantic knowledge and building ontologies based on classification results and open available data sources, and visualization.

On the first step, we develop algorithms for classification and segmentation of satellite orthoimagery using convolutional neural networks. From multi-band true-ortho satellite imagery covering the range from 400 to 1040nm, along with high detailed digital elevation models (DEM), we produce rapid classification of the area into five classes: vegetation, ground, water, road, building. Although not perfectly precise, the results of classification allows to build reasoning on the data and incorporate other data sources, such as Open Street Map (OSM). The reasoning will in tern allow to improve classification (for instance, to solve confusion in the difficult areas of the map, for example, in shadows).

Example of SemMap Viewer 2.0 with flood simulation and flood affected areas. Labels show the semantic data of points, when not flooded.



Marjan Alirezaie; Martin Längkvist; Michael Sioutis; Amy Loutfi

Semantic Referee: A Neural-Symbolic Framework for Enhancing Geospatial Semantic Segmentation Journal Article

In: Semantic Web Journal, 10 (5), pp. 863-880, 2019.

Abstract | Links | BibTeX


Marjan Alirezaie; Martin Längkvist; Michael Sioutis; Amy Loutfi

A Symbolic Approach for Explaining Errors in Image Classification Tasks Workshop

IJCAI Workshop on Learning and Reasoning: Principles & Applications to Everyday Spatial and Temporal Knowledge, 2018.

Abstract | Links | BibTeX


Marjan Alirezaie; Andrey Kiselev; Martin Längkvist; Franziska Klügl; Amy Loutfi

An Ontology-Based Reasoning Framework for Querying Satellite Images for Disaster Monitoring Journal Article

In: Sensors, 17 (11), 2017, ISSN: 1424-8220.

Abstract | Links | BibTeX

Marjan Alirezaie; Andrey Kiselev; Franziska Klügl; Martin Längkvist; Amy Loutfi

Exploiting Context and Semantics for UAV Path-finding in an Urban Setting Inproceedings

In: 1st International Workshop on Application of Semantic Web technologies in Robotics (AnSWeR), pp. 11–20, Portoroz, Slovenia, 2017, ISSN: 1613-0073.

Links | BibTeX


Marjan Alirezaie; Martin Längkvist; Andrey Kiselev; Amy Loutfi

Open GeoSpatial Data as a Source of Ground Truth for Automated Labelling of Satellite Images Inproceedings

In: Proceedings of the Workshop on Spatial Data on the Web (SDW 2016) co-located with The 9th International Conference on Geographic Information Science (GIScience 2016), pp. 5-8, 2016.


Martin Längkvist; Andrey Kiselev; Marjan Alirezaie; Amy Loutfi

Classification and Segmentation of Satellite Orthoimagery Using Convolutional Neural Networks Journal Article

In: Remote Sensing, 8 (4), 2016.

Links | BibTeX

Martin Längkvist; Marjan Alirezaie; Andrey Kiselev; Amy Loutfi

Interactive Learning with Convolutional Neural Networks for Image Labeling Inproceedings

In: IJCAI 2016 workshop on Interactive Machine Learning: Connecting Humans and Machines, 2016.

Links | BibTeX

Marjan Alirezaie; Franziska Klügl; Amy Loutfi

Knowing without telling: integrating sensing and mapping for creating an artificial companion Inproceedings

In: Proceedings of the 24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, GIS 2016, Burlingame, California, USA, October 31 - November 3, 2016, pp. 11:1–11:4, 2016.

Links | BibTeX