Data Management
The Data Division of the Computing and Data Centre of Alfred Wegener Institute is responsible for the publication of data products of environmental research and the research infrastructures from AWI, MARUM and beyond. It provides services and support for data workflows, software tools and data science. The service is provided by the following groups.
Question? Need support? Contact us at: data@awi.de
Data Logistics Support
The Data Logistics Support (DLS) group provides services to transfer raw data from measuring devices (SENSOR, DShip, Data-INGEST, sensor network, satellite data transmission, NRT, ...) to the AWI's central storage and archive systems. The corresponding measuring devices can be used either mobile or installed on board the research platforms as well as in the laboratory. It does not matter whether the devices transmit the raw data via satellite from e.g. the Antarctic or whether they are transferred after the expedition using mobile data media like hard disks or USB sticks.
The group members provide comprehensive help and advice in connecting your devices. Our service also includes the metadata description of the measuring device and its use (SENSOR and DShip).
The aim of the data support group is to relieve the scientists of the technical workflows to transmit their data to leave more time to focus on their scientific questions. Practically this means: You describe the device and the raw data transfer just once, after this each deployment is automatically documented. That's all. Subsequently, all raw data are available in the AWI’s workspace and archived in PANGAEA with a rich set of metadata.
Data Science Support
The Data Science Support group (DSS) advises and supports the scientists in all issues concerning the work with scientific data using the IT infrastructure of AWI’s Computing and Data Centre. New developments like modern collaborative work environments on cloud-based infrastructures are offered to AWI’s scientific community. On the other hand user requirements and expectations of the scientists are gathered and discussed. In case they are of general interest, these user requirements will be realized as permanent services. For this reason, the DSS-Group designs and implements the technical concepts and develops the data flows and software together with the DLS and SE-Groups.
The main focuses of the DSS group are (meta)data management and visualization using GIS and WebGIS, as well as data processing using Artificial Intelligence (AI) and machine learning (neural network) algorithms. DSS has in-depth expertise in the handling and processing of large and compute intensive remote sensing and bio-informatic data sets, as well as supporting scientific work and development on collaborative platforms (WORKSPACE, Jupyter, GitLab, …).
Another domain of the DSSGroup is the training of the scientists in the field to handle and process scientific data sets and their visualization using the IT-infrastructure of the Computing and Data Centre at AWI.
Together with colleagues from partner institutions as well as within the scope of national scientific associations and initiatives the DSS group develops comprehensive data infrastructures and standards.
Interactive visualization platform (maps.awi.de) and analysis platforms in collaborative development environments (jupyterhub.awi.de and cloud.awi.de)
Software Engineering
The Software Engineering (SE) group develops infrastructure components and systems for scientific data management and core compute services. Following SCRUM-based principles during the development process, we are agile in coping with new requirements from science. As an integrative DevOp team, we are also maintaining and running developed applications.
The software development portfolio ranges from metadata management to describe platforms, devices and sensors to automatic data acquisition and transformation up to long-term archiving and publication with PANGAEA. This also includes solutions for web portals and data visualization. In short: Observation to Archive and Analysis - O2A.
Most applications are developed and provided as services for the science community as web applications. The technology stack ranges from Python scripting for data intensive processes over strong Java middleware and backend developments to web technologies. Of course, data modelling, databases and big data approaches are part of our daily work.
We are involved in projects with strong technology focus exploring and using the newest technology for data storage, handling, processing and analyses in tight collaboration with system experts inside and outside of AWI.
PANGAEA
The PANGAEA group provides essential services for scientific project data management, long-term data archiving and preservation, data publication, and dissemination of quality approved metadata according to the FAIR data principles. Every dataset published is fully citable including a persistent unique Digital Object Identifier (DOI).
PANGAEA - Data Publisher for Earth & Environmental Science is a joint facility of the Alfred Wegener Institute Helmholtz Centre for Polar and Marine Research (AWI) and the Centre for Marine Environmental Sciences (MARUM) at the University of Bremen. It is collaboratively developed with the DLS, DSS and SE group as well as its thousands of international users to provide comprehensive data archiving and publication following the FAIR.
Founded in 1992, PANGAEA has demonstrated its long-term perspective by a certification of the ICSU World Data System and the CoreTrustSeal, and is accredited by the WMO as Data Collection and Processing Center (DCPC) (link zu. The system is operated in compliance with the Berlin Declaration on Open Access to Knowledge in the Sciences and Humanities.
The successful cooperation between PANGAEA and the publishing industry enables the cross-referencing of scientific publications and archived datasets. PANGAEA is the recommended data repository of numerous international scientific journals.
Screenshot of the PANGAEA landing page indicating the dynamic and diversity of datasets published in this long-term archive.
Head of Data Division
Prof. Dr. Frank Oliver Glöckner
Deputy Head of Data Division
Dr. Angela Schäfer
Data Logistics Support
Sebastian Immoor
Data Science Support
Dr. Sonja Hänzelmann
Software Engineering
Dr. Roland Koppe
PANGAEA
Dr. Janine Felden
O2A: The Observation to Archive and Analysis Framework
A generic and sustainable framework enabling the seamless flow of device (sensors) observations to archives and analysis. This framework builds upon international OGC standards for metadata and data interoperability and is meant to assist scientists in developing enhanced data products and in facilitating the data re-use. AWI's data flow framework consists of seven modular and extensible components as depicted below.
Learn more about our Data Flow Framework.
Have also a look to technical details and documentation in our Wiki here.
Marine Data
The Marine Data Portal is the single-entry point to near real-time data, platforms, expeditions and data visualisation of the German marine research vessels, infrastructures and beyond. Explore the large collection of sensors and our interactive maps. Create your own dashboard to explore the real-time data flow and analyse your data in the workspace.
