Information Systems Group - Prof. Jens Dittrich

Overview

Stefan Richter
Ph.D. student
Information Systems Group
Campus E1.1, room 3.08.02
66123 Saarbrücken, Germany

e-mail: stefan.richter at infosys.uni-saarland.de

I am a Ph.D. student at the Saarland University's Computer Science Department. My advisor is Prof. Jens Dittrich. My current focus of research is large-scale analytics and Hadoop MapReduce. I am a member of the Saarbrücken Graduate School of Computer Science.

Publications

Stefan Richter, Jorge-Arnulfo Quiané-Ruiz, Stefan Schuh, Jens Dittrich
Towards Zero-Overhead Static and Adaptive Indexing in Hadoop
The VLDB Journal. 2013. Abstract / [bib]
Towards Zero-Overhead Static and Adaptive Indexing in Hadoop
Hadoop MapReduce has evolved to an important industry standard for massive parallel data processing and has become widely adopted for a variety of use cases. Recent works have shown that indexes can improve the performance of selective MapReduce jobs dramatically. However, one major weakness of existing approaches are high index creation costs. We present HAIL (Hadoop Aggressive Indexing Library), a novel indexing approach for HDFS and Hadoop MapReduce. HAIL creates different clustered indexes over terabytes of data with minimal, often invisible costs and it dramatically improves runtimes of several classes of MapReduce jobs. HAIL features two different indexing pipelines, static indexing and adaptive indexing. HAIL static indexing efficiently indexes datasets while uploading them to HDFS. Thereby, HAIL leverages the default replication of Hadoop and enhances it with logical replication. This allows HAIL to create multiple clustered indexes for a dataset, e.g. one for each physical replica. Still, in terms of upload time, HAIL matches or even improves over the performance of standard HDFS. Additionally, HAIL adaptive indexing allows for automatic, incremental indexing at job runtime with minimal runtime overhead. For example, HAIL adaptive indexing can completely index a dataset as byproduct of only four MapReduce jobs while incurring an overhead as low as 11% for the very first of those job only. In our experiments, we show that HAIL improves job runtimes by up to 68x over Hadoop. This article is an extended version of the VLDB 2012 paper “Only Aggressive Elephants are Fast Elephants” (PVLDB, 5(11):1591-1602, 2012).
```
@article{RQS+13,
   author    = {Stefan Richter and Jorge-Arnulfo Quian{\'e}-Ruiz  and Stefan Schuh and Jens Dittrich},
   title     = {{Towards Zero-Overhead Static and Adaptive Indexing in Hadoop}},
   journal   = {VLDB Journal},
   year      = {2013},
   }
```
Stefan Richter, Jens Dittrich, Stefan Schuh, Tobias Frey
Mosquito: Another One Bites the Data Upload STream
VLDB 2013, Riva, Italy. (Demo paper) Abstract /[bib]
Mosquito: Another One Bites the Data Upload STream
Mosquito is a lightweight and adaptive physical design framework for Hadoop. Mosquito connects to existing data pipelines in Hadoop MapReduce and/or HDFS, observes the data, and creates better physical designs, i.e. indexes, as a byproduct. Our approach is min- imally invasive, yet it allows users and developers to easily improve the runtime of Hadoop. We present three important use cases: first, how to create indexes as a byproduct of data uploads into HDFS; second, how to create indexes as a byproduct of map tasks; and third, how to execute map tasks as a byproduct of HDFS data up- loads. These use cases may even be combined.
```
@article{DRS+13,
   author    = {Stefan Richter and Jens Dittrich and Stefan Schuh and Tobias Frey},
   title     = {{Mosquito: Another One Bites the Data Upload STream}},
   journal   = {PVLDB},
   volume    = {6},
   number    = {12},
   year      = {2013},
   pages     = {unknown},
   }
```
Jens Dittrich, Stefan Richter, Stefan Schuh
Efficient OR Hadoop: Why Not Both?
Datenbank Spektrum, January 2013 Abstract /[bib]
Efficient OR Hadoop: Why Not Both?
In this article, we give an overview of research related to Big Data processing in Hadoop going on at the Information Systems Group at Saarland University. We discuss how to make Hadoop efficient. We briefly survey four of our projects in this context: Hadoop++, Trojan Layouts, HAIL, and LIAH. All projects aim to provide efficient physical layouts in Hadoop including vertically partitioned data layouts, clustered indexes, and adaptively created clustered indexes. Most of our techniques come (almost) for free: they create little to no overhead in comparison to standard Hadoop.
```
@article{DRS13,
   author    = {Jens Dittrich and Stefan Richter and Stefan Schuh},
   title     = {{Efficient OR Hadoop: Why Not Both?}},
   journal   = {Datenbank-Spektrum},
   volume    = {13},
   number    = {1},
   year      = {2013},
   pages     = {17-22},
   }
```
Stefan Richter, Jorge-Arnulfo Quiane-Ruiz, Stefan Schuh, Jens Dittrich
Towards Zero-Overhead Adaptive Indexing in Hadoop
TR, arXiv:1212.3480 [cs.DB], 2012 Abstract / [bib]
Towards Zero-Overhead Adaptive Indexing in Hadoop
Several research works have focused on supporting index access in MapReduce systems. These works have allowed users to significantly speed up selective MapReduce jobs by orders of magnitude. However, all these proposals require users to create indexes upfront, which might be a difficult task in certain applications (such as in scientific and social applications) where workloads are evolving or hard to predict. To overcome this problem, we propose LIAH (Lazy Indexing and Adaptivity in Hadoop), a parallel, adaptive approach for indexing at minimal costs for MapReduce systems. The main idea of LIAH is to automatically and incrementally adapt to users' workloads by creating clustered indexes on HDFS data blocks as a byproduct of executing MapReduce jobs. Besides distributing indexing efforts over multiple computing nodes, LIAH also parallelises indexing with both map tasks computation and disk I/O. All this without any additional data copy in main memory and with minimal synchronisation. The beauty of LIAH is that it piggybacks index creation on map tasks, which read relevant data from disk to main memory anyways. Hence, LIAH does not introduce any additional read I/O-costs and exploit free CPU cycles. As a result and in contrast to existing adaptive indexing works, LIAH has a very low (or invisible) indexing overhead, usually for the very first job. Still, LIAH can quickly converge to a complete index, i.e. all HDFS data blocks are indexed. Especially, LIAH can trade early job runtime improvements with fast complete index convergence. We compare LIAH with HAIL, a state-of-the-art indexing technique, as well as with standard Hadoop with respect to indexing overhead and workload performance.
```
@article{RQS+12,
   author    = {Stefan Richter and Jorge-Arnulfo Quian{\'e}-Ruiz and Stefan Schuh and Jens Dittrich},
   title     = {{Towards Zero-Overhead Adaptive Indexing in Hadoop}},
   journal   = {CoRR},
   volume    = {abs/1212.3480},
   year      = {2012},
   }
```
Jens Dittrich, Jorge-Arnulfo Quiane-Ruiz, Stefan Richter, Stefan Schuh, Alekh Jindal, Jörg Schad
Only Aggressive Elephants are Fast Elephants
VLDB 2012/PVLDB, Istanbul, Turkey. slides Abstract / [bib]
Only Aggressive Elephants are Fast Elephants
Yellow elephants are slow. A major reason is that they consume their inputs entirely before responding to an elephant rider’s orders. Some clever riders have trained their yellow elephants to only con- sume parts of the inputs before responding. However, the teaching time to make an elephant do that is high. So high that the teaching lessons often do not pay off. We take a different approach. We make elephants aggressive; only this will make them very fast. We propose HAIL (Hadoop Aggressive Indexing Library), an enhancement of HDFS and Hadoop MapReduce that dramatically improves runtimes of several classes of MapReduce jobs. HAIL changes the upload pipeline of HDFS in order to create different clustered indexes on each data block replica. An interesting feature of HAIL is that we typically create a win-win situation: we improve both data upload to HDFS and the runtime of the actual Hadoop MapReduce job. In terms of data upload, HAIL improves over HDFS by up to 60% with the default replication factor of three. In terms of query execution, we demonstrate that HAIL runs up to 68x faster than Hadoop. In our experiments, we use six clusters including physical and EC2 clusters of up to 100 nodes. A series of scalability experiments also demonstrates the superiority of HAIL.
```
@article{DQR+12,
   author    = {Jens Dittrich and Jorge-Arnulfo Quian{\'e}-Ruiz and Stefan Richter and Stefan Schuh and Alekh Jindal and J{\"o}rg Schad},
   title     = {{Only Aggressive Elephants are Fast Elephants}},
   journal   = {PVLDB},
   volume    = {5},
   number    = {11},
   year      = {2012},
   pages     = {1591-1602},
   }
```

Stefan Richter

Overview

Projects

Publications

Positions

Studies