Skip to content
This repository has been archived by the owner on Apr 5, 2022. It is now read-only.

Sample Applications for getting started with Spring for Apache Hadoop

License

Notifications You must be signed in to change notification settings

spring-attic/spring-hadoop-samples-old

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

spring-hadoop-samples-old is no longer actively maintained by VMware, Inc.

Sample Applications for Spring for Apache Hadoop

This repository contains several sample applications that show how you can use Spring for Apache Hadoop. The applications are located in the directory samples.

The directory original-samples and docs are copies of the example application code and documentation that were shipped in Spring Hadoop 1.0.0.M2. They will be migrated into the samples directory and removed soon.

Overview of Spring for Apache Hadoop

Hadoop has a poor out of the box programming model. Writing applications for Hadoop generally turn into a collection of scripts calling Hadoop command line applications. Spring for Apache Hadoop provides a consistent programming model and declarative configuration model for developing Hadoop applications.

Together with Spring Integration and Spring Batch, Spring for Apache Hadoop can be used to address a wide range of use cases * HDFS data access and scripting * Data Analysis MapReduce Pig Hive Cascading * Workflow * Data collection and ingestion * Event Streams processing

Features

  • Declarative configuration to create, configure, and parameterize Hadoop connectivity and all job types (MR/Streaming MR/Pig/Hive/Cascading)

  • Simplify HDFS API with added support for JVM scripting languages

  • Runner classes for MR/Pig/Hive/Cascading for small workflows consisting of the following steps HDFS operations → data analysis → HDFS operations

  • Helper “Template” classes for Pig/Hive/HBase

    • Execute scripts and queries without worrying about Resource Management Exception Handling and Translation

    • Thread-safety

  • Lightweight Object-Mapping for HBase

  • Hadoop components for Spring Integratio and Spring Batch

    • Spring Batch tasklets for HDFS and data analysis

    • Spring Batch HDFS ItemWriters

    • Spring Integration HDFS channel adapters

Additional Resources

The samples were taken from the O’Reilly book Spring Data. Using the book as a companion to the samples is quite helpful to understanding the samples and the full feature set of what can be done using Spring technologie and Hadoop.

The main web site for Spring for Apache Hadoop

About

Sample Applications for getting started with Spring for Apache Hadoop

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published