Spring XD, a Runtime Environment for Big Data!

Spring XD is a unified, distributed, and extensible system for data ingestion, real time analytics, batch processing, and data export.

Much of the complexity in building real-world big data applications is related to integrating many disparate systems into one cohesive solution across a range of use-cases. Common use-cases encountered in creating a comprehensive big data solution are

• High throughput distributed data ingestion from a variety of input sources into big data store such as HDFS or Splunk
  
• Real-time analytics at ingestion time, e.g. gathering metrics and counting values.
  
• Workflow management via batch jobs. The jobs combine interactions with standard enterprise systems (e.g. RDBMS) as well as Hadoop operations (e.g. MapReduce, HDFS, Pig, Hive or Cascading).
  
• High throughput data export, e.g. from HDFS to a RDBMS or NoSQL database.

The Spring XD project aims to provide a one stop shop solution for these use-cases.

Features of Spring XD.  

• Unified platform - Stream Processing and Batch Jobs
  • Off-Hadoop Batch Jobs
  • Hadoop Batch workflow orchestration
  • NoSQL Analytics
  • Scoring of Machine Learning algorithms
• Runtime that provides critical non-functional requirements
  • Scalable, distributed, Fault-Tolerant
  • Portable. On prem DIY cluster, YARN, EC2.
• Easy to use
• Easy to extend and integrate other technologies

Minimum requirements: System should have a minimum Java JDK 6 or newer installed. Java JDK 7 is recommended.
To download the current release, you can download the distribution spring-xd-1.0.0.RELEASE-dist.zip.