Apache Hadoop Ecosystem

Ambari:

A web-based tool for provisioning, managing, and monitoring Apache Hadoop clusters which includes support for Hadoop HDFS, Hadoop MapReduce, Hive, HCatalog, HBase, ZooKeeper, Oozie, Pig and Sqoop. Ambari also provides a dashboard for viewing cluster health such as heatmaps and ability to view MapReduce, Pig and Hive applications visually alongwith features to diagnose their performance characteristics in a user-friendly manner.
The Apache Ambari project that is used to simplify the management of Apache Hadoop clusters using a web UI. It also integrates with other existing applications using Ambari REST APIs

Avro:

A data serialization system. Avro is a row-oriented remote procedure call and data serialization framework developed within Apache's Hadoop project. It uses JSON for defining data types and protocols, and serializes data in a compact binary format.
Avro has a schema-based system. A language-independent schema is associated with its read and write operations. Avro serializes the data which has a built-in schema. Avro serializes the data into a compact binary format, which can be deserialized by any application

Cassandra:

A scalable multi-master database with no single points of failure.

Chukwa:

A data collection system for managing large distributed systems.

HBase:

A scalable, distributed database that supports structured data storage for large tables.

Hive:

A data warehouse infrastructure that provides data summarization and ad hoc querying.

Mahout:

A Scalable machine learning and data mining library.
It implements popular machine learning techniques such as:

Recommendation /Collaborative Filtering

• Item‐based Collaborative Filtering
• Matrix Factorization with Alternating Least Squares
• Matrix Factorization with Alternating Least Squares on Implicit Feedback

Classification

• Naive Bayes
• Complementary Naive Bayes
• Random Forest

Clustering

• Canopy Clustering
• k‐Means Clustering
• Fuzzy k‐Means
• Streaming k‐Means
• Spectral Clustering

Pig:

A high-level data-flow language and execution framework for parallel computation.

Spark:

A fast and general compute engine for Hadoop data. Spark provides a simple and expressive programming model that supports a wide range of applications, including ETL, machine learning, stream processing, and graph computation.

Submarine:

A unified AI platform which allows engineers and data scientists to run Machine Learning and Deep Learning workload in distributed cluster.

Tez:

A generalized data-flow programming framework, built on Hadoop YARN, which provides a powerful and flexible engine to execute an arbitrary DAG of tasks to process data for both batch and interactive use-cases. Tez is being adopted by Hive™, Pig™ and other frameworks in the Hadoop ecosystem, and also by other commercial software (e.g. ETL tools), to replace Hadoop™ MapReduce as the underlying execution engine.

ZooKeeper:

A high-performance coordination service for distributed applications.
Some of the prime features of Apache ZooKeeper are:
Reliable System: This system is very reliable as it keeps working even if a node fails.
Simple Architecture: The architecture of ZooKeeper is quite simple as there is a shared hierarchical namespace which helps coordinating the processes.
Fast Processing: ZooKeeper is especially fast in "read-dominant" workloads (i.e. workloads in which reads are much more common than writes).
Scalable: The performance of ZooKeeper can be improved by adding nodes.

Apache Flume

Apache Flume is a distributed, reliable, and available system for efficiently collecting, aggregating and moving large amounts of log data from many different sources to a centralized data source. Review this Flume use case to learn how Mozilla collects and Analyse the Logs using Flume and Hive.
Flume is a framework for populating Hadoop with data. Agents are populated throughout ones IT infrastructure – inside web servers, application servers and mobile devices, for example – to collect data and integrate it into Hadoop.