Intel on Tuesday revealed an Apache Hadoop distribution of its own - the Intel Distribution for Apache Hadoop software (Intel Distribution).

The offering includes Intel Manager for Apache Hadoop software and is built from the silicon up to deliver high performance and improved security features.

Like EMC and Hewlett-Packard earlier this week, the idea behind Intel's distribution is to massive amounts of big data for the purpose of enabling better business decisions while also identifying potential security threats quickly.

The ability to analyze and make sense of big data enables new scientific discoveries, business models and consumer experiences. Yet, only a small fraction of the world is able to extract meaning from all of this information.

Hadoop is an open source framework for storing and processing large volumes of diverse data on a scalable cluster of servers that has emerged as the preferred platform for managing big data. With even more information coming from billions of sensors and intelligent systems also on the horizon, the framework must remain open and scalable as well as deliver on the demanding requirements of enterprise-grade performance, security and manageability.

"People and machines are producing valuable information that could enrich our lives in so many ways, from pinpoint accuracy in predicting severe weather to developing customized treatments for terminal diseases," said Boyd Davis, vice president and general manager of Intel's Datacenter Software Division. "Intel is committed to contributing its enhancements made to use all of the computing horsepower available to the open source community to provide the industry with a better foundation from which it can push the limits of innovation and realize the transformational opportunity of big data."

Intel's open source software framework is backing Hadoop directly into the silicon level. The Intel Distribution provides complete encryption with support of Intel AES New Instructions (Intel AES-NI) in the Intel Xeon processor. By incorporating silicon-based encryption support of the Hadoop Distributed File System, organizations can now more securely analyze their data sets without compromising performance.

The optimizations made for the networking and IO technologies in the Intel Xeon processor platform also enable new levels of analytic performance. Intel claims that analyzing one terabyte of data, which would previously take more than 4 hours to fully process, can now be done in 7 minutes thanks to the combination of Intel's hardware and the Intel Distribution. Considering Intel estimates that the world generates 1 petabyte (1,000 terabytes) of data every 11 seconds or the equivalent of 13 years of HD video, the power of Intel technology opens up the world to even greater possibilities.

Intel also said that it's Hadoop distribution framework will be optimized with solid state drive and cache acceleration, up to 8.5 times faster queries in Hive, hardware-enhanced compression with AVX and SSE 4.2, and automated tuning on clusters with Active Tuner from Intel Labs.

The addition of the Intel Manager for Apache Hadoop software also simplifies the deployment, configuration and monitoring of the cluster for system administrators as they look to deploy new applications. Using the Intel Active Tuner for Apache Hadoop software optimal performance is automatically configured to take the guesswork out of performance tuning. Until now, this required a specialized understanding of each application's use of system resources along with the Hadoop configuration and performance benchmarks.

Intel is working with other companies to integrate this software into a number of next-generation platforms and solutions, and to enable deployment in public and private cloud environments. Intel's partners supporting the launch include 1degreenorth, AMAX, Cisco, Colfax Corporation, Cray, Datameer, Dell, En Pointe, Flytxt, Hadapt, HStreaming, Infosys, LucidWorks, MarkLogic, NextBio, Pentaho, Persistent Systems, RainStor, Red Hat, Revolution Analytics, SAP, SAS, Savvis, a CenturyLink company, Silicon Mechanics, Simba Technologies, SoftNet Solutions, SuperMicro Computer, Inc., Tableau Software, Teradata, T-Systems, Wipro and Zettaset.

The Intel-Hadoop deployment will be delivered through an annual subscription with technical support through solution vendors and service providers.