Home Database Mysql Tutorial 超越Hadoop的大数据分析之前言

超越Hadoop的大数据分析之前言

Jun 07, 2016 pm 04:34 PM
da hadoop Preface data analysis transcend

本文翻译自《BIG DATA ANALYTICS BEYOND HADOOP》译者:吴京润 校对:方腾飞 我试图给人们学习大数据留下的一点深刻印象:尽管Apache Hadoop很有用,而且是一项非常成功的技术,但是这一观点的前提已经有些过时了。考虑一下这样一条时间线:由谷歌实现的MapR

本文翻译自《BIG DATA ANALYTICS BEYOND HADOOP》译者:吴京润 校对:方腾飞

我试图给人们学习大数据留下的一点深刻印象:尽管Apache Hadoop很有用,而且是一项非常成功的技术,但是这一观点的前提已经有些过时了。考虑一下这样一条时间线:由谷歌实现的MapReduce投入使用的时间可追溯到2002年,发表于2004年。Yahoo!于2006年发起Hadoop项目。MR是基于十年前的数据中心的经济上的考虑。从那时以来,已经有太多的东西发生了变化:多核心处理器、大内存地址空间、10G网络带宽、SSD,而至今,这已经产生足够的成本效益。这些极大改变了在构建可容错分布式商用系统规模方面的取舍。

此外,我们对于可处理数据的规模的观念也发生了变化。成功的公司诸如亚马逊、eBay、谷歌,它们想要更上一层楼,也促使随后的商业领袖重新思考:数据可以用来做什么?举个例子,十年前是否有为大型图书出版商优化业务的大规模图论用例?不见得有。出版社高层不可能有耐心听取这样一个古怪的工程建议。这本书本身的营销将基于大规模数据、开源、图论引擎,它们也将在本书后续章节讲到。同样的,广告科技和社交网络应用驱动着开发技术,而如今在工业化的因特网,采用Hadoop将显的捉襟见肘,也就是所谓的“物联网”——在某些情况下,会有几个数量级的差距。

自从MR的商用硬件规模首次制定以来,底层系统的模型已发生了巨大变化。我们的商业需求与期望模型也发生了显著的变化。此外,应用数学的数据规模与十年前的构想也有巨大的差异。如今主流编程语言也能为并行处理的软件工程实践提供更好的支持。

Agneeswaran博士认为这些视图,以及对它们的更多关注和系统方法,呈现了如今大数据环境的全景视图,甚至还有超越。本书引领我们看到过去十年如何通过MapReduce做批处理数据分析。这些章节介绍了理解它们的关键历史背景,并为应用这些技术提供了清晰的商业用例的至关重要的方面。这些论据为每个用例提供了分析,并指出为什么Hadoop不是很适合应用于此——通过对例证的彻底研究、对可用开源技术的出色调查、以及对非开源项目的出版文献的回顾。

本书研究了如今的商业需求中除Hadoop以外的最佳实践以及数据访问方式的可用技术:迭代、流式处理(译者注:原文是streaming)、图论,以及其它技术。比如,一些企业的收入损失计算可精确到毫秒级,以至于“批处理窗口”这样的概念变的毫无意义。实时分析是惟一可以想到的可行方案。开源框架诸如Apache Spark、Storm、Titan、GraphLab,还有Apache Mesos可以满足这些需求。Agneeswaran博士引导读者们了解这些框架的架构和计算模型、研究通用设计模式。他在书中提到了业务范围的影响以及实现细节还有代码样例。

伴随着这些框架,本书也为开放标准预测模型标记语言提出了一个引人入胜的例子,使得预测模型可以在不同平台与环境之间迁移。本书还提到YARN以及下一代超越MapReduce的模型。

这正是当今业界的焦点——Hadoop基于2002年以来的IT经济,然而更新的框架与当代业界的用例更为密切。另外,本书既提供了专家指导,也热烈欢迎由大数据分析开启的无限可能。

Paco Nathan

图书《Enterprise Data Workflows with Cascading》的作者 ? (校对注:样章下载)

Zettacap的顾问以及Amplify的合作伙伴

(全文完)如果您喜欢此文请点赞,分享,评论。


  • 原创文章转载请注明出处:超越Hadoop的大数据分析之前言
  • 小额赞助本站::我要赞助

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Read CSV files and perform data analysis using pandas Read CSV files and perform data analysis using pandas Jan 09, 2024 am 09:26 AM

Pandas is a powerful data analysis tool that can easily read and process various types of data files. Among them, CSV files are one of the most common and commonly used data file formats. This article will introduce how to use Pandas to read CSV files and perform data analysis, and provide specific code examples. 1. Import the necessary libraries First, we need to import the Pandas library and other related libraries that may be needed, as shown below: importpandasaspd 2. Read the CSV file using Pan

Introduction to data analysis methods Introduction to data analysis methods Jan 08, 2024 am 10:22 AM

Common data analysis methods: 1. Comparative analysis method; 2. Structural analysis method; 3. Cross analysis method; 4. Trend analysis method; 5. Cause and effect analysis method; 6. Association analysis method; 7. Cluster analysis method; 8 , Principal component analysis method; 9. Scatter analysis method; 10. Matrix analysis method. Detailed introduction: 1. Comparative analysis method: Comparative analysis of two or more data to find the differences and patterns; 2. Structural analysis method: A method of comparative analysis between each part of the whole and the whole. ; 3. Cross analysis method, etc.

11 basic distributions that data scientists use 95% of the time 11 basic distributions that data scientists use 95% of the time Dec 15, 2023 am 08:21 AM

Following the last inventory of "11 Basic Charts Data Scientists Use 95% of the Time", today we will bring you 11 basic distributions that data scientists use 95% of the time. Mastering these distributions helps us understand the nature of the data more deeply and make more accurate inferences and predictions during data analysis and decision-making. 1. Normal Distribution Normal Distribution, also known as Gaussian Distribution, is a continuous probability distribution. It has a symmetrical bell-shaped curve with the mean (μ) as the center and the standard deviation (σ) as the width. The normal distribution has important application value in many fields such as statistics, probability theory, and engineering.

Machine learning and data analysis using Go language Machine learning and data analysis using Go language Nov 30, 2023 am 08:44 AM

In today's intelligent society, machine learning and data analysis are indispensable tools that can help people better understand and utilize large amounts of data. In these fields, Go language has also become a programming language that has attracted much attention. Its speed and efficiency make it the choice of many programmers. This article introduces how to use Go language for machine learning and data analysis. 1. The ecosystem of machine learning Go language is not as rich as Python and R. However, as more and more people start to use it, some machine learning libraries and frameworks

How to use ECharts and php interfaces to implement data analysis and prediction of statistical charts How to use ECharts and php interfaces to implement data analysis and prediction of statistical charts Dec 17, 2023 am 10:26 AM

How to use ECharts and PHP interfaces to implement data analysis and prediction of statistical charts. Data analysis and prediction play an important role in various fields. They can help us understand the trends and patterns of data and provide references for future decisions. ECharts is an open source data visualization library that provides rich and flexible chart components that can dynamically load and process data by using the PHP interface. This article will introduce the implementation method of statistical chart data analysis and prediction based on ECharts and php interface, and provide

Integrated Excel data analysis Integrated Excel data analysis Mar 21, 2024 am 08:21 AM

1. In this lesson, we will explain integrated Excel data analysis. We will complete it through a case. Open the course material and click on cell E2 to enter the formula. 2. We then select cell E53 to calculate all the following data. 3. Then we click on cell F2, and then we enter the formula to calculate it. Similarly, dragging down can calculate the value we want. 4. We select cell G2, click the Data tab, click Data Validation, select and confirm. 5. Let’s use the same method to automatically fill in the cells below that need to be calculated. 6. Next, we calculate the actual wages and select cell H2 to enter the formula. 7. Then we click on the value drop-down menu to click on other numbers.

What are the recommended data analysis websites? What are the recommended data analysis websites? Mar 13, 2024 pm 05:44 PM

Recommended: 1. Business Data Analysis Forum; 2. National People’s Congress Economic Forum - Econometrics and Statistics Area; 3. China Statistics Forum; 4. Data Mining Learning and Exchange Forum; 5. Data Analysis Forum; 6. Website Data Analysis; 7. Data analysis; 8. Data Mining Research Institute; 9. S-PLUS, R Statistics Forum.

Golang: The dark horse in the programming world, can it surpass traditional languages? Golang: The dark horse in the programming world, can it surpass traditional languages? Mar 05, 2024 pm 04:42 PM

Golang, the Go language, is an open source programming language developed by Google and has attracted much attention since its inception. As a statically typed, compiled language, Golang has shown strong advantages in handling concurrency, network programming, etc., so it is considered a language that is very suitable for high-performance server-side application development. After all, can Golang surpass traditional languages ​​and become a leader in the programming world? This article will discuss language characteristics, performance, ecology and future prospects, and demonstrate through specific code

See all articles