如果企业或组织仍找不到办法来从大数据技术中获取到商业价值，大数据可能就逃不掉“来也匆匆，去也匆匆”的命运。要摆脱这样的命运，仅有一些数据科学家的沙盒分析、实时分析或基于云存储可扩展性的技术是不够的。我们需要开发出能够驱动业务发展的应用程序，从而把数据转化为能够产生实际价值的真实行动。

如Concurrent的首席技术官Chris Wensel所说的，要从大数据中获取商业价值，我们的动作必须要足够快，快人一步地找到一种持续的、完善的从大数据中获取价值的方法：

“你现在拥有这样一个机会，通过纯粹的创新就可以让你与众不同——不是因为你比别人更有钱，而是因为你比别人更聪明。你已经有了优秀的工程师。你现在需要的是一些强大的工具以支持你可以更快地把你的产品推向市场，从而让你先人一步，与众不同。“

Concurrently最近推出了其最新的工具（ Driven ），以协助大数据应用的生产和监控，宣称是世界上第一个针对大数据的绩效管理产品。作为一个云服务，在2月4日，Driven开始在它的用户群中传播开来。它的目的是完善那些运行于Apache Hadoop上的流行的开源大数据应用程序开发框架（ Cascading ）。

Driven直接影响着大数据应用程序产生的商业价值：

缩短开发时间：通过Driven详细的用户界面，它提供了一种企业应用（包括综合的和专用的）进程可视化的方式，这大大提升了研发速度从而减少了所需花费的时间，可以更快地把应用推向市场。

加强应用程序的可靠性：因为在开发阶段就能够方便地看到应用程序的可视化效果，工程师将能够快速发现问题，甚至是预见问题，这样他们在把产品推向市场之前可以创建出更可靠的应用程序。可视化功能还有助于确定最合适的算法和应用程序指标，以确保得到的结果准确一致。

故障排查和优化：Driven的可视化功能可以从用户级别检查出应用程序的问题——包括被操控的数据集有哪些，有哪些工作负载，以及哪些应用程序正在运行等等。因此，确定应用程序故障点的进程得到了非常大的加速，操作人员在短短几分钟内就能识别出缓慢的时期和优化峰值，而不是像过去那样得花上几天或几周的时间。Driven 1.0企业版（预计于第二季度发布，按年收费）包含了通知功能，可以告诉开发人员哪些应用程序出故障了，故障出现什么地方，以及谁在使用这些应用程序。

当Driven与Cascading提供的框架配合使用时，通过创建加强业务目标的应用，Driven的管理功能能够帮助企业从大数据的使用中获得最大的价值。

Wensel指出，这种行动超越了单纯的见解：“你并不只是向数据提出一个问题，实际上你是在创建产品、模型或是由数据衍生出来的一些东西，并且你使用它们来促进你的业务的发展“。

利用好Hadoop

Driven目前只兼容使用Cascading创建的应用程序，后续的版本将支持Pig和Hive。这其中的一个原因是Cascading的流行度较高，在全球有约6,000项产品部署，其中很多不同行业的知名公司比如Visa、CBS和百思买等等都使用了Cascading，另外每月在开源社区还有超过13万的下载，这主要是由于其对MapReduce和Hadoop的支持（计划于今年晚些时候将支持Quartz和Sparc）。

此外，Cascading包括了ANSI SQL JVC驱动器，用户能够使用兼容Java与SQL的语言在Hadoop中创建应用程序，另外也可以使用任何第三方工具（如商业智能和分析平台）。通过整合Driven，开发人员可以用他们最熟悉的语言设计应用程序，并让每个编程语句可视化。这样的组合使他们能够利用最流行的框架来访问大数据（Hadoop），并查看与业务关注点直接相关的应用程序细节的可视化效果：Hadoop集群上的负荷有多大，哪些数据正在被访问，以及多少数据消耗正在发生。

最重要的是，Driven提供了一定程度的透明度，这增加了它的易用性。一旦用户初始连接了Driven所包括的插件，所有的遥测数据（如元数据和每个特定应用程序的功能或用途的描述）将在应用程序性能管理平台上可以被访问，然后用户可以对其进行可视化操作。Concurrent的首席执行官Gary Nakamura指出：

“Cascading的这个计划的目的是简化在类似Hadoop的结构上创建数据应用程序的过程。框架所做的事情是把复杂性抽象化，所以你可以按业务逻辑的单位进行思考；也可以像数据集成那样把内容分离，把每个问题隔离开来作分析。然后Cascading将把它们聚合在一起并在Hadoop上运行。“

付诸行动

虽然大数据计划的目标很大程度上取决于组织的行业和业务目标，不过最常见的大数据用例就是利用大数据技术以提高广告收入。比如，关于情绪的数据源多如牛毛，但要对这些数据进行分析，需要使用特别的算法和涉及特定运行程序的分析要求。

Concurrent与Twitter开始合作的一个原因是后者需要为用户显示基于各种数据的相关的匹配的广告内容。其收益部门进入Cascading创建一个API，这减少了定义工作量的复杂性，也减少了使用领域专用查询语言整合用户功能的测试数据源。现在，收益部门的人员可以根据最相关的数据进行分析，为广告商选择出目标消费者。

Wensel指​​出了为大数据构建应用程序的必要性：

“大数据的关键是能够运用不同的算法或一些无法使用SQL或其他的Hadoop语言所能表达的计算方式对数据进行处理。如果你是一个基因测序公司，而且你已经有了一个更好的流匹配算法，你真正想要的是充分发挥Java的全部能力，让Java工程师来解决问题。或者，你实际上在创建一个商品推荐引擎，创造一个评分模型，然后把它部署到网站上从而引导人们购买更多的商品。没有现成的工具可以很好的完成这些工作。“

智能的应用程序管理

最重要的是，Driven的发布和Cascading在短时间内取得的成功表明，只有大数据的应用程序专门定位于引导商业的发展，大数据的能动性才能体现出来。正因为有了这样的应用，分析工作才显得非常必要。而算法是许多数据驱动的进程中不可缺少的组成部分。有了合适的工具，比如Driven和Cascading，这些以业务为中心的应用程序使企业能够改进产品和服务，甚至能在竞争对手前面更早地开展这些工作。越来越多的开源用户已经意识到了这个事实。而随着更多的企业越早的开展这块的工作，大数据将越早地成为主流应用并蓬勃发展。Nakamura谈到这样一个事实：

“企业已经在他们的Hadoop需求上投入了大量资金，所以加快云应用在他们的Hadoop集群上的部署并获得收益，这对于他们的业务来说意义重大。确保业务的可靠性也非常重要，如有需要，则可根据业务需求优化应用程序。“

英语原文：

The Big Data hype will dissipate as quickly as it arrived, unless organizations find a way to produce business value from these new technologies. Doing so requires more than sandboxes for Data Scientists, real-time analytics, or Cloud-based storage and scalability.

It requires the development and production of business-driven applications that readily convert data into monetary action.

According to Chief Technology Officer of Concurrent, Chris Wensel, it requires doing so at a rate and in a manner that is more consistent, thorough, and swifter than those of the competitors:

“You have the opportunity now to differentiate yourself purely through your innovation – not because you have more money than the other guy, but because you’re smarter than the other guy. You have great engineers. You now need great tools to get those products to market much, much quicker. That’s the difference.”

Concurrent recently unveiled its latest tool (Driven) to assist in the production and monitoring of Big Data applications, which it touts as the world’s first performance management product specifically for Big Data. Driven was disseminated as a Cloud-based service to its user community on February 4. It was designed to complement its popular open source Big Data application development framework (Cascading) which principally runs on Apache Hadoop.

Driven directly impacts business value produced by Big Data apps by:

Reducing Development Time: Driven provides a means of visualizing the processes for enterprise apps (both comprehensively and individually) via a detailed user interface, which considerably speeds up the time spent in development and production, and gets apps to the marketplace quicker.
Strengthening App Reliability: Due to the ease of visualizing apps during the development phase, engineers will be able to identify problems and even readily anticipate them so that they can build more dependable apps before the product reaches the market. The capacity to visualize also assists with determining the most appropriate algorithms and application metrics to ensure accurate and consistent results.
Pinpointing Failures and Optimization: The visualization capacity for Driven is ideal for scrutinizing apps at the user level – including which data sets are manipulated, what the workloads are, and which apps are running. As such, the process of identifying points of failures in apps has accelerated considerably, enabling operations personnel to identify slow periods and optimization peaks in a matter of minutes – as opposed to days or weeks. The enterprise version of Driven 1.0 (which is projected for a second quarter release and has an annual fee) contains notifications which tells developers which App has failed, where the failure is, and who owns the app.
When used in conjunction with the framework provided by Cascading, the management features of Driven help organizations to ensure that they are exploiting Big Data to derive maximum value by building applications that reinforce business objectives.

As Wensel noted, such action transcends mere insight: “You’re not just asking data a question, you’re actually building products, models, or whatever it is out of the data, and you’re using that to make your business better.”

Leveraging Hadoop

Driven is compatible with apps created through Cascading, although subsequent versions are projected to support Pig and Hive. Part of Cascading’s popularity – which includes approximately 6,000 production deployments in a range of industries and companies such as Visa, CBS, and Best Buy, as well as over 130,000 downloads from the open source community each month – is attributed to its support for MapReduce and Hadoop (support for Quartz and Sparc is planned for later in the year).

Additionally, Cascading includes an ANSI SQL JVC driver which enables users to create apps in Hadoop with languages that are compatible with both Java and SQL, as well as to utilize any third-party tool that is as well (such as Business Intelligence or analytics platforms). By including Driven, developers can design apps with the language they are most familiar with and visualize each statement while doing so. The combination allows them to utilize one of the most popular frameworks for accessing Big Data (Hadoop) while visualizing the particulars of the app which directly relate to business concerns: how great the load is on the Hadoop cluster, what data is being accessed, and how much data consumption is taking place.

Best of all, Driven provides a degree of transparency which facilitates an innate ease of use. Once users initially connect its included plug-in, all telemetry data (such as metadata and other descriptions of what each particular app is doing or is used for) becomes accessible to the app performance management platform, which then enables users to visualize it. Concurrent CEO Gary Nakamura observed that:

“The plan for Cascading was to make it easy for the rest of the world to build data applications on top of fabrics like Hadoop. What the framework does is it abstracts the complexity away so you can just think in terms of business logic and also separate things like data integration so you can isolate each of the problems. Then at one time Cascading brings them together and runs them on Hadoop.”

In Action

Although the objective of a Big Data initiative largely depends on the industry and business objectives of the organization, one of the most common use cases for Big Data is to leverage its technologies for increased advertising revenue. The plethora of sentiment data sources, however, requires particular algorithms and analytics requirements that involve specific applications to process.

Concurrent’s partnership with Twitter began in part due to the latter’s need to pair users with the most relevant advertising based on a variety of data pertaining to advertising content and trending topics on its site. Its revenue department accessed Cascading to create an API that reduced the complexity of defining workloads and testing data sources while integrating user functionality with a domain-specific query language. Now, revenue personnel can readily analyze the most relevant data for advertisers to target consumers.

Wensel pointed out the necessity of application building for Big Data:

“The point of Big Data is to apply different algorithms or computations that just aren’t expressible in SQL or in other Hadoop languages. What you really want is the full power of Java to leverage Java engineers to solve problems like if you’ve got a better stream matching algorithm if you’re a gene sequencing company. Or you’re actually building a recommender engine, creating a scoring model and getting that deployed to a website so people buy more. There are no tools that do that.”

Smart Application Management

More than anything, the release of Driven and the burgeoning success of Cascading indicate that a Big Data initiative is only as successful as its applications are specifically targeted towards generating business. Such applications are the real reason that analytics are necessary and that algorithms are such an integral component of many data-driven processes today. Business-focused apps enable organizations to refine products and services and, with tools such as Driven and Cascading, allow them to do so before the competition does. There is a growing community of open-source users that realize this fact; the sooner more enterprises do as well the sooner the mainstream adoption of Big Data will flourish. Nakamura commented on this fact:

“Businesses have made tremendous amounts of investments in their Hadoop requirements, so accelerating the time it takes them to deploy Cloud applications on their Hadoop clusters and get yield from it are extremely impactful to the business. Making sure that they are reliable is also very important to them, as is being able to optimize applications when needed based on business requirements.”

【敬请添加微信公众帐号：数据堂 微信号：datatang】