We all heard the news of Facebook’s $100 billion IPO a few weeks ago and according to some analysts Apple will be the first company with a trillion dollar market value courtesy of their iPhones and iPads. It is amazing the scale and speed with which technology is changing and how it is shaping the world. These changes and pace bring new challenges to IT organizations. IT organizations have to adjust to the new ways users interact and communicate and support decision-makers in organizations and governments by providing intelligence from the large amounts of business data that are being generated.Social Media:As of April 2012, Facebook has over 900 million users. That is more than one in seven people who are a part of this single community. What can such a large community of seemingly common people accomplish? People with the help of social media, have most recently forced dictators and rulers out of power from Tunisia, Egypt, Libya and Yemen; civil uprisings supported by social media have erupted in Bahrain and Syria too. No organization or government can ignore its power. Since its’ inception, a single unhappy customer can create enormous damage to the reputation of a company … and it can happen very quickly. Organizations across the world are now faced with the challenge of managing this new medium that can be volatile and does not often give one much time to respond. Intelligent marketers now handle social media challenges by quickly mining the ‘intelligence’ from the data.
This allows them to make this a very effective channel to create positive opinion about their products and services. IT in many organizations is now asked to provide the intelligence from internet and social media to business teams.Informatica understands this and has developed a number of ways for IT to easily access the relevant information from social media sites. Look at the details for connectivity for Facebook, Twitter, LinkedIn and in fact any website on Informatica Marketplace. Informatica Marketplace has actually gone beyond just connectivity to social media and now has multiple sentiment analysis solutions for customers. These include the Marketplace innovation award winning solution by Deloitte which uses Informatica PowerExchange connectors for social media to extract data and Deloitte’s own sentiment analysis technology that allows users to train the algorithm for any industry. These solutions allow customers to mine the business intelligence from the social media data and accurately measure the sentiment about the products.Mobile:According to IDC report published in June 2011, over 470 million smartphones have shipped globally in 2011 alone. Many people use smart phones and tablets like iPads for their routine jobs. These are often easier to carry than laptops and are available wherever one might travel. How would it be if Informatica customers could carry out some of their key activities on their phones which would allow them to be on top of any situation while away from the workplace? Informatica Marketplace has brought some mobile solutions to market which will do just that.
PowerCenter Mobile Monitor is a solution which allows users to monitor PowerCenter and even restart the workflow from an Android phone. Informatica Cloud for the iPhone allows users to extend the self-service benefits of Informatica Cloud via iPhones and iPads. Informatica MDM-RE Search Client for Android is a free tool that allows users to extend the power of MDM-RE search to Android devices.Big data and Hadoop:With the explosion of compute power and the availability of cheap storage, companies are now reaching heights which are not easy to fathom. The New York Stock Exchange is generating one terabyte of data a day and LHC Geneva which is searching for the elusive “God particle” generates 15 petabytes per year. Even for a person like me who made a career in IT and has been in this industry for the last 13 years, these numbers are mind blowing.McKinsey in their report published in May 2011, actually estimates that by now, almost all sectors of industry will have an average of 200 terabytes of data. The good news is that with Hadoop, it is now possible to mine such large amounts of data. Trillions of dollars can be saved by companies and governments by processing such large amounts and leveraging the intelligence from it. The report also mentions that, if US healthcare were to use big data creatively and effectively to drive efficiency and quality, the sector could create more than $300 billion in value every year.
Two-thirds of that would be in the form of reducing US healthcare expenditure by about eight percent.Naturally, many companies have started to work with big data by leveraging Hadoop-based technologies. However, the challenges are enormous. Most of the Hadoop implementations are extensions to existing implementations. This creates the challenge of integrating the two environments and maximizing the synergies between them. We in the Informatica Marketplace team have brought solutions to market from within Informatica and from partners. For example check out PowerExchange for Hadoop which allows developers to treat HDFS alike any other source or target. We also have brought in the HParser commercial edition, which runs data transformation services as distributed applications. The transformation runs as a Hadoop Map Reduce job and within the MapReduce engine. This solution can be tried for a 30-day evaluation. The community edition of HParser is free and allows users to design and deploy up to five transformations and handle web logs, XML files and JSON objects.
There are other free tools like File and Table compare utility for HDFS and Hive which allows table and file comparisons on Hive and HDFS. Check out the Big Data Mall for these applications.It is fascinating to watch how people from the data integration industry try to address these challenges.