“Integrate, Analyze, Visualize & Socialize” – Visualization Tools & Techniques

Turning raw data into insights often involves integrating data from multiple disparate sources (not just limited structured one), analyzing the data, visualizing it and socializing the results/insights to a broader audience to whom the results are of interest. In this cycle of turning data into insights, Visualization plays a vital role and hence would be the topic of my discussion in this blog post . Visualization could aid in analyzing huge data by identifying patterns which are easily interpretable visually as compared to tabular layout of numbers.Second, Visualization could help  represent the numbers using visuals which are easy for everyone to read and understand. One could easily convey the insights of the analysis by visuals, grasped in a minute or two, which might have possibly took 3-4 mins using textual aid/table of numbers.This is a important factor to consider especially when are you delivering the findings to the CEO/CFO/CXO/CIO of a company, as often they have limited time.

London Cholera Outbreak visualized

Going back to history of visualization. The most famous, early example mapping epidemiological data was Dr. John Snow’s map of deaths from a cholera outbreak in London, 1854, in relation to the locations of public water pumps. The original (high-res PDF copies from UCLA), spawned many imitators including this simplified version by Gilbert in 1958. Tufte (1983, p. 24) says,”Snow observed that cholera occurred almost entirely among those who lived near (and drank from) the Broad Street water pump. He had the handle of the contaminated pump removed, ending the neighborhood epidemic which had taken more than 500 lives.”
The following pointers should help anyone analyze data and socialize finding by effective newer visualizations techniques:

1. Fusion Charts – involves basic chart types, all it needs is a data file, configuration file and can link the chart to the data file, flash based  & supports interactive charts, web supported.
2. Fusion Maps – contains maps of all counties and major cities world wide, interactive, flash based, involves data file and configuration file, web supported.
3. Fusion Widgets – involves coolest visualization techniques like angular gauge, spark line/column,gant chart, pyramid, cylindrical & thermometric gauge & bulb gauge. Some of these charts have power to do real time streaming generally used in stock market analysis.
4. Power Charts – contains some of the rare chart types like node chart, heat map, waterfall chart, multilevel pie chart, candlestick chart,etc. again flash based and hence web supported.
4. R – Revolution Computing – a powerful open source data mining/stat language which can generate stacked multi-combinatorial charts using a single line of command.
5. Google Visualization – javascript based, web supported, involves some of the coolest viz techniques like motion chart which can display data in 5 dimensions, geomap, word cloud, money pile, 3D chart, QR code, etc.
6. Google Charts – contains all basic chart types, from google.
7. Custom Flex Charts – Using customer written flex code and action script code.
8. Microsoft Excel – famous for its quick and ease of chart creation , latest version now has spark line chart support.
9. Tableau – Data Exploration- would recommend this tool for rapid fire analytics involving various dimension, it is just as easy as drag and drop to change views of the metrics by dimension hierarchy.
10. BI Report Tools – BOXI, Cognos – commercial BI tools with support for creation of various report type based on charts and tabular layouts.

Industry Trends involve real time streaming of charts – used in supply chain analytics, interactive charts, mobile supported charts, Creating alerts in charts(for example alert biz. users sending an email, as the sales of any product goes below $x on three consecutive days and so on..), video & audio supported charts.

Beyond BI & Analytics

For the last 6 months, i have been closely following trends in information management. Below are few of my observations.

• Data source explosion: Business Problems are gaining complexity day by day, hence there is a huge demand for analyzing data from multitude of sources to help companies frame strategies for growth.  GPS data accumulated by Telecom companies offer insights into customers current location and provide context aware recomendations. Infact, some of the telecom companies have introduced location based pricing. Sensor data helps identify security threats to secure networks. Social network data has opened up as a channel for marketing services/product. Analysis of such closely knit data leads to behavioral & Contextual targeting. Traditional data analysis tools/algorithms fail to perform efficiently because such data are of huge sizes and needs newer datastructures for efficient analysis.
• Databases going beyond relational is gaining popularity. NoSQL dbs and Graph/Tree/XML based databases.
• Open Source tools continue to emerge.(R, RapidMiner, Weka)
• Growing need for massive dataset analysis.
• Artificial Intelligence(AI) and NLP gaining popularity among data analysts( in additional to ML techniques)
• Multimedia Analytics: Need for gathering critical metrics like customer footfalls, quantifying customers satisfaction by using facial expressions. All these applications demand high end signal processing( both Image & Video). There is a lot of scope for innovation in this area.
• Privacy preserving techniques for data analysis. This in turn encourages companies to outsource some of the critical data analysis to third parties.
• Agile Methodologies for Analytics Project to cope up with rapidly changing customer/business needs.
• Bio-Inspiration/Bio-Imitation: To learn from nature/natural processes and develop analogous techniques which could potentially solve a real-world problem. Some classic examples are development of Neural network inspired by working of a human brain, solving path optimization problem from Ant colonies, 280 degree view of honey bee(vision) etc.
• More and more data are made publicly available.
• Real Time data integration, insight generation and business decision.
• Complex visualization techniques through new technology like Adobe Flex , MS Silverlight,etc which are known for generating RIA.(Rich Internet Applications)

And I am sure these are just few items in the list and really not exhaustive. Feel free to share your comments.