Transforming Big Data Information into Deep Data Insights
Yesterday I posted a question to several of the groups I belong to on LinkedIn. It was related to several of the things I’m interested and involved in: Systems Thinking, Knowledge Management, and Decision Modeling. It was somewhat informed, as well, by an article appearing in the Huffington Post, where Otto Scharmer, a Senior Lecturer at MIT and founder of the Presencing Institute, talks about the need to make sense of the huge and growing amounts of data we have available to us. He argues the importance of turning from “Big” data, where we mainly look outward in our attempt to understand what it is telling us about markets and our external influence, to “Deep” data, where we begin looking inward to understand what it’s telling us about ourselves and our organizations and how we get things done.
The question I asked was designed to seek out capabilities and functionality that people would like to have, but that is currently unavailable. My interests include working with others to understand and provide for those needs, if possible. I thought I would present the question here as well, where it will remain a part of my online presence and, hopefully, might elicit some useful responses. Here it is:
With the growing proliferation and importance of data — a development at least one author and MIT Lecturer has suggested is moving us from the information technology era to the data technology era — what tools would you like to see become available for handling, understanding, and sharing the new types of information and knowledge this development will bring?
In other words, what would you need that you don’t have today? What types of technology do you think would offer you, your colleagues, and your organizations a greater ability to make use of data to bring about a transformation from primarily siloed, outward looking data to collaborative, inward looking data as well?
I would love to hear of any ideas you might have regarding the kinds of tools or apps you could use to better deal with data by turning it into useful information and knowledge . . . perhaps even a smidgen of understanding and wisdom.
Systems Savvy 
October 29th, 2014 at 8:50 pm
When you look at the “data” that each person encounters as they go through life, it can be broken down into two basic categories: that which we are consciously aware of and have to process to make sense of; and that which we are not consciously aware of but our brains and bodies know how to use. Seven or so bits that we can consciously manipulate compared to the millions (billions?) of bits that “we” respond to without even thinking about it. Do these correspond to the ideas of “surface” and “deep” data? Not sure, but it is worth a bit of exploration.
When you look at a company, consider the executive(s) the “conscious” aspect, responsible for being aware of and responsible for actions and interactions with the environment. The rest of the company would then comprise the senses and unconscious data processing that informs the decisions and actions of the executive. If you take the comparison as exact and to the extreme, the decisions and actions taken by the executive / consciousness of the company will actually be the decision/action that the company has decided on and that the executive only thinks is their idea as they execute it.
Unfortunately, many companies don’t have this connection between their “senses” and their “consciousness” and rely on just the surface data. Executives have dashboards and other similar tools available to them now, but despite the depth of the data it may consolidate I would still consider that surface data, a conscious attempt at making sense of what the company knows. (Again, imagine a person – say an athlete – trying to mentally process a dashboard as they were trying to compete in an event of some sort.) The executives make decisions based only on what they can make sense of for themselves.
Or they have a connection, but there is no filter and they try to consciously process and understand all of this deep data that comes from within the company, the financial, the environment, the political. Imagine if you felt every piece of dust that fell on your skin throughout the day, or heard every little sound or saw every bit of light and had to make sense of it without the assistance of the relevant centers in the brain. And still had to drive your car to work in the morning.
I’m pretty sure I had a point I was building to when I started this, but that was a couple of Tasmanian IPAs ago and I’ve kind of lost the thread. Will come back tomorrow and see if I can’t make some more sense of this.
LikeLike
October 30th, 2014 at 7:24 pm
Hello Brett – Glad you stopped by. I asked this question for a very specific reason and, frankly, I’m thinking I may need to reword and ask it again. Maybe multiple times in multiple ways. I’m not surprised by this. In fact, it’s kind of what I expected.
I represent an organization that has some very strong – and pretty deep – expertise in decision modeling and what we call predictive intelligence. The name of the company is Quantellia LLC (http://quantellia.com). You might find what we offer of interest, as our approach is to offer a tool that can look at a huge amount of dust, listen to numerous sounds, and perceive a large portion of the light you refer to. The main tool is called World Modeler and, along with the software development and AI expertise of our two co-founders, there have been some useful customizations built on top of WM.
Imagine being able to not merely look over the horizon, but to also look around corners and (if I may stretch the metaphor) into closets. What we are after is to understand what capabilities organizations would like to have in terms of gathering and understanding the huge, and growing, amounts of data that’s available to analyze and, especially, synthesize. Here are some links to further info you may find of interest. I welcome any feedback or questions you may have. We’re looking for some challenges to meet.
Click to access DEEPM_Fact_Sheet.pdf
LikeLike
October 29th, 2014 at 3:02 pm
Thanks for the comment. What you’re looking for isn’t really what I was asking about. My interest is more in the development of tools that can make sense of what is being called “Big Data”, i.e. being able to discern patterns from huge amounts of data and using them to make large-scale decisions on important projects.
What you’re looking for I believe can be done with something like Dragon Naturally Speaking. You can get the basic tool for less than $50. You may also need a scanner and OCR software, unless you receive the essays already scanned and converted to digital files. I hope this is helpful.
LikeLike
October 29th, 2014 at 11:57 am
I have OCD and I dislike writing, even though I teach English. I would love to be able to read scanned essays then verbally correct them on line. I would like my ideas to be accurately be recorded in a document or blog without the need for typing.
LikeLike