"I love software."
Big ideal: Automated Insight Generation through Analytics processes is starting to trump the Software Development process in terms of value generation.
Instead of Hierarchical Value Generation/Value Flow, we need to do horizontal flow.
People are starting to get rid of middle person between human knowledge and data scientist and replace with Insightful Applications/Machine Learning.
A computer that used machine learning reached Israeli Master level in chess in 3 days.
Two Primary Drivers of Contemporary Software
1. Internet of Things
2. Machine Learning
The only way to cope with vast amounts of data from IoT is machine learning.
1. Foundation of software is value generation
2. Current software methodologies and frameworks will need to change in 2020
3. End State (2025? 2030?) is to have machine learning do 70% of value generation process.
4. By 2018 will face a shortage of people with deep analytical skills to extract insights from collected data. We will need 1.5 million managers who have the quantitative skills to make decisions based on analytical data.
What to do when data quality is bad?
How to manage poor software quality in data analytics?
What will happen to all the people replaced by analytics?
What is Insight Generation?
Insight generation is the identification of novel, interesting, plausible, and understandable relations among element of a data set that
a) need to formulate action plan
b) it results in an improvement as measured by KPIs.
Case for Automated Insight Generation:
1. The amount of data is so huge, humans can't process it
2. We don't know how to really analyze and utilize Big Data
3. Time to decision is decreasing while data velocity is increasing - we don't have time to process anymore
What to do?
Don't let today's method/process/framework debates absorb all you attention cycles
Start in-house training programs in IoT and Machine Learning
Young kids want bleeding-edge projects