Three Big Ideas for Beating Innovator's Block

I was invited to speak at the IBM dev@Insight conference last October.  I just realized last week that the event was recorded and that the video is available online!  My talk was about how my team overcame our "innovator's block" and created a set of integration patterns for using the IBM Watson Developer Cloud services within Watson Explorer applications.  We see these patterns as a small start of something much bigger and more important.  I hope that history sees our contributions as extremely minor and rather silly compared to the enormously awesome and amazing ideas that follow us in the coming months and years.
The talk is an Ignite talk so the slides auto-advance every 15 seconds.  It makes for a fast, intense, information packed talk.  I've also written a rough transcript of the talk for those who prefer to read (though the video is only five minutes long).

The Problem with Innovation

The phonautograuph was invented in 1857 by a Frenchman by the name of Edward-Leon Scott. What's interesting about the phonautograph is that it was the first ever sound recording device and it was a rather elegant invention for its day.  The device works by picking up sound waves from the air, similarly to how the inner ear drum works and then transcribes the raw sound waves onto paper or smoked glass using a needle.

The most fascinating thing about the phonautograph is that there was no playback capability.  Sound waves were written to paper much like music notes might be written to a page.  Scott had envisioned a world in which trained professionals would some day be able to read the raw sound waves and be able to translate speech or music directly from the written wave form.

In retrospect, such a device is almost laughable but in context it is easy to forgive Scott for such an oversight as no playback.  To start, this is the first device capable of sound recording, so that is an impressive feat.  Next, up to that point the only means of capturing sound was through written forms such as sheet music for an opera or symphony.  It was as if Scott was wearing innovation blinders based on his current context and physical reality.  In the world he knew, people read sound from paper.  It was certainly not created automatically from a recorded medium.

In the technology world we have a saying that summarizes this phenomena pretty well:

"I suppose it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail." (Abraham Maslow, 1966).

The important lesson here is that it is important to know how to use a variety of tools, otherwise your view on the world will be limited by your own capability or the context of your surroundings. Without a set of broad experiences you will be incapable of seeing different solutions to a given problem as everything will appear solvable only through the tools that you know how to use.

Search is my Hammer...

This was the problem that my team was facing when we were fist given the task of integrating the Watson Developer Cloud cognitive services with Watson Explorer.  We know search and we know Watson Explorer extremely well. While we were aware of cloud application basics and many of the theories behind cognitive computing, we had no idea what to do with cloud-based cognitive services we were tasked with using, within the context of the search applications that we were used to deploying.

Focusing on the capabilities in the new technology we struggled to come up with good ways of using these new capabilities.  And our first ideas were pretty bad.

"Maybe we could use Watson to 'cognitively enhance' text clustering?" we discussed.  Text clustering, the ability to dynamically extract relevant themes from a set of search results such that it takes into account natural language including grammar (e.g. stemming, tense, useful vs. noise words, etc.) and knowledge (e.g. acronyms, synonyms, stop words, etc.), was one of Watson Explorer's original core capabilities born over a decade ago back when the product was still known as Vivisimo Velocity and Vivisimo was just a tiny startup.  In and of itself this is not a terrible idea, it's just not super interesting.

"OK, well what about some kind of recommendation system like what Google and Bing do with ad words and user profiles?" This is getting warmer, but the more we talked it out, the less it made sense.  How would this play out behind the firewall in an enterprise setting?  Is this really the best we can do?

We have access to the most computing power and the most powerful algorithms in the history of the world and the best we can come up with is optimizing ads?  Certainly we can do better than optimizing the consumerist machine!

Focus on the User

Success finally came when we stopped thinking about the technology and started focusing on the user.  What problems did our users have?  What kinds of things do they ussually do with Watson Explorer?  How can we take solutions to those problems to the next level with the new tools that we had in the Watson Developer Cloud?

A common use case we see with Watson Explorer focuses on creating an "enhanced 360 degree view" of some entity.  For example, you might look at a person entity and then from a single UI view search results pushed to you contextually based on metadata from a variety of sources, or computed analytics, or really any number of facets of that entity.

With a solid use case in hand, it was extremely easy to figure out wich of the cognitive services in the Watson Develper Cloud worked for our use cases and which ones did not.  For example, Watson Personality Insights is a perfect complement to the 360 degree view of a person from either a real-time or pre-computed perspective.  Question and Answer is a natural extension of search and might apply in a number of closely related use cases.

Rather than looking at our tools and figuring out what do with our hammers, we looked out into the world to understand what tools we needed first for the problems that were out there to be solved.

Play to your Strengths

My team is an engineering team.  We're not product managers and not always super great at coming up with innovative, disruptive ideas.  But we are extremely good at software design and development. It's what we do.  Once we had some plausible use cases in hand we immediately shifted gears to bring the integration project into our wheelhouse: software development.

We immediately set about writing code and creating a useful architecture for how the Watson Developer Cloud might be used from Watson Explorer.  The most important thing to emerge from this exercise (other than working examples) was a set of integration patterns for when and how you can use Watson Developer Cloud services from Watson Explorer.  These patterns are important because it helps you to classify and quickly understand the nature of the solution once you know you're dealing with a certain kind of problem.  Further, knowing the patterns offers another way to navigate the available services as not all Watson Developer Cloud services are applicable to all patterns.

Software architecture is one of our strengths.  Once we fully embraced this, the project became much easier.

Collaborate

Reflecting on this project, one of the most valuable tools we had available to us was each other.  Our best integration ideas and the best work we did were the a result of my team and IBM Watson as an organization coming together to collaborate.  One of the most valuable activities we did was brainstorming and spit balling ideas on how Watson Developer Cloud services might work and what we could do with them.  When we were working together it was fun and the results were so much better than working alone.

In the spirit of this collaboration we've released our sample code, documentation, and high-level ideas for use cases under an open source license on GitHub.  Please take advantage of the work that we did. Build on what we started and create something truly amazing.

Let's Create Something Amazing

In 1878 Thomas Edison invented the phonograph, finally bringing playback capabilities - a full 20 years after Scott's phonautograph.  The lesson here is that you have to start somewhere. Even if it seems silly.  Even if it's not perfect.  Even if all you know is that you aren't satisfied with the first steps you took.  We're all in this together and defining the next era of computing is not going to happen by one person in a couple of months.  It's going to take all of us working together, playing to our strengths and never letting our focus wander from creating something useful for people that helps them solve real problems.