Design Inspiration: Smart TV, Internet TV & Gaming

In my Feb article Design Inspiration: Second Screen Designs I wrote about some TV applications.  In this article in the Design Inspiration series, I’ll talk about some of the many cool opportunities that I think are in the TV and gaming tech market, but no one is taking full advantage of yet.

It’s called by many names – smart TV, Internet television, IPTV, Web television and others.  These links show that Wikipedia’s definitions vary slightly.  But for the purposes of this article, let’s group all of the TV related functionality under Internet TV.  And define it simply as a device that you use for playing video and audio, connected to the Internet and to your TV set in your living room.  The Internet TV device could be a standalone device or integrated into a television.

Here are some of the features that would typically fall under this Internet TV device category:

I started using and experimenting with Internet TV type devices back in the late 90s with WebTV, the first device to combine the Internet with television in 1995.


Back when dial up Internet was common and before broadband Internet via cable and DSL.  My WebTV device connected to the Internet via dial up.  So it had a modem, a tuner, a hard drive and had features such as a crude web browser, email, program guide and Digital Video Recorder (DVR) so it could record broadcast programs and store them on it’s hard drive for later playback, in addition to viewing live broadcast television channels.

WebTV Hardware

In 1997 Microsoft acquired WebTV and in 2001 renamed it MSN TV.  The product evolved as technology did, and was rebranded and resold by many major consumer electronics companies like RCA, Samsung, Phillips, Sony and more.

My next Internet TV device was a TiVo, back before anyone knew what a TiVo was.  It wasn’t long after I bought it that I hacked the TiVo and upgraded it’s hard drive and added a second drive to significantly increase it’s recording capability.

Now that I was addicted to time shifted television viewing using Digital Video Recorder (DVR) tech and being able to watch what I wanted, when I wanted, and skip over the commercials, all at the touch of a button, it wasn’t long after that wanted more.  So in 2004 I built my own PC based DVR, and over the years added more and more hard drives into the terabytes, satellite and HD capabilities.  I used DVR software called SageTV, which was acquired by Google in 2011 and their tech is thought to have been integrated into Google TV.

With the increasing sources of programming streaming over the Internet, several years ago I finally cut the cord completely, abandoning DVR and broadcast TV, pulling the plug on cable, and using on demand video services like Netflix, Amazon Prime and the channel web sites like ABC, Comedy Central, etc.  I evolved through multiple TV connected devices like a PC, Mac and Playstation.

There are a ton of devices out there now that can fall into the Internet TV definition.  Too many in fact, and not enough that are very complete and take full advantage of what the convergence of TV and Internet tech could potentially do.

Here are some of the devices on the market today.  There are so many devices, I could write an entire article just to provide a complete list.  So I’ve focused on the most well known and feature rich devices that are significant players in this space.

TiVo Box

TiVo really made the DVR go mainstream, turning their brand into a commonly used verb – as in “Did you TiVo that show ?”.  They were made a cultural icon by the phrase’s use and the device’s appearance in popular entertainment programs.

TiVo has always been one of the most feature rich and easy to use DVRs on the market.  Far more so than proprietary DVRs offered by cable and satellite providers.  And also one of the most expensive with the combination of the device cost and monthly subscription fees.  Over the years they’ve expanded the device’s capabilities with streaming, multiple TV and mobile device integration, and other cool stuff.

Roku 3 Box

Roku has probably the largest offering of subscription and free streaming channels, claiming now over 1,000.  Their devices have been very inexpensive, currently all under $100.  Until recently, their devices have been focused entirely on streaming TV services.  Their newest offerings are adding casting from mobile devices running Android and iOS, and motion control.  So they are very clearly making a play for gaming and a fuller feature set.

In 2013, Roku and Chromecast were the two most successful streaming devices sold in the US.  With Apple TV in third.

Chromecast Device

Google’s Chromecast device is ridiculously cheap, currently at only $35.  But it is also by design, extremely limited.  It’s designed to be focused only on casting from a mobile device.  It’s basically a wireless video connector for your TV.

Using mobile apps on your Android and iOS device that have been specially enabled to support Chromecast, you can cast sources from your mobile app to your TV.  So the user experience is not very good compared to what people are used to and expect in the living room.  The user has to load up a video or game on their tablet or phone, cast it to the TV, control it with their tablet and phone rather than a simple remote control or game controller, and all the while they’ll need to keep the tablet or phone plugged in to power because typical use will drain the battery fairly quickly.  Control via the tablet/phone, while unconventional, can offer some innovation opportunity however, because a touch screen can be designed with whatever controls the app needs and isn’t limited to physical buttons.

Beyond that the Chromecast device has no software that runs on it, so it can’t do any of the things that a Roku, TiVo or Apple TV can do.  But again, that’s by design and simplicity certainly has it’s virtues.  And due to it’s simplicity or low price or both, the Chromecast has become very popular, very quickly.

Apple TV Box

Apple TV has a respectable list of streaming channels and capabilities and has casting, mirroring and dual screen capabilities called AirPlay that support iOS snd Mac devices.  It’s been around since 2007 and while Apple has, until recently, said that it is a hobby, it has consistently had rumors circling strongly around it, that Apple is planning something very very big in the TV space.  And I agree.

There are a number of players trying to make Internet TV devices based on the Android OS.  But Google’s Android TV has the most promise.  Back in April The Verge reported that Google was planning an Android TV device and in June Google officially announced it and Engadget’s hands on review indicates it has promise.

Android TV UI

Android TV is offering a platform that can support streaming, casting, gaming and the vast app ecosystem of Android.  And since it’s based on the Android OS and is open for developers to build apps, it’s possible that a device could be built to support DVR features, live broadcasts and more.

Google is offering an Android TV Developer Preview and docs in the Android TV Apps guide.  Consumer hardware is expected to ship this fall.

Amazon Fire TV Box

Most recently Amazon launched Fire TV.  It’s focused currently on some of the features similar to Apple TV and Android TV including streaming and mirroring.  Amazon has also included casual gaming – as opposed to console class gaming offered by Playstation and Xbox..

An ambitious device with impressive specs and a low price of just $99.  Although the first release did not get the best reviews because it’s user experience does not live up to the hype.  According to reviews from CNet, Gizmodo and ReadWrite to name a few.  

Fire TV is based on Android, as are other Amazon devices, but that may limit it because it’s goal is to support Amazon’s services and it’s not open to the broader innovations happening in the Android space.

Over the last several years there’s been a blurring and convergence of product categories and devices.  Such as:

Integration between devices offering second screen or multiscreen user experiences such as:

I wrote about this in my Feb article Design Inspiration: Second Screen Designs.  This is one of the areas where innovation in the design of the user experience can be transformative.  And there is a ton of opportunity to innovate here.

Second screen or multiscreen designs have the potential to revolutionize many types of applications.  And to date, app developers have barely scratched the surface of that enormous potential.  Second screen and multiscreen refers to apps that are designed to extend or enhance a User eXperience (UX) by using additional devices to interactively connect to a primary device.

Here are some current second screen / multiscreen apps.

Television program based apps on smartphones, tablets or web sites that provide extended content for television shows and movies.  The most innovation that we see in this type of app is maybe real time synchronization with the program.  But even that does not have much usefulness and the user experiences are pretty bad.

These apps provide guides to what programs are available on broadcast television or via on demand services.  Most implementations don’t do anything more useful than provide an enhanced electronic version of a paper based TV guide.

But Netflix has an interesting and useful implementation of second screen interactivity with their streaming service, that just begins to show the potential that second screen designs can offer.  The second screen mobile app can control the streaming on a primary screen.  Effectively turning your mobile device into a sophisticated remote control for your TV.

I’ve used this myself using the Netflix app on my iPad to control the stream playing on the Netflix app on my PlayStation 3.  This is really handy.  You can not only manage and play programs on your iPad, but also send the stream to play on your PlayStation.  And then control the playing of the stream from the iPad.  It’s very cool.  Sadly this same functionality does not work if you use a Mac or PC connected to your TV, using a web browser.  Netflix needs to implement this real time interactivity in their app across all of the platforms they support.

Here’s a Netflix doc on this second screen capability.  Netflix does support an enormous number of devices – a huge competitive advantage for them.  So this feature may be supported on other primary screen based devices.  For more on Netflix device support check out the Watching Netflix section of their Help Center.

Apple TV UI

This is simply mirroring a device’s screen on another device.  Apple’s AirPlay mirroring on Apple TV and Google’s Chromecast are examples.  Although not a terribly innovative second screen design, it is very useful.

Now here’s where second screen designs start to get interesting.  These implementations use a mobile device as an fully interactive second screen to a video game’s primary screen.

Apple TV Dual Screen Gaming

The Dual screen capability of Apple TV is an excellent example of this type of second screen design.  In this scenario the game app runs on an iPhone or iPad which displays a second screen UI with controls, info and other supporting functionality, while it simultaneously broadcasts a primary game screen with the action to an Apple TV device.  This is wicked cool gaming and is the future of how games should be.  But I think Apple may do even more with the next generation Apple TV.

Sony is starting to do some interesting things in this area with their PlayStation 4 app.  And so is Microsoft with Xbox SmartGlass.  But I think Apple has the lead in second screen gaming because the tech has been out longer and is more robust.  Not to mention that mobile device gaming has exploded and has taken a big bite out of console gaming markets like the Xbox and PlayStation.

There are a lot of opportunities for innovation in smart TV, Internet TV, gaming and second screen / multiscreen technologies.  Here are some of those opportunities.

Apple is getting ready for a big innovation in TV.  They could dominate in Internet TV and gaming.  They are boasting their tech that could be used in TV.  And they can leverage their enormous ecosystem of apps, developers and services infrastructure.

Apple’s upcoming iOS 8 offers some big leaps in tech for developers.  Like Metal to dramatically increase graphics performance and it is targeted at support for console level gaming performance.  SceneKit and SpriteKit to enable game developers to build 2D and 3D games even faster.

A huge number of new gaming controllers for iOS were announced at E3 in June 2014.  Apple has been enhancing it’s Made For i[devices] – MFi program to encourage more hardware accessory development.

I also think that Apple is working on motion control and 3D vision tech that could appear in Apple TV and mobile devices.  Last Nov Apple acquired PrimeSense, the company that created the original 3D body sensing technology used by Microsoft for it’s Kinect motion gaming tech.  In a TV product this kind of tech can compete directly against Microsoft’s Xbox Kinect and Sony’s Playstation Move tech.

In addition, Apple has laid the groundwork for another giant leap in processor technology with last Sept’s release of the 64 bit A7 processor in the latest iPhone and iPad.  This took the mobile processor developer Qualcomm by surprise and the mobile industry in general – to have 64 bit in a mobile processor so soon.  Power that today’s apps don’t yet need.  I think this move was to ready iOS and app developers for huge innovations this year and next.  These ideas are explored a bit in the Macworld article Why did Apple go 64-bit? One iOS, to rule them all.

Apple TV software betas are already part of Apple’s iOS Developers program.  Apple TV is based on the same A series processors with which iOS devices are built.  Currently using the 2 year old single core A5.  It has not been updated for over 2 years.  There’s an enormous amount of developments and rumors that indicate Apple is planning something.  In April Tim Cook removed the hobby label from Apple TV, with it generating over a $1 billion for them in 2013.

So it’s time for a new Apple TV product that will be big.

Although Android is way behind Roku and Apple TV, and Google has had it’s challenges with the Google TV attempt 4 years ago, there is clearly massive potential, momentum and built up demand for Android TV.  And hardware technology is now so advanced and at price points that consumers will be willing to pay.

So I think Android TV is gonna hit big.  This is going to hit everybody in the Internet TV space pretty hard.  But hardest hit is going to be products built on proprietary platforms.  Next year when there are several Android TV boxes on the market and a ton of third party apps for them available in simple to get app stores, no proprietary TV box will be able to compete.  This might even happen as soon as this holiday season, since Android TV products are expected to ship this fall.  I think proprietary products are likely to move to the Android TV platform in order to stay competitive.  The same way that proprietary smartphones have moved to Android.

The gaming market is changing very very fast due to the power of mobile devices and the openness of iOS and Android.  Android and iOS devices now fully dominate computing devices world wide.  And the depth and strength of their app ecosystems means that thousands of apps and app developers are building apps for these platforms.

Because of the massive shifts in just the last few years, created by mobile tech, iOS and Android, gaming is bigger than it ever was.  But not gaming consoles.  And gaming console market players have missed this huge shift in the market.

What is happening in console gaming for a player once dominate in that market, Nintendo, is summed up in this from 3 Reasons the Nintendo Wii Failed:

I think not just Nintendo is falling victim to this way of thinking.  Sony’s Playstation and Microsoft’s Xbox are also vulnerable to what’s happened to Nintendo.

Android and iOS are positioned to make the leap to the TV.  And they will take over most gaming categories.  The ones that make the most money and sell game apps to the most people.  Game consoles will hold on to the hard core gaming for a while longer, but everything else is already and will continue to go to iOS and Android.

The hardware tech for mobile and TV devices based on Android and iOS have now advanced to a level that is capable of challenging much of what previously only gaming consoles could do.  Take for example, the powerful multi-core processors being used in Android and iOS devices.  The graphic capabilities and co-processors can do things with 3D rendering that rival consoles.

Voice control is a reality and getting better and better.  Google has Google Now.  Apple has Siri.  Microsoft’s Xbox has Kinect Voice control.  Amazon’s Fire TV has voice search capabilities.  All of this tech does need improvement, but that is the opportunity.  Offered by the OS as a service that app developers can integrate in to their own apps, makes voice control an ever bigger opportunity.

Microsoft’s Xbox One Kinect and Sony’s Playstation Move already offer motion control tech for game consoles.  That was first released in 2010 and is a great deal more advanced now.

I think that Apple could build motion tech in to the next Apple TV.  And it’s already starting to show up in Android hardware, so it’s just a matter of time before we’ll see it in Android TV.

Often program guides and search are service specific.  For example, Netflix has their search and guide, Amazon Prime has another, Comcast has more.  And none are integrated.  So there is a huge opportunity for innovation.

Not just integrated search, but integrated recommendations.  Recommendations based on rating data from multiple services and viewing behaviors.  Personalized by person, so it knows who in the house watched which programs.  Integration of additional sources of info such as IMDB.  And more.

Some food for thought on smart TV, Internet TV and gaming: