A great TV User eXperience (UX) is dependent upon primarily 2 areas:
Whether the next generation Internet TV / Smart TV devices are successful in a broad consumer electronics device like TV, will greatly depend on how well the entire User eXperience (UX) is designed – both content and technology. And the UX on Internet TV is one of the major problems that have kept them from being great, and a game changer in TV. Because creating a great TV UX is really hard stuff.
One part of the TV UX that is a ginormous issue that is preventing huge innovation in TV, is the licensing of the content from networks, channels and studios that produce it. As the music industry did, the TV content industry is resisting the massive paradigm shift to Internet delivery. It’s slowly working itself out – very very slowly.
ABCNews on Apple TV – First 30 Days
As more and more networks, channels and sources move towards Internet TV delivery, the momentum to change paradigms will grow exponentially larger. Take the recent experience of ABC News with Internet TV. In June an ABC News channel was launched on Apple TV. After just the first 30 days, the stats supporting Internet TV use were very impressive. Live video usage on Apple TV was 50% higher than on mobile and desktop devices combined. And viewers spent 65% more time per visit than desktop users.
But also similar to music, mediocre technology is limiting how we find and get content. It’s a chicken and the egg thing. Great technology is required to push the paradigm shift faster and pressure the TV industry to change it’s thinking around content licensing and delivery – and the TV content industry must change in order for the great technology to have great stuff to play.
While the content problem is all about needing a paradigm shift in the TV content business and letting go of last century thinking, the tech in TV really does stink right now and needs some major work. And that’s something that presents a giant opportunity that smart companies can do something about right now. So let’s focus on the tech in this article.
A Game Changer is Needed
Apple TV Device
A game changer is what Apple has been working on, and therefore why up until now Apple TV has only been considered a hobby. In April Tim Cook removed the hobby label from Apple TV, since it generated over a $1 billion for them in 2013. And why it seems likely that, as they did in smartphones and other product categories, they will launch a new TV product very soon.
Apple has a pattern of product innovation. They are not the first in a product category. But they spent the time focusing on creating a great end to end UX. So when the iPhone for example, was finally launched, it changed everything. That same opportunity exists in TV.
The TV UX has not been radically improved in over a decade – just incrementally added to and tweaked. TiVo came the closest to a major paradigm shift on par with the iPhone. It was a big innovation and got mainstream, with the word TiVo become practically a verb. As in “Did you TiVo that show ?”. But it lacked really giant wide spread adoption, due to price and issues penetrating a market dominated by cheap, proprietary and limited devices sold by cable and satellite companies.
Since TV devices haven’t changed much in a very long time and have been held captive by devices distributed by cable and satellite companies, there is huge room for innovation. And I’d argue, a giant pent up demand in consumers for real innovation.
500 channels and nothin’s on
Finding content is the biggest problem for a consumer. The phrase “500 channels and nothin’s on” seems to sum up the problem. Channel surfing was never a productive way of finding something you want to watch. And the problem just got worse over the years as cable TV exploded, and the number of channels available got larger and more and more unmanageable.
What we need is a a revolutionary and complete re-design of how we find content.
There are too many channels and program guides. Back when all we had was the package of channels offered by the cable and satellite companies, getting a single program guide with everything in it was doable. The UX on the program guides was often pretty awful, and based around old ideas developed many decades ago. Essentially based on the design of the TV Guide first published in 1953. Basically just electronic versions of the paper TV Guide. And no innovation. And because it’s was an extremely closed market, with cable and satellite providers in a near monopoly, competition was severely limited, and therefore innovation was straggled.
So you’ve got to hand it to TiVo, for accomplishing what they did, in such a closed market. And it was innovation that allowed them to do that. With Digital Video Recorder (DVR) capabilities, they eliminated channel surfing and allowed you to watch what you wanted, when you wanted it. And they designed an simple to use and intuitive program guide, and innovated way beyond what anyone had done with guides before.
With the growth of Internet delivered content from many many services, we now have a giant problem finding content all over again. Program guides and search are service specific. For example, Netflix has their search and guide, Amazon Prime has another, Comcast has more, DirectTV has theirs. And none are integrated. So there is a huge opportunity for innovation here.
First of all, search needs to be integrated. One search finds content in every service that the consumer subscribes to. Categorization has to be integrated, so they can surf categories and refine searches with them.
And we need not just integrated search, but integrated recommendations. Recommendations based on categories, and consumer viewing habits such as other people who liked this program, also liked those programs. Recommendations based on rating data from multiple services and viewing behaviors. Personalized by person, so it knows who in the house watched which programs. Integration of additional sources of info such as IMDB. And more there’s more that can be done here.
Part of the reason that Amazon’s Fire TV got some bad reviews is because they over hyped and under delivered on program discovery. They did something interesting by adding voice search but didn’t allow it to work with other services nor offer an integrated discovery user experience.
Other Internet TV devices don’t do any better with this. And the content delivery services are not making it easy for device UX designers and engineers, but that is improving.
While new designs for finding content will do a ton to finally move Internet TV forward and is the foundation of a great TV UX, there’s a lot more opportunity than just that. And the more that a next gen TV product uses, in the pure pursue of a great user experience, the more likely it is to finally re-invent TV.
Let’s be clear though. This should not be a checklist war – jamming in technologies to make your product feature checklist bigger, in order to look more impressive. And it can’t be, because your everyday, jane and joe public, typical consumer, doesn’t give a crap. They care about something that’s simple and adds a giant leap in what it can do for THEM.
In the history of TV, the TV Guide did that. TiVo did that. And they achieved Kleenex level status – a product brand name that becomes a commonly used noun or verb in our language. Known as a generic or genericised trademark. And TiVo did that without billion dollar marketing budgets. They were a small company, and they accomplished that enormously impressive goal by word of mouth, because they designed a product that was incredibly useful in a market, TV, that was utterly devoid of innovation. TiVo first shipped over 15 years ago.
With the explosion of Internet delivered content services, TV is so bad again, that there is an opportunity for something truly revolutionary, to become the next Kleenex in TV.
Motion Control Gaming Tech Circa 2010
Let’s take motion control for example. Most current motion UXs are very bad for the type of use that Internet TV presents. For example the motion control UX of game consoles like Xbox, PlayStation and Wii are very awkward to use to point to something, like navigating a cursor, clicking a button, navigating a menu or checking an option. Experienced gamers adjust to this awful UX quite quickly. But the average person that will be buying and using Internet TV in the 100s of millions will not have the patience for that kind of motion UX. And nor should they have to endure that.
So a great UX design for motion control is especially important for this type of consumer device. Gestures must be natural, so they are easy to use. They must be available with and without the use of a remote control device with motion sensors in it, like a dedicated hardware remote or a virtual remote implemented in an app on a smartphone.
Without use of a remote, the device would use cameras and/or other motion sensing tech. The motion control tech will need to be accurate enough to distinguish fingers. As the tech like Leap Motion can do. Finger detection opens up all kinds of sophisticated motion control techniques, that can do much more than a physical remote with motion sensors could do. Just as multi touch on screens can do. Resulting in a much more intuitive UX.
Imagine gestures to pause, play, fast forward and rewind a video. And to skip a music track on a play list or radio station. Perhaps a halt gesture for pause. Swipe gestures for playback and skip controls. Thumbs up or down for like and unlike controls. Drag and tap like gestures to duplicate touch screen actions.
Gestures should be context sensitive so the same gesture can do multiple things, depending on the current activity. Take a swipe to the right gesture, for example. When a video is paused, the gesture can play the video. When a video is playing it can initiate fast forward. When a music track is playing, the same gesture can skip to the next track. When an information panel is displayed, it can slide the panel to the left as it does on a touch screen.
Motion control can be used in many ways. Obviously in gaming apps, as it’s used on gaming consoles. In app UI controls in all types of apps. A UX that behaves like touch screen controls will make it easier to implement by app developers.
An example of what finger detection can do when music is playing. A pointed finger swipe can skip tracks. While multiple fingers could manipulate the visualizer graphics being displayed. Adding interactivity to music playing, as a simple example of a more immersive experience that can be possible with this kind of tech.
Voice control can be used to improve some of the every day uses of Internet TV. But not all. Navigating for example is easiest to do with a gesture of your finger, hand or motion controller.
Voice control can be a great way of executing quick actions, such as “Show me movies with Julia Roberts in them ?”. But voice control can only be useful if it can think and comprehend. Language is very complex, and the same action can be said in many many different ways. It’s not just a matter of translating speech to text. It must have a level of artificial intelligence in it that can interpret speech in to intention and meaning. Not just in to words.
Also, dealing with noise will be an issue for voice control. A room with a TV is a very noisy place. So noise cancellation tech will be essential. Building a microphones in to the device and remote is one way to deal with this problem. The audio being played back by the device can be used by cancellation algorithms so that it can clearly distinguish the user’s speech from the audio being played by the device.
Leverage existing UX Designs
One software and UX design strategy for app and web site developers to immediately be able to offer an acceptable TV UX, is to leverage the existing touch based tablet UX that has been built into apps and web sites already. Combined with a motion control UX that offers an excellent pointing UX and really really smart voice control, the touch screen designed tablet UX could provide a good, and maybe even great UX on a TV.
This is not to say that app developers will not have to optimize their UX for TVs. They will and should, in order to offer the best UX for customers. Competitors will be doing just that, and quite quickly too. So start designing a TV UX now.
PlayStation Move Controller
Game developers have already invested a great deal of time into UX designs that utilize motion control, game controllers, cameras, force feedback, Internet services and more. When PlayStation Move was launched in 2010, in wasn’t until 2011 before games with great motion UX designs started shipping. There is now an enormous amount of games for mobile devices with motion control. So leveraging the existing mobile OS APIs and the motion UX designs that developers have already built, will dramatically increase the speed with which new apps will ship that can take full advantage of all the TV tech.
Ultimate Internet TV Feature Set
The killer feature set for the next generation Internet TV that I think we’ll see later this year, could include the following features:
This will be one of the competitive advantages that a great Internet TV will need to have. Because in this new age of thousands of sources of programming, finding something worth watching is really difficult.
Great Motion and Voice Control UX
The remote control could be used as a motion controller and have microphones for clear and easy voice control. Multiple microphones allow for excellent noise cancellation. For example, Amazon’s Fire TV remote includes microphones and Roku offers remotes with motion tech.
Even greater motion control possibilities can be accomplish with a remote-less implementation, that uses visual tech, like cameras, to sense fine levels of motion. This would be accurate enough to distinguish fingers that opens up all kinds of sophisticated motion control techniques, that can do much more than a physical remote with motion sensors could do. Just as multi touch on screens can do. Resulting in a much more intuitive UX.
Apple has been investigating the use of motion control tech for a future Apple TV since well before 2013. An Apple patent surfaced in 2009 for a “remote wand”, also referred to as a “Magic Wand”. A remote control that could be used to control a “media system”. It surfaced again this year in an internal email from Steve Jobs that he wrote in 2010, describing a possible future “Apple TV 2” that involved subscriptions, apps, web browser and a magic wand control device. With the acquisition of PrimeSense tech, use of motion control in existing products like Xbox and Playstation for some time now, use of motion control for a while in games for mobile devices and the dual screen gaming tech already in Apple TV, the addition of the M7 motion co-processor and more – Apple seems even more likely to implement motion control in to the next Apple TV.
Full Powered Gaming Graphics
Graphics capable of performance levels to support casual to near console class 3D gaming.
For example, Apple’s upcoming iOS 8 offers some big leaps in tech for developers. Like Metal to dramatically increase graphics performance and it is targeted at support for console level gaming performance. SceneKit and SpriteKit to enable game developers to build 2D and 3D games even faster.
Full Web Browser
Next generation Internet TV devices should include a fully functional Chrome or Safari web browser with a User eXperience (UX) adapted to a TV User eXperience (UX) where the user is in a living room operating the device from the sofa. An inherently very different UX than mobile and desktop, so it needs to be re-imagined.
To immediately be able to offer an acceptable TV User eXperience (UX) support for web sites that are not yet optimized for a TV UX, the web browser can be designed to default to the tablet optimized UX of the web sites that supports tablets. Combined with a motion control UX that offers an excellent pointing UX, the touch screen designed tablet web site could provide a good, and maybe even great UX on a TV.
Physical Remote Control
There is no substitute for a physical remote with physical buttons on it. The passive viewing UX inherent in the experience of watching video on a TV, is best done with a simple physical remote with play, pause, skip and other control buttons that they can feel. People don’t want to have to look at a touch screen, to control playback.
The remote should use bluetooth so it does not require a line of sight with the Internet TV device. And ideally it should be rechargeable, although this adds costs, so many bundled remotes currently available are not rechargeable and use standard batteries.
In the future smartphones and tablets may have flexible touch screen displays with haptic feedback that simulate physical buttons. A recently discovered Apple patent for example, describes such a technology that we might see in the years to come. This type of research emphasizes the importance of the kind of physical feedback in our devices, that buttons provide.
Apple Haptic Patent Diagram
Advanced Remote Control App
More sophisticated remote control features will be more easily done with a touch screen device like a tablet or smartphone. And with second screen designs in the app, it can offer a huge competitive advantage to product designers that do this well.
This app can also provide quick and easy support for multiple game controllers. Get a game going by having guests simply download the app and pair it to the Internet TV so they can all play in party and family oriented games. The Nintendo Wii sold millions of consoles based on this capability alone.
Apple Remote App Play Panel
The app can also offer some simple force feedback support using the vibration capable of the device. Bumps, shots and explosions can vibrate the device, creating a more immersive gaming experience.
The current Remote app for use with Apple TV already supports using simple gestures on an iOS device. You use the screen similar to the way you would use a trackpad. You can swipe left and right between options, and then tap to select.
The current Android TV Remote Control app supports voice search. Other than that, it is relatively simple right now. But that’s to be expected since the TV products have not shipped yet.
Fast, cheap, flexible and offers remote control that does not require line of sight. The remote control bundled with the Internet TV should use Bluetooth. This needs to be at least Bluetooth v4.0. The latest 4.1 spec was formally adopted in Dec 2013, so that may be widely supported soon. Bluetooth software should support a full range of device types including remotes, game controllers, headphones, keyboards, mice, etc.
IR – Infrared
Logitech Harmony Ultimate Remote
Infrared provides compatibility with universal remote controls, like the very popular Logitech Harmony. Should support standard IR code sets, so setup is easy.
BTW Apple TV and Roku, as examples, have support for both Bluetooth and IR. Roku offers both IR and Bluetooth remotes. While Apple’s remote is IR only. Amazon’s Fire TV remote uses Bluetooth.
Optionally game controllers should be available because sometimes there is no substitute for a physical controller in a game. They should use bluetooth and be rechargeable. Force feedback hardware is common on game console controllers and adds vibration feedback in the controller and makes the gaming experience more engaging.
Sony Dualshock Game Controller
Apple has an advantage in controllers because of the huge number of third party controllers already available for iOS devices, due to Apple’s Made For iOS program. The number of controllers shipped and announced this year at E3 is clear indication that game controllers are big. And an iOS based Apple TV would have a ton of controllers supporting it, from the day it was launched.
Multiuser Game Control
Support for multiple game controllers, either physical or virtual controllers via a tablet or smartphone, will add tremendously to the casual and family gaming experience. Nintendo Wii was enormously successful in it’s day, partly because of these types of games. Wii Sports anybody ? But sadly, Nintendo didn’t keep up. This Motley Fool article takes a look at why the Nintendo Wii failed and makes this observation:
“Nintendo, which was once considered the “Apple of video games,” could soon become the “BlackBerry of video games”…
Obviously HDMI supporting 1080p and digital audio is required. Some current devices also include other connectivity like optical audio and older video connectivity. These days HDMI is so widely available and cheap, that the older tech is not necessary.
With today’s technology it should be a no brainer to include support for gigabit Ethernet and WiFi 802.11ac. Sadly, the brand new Amazon Fire TV and Roku 3 do not support gigabit Ethernet nor 802.11ac. Neither does Apple TV, but it’s over 2 years old, so there’s a good reason why it doesn’t.
Although that level of speed is not necessary for Internet TV, a great device should support the latest tech and since the tech is well established it should add little cost.
Massive App Ecosystem
This is one of the capabilities that will allow an Internet TV device to be a true breakthrough. A great User eXperience (UX) and an open app ecosystem, will create a breakthrough. A game changer.
Apple and Google have massive app ecosystems and will have an enormous competitive advantage over proprietary devices.
Apple is getting ready for a big innovation in TV. They could dominate in Internet TV and gaming. They are boasting their tech that could be used in TV. And they can leverage their enormous ecosystem of apps, developers and services infrastructure.
Obviously, in order for third party developers to create apps, development tools are needed. The more mature and robust those tools are, the faster apps will be created and the better they will be. Once again, Apple and Google have an enormous competitive advantage in this area as well.
To accelerate the creation of apps optimized for a TV experience, development tools should be extended with TV tech APIs that will allow developers to easily adapt apps to a TV User eXperience (UX). And APIs for existing tech in iOS and Android smartphones and tablets, like motion and voice control, should be extended to support the new capabilities of the TV tech.
USB is important for expandability and adds little cost to the device. External hard drives, flash drives, peripherals of all sorts could be added. Several Internet TV devices currently on the market, such as the Roku 3 and Amazon Fire TV, have at least some support for USB and playing media files.
USB 3, rather than USB 2, is important because it’s dramatically faster and has been around long enough now that it should be used in any new devices. That added performance is especially useful for playing and recording HD video. And would be important for supporting DVR apps.
In the discussions of Internet TV features there are some that will be discussed and abandoned. And others that should be discussed and planned for and added in the near future. All of these are supported via software, so they can be added through OS updates and apps.
Flash Support ?
Flash support on a TV device is a real long shot because of Flash’s lousy performance, massive crashing and security problems. But it would increase the number of television network and channel web sites supported, until they all convert to HTML 5 standards. And also increase the number of games that will run on the platform, until game developers update their games to support the TV platform natively.
There are still many network and channel web sites that require Flash to play video. They have not yet converted to HTML 5 standards. Although the massive paradigm shift to mobile devices has pushed many to convert, there are still big media companies that are slow to evolve to new technology – even though now it’s not so new any more. This sluggishness to move and innovate, is not uncommon for media companies – can you say “digital music downloads” anyone ?
Now while you can get third party web browsers for Android and iOS that support Flash, it’s not a good solution. It’s still Flash. So it’s handicapped. It’s old tech. It’s not using the advancing and open industry standards of HTML 5. The newer Flash Player versions are also not supported on Linux. The current version is v14.x and v11.2 is the last version to target Linux as a supported platform. Interestingly if you use FlashFox, one of those browsers on Android that supports Flash, and hit Adobe’s get Flash page with it, the page thinks it is detecting Linux and reports this warning.
Flash has been and is still today, a performance pig on PCs and Macs. And these are devices with extremely fast desktop class processors. It’s been 7 years since the first iPhone, and Flash still has all the same problems it did then – pitiful performance, it’s very unstable and has security holes galore. So it’s not likely to be allowed on Internet TV devices either.
But the upside, is that when Internet TV explodes in growth later this year, all those media company hold outs will finally be forced in order to compete with other media companies, to convert to HTML 5 standards for video playing and build native apps for the next generation of Internet TV devices.
For example, I still like to use web sites to play video from CBS, ABC, Comedy Central and others. And they still use Flash. It’s performance is awful, it’s unreliable as hell especially when dealing with commercials, and is very flakey so it’s not uncommon that I have to reload the web page to restart the video playing from the beginning then skip back to were it was when it lost it’s mind – very annoying. Although lots of their shows get re-distributed via streaming services like Netflix and Amazon Prime, it gets delayed. Sometimes delayed for quite some time. So although I avoid services that require Flash, sometimes I’m stuck and have to use these channel sites dependent on Flash.
The move to standards was been slowly progressing for a while now. Netflix for example, uses Microsoft’s Silverlight plugin when you play video in a desktop web browser – as does Amazon Prime. And when Yosemite was announced, Netflix confirmed that it supports HTML5 video streaming on Safari in OS X Yosemite. Initially it was flaky due to Netflix server issues – to be expected, it’s a beta. Now it works very well. This isn’t going to be an issue with Netflix, since they have native apps on like everything. I just point it out as an example of a giant player that is completely doing away with plugins and going with standards.
Optical Drives ?
Since this is an Internet device, a DVD or Blu-ray drive is totally unnecessary. It’s old tech. However, it could be supported via an optional external USB connected drive and an app to play movies.
However, studios are still delaying the availability of new movies and programs on streaming services, while making them available only on optical disk media. So having a drive to play content would allow it to be a complete device. And not require consumers to also buy and use DVD and Blu-Ray players.
Digital Video Recorder (DVR) ?
DVR capabilities are needed much less now than just a few years ago. Because of the huge number of streaming services and download options available. It also adds cost requiring hard drives and tuners. However, DVR functionality could be provided via an app and optional hardware like USB 3 connected external hard drives and external tuners.
Having DVR capabilities also makes it a more complete device, and provides a full featured device that can transition from today’s broadcast model of cable and satellite services, to the streaming, on demand services that TV is moving towards.
Wearable Integration ?
One of many iWatch concepts
Wearable devices, like smartphones, are expected to have motion sensors and voice control tech. Seems like they might be a natural remote control device for Internet TV. The hardware tech will be there. So to support TV, it will need the app, then it can do anything, any other remote could do.
No need to pick up a remote control, or launch a remote control app on your phone or tablet and press buttons, when you’re already wearing one. The motion control tech in a smart watch can augment or completely duplicate motion tech in a remote control, phone or tablet. Make gestures with your arm to play, pause, rewind and fast forward a video.
Voice control tech in the wearable, like the microphones built in to a smartphone, can do the same for Internet TV voice control. Raise your wrist and speak to navigate the features or search for anything.
A touch screen on a smart watch can provide all the basic playback controls too. Raise your wrist when the TV is playing something and touch playback controls appear.
Home Automation Integration ?
There have been a lot of innovations lately in home automation. From Philips Hue lights to Nest thermostats to remote locks and tons more. With all the home automation devices popping up like crazy now, with their own proprietary control apps for iOS and Android, it’s getting kind of nuts. There is no central control or integration standards for all of these innovative products. Islands and more islands of home automation. This somewhat defeats one of the purposes of home automation – central control.
At WWDC at the beginning of June, Apple also announced a tech called HomeKit. So clearly they recognize this problem. To accomplish central control, there needs to be a central device to manage it. To date there are a ton of competing hardware devices and software apps to act as this central device or hub, to monitor and control all of the many devices being automated. Way too many of them. Some are even sold with cloud services with and without subscription fees attached.
There are several methods of controlling devices. Some open ones like X10, Insteon, Z-Wave and others. And many proprietary control methods, like those used in many security systems and newer tech like Hue and Nest. So a hybrid of control schemes is the best of all worlds. Devices controlled by a central hub in the home, connected to multiple cloud services for varying degrees of service offerings.
So….. Why not run the central home automation software on your Internet TV device ? Make it the central hub that connects everything to everything else and manages and executes schedules and automation scripts. It’s got the processing horsepower. It’s got the open platform based on the widely supported iOS or Android OSes. It’s cheap. It will be available 24/7. There is already a ton of home automation app development invested in control apps for Android and iOS.
It makes sense.
Some food for thought on smart TV, Internet TV and gaming:
The biggest opportunities are in open platforms like Android and iOS, with huge app and services ecosystems, massive installed base, robust and mature development tech, and an enormous developer community.
OEMs and hardware designers have a huge opportunity to build extremely robust products that consumers will buy, using these open platforms.
App developers will have a huge opportunity in TV apps.
Start designing a TV optimized UX for your apps and web sites now.