User Name
Password
AppleNova Forums » Speculation and Rumors »

The future of iMac control is...


Register Members List Calendar Search FAQ Posting Guidelines
The future of iMac control is...
Thread Tools
Matsu
Veteran Member
 
Join Date: May 2004
 
2009-11-07, 06:20

in the camera stupid!

A couple of weeks ago I watched the Jobs/Gate/Mossberg interview from 2007. Gates made an excellent point about computer controls. CCD cameras are cheap and the software for scene, facial, eye and gesture recognition is powerful and readily available. Doesn't this make a heap of sense as the ultimate in desktop computer interfaces? Microsoft has demonstrated and will introduce camera based motion control on their xBox. I can't believe I never thought of this, even after Microsoft's gaming related announcements. I had to hear it in the context of that discussion, and then again it sounded almost as if most of the audience and Mossberg himself glossed over it. A sort of, "ho-hum, still won't replace my mouse keyboard etc..." Maybe not, but the discussion veered a bit into people's comfort levels and expectations for interfaces, and as a supplement to those, the idea is potentially brilliant.

In the case of something like the iMac, the camera is already there, maybe some software is all it takes to introduce some basic motion control.

One problem that might be easy to anticipate involves unintended motion being interpreted by the machine as a command. That could get frustrating in a hurry. It would need some kind of signal that you intend to use motion control, but doing so would take something that should be fluid and make it tedious.

There is a possible, simple and cheap, answer: Add a second iSight. With two "eyes" your iMac would have stereoscopic vision and the ability to very accurately judge distance. It could be programmed to ignore everything behind you; or to watch your hands in the bottom of its peripheral vision (would require wide-angle lenses); or to only allow certain UI controls when you raised your hands within six inches of the screen, or some combination of all three... Make it smart enough and you can fluidly move between keyboard, mouse, and gesticulating wildly in front of your iMac.

Thoughts?

.........................................
  quote
chucker
 
Join Date: May 2004
Location: near Bremen, Germany
Send a message via ICQ to chucker Send a message via AIM to chucker Send a message via MSN to chucker Send a message via Yahoo to chucker Send a message via Skype™ to chucker 
2009-11-07, 06:29

Gates has always been a little… excessively "visionary" when it comes to input devices. mpt has a good account of that.
  quote
Matsu
Veteran Member
 
Join Date: May 2004
 
2009-11-07, 06:53

There are important differences between speech recognition and cameras.

People are still figuring out the former, together with other forms of auditory recognition/analysis. And while people are figuring it out, writing/typing (themselves different experiences) are altogether different thought processes from speaking. So, while speech recognition will get sorted one day, how and where people will use it depends heavily on both their own needs/limiations, for example in the case of disabled users, against the natural predelictions of the brain (the act of writing and reading vs speaking and listening... The people figuring this stuff out care specifically about those aspects, or they should, they won't succeed otherwise/

For the latter, cameras, the people figuring that out have interests in another large area of direct commercial application, mostly image/video. The technology is ready yet still improving at considerable pace, it's more a question of using it differently that inventing it in the first place. And where human interaction is concerned, it doesn't tread in neurologically confounding territory of the visual/auditory perception of language. Gestures are both easier for the machines to understand, and more natural to the humans as a form of control...

.........................................
  quote
chucker
 
Join Date: May 2004
Location: near Bremen, Germany
Send a message via ICQ to chucker Send a message via AIM to chucker Send a message via MSN to chucker Send a message via Yahoo to chucker Send a message via Skype™ to chucker 
2009-11-07, 07:18

Having to move your arms up to gesticulate sounds like an idea full of discomfort and health problems, with little benefit over doing gestures on a touch pad area (trackpad, Magic Mouse, Wacom Bamboo Touch, etc.).
  quote
Matsu
Veteran Member
 
Join Date: May 2004
 
2009-11-07, 07:50

Perhaps not even up, the cameras could follow your movements on the surface of a desk as well... I don't think you'd want it to replace your keyboard and mouse, but I could see a few instances where quick gestures would work nicely. widgets, games, maps, selecting images, manipulation of 3-d elements. The key to making it work would lie in being sensitive enough to not require overt motions, but smart enough to discard unintended ones. If I have to lift my arms above the mac for everything, it's not going to be of much use.

.........................................
  quote
Robo
Formerly Roboman, still
awesome
 
Join Date: Jul 2004
Location: Portland, OR
 
2009-11-07, 09:28

Quote:
Originally Posted by Matsu View Post
The key to making it work would lie in being sensitive enough to not require overt motions, but smart enough to discard unintended ones.
Sounds like...speech recognition.
  quote
Matsu
Veteran Member
 
Join Date: May 2004
 
2009-11-07, 11:07

Yeah, I know... but I think the task remains quite a bit simpler than trying to create the form of digital amanuensis necessary for speech to text to work. Speech commands would likewise be simple. The camera and mic are already there, it's be interesting to see how else they could be utilized...

.........................................
  quote
Wrao
Yarp
 
Join Date: May 2004
Location: Road Warrior
 
2009-11-07, 13:09

We'll probably get some hints(about the desirability for this sort of control scheme) when MS's Project Natal is out and active.
  quote
joveblue
Veteran Member
 
Join Date: Aug 2006
Location: Melbourne
 
2009-11-07, 18:32

Quote:
Originally Posted by Matsu View Post
Perhaps not even up, the cameras could follow your movements on the surface of a desk as well... I don't think you'd want it to replace your keyboard and mouse, but I could see a few instances where quick gestures would work nicely. widgets, games, maps, selecting images, manipulation of 3-d elements. The key to making it work would lie in being sensitive enough to not require overt motions, but smart enough to discard unintended ones. If I have to lift my arms above the mac for everything, it's not going to be of much use.
I can't see the benefit over a large trackpad in this case. Interacting with a trackpad however would give you a greater degree of accuracy and control, IMO.
  quote
Swox
OK Mr. Sunshine!
 
Join Date: Oct 2006
Location: Toronto
 
2009-11-07, 19:19

Quote:
Originally Posted by chucker View Post
Having to move your arms up to gesticulate sounds like an idea full of discomfort and health problems, with little benefit over doing gestures on a touch pad area (trackpad, Magic Mouse, Wacom Bamboo Touch, etc.).
My thoughts exactly. I just don't see the appeal of moving around like crazy to do something that is already done just as well with a mouse/trackpad/etc/. I don't doubt that there are going to be awesome new ways of controlling computers in the future, but I don't think this is it.

I think Microsoft are going to find out that people don't want to move around like crazy to play most video games, too (i.e. pretty much everything but sports games).

Do not be oppressed by the forces of ignorance and delusion! But rise up now with resolve and courage! Entranced by ignorance, from beginningless time until now, You have had more than enough time to sleep. So do not slumber any longer, but strive after virtue with body, speech, and mind!
  quote
Matsu
Veteran Member
 
Join Date: May 2004
 
2009-11-08, 09:17

A bit of an uncharacteristic fit of enthusiasm on my part, but as a few have pointed out, visual recognition isn't going to best traditional input. I agree. That said, it seems a shame not to use the ubiquitous camera in unique, fun, and potentially useful ways, sort of like the gyroscopic control has enabled a range of fun possibilities in the iPod touch, something that was only partially intended, and seemingly not the primary focus has become one of the great aspects of the system.

A camera that recognizes you at login, for instance, and loads your settings and login information. A system that lets you virtually hold a 3-d object and manipulate it as you would with your hands, without additional specialized peripherals. Or one that adjusts volume and brightness based on ambient lighting and seating position. Something that interprets your mood based on facial expression and juggles your iTunes random playlist accordingly (not everything has to be purely functional...)

.........................................
  quote
chucker
 
Join Date: May 2004
Location: near Bremen, Germany
Send a message via ICQ to chucker Send a message via AIM to chucker Send a message via MSN to chucker Send a message via Yahoo to chucker Send a message via Skype™ to chucker 
2009-11-08, 09:41

Quote:
Originally Posted by Matsu View Post
Or one that adjusts volume and brightness based on ambient lighting and seating position.
Note that a lot of Apple products already adjust screen brightness and keyboard illumination based on ambient lighting.

(I'd love for iPhone's volume to adjust itself based on ambient noise, though.)

Quote:
Something that interprets your mood based on facial expression and juggles your iTunes random playlist accordingly (not everything has to be purely functional...)
That would be absolutely awesome.
  quote
Luca
ಠ_ರೃ
 
Join Date: May 2004
Location: Minnesota
 
2009-11-08, 16:13

Sounds like Microsoft's Project Natal for the Xbox 360. I think it's a stupid-ass idea for gaming, but that type of technology could possibly be useful for certain things. Actually, wasn't Kickaha working on some sort of camera-type control for a computer interface?
  quote
Kickaha
Veteran Member
 
Join Date: May 2004
 
2009-11-08, 16:37

Yup, five years ago we had a working system for Macs.

http://www.cs.unc.edu/~smithja/FaceTop.html

For general computer control it was workable, but not something you wanted to use on a 24/7 basis. (OTOH, using a camera the way we did let you not have to wave your hands in front of the whole display - the camera could be off to one side, and you could make little finger gestures in its direction as needed without needing to keep your whole arm suspended.)

What it was *stellar* for, however, was collaborative document editing. It combined video conferencing, document editing, and natural gesturing all at one shot. It was really like standing with someone in front of a whiteboard and working together.
  quote
Matsu
Veteran Member
 
Join Date: May 2004
 
2009-12-20, 05:27

check out the link below.

http://www.youtube.com/watch?v=Jd3-e...layer_embedded

I got it from an AI story about Apple filing a head tracking patent application. I'm not saying that they would do this, but it certainly falls into the camera's already there so might as well sue it scenario.

The 3-d effect is quite convincing, but it only works for the person being tracked, which is just fine for a personal computer... What's really neat is that it allows the display to be much more like a virtual window than the current metaphor, which is more like a desktop or corkboard.

Properly constrained it could make for a very neat UI experience too. Imagine looking behind open windows, or pushing some menus off to one side and just leaning over to get a look at them when needed...

.........................................
  quote
Kickaha
Veteran Member
 
Join Date: May 2004
 
2020-06-24, 13:36

Well, this is interesting...

https://www.macrumors.com/2020/06/24...ody-detection/

  quote
kscherer
Which way is up?
 
Join Date: Aug 2004
Location: Boyzeee
 
2020-06-24, 13:40

Yeah, I saw that business. Can't wait for the "control your iPhone by unzipping your fly" app.

You know it's coming.
  quote
PB PM
Sneaky Punk
 
Join Date: Oct 2005
Location: Vancouver, BC
Send a message via Skype™ to PB PM 
2020-06-24, 14:41

It will be a big hit with hookup apps.
  quote
Kickaha
Veteran Member
 
Join Date: May 2004
 
2020-06-24, 22:53

"Swipe down to get down"
  quote
kscherer
Which way is up?
 
Join Date: Aug 2004
Location: Boyzeee
 
2020-06-24, 22:59

Quote:
Originally Posted by Kickaha View Post
"Swipe down to get down"
Obviously, that's the boy side of things.

The girl side will be "swipe up to get …"

Oh never mind. You figure it out.
  quote
Kickaha
Veteran Member
 
Join Date: May 2004
 
2020-06-24, 23:05

You presume much of this crowd.
  quote
kscherer
Which way is up?
 
Join Date: Aug 2004
Location: Boyzeee
 
2020-06-25, 11:21



I give them the benefit of the doubt.
  quote
Posting Rules Navigation
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Post Reply

Forum Jump
Thread Tools
Similar Threads
Thread Thread Starter Forum Replies Last Post
What does future hold for the iMac? Unbeliever Speculation and Rumors 128 2006-09-06 14:54
The future of iMac graphics adapter sonichart Speculation and Rumors 8 2006-04-11 08:52
We don't control what we do. macgeek2006 AppleOutsider 118 2006-01-15 13:08
Control Your Pda With Your Mac? MacMan05 Third-Party Products 0 2006-01-08 12:46
Help me (I'm out of control)! tskwara Purchasing Advice 44 2005-12-13 23:52


« Previous Thread | Next Thread »

All times are GMT -5. The time now is 07:46.


Powered by vBulletin®
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004 - 2024, AppleNova