User Name
Password
AppleNova Forums » General Discussion »

BareFeats Baninated


Register Members List Calendar Search FAQ Posting Guidelines
BareFeats Baninated
Thread Tools
murbot
Hoonigan
 
Join Date: May 2004
Location: Canada
 
2004-10-05, 11:31

Barred From the Apple Store

Just came across this interesting article. Seems that BareFeats' "Chief Mad Scientist", Rob-ART Morgan, has been banned from using his local Apple Store as a test lab for his site.

Looks like it was because of his less than stellar comments on the new iMac G5... but then again, they could just be tired of the dude doing performance tests on in-store Macs, which are probably not a well maintained as a computer should be prior to testing.

Just thought it would be an interesting tidbit to toss out.

Another interesting thing - his site is the sole source of his income? I'm surprised he could generate that kind of money from there, though I don't know squat about running sites, ads, etc. I should get off my butt and get www.appleinsideher.com running!
  quote
BuonRotto
Not sayin', just sayin'
 
Join Date: May 2004
Location: Durham, NC
Send a message via AIM to BuonRotto Send a message via Yahoo to BuonRotto  
2004-10-05, 11:36

From what I read, he's not banned from the store, just from using the store machines for his tests. I'm surprised they let him do that in the first place.

Bare Feats' testing isn't what I would call scientific either, and this kind of proves as much too.
  quote
psmith2.0
Mr. Vieira
 
Join Date: May 2004
Location: Tennessee
 
2004-10-05, 11:39

That's pretty funny. I do wonder what the outcome would've been if he'd given the iMac glowing praise?

Would be like going to the local Audi dealership, taking multiple test drives (with never any intention of actually BUYING one), then writing a damning review of the new model.



I'd ban his silly ass too, frankly.
  quote
johnq
Multi-touch Piñata
 
Join Date: May 2004
 
2004-10-05, 12:20

Not defending him at all, but...

Shouldn't any of us be able to test Macs to our liking (within reason, I don't mean putting in 8 hour shifts)? Purchase is not a requirement for dabbling, testing, playing on Macs in an Apple Store. A potential buyer will want to run them through their paces, even make comparisons between each model.

Wonder what he did that made it obvious he was running tests. Usually you can work on the Macs for quite a while without being pestered. Hell, I waited for 45 minutes to get someone's attention to buy a damn Mac.

His Apple Store must have been dead slow and had too many overly helpful staffers standing around.

"Peace cannot be kept by force. It can only be achieved by understanding." - Albert Einstein
  quote
alcimedes
I shot the sherrif.
 
Join Date: May 2004
Send a message via ICQ to alcimedes  
2004-10-05, 12:35

i think the problem is that by using in-store models to run his "tests" on, he's giving people the impression that he's actually testing these machines to some respectable standards.

i mean, when you're supposed to be testing gaming performance on a machine, you can't just walk up to a demo model, that had god knows what running on it, test it out and then say "Apple's numbers are way different than mine, are they fudging it?"

cripes, every store model has ARD installed on it. if nothing else you could easily have store staff watching him when he's on the machines, assuming they don't exactly trust him not to try and break crap. that right there would skew your supposed results, and he'd never know it. why? because he didn't set the machines up and doesn't know for sure what all is on them and running.

i always knew barefeats had crappy testing methods, but this just puts another nail in an already buried coffin.

Google is your frenemy.
Caveat Emptor - Latin for tough titty
I tend to interpret things in the way that's most hilarious to me
  quote
AirSluf
Member
 
Join Date: May 2004
Location:
 
2004-10-05, 13:33

......

Last edited by AirSluf : 2004-11-15 at 23:54.
  quote
johnq
Multi-touch Piñata
 
Join Date: May 2004
 
2004-10-05, 13:44

Right I agree with all of that, I just wonder, what, from Apple's perspective, differentiated this guy from a serious prospective buyer. Maybe the Cambridge store here is more tolerant of people (MIT kids) coming in and stress testing things in arcane ways than the store he went to.

I'm just picturing him wearing a BareFeats t-shirt, running a program called BareFeats TESTZ 1.0 on all the machines simultaneously and loudly saying "Man, these iMacs suck for performance!" with the powerdrain from the CPUs making the store lights dim to a brownout.

"Peace cannot be kept by force. It can only be achieved by understanding." - Albert Einstein
  quote
curiousuburb
Antimatter Man
 
Join Date: May 2004
Location: that interweb thing
 
2004-10-05, 13:46

Quote:
Originally Posted by johnq
Not defending him at all, but...

Shouldn't any of us be able to test Macs to our liking (within reason, I don't mean putting in 8 hour shifts)? Purchase is not a requirement for dabbling, testing, playing on Macs in an Apple Store. A potential buyer will want to run them through their paces, even make comparisons between each model.

Wonder what he did that made it obvious he was running tests. Usually you can work on the Macs for quite a while without being pestered. Hell, I waited for 45 minutes to get someone's attention to buy a damn Mac.

His Apple Store must have been dead slow and had too many overly helpful staffers standing around.
The first release 2.0GHz G5 had a "Test Drive" promo in store.

And IIRC, the purchase of a "Pro" machine from an Apple Store comes with a 'pro' card which equates to a 30 day money-back guarantee for you to take it home and test drive it all you want.

Sounds like he's too cheap to buy machines or buy and return like some other reviewers do.
Real reviewers ask Apple for machines to test and often get machines for test drive, sometimes for weeks.

Maybe he's just lazy.
  quote
bassplayinMacFiend
Banging the Bottom End
 
Join Date: Jun 2004
 
2004-10-05, 13:50

Quote:
Originally Posted by alcimedes
i think the problem is that by using in-store models to run his "tests" on, he's giving people the impression that he's actually testing these machines to some respectable standards.

i mean, when you're supposed to be testing gaming performance on a machine, you can't just walk up to a demo model, that had god knows what running on it, test it out and then say "Apple's numbers are way different than mine, are they fudging it?"

cripes, every store model has ARD installed on it. if nothing else you could easily have store staff watching him when he's on the machines, assuming they don't exactly trust him not to try and break crap. that right there would skew your supposed results, and he'd never know it. why? because he didn't set the machines up and doesn't know for sure what all is on them and running.

i always knew barefeats had crappy testing methods, but this just puts another nail in an already buried coffin.

It's obvious you didn't read the article which you should probably do first before replying. I say this because it's clearly stated in the article that Rob brings in a bootable firewire hard drive with OS X + benching programs. So it doesn't matter how the store set it up, Rob has a consistant OS / program setup.

Also (in the article but not directed at you, alcimedes) there was a memo from One Infinite Loop right after Rob's iMac G5 benchmark results specifically stating that Apple store Macs could not be used for benchmarking purposes.

Please people, read the article, then post.
  quote
psmith2.0
Mr. Vieira
 
Join Date: May 2004
Location: Tennessee
 
2004-10-05, 13:56

That's what I was thinking earlier. If you're a legit, credentialed (is that the right word?) writer/reviewer type, you can get machines for review, right? Everytime I read an article from MacMinute (there's one there today, actually, from the Detroit Free Press and a guy reviewing the iMac G5), the reviewer often mentions "before I returned the machine" or "upon receiving the review unit" and so forth.

So he could work toward going through legit channels, doing it a bit more professionally and scientifically.

I wouldn't walk into Guitar Center, plink around on four or five Strats for 20 minutes and use that to right a serious, indepth review. I'd arrange to have one sent to me, and I'd play it for a month in all kinds of situations. Otherwise, you're just a dork in a store, putzing around with stuff in an uncontrolled, subjective environment.



Bassplayer, bringing his own FireWire drive in - exposing the floor models to God-knows-what, probably irks the employees and Apple, regardless of how "good" that makes his testing. Does he ask permission, or just start hooking his shit up to their stuff? I'd DOUBLE ban him for that! You make it sound like a plus.

"Well, he farted all over the place, but he DID bring some scented candles, bless his heart...".


Last edited by psmith2.0 : 2004-10-05 at 14:02.
  quote
AirSluf
Member
 
Join Date: May 2004
Location:
 
2004-10-05, 22:35

......

Last edited by AirSluf : 2004-11-15 at 23:54.
  quote
alcimedes
I shot the sherrif.
 
Join Date: May 2004
Send a message via ICQ to alcimedes  
2004-10-05, 23:45

Quote:
It's obvious you didn't read the article which you should probably do first before replying. I say this because it's clearly stated in the article that Rob brings in a bootable firewire hard drive with OS X + benching programs. So it doesn't matter how the store set it up, Rob has a consistant OS / program setup.
i assumed he had to be playing the games off of the internal HD if he's supposed to be testing their framerates. you would get wildly different results playing Halo off of the internal drive vs. an external 1394 drive.

anything that can't load completely into RAM is going to give you different results when running off external vs. internal drives.

Google is your frenemy.
Caveat Emptor - Latin for tough titty
I tend to interpret things in the way that's most hilarious to me
  quote
FFL
Fishhead Family Reunited
 
Join Date: May 2004
Location: Slightly Off Center
 
2004-10-06, 01:08

Both the methodology and the ethics of "testing" this way are, in a word, fucked.

-2 Cred Points for barefeats in my book.
  quote
cudaboy_71
Member
 
Join Date: Aug 2004
Location: sacramento, ca
Send a message via AIM to cudaboy_71  
2004-10-06, 10:25

which article are you all reading?

the one i read implied a) he had a good working relationship with the employees of the store, often letting him modify the hardware (e.g. memory) b) it was company policy not to allow testing of store machines. apple finding out about his tests were the cause for enforcement, rather than new policy being instituted as a reaction to the findings. and last (and certainly not least) c) apple will be giving him a imac G5 for testing purposes.

from wired:
Quote:

To nurture a community vibe at its retail stores, Apple Computer encourages customers to hang out and play around with the equipment. The stores are pretty liberal: Customers can stay as long as they want, surfing the net, checking e-mail or videoconferencing for free.

But when it comes to running an unofficial hardware test lab, Apple draws the line.

Last week, Apple barred webmaster Robert Morgan from using the Macs at his local Apple Store as a performance test lab -- right after Morgan published tests results for the iMac G5 that contradict Apple's claims for the machine.

Since 1995, Morgan's Bare Feats site has often been the first -- and sometimes only -- source of performance data on Apple's newest machines.

But on Sept. 21, Morgan published tests examining the 3-D games performance of the new iMac G5.

Conducted on a display iMac at the Fashion Valley store in San Diego, the tests' results implied the machine is only moderately faster than its predecessor, the Luxo-lamp iMac G4.

But according to Apple, the iMac G5 is three times faster than the previous model.

Morgan's results sparked a minor storm of controversy on the web. His results were widely linked on Mac websites and hotly debated on forums.

Morgan has been using display machines in various computer stores since 1995, and those at the Apple Store in Fashion Valley since March 2003, with the full knowledge and consent of staff, he claimed.

In fact, he was such a regular at the Fashion Valley store that he was given a special discount and was invited to give an in-store class, "Secrets to Squeezing Speed From Your Mac."

"I've been testing at stores for years," he said.

Morgan said he'd typically run a battery of performance benchmarks. He'd visit during off hours, boot the machines from an external FireWire hard drive and keep his time short to avoid monopolizing machines.

Morgan used to run Bare Feats from Honolulu, where he'd test machines at several stores, including a CompUSA that allowed him to tinker with the hardware.

"The staff not only let me test on their Macs, but let me add memory," Morgan said. "The agreement was, 'If you break it, you buy it.'"

Morgan claims to be a veteran Mac performance expert, having worked for four years as chief Mac scientist at the San Diego Supercomputer Center.

"The new iMac G5 is a wonderful machine. I'm just concerned that the advantage over the previous model has been overstated," Morgan said.

According to Morgan, a couple of days after publishing his results, he was contacted by Apple. Over the next few days, Morgan discussed the results with a couple of people from Apple, including the company's PR department. In prior years, company representatives generally haven't responded when he's notified them of his test results, Morgan said.

And then last week, he received an e-mail from the local Apple Store containing the line, "I have some bad news." The note said the store would no longer be available for equipment testing.

Morgan said the ban came from Apple HQ in a "policy memo."

Apple confirmed that Morgan is no longer allowed to test store machines, but strongly denied the ban is related to Morgan's iMac G5 test results. Rather, the stores aren't appropriate venues for conducting performance tests, said Natalie Kerris, Apple's director of products and technology PR.

"Our retail stores are for customers to try before they buy," she said. "They are not test labs."

Kerris said Apple may be willing to send Morgan proper test machines, but because his site is relatively small, he likely won't be first in line.

According to Morgan, Bare Feats attracts 300,000 unique visitors a month. Morgan said the site is his sole source of income.

After Wired News talked to Apple, however, Morgan said the company has promised to send him an iMac G5 for testing. "They're making nice with me," he said. "It's a lot better than going to the store. I didn't expect this outcome, but I'll take it."
  quote
psmith2.0
Mr. Vieira
 
Join Date: May 2004
Location: Tennessee
 
2004-10-06, 10:32

Well that's good. If he gets his own machine and can test it properly and reliably, maybe he'll have something to contribute.
  quote
AirSluf
Member
 
Join Date: May 2004
Location:
 
2004-10-06, 11:28

......

Last edited by AirSluf : 2004-11-15 at 23:54.
  quote
Moogs
Hates the Infotainment
 
Join Date: May 2004
Location: NSA Archives
 
2004-10-07, 22:28

Barefeats has always been a completely non-rigorous operation IMO anyway. Their benchmarks are skewed / worthless most of the time, and their conclusions all of the time.
  quote
UnixMac
Member
 
Join Date: Jul 2004
Location: Phoenix, AZ
 
2004-10-07, 23:14

Tough crowed you all are. I don't know of many good benchmarking site period. His is no different.

I'll bet if someone wanted to place all bias aside, and was able to get machines without relying of freebies from the manufacturers, then we could have a really good benchmark site to reference. But A) we're not gonna find someone with time and money to invest in this money loosing venture, and B) everyone is biased!
  quote
alcimedes
I shot the sherrif.
 
Join Date: May 2004
Send a message via ICQ to alcimedes  
2004-10-07, 23:29

www.xlr8yourmac.com

unbiased in every way i've ever seen, and i trust their reviews on everything.
  quote
Moogs
Hates the Infotainment
 
Join Date: May 2004
Location: NSA Archives
 
2004-10-08, 08:15

Ditto that, and storagereview.com also, though they're not systems benchmarkers obviously.
  quote
UnixMac
Member
 
Join Date: Jul 2004
Location: Phoenix, AZ
 
2004-10-08, 09:42

Quote:
Originally Posted by Moogs
Ditto that, and storagereview.com also, though they're not systems benchmarkers obviously.
This I can agree with 100%
  quote
Posting Rules Navigation
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Post Reply

Forum Jump
Thread Tools

« Previous Thread | Next Thread »

All times are GMT -5. The time now is 17:01.


Powered by vBulletin®
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004 - 2024, AppleNova