A hotel management tries to engineer the most pleasant room to stay in. They work with temperature, light and music. On floor 5, they test temperature, light on 6 and music on 7.

The test works like this: walk down the corridor. Enter a room to your left, and then a room to your right. Each room might or might not have a different temperature (on floor 5), a different type of lightbulb (floor 6) or a different music system (floor 7).

On floor 5, can they tell whether two rooms are one degree apart in temperature? Five degrees? 0.5? 0.1?

On floor 6, can they tell a 60 watt lamp with luminance factor X from a different one?

On floor 7, rooms to the left have a certain DAC, rooms to the right another. Or the music was recorded with a Neve 1073 vs a clone. Or it’s mp3 vs Wav. Or tubes vs solid state. Or anything we discuss here. Can they tell them apart? Can you?

Now, here’s what I assume: when it comes to temperature and light, there can be and typically are differences in how pleasant a customer experiences one room vs another, that come down to one degree warmer or colder, or different lightbulbs.

Even in cases when the customer can not reliably report that room A is warmer than room B. And even for individual customers who fail the test.

Why would it be different for sound? Why would being able to consciously tell A from B rule over how pleasant a listener finds the music?

Note - I am ONLY talking about cases where equipment (like thermometers or null tests) CAN reliably tell A from B. So where there IS a difference, that human perception can in principle pick up (like temperature), and could affect their mood or experience, but where most people can’t say the exact value or pinpoint a change in values.

Not quite following, but if you can hear it, you can measure it, but if you can measure it, you might not be able to hear it. With the former, this has nothing to do with preference... like your room: when I walk into a hotel room, I want it to be about 2-3 degrees cooler than it is, even in the dead of winter.

I also want the shades to close tightly, because that's literally all I'm doing in there, is sleeping.

However, a person staying at a hotel for an extended period, possibly working in there during the day, might want natural light and for it to be cozy/warmer.

I hated living in hotels, its not what you think.

And I could spot the differences.

All that matters, is what comes out of the speakers. - Andrew Scheps

An A/B test is like transporting you instantly from one hotel room to the next, so you can actually compare the two rooms directly without first adjusting to whatever is going on in the hallway/elevator/etc.

When comparing the temperature of one room versus another, you're not actually comparing the temperature of room B versus room A, but rather your memory of room A's temperature versus room B.

An A/B test is only as good as your memory.

For hotel room sound two things matter. The most imprtant one is isolation. Isolation from other rooms, mechanical systems noise, and outside noise. The second thing is how live or dead the room sounds to the occupants with either end of the spectrum unpleasant. Architects and Engineers have balanced cost of construction with spec'ed acoustics for hotels for centuries now, hopefully improving incrementally along the way.

I’m not sure if you read the post, but the hotel room is just an analogy. The topic is A/B testing of sound equipment like DACs, preamps and other things discussed a lot on this site.

Here’s an attempt to clarify what I’m after.

There’s a difference between:

A) Our ears/brains not being physiologically able to pick up a difference

B) Differences we are fully conscious of, that we can verbally report the details of

C) Differences picked up by our ears/brains but that we aren’t fully conscious of and/or can’t report the details of

When it comes to perceving sound for enterainment/art, both conscious and subconcious differences matter. But a test that relies on the listener being able to pinpoint and report the difference is only about differences we are entirely conscious of and attend to.

But, just like a difference in temperature can/will matter to the overall pleasantness of a hotel night, a difference in sound can/will matter to the overall pleasantness of music. If the difference somehow affects us, even if we can’t say how.

The real ”test”, or the real result a hotel manager or music producer is after is the customer saying ”yes” to the question ”do I want to come back to this?” and/or recommending it to others.

Which is often due to a subtle sense of liking it for some reason one can’t really pin down. It might feel like a whim. But there are still causes for that whim or that sense of liking it, like the temperature of a hotel room being just right, or the sound of a track being just right.

I think I'm more confused, but don't take that personally... on about 4 hours sleep.

With all the Hotel comments I feel like we're in The Shining.

It really really really depends on the hotel.

I've found living in W's, Omni's and Hyatt Place's to be pure Hades.
While I wish I never had to move out of a few Mandarin Orientals, Amans and Waldorfs I've spent more than a week in.
Like, would have abandoned my three-level home without renumeration just to be able to stay in a suite there for the rest of my life.

Similar to audio gear, one gets the life one pays for. (Or can afford).

Boy, that hotel management is on the wrong track. Of course there is no one temperature that everybody is comfortable with. Put a thermostat in every room, that will make the customers happy.
By the way, it is remarkably difficult to accurately measure the temperature in a room. It depends on where you measure, air currents in the room etc.

With lights my advice would be similar. Give the customer as much control as possible. Multiple light sources, make them dimmable, colored lights maybe if you really want to get fancy.

And music in a hotel room, is that a thing? Surely you would be able to turn it off I would hope?

But I guess your point is that A/B tests are useless because there are certain unconscious processes that influence how we feel and they can't be detected with the test. I don't see why not. It's a matter of what you want to find out. Nothing is stopping you from subjecting people to different sound sources and afterwards asking them how it made them feel. All psychological tests work sort of like that. If people consistently prefer A over B in a significant way that will be scientifically relevant. Write it down in a paper and have it peer reviewed.

My point isn’t that A/B tests are useless

My point is that people on this forum need to be wary of false negatives. The absence of positive evidence does not constitute evidence of the negative conclusion

Thinking that there is no difference on the listener because the listener does not report one is a false negative

Thinking that a listener does not prefer the sound of some gear because they can’t say that A is better than B is a false negative: there might still be a preference

Most scientific ABX tests require "forced choice" over many trials. That is to say you are not allowed to "pass". You have to pick one or the other. If the difference is real, it will often show up statistically even when the subject feels unsure. This happens a lot. In any case in a true ABX, the listener can practice with the labeled examples. "This is A, this is B" - back and forth Then when you are ready: "this is X".

Then there is the question: are they being asked which room they prefer? Or are they being asked which room is warmer? Because if you are studying preference, you probably should not even tell them what is different about the two rooms!

The ability of humans to distinguish minute temperature differences might be interesting to doctors and medical researchers. But what is the point of 'proving' preference for anyone but yourself? Certainly the hotel cannot know in advance if you are the 69º guest or the 70º guest.


sure but what would probably affect their mood and experience even more is expectation bias. Put fake thermostat with an LED saying whatever number you want it to say. Put some food dye in the white wine, and even a lot experts will think it's a rosé. Mount a Behringer preamp in a vintage Neve chassis...

...if the musician is happy, maybe he performs better, even
if he is deluded. The audience too, may like happy music more than better gear. That is just psychology, though. Not physiology. Not audio. I forget who it was, but some famous engineer said something to the effect of: "if I can't readily hear the difference blindfolded, I quickly lose interest in that difference".

Over a week's time, the 1 degree difference might become a much more significant thing, and probably not something that could be reliably picked out in a minute or less of visiting each room. Also, if you don't tell the person what the variable is you're testing for, they might start looking for other differences that aren't there -- simply because they're expecting a difference.

It all seems to me a case of asking the wrong questions. If you want to know if a person prefers A over B then ask him that. If you want to know if they can tell the difference between A and B then find out if they can. The two things are not necessarily related so no conclusion about either can be derived from the other.

My point that asking them is not a useful method to find put if they can percieve a difference. Whether they can tell a difference as in telling us there is one, can only show a result if they can in fact do so. If they can’t say, all bets are off. ”I can’t tell” means nothing about whether they perceive one or not.

Perception here including unconscious or unattended perception. Like the refrigerator we don’t think about making a sound until it stops. Or the alarm clock that wakes us (proving we perceive sounds even when we sleep, otherwise alarm clocks couldn’t awaken us).

The wording is tricky because ”can they tell a difference” can mean either: ”is a difference perceived by them, somehow” or ”can they spot it and tell us”.

As the relation to preference, I find it puzzling. A forced preference like Joe referred to might statistically show something. But again if it does not do so, that’s not evidence that there is no preference. It might be under that radar too.

Don’t overcomplicate. AB is no more than listening to this, then that, and deciding which you like best.

Entirely disagree, it the analogue digital argument all over again.

Maybe we get up to different tings in hotel rooms and my house is nicer than yours?

I wanted to play baseball in the bathroom...

My emphasis here is listener satisfaction.

In nearly every thread on GS, there’s the ”but 95% of people can’t tell the difference”.

The Hotel analogy here is supposed to show that:

1. Them being able ”tell” as in ”tell someone” if there is a difference or preference is not what matters.

What matters is whether it makes a difference to their experience whether they’re aware of it or not. And nearly always, they are not aware.

2. What ultimately matters is how seemingly small nuances affect precisely the 95% who cannot consciously detect, judge or explain these differences.

3. We should trust what is in fact known about the physics and engineering of equipment, methods and signal flow, as well as its proven record in actual music, rather than draw far-reaching conclusions from artificial and constrained experiments, that only show limited results, as they cannot show the absence of an impactful difference.

They can show a positive difference, but that might be so blatant it might seem silly to even make the experiment.

If your hotel room feels too cold, you can put on a sweater. If you don't like what your compressor is doing, just add a distortion plugin.

When it comes to the majority of listeners, the A/B methodology idea breaks down precisely in the cases where we would want it to hold up. I’m not saying the experiments have no use, but they need to be interpreted with their limitations in mind.

I’m also not saying you shouldn’t A/B your chains or plugins when you make your music. It is good however to be aware that unconscious perception is real even for professionals. Nobody has a 100% transparent perception of how sound affects them.

Also, working professionally with the same track repeatedly introduces new blind spots as you are all aware. Just like an author cannot proof read their own text. Their brain corrects it so they don’t see the typos. Same with sound. You know this. That’s why taking a break from a mix, listening on other systems etc is a thing.

there is this strange obession with 'testings' and 'shoot outs' here on gs while in fact, people perform simple

comparisons

for which switching between signals or using the bypass switch is enough - carrying out correct and meaningful tests is a completely different matter...

As I have mentioned before ddy, in times of uncertainty and lack of knowledge people will fill in with with whatever they can.

I can tell the difference between good and bad hotels as I lived in them for 6 months of the year for about 15 years.

I can tell the difference between good and bad audio gear as I have been playing with it for 40+ years.

I'm confused why so many reject the opinions of highly experienced people who can hear things that are quite obvious to them ( funny how 40+ years of training gives you that, as with any topic or subject.)

So fair enough if you want to ignore the opinions of multi award winning audio types and just go with the graph marketing published, fine if that your thing (not you ddy.)

At least we don't have to worry about upcoming audio types superseding us!

Simple example: other day had super rush job for BBC. There happened to be a young audio type hanging about who was blown away how I set up from scratch recorded and edited in about 10 minutes flat. X/Y > DUREC > BBC.

His little face when he asked what plugins I used as it sounded amazing. None, no Eq just the mics in right place.

People's ears and hearing are different. Their discernment of audio information in a piece of sound is different. So these tests often fail on the receiving end .. and there's room for error in the test methodology as well. It's like pointing a number of uncalibrated instruments somewhere and wondering why the readings are all over the place.

quote of the year!

___


trouble is (around here) that people mostly fill the gap by ascribing mystical qualities to gear...

Well, I've got two residences. One in the greater NYC area, one in L.A. Neither of them are at all impressive. And I don't often get up to much 'fun' in either of them.
On the other hand, I stayed for over a month in a Marrakech hotel on a gig about 10 years ago tho, and I could have played half-court hoops in my room's "bathroom". My 'room' wasn't even a room, it was a private 3-story riad within a hotel.
I've gotten to stay a few times at places like that, and it is sooo much better than home.
Horses for courses.

One of my favorite colloquial phrases. So many journalists and politicians drop it: "In these uncertain times".
Which times were even the least bit 'certain'?
Whether we have free will or we're on a set of determinist rails, the future always remains 100% uncertain.

I'm enjoying this a/b test exploration.
It's really way above my head though. Not computing. Whether we're trying to come up with a 'better' (or more exacting) way to a/b audio gear, or whether we're just commenting on how flawed our current test mechanisms are.
I'll also admit to not really getting the hotel room scenario in which to stage the analogy. Is it because all the rooms are the same layout? In this case, the auditioner might hate the layout, and that might factor into his comparisons of lighting, temperature, sound..... Maybe he/she thought she was auditioning room temps, but couldn't get over the fact that she only wanted a single room and you showed her three 1BR suites. Maybe test-driving a car a better stage? Not sure.

But I'm sure it's not in trem's explaining of the scenario and purposes. It's likely down to me just not having a big enough brain.

Listener satisfaction is 100% subjective. A/B test results are generally subjective and personal. This is borne out by the fact that results on most of these tests people post on GS end up no better than guessing games.

I forget who said, and I paraphrase.... 'If I listen back and forth 3 or 4 times, and I still can't tell which is "better," I lose interest.'

When deciding if something is better than something else gets this nebulous and complex, it no longer matters. This goes for anything, not just music gear. I know that an RE20 beats a 1960's Radio Shack mic. I can hear it instantly, and then I can quantify why the RE20 wins with specs. I know that a BMW 330 is a higher quality, more satisfying driving experience than a Pinto. Chef-made Italian food is better than canned Chef Boyardee crap. But two different dishes of gourmet Italian food? IDK, what do you like? A bit too much oregano? The guy next to you loves oregano.

Is a Stam 1176 better than a Warm 1176, or even a UA 1176? Is a Warm Centavo better or worse than a Klon Centaur? AB them, and it becomes a guessing game which is which. At that point, it just doesn't matter anymore sonically. What it sounds like is what we're talking about, right?

The problem for me is home is full of my stuff and friends live near by. Hotels, not so much. Oh and you cant smoke!

I do think it worth having these discussions as we should really pay attention to what we do day to day and understand some of the reasons and science behind it.

Big brains are not the issue, it is clarity of explanation.

I'm looking at the other AB bunfight thread about all DACS being exactly the same, apart from the things you cant hear and thus not the same?

There is a hoohah video that illustrates via awful music and a crap ADC. No idea what is trying to be proved and attached file on video is not the same and you can see bad cuts on the waveform (along with DC offset and weird Low End drop off.)

I even found a pseudo science paper that offers statistical analysis formula ?

You know when your talking to someone and you start to release they are a bit special...