I would counter that losing to crappy teams on your home floor has a larger negative effect.
In professional sports yes. These guys aren't playing for money, they're playing for pride and the fans. Hard to have pride in what you're doing when the fans don't care enough to show up.