On top of just being good and having good songs - a lot of it has to do with marketing and general exposure. You get out and you play CONSTANTLY. Take Moe for example - they had regular weekly (weekday) gigs all over NY state. they used to play every Monday at Broadway Joes in Buffalo - a weekday slot that no local band would touch - but they did. They were probably playing almost every day of the week in different parts of the state. They ended up over time generating a large following.
If you only stick to one area then your pool of people to draw from is relatively small. If you spread out then you increase that pool.
And don't forget marketing - it's about getting the word out. There's an infinite amount of varying tactics for marketing. But - flyers, ads, facebook... all of it needs to be done continuously. And as far as flyers - they need to be put up in places that your crowd will be at (at least 2 weeks in advance of your show) - hippie shops, head shops, colleges - it's an enormous amount of work and money but it has to be done to get a crowd. Word of mouth will eventually help out a lot too - but you first have to get the word out to begin with.
It takes a lot of time, work and money to get it off the ground.
And you are right - the jamband scene at bars is shrinking. Boston is awful for jambands. All of the clubs go for hard rock formats with 4 - 5 bands a night. They charge $5 cover, bands play for 45 minutes with no sound check - and nobody makes any money except the bar. It sucks. But - once you've established a good following, then the bars will listen. If you can prove that you can bring in 150 - 200 people then they'll give you a night. After you get in and show them what you can do - they love it. It's easier for them to handle one band's set-up and the jamband crowd is usually a whole lot easier to deal with that a metal crowd.