You Need, Go Search,
Find Take, Return Changed.
It's not that stories have to follow this structure, it's that, without some semblance of this structure, it's not recognizable as a story.
When I talk about "story structure" I'm talking about something very scientific, like "geometry." Your story could have "perfect" structure, in that it hits all the resonant points craved by the audience mind, but that won't make it a perfect piece of entertainment.
Example:
Once upon a time, there was a thirsty man on a couch. He got up off the couch, went to his kitchen, searched through his refrigerator, found a soda, drank it, and returned to his couch, thirst quenched.
That was "perfect story structure." On the other hand, the story sucked.
Here's a converse example:
Once upon a time, a car exploded. A Navy Seal killed a werewolf, then a robot shot the moon with a Jesus-powered laser. The world became overpopulated by zombies. The End.
Lot of exciting, creative stuff is happening, but very little structure. Again, boo. What do you want? You want both. You want to be cool, but you're going to be cooler if the structure is there. Cool stuff with no structure is like that perfect scene you recorded when you left the lens cap on. "Guess you had to be there." Show me an army of zombies and I might say "cool zombies," but I'm not going to "be there."
1. YOU (a character is in a zone of comfort)
ESTABLISH A PROTAGONIST... Who are we? A squirrel? The sun? A red blood cell? America?
2. NEED (but they want something)
SOMETHING AIN'T QUITE RIGHT… Something is wrong, the world is out of balance. This is the reason why a story is going to take place. The "you" from (1) is an alcoholic. There's a dead body on the floor. A motorcycle gang rolls into town.
3. GO (they enter an unfamiliar situation)
CROSSING THE THRESHOLD… For (1) and (2), the "you" was in a certain situation, and now that situation changes. A hiker heads into the woods. Pearl Harbor's been bombed. A mafia boss enters therapy.
4. SEARCH (adapt to it)
THE ROAD OF TRIALS… Adapting, experimenting, getting shit together, being broken down. A detective questions suspects. A cowboy gathers his posse. A cheerleader takes a nerd shopping.
5. FIND (find what they wanted)
MEETING WITH THE GODDESS… Whether it was the direct, conscious goal or not, the "need" from (2) is fulfilled. We found the princess. The suspect gives the location of the meth lab. A nerd achieves popularity.
6. TAKE (pay its price)
MEET YOUR MAKER… The hardest part (both for the characters and for anyone trying to describe it). On one hand, the price of the journey. The shark eats the boat. Jesus is crucified. The nice old man has a stroke. On the other hand, a goal achieved that we never even knew we had. The shark now has an oxygen tank in his mouth. Jesus is dead- oh, I get it, flesh doesn't matter. The nice old man had a stroke, but before he died, he wanted you to take this belt buckle. Now go win that rodeo.
7. RETURN (and go back to where they started)
BRINGING IT HOME… It's not a journey if you never come back. The car chase. The big rescue. Coming home to your girlfriend with a rose. Leaping off the roof as the skyscraper explodes.
8. CHANGE (now capable of change)
MASTER OF BOTH WORLDS… The "you" from (1) is in charge of their situation again, but has now become a situation-changer. Life will never be the same. The Death Star is blown up. The couple is in love. Dr. Bloom's Time Belt is completed. Lorraine Bracco heads into the jungle with Sean Connery to "find some of those ants."