Here are the rules I use...
Retreat: Your hero may flee before combat starts, if it looks like he may lose. Make a note of the entry, and return to the map. (This covers the moments where you realize that you're in over your head.)
Knocked-out: If your hero loses a combat, all his consumable items are destroyed. Make a note of the entry, and return to the map. (This adds consequence to failure, but not so much that your hero is gimped. -It also shows that your hero gave it his all by using every consumable.)
Legendary monsters exception: The hero may retreat from a legendary monster. But if your hero enters combat and is defeated, he is knocked-out and may not attempt to defeat the same legendary monster again. (This makes legendary monsters a high-stakes affair where winning grants your hero an exciting reward.)
Other exceptions: If the entry specifies that quest is over when the hero is defeated, then the hero cannot retreat. (Some quests are over if a combat is lost. Retreat undermines the spirit of the rules in this case.)
These rules have increased the tension for me, and added an element of strategy where I need to evaluate my opponent before committing to combat.