Add thrills
... okay, so what is a thrill?
First: it is not suspense.
Suspense is what you get when the outcome is uncertain, and you're worried about the fate of the protagonist. Alfred Hitchcock was known as the "master of suspense," and (IMHO) his most suspenseful movie was Torn Curtain, which is a movie that takes the concept of "out of the frying pan and into the fire" and daisy-chains it to an absurd degree. The hero has a problem, and then something happens to pause that problem and insert a bigger problem in front of it, and then that problem gets put on pause because of the intervention of another problem, and then that problem is also pushed to the side because an even more urgent problem has interposed itself; this repeats like a dozen times, with none of the problems having been resolved along the way, still looming in the background, each ready to destroy the hero's life. (It's a really good movie.)
Suspense can be an effective way to pull the reader along, but it does not a thriller make.
Second: it is not action or combat.
Buffy the Vampire Slayer had a lot of physical fighting, but it was not a thriller. Buffy was a high schooler who basically stayed in her hometown and skulked around at night. If you did all the things Buffy did except for staking demons, you would not feel the need to tell people: "OMG last night I walked past the cemetery." Anybody can walk past a cemetery, and they generally don't give it another thought. The same goes for being out-of-doors at night (except for little kids, who have early bedtimes).
Third: it is not high stakes.
In State and Main, Phillip Seymour Hoffman discovers that he's the only witness who can corroborate the testimony of a statutory rape, but his employer wants the rape charges to go away, so they pressure him to lie by offering him a successful career. But Hoffman has also fallen in love with Rebecca Pidgeon, and she is an honest person who is friends with the victim. So when he does lie in his deposition, she loses her respect for him. Afterward, he goes to the train station to leave town, a broken man, saying "I've ruined my entire life, and all in a moment." Because now he understands that the love of a good woman is worth more than a big movie career. He begs a stranger to wave his magic wand and turn back time so he can do it all over again the right way.
Those are high stakes: a lifetime of real happiness and companionship versus a lucrative career without the love. But the movie is not a thriller. It's an entirely realistic comedy of errors.
What is thrilling is when the characters do things that are extraordinarily exciting.
For example, in the beginning of The Matrix, before Neo is pulled out, there's that bit in the office where he has to climb out the office window and onto the ledge of a tall skyscraper. Imagine if you had to do that for some reason. That episode would definitely end up in your diary, and you'd probably tell your friends about it later. The 30 seconds you spent on the ledge would probably be the most exciting thing you do all year, even if the reason you did it was to grab a pen that had rolled out the window. The pen is irrelevant.
If you look up "thrill-seeking behavior," you'll commonly see mentions of extreme sports, use of hard drugs, exotic locales, and unusual modes of transport. Sky-diving, snowboarding, SCUBA diving, visiting rare and majestic sites (e.g. Taj Mahal, the Pyramids, Angkor Wat), that kind of thing.
Importantly, thrills are generally at least plausibly enjoyable. They are the kind of thing a person might at least fantasize about doing.
So, what might be thrilling in a supernatural story?
- Infiltrating a cult to get a look at their rituals and see if they have the "real" artifact vs just being posers
- Traveling to a famous place that everybody has always assumed was fictional, like Atlantis, or Xanadu (Coleridge's, not Newton-John's)
- Breaking into a Natural History Museum to rifle through their warehouse for a book or artifact, without being caught by the (mortal) guards
- Fleeing your hometown in the middle of the night with just the clothes on your back, because you've been marked for death and you want to lead your pursuers away from your family and friends (sneaking out at night is exciting!)
- Meeting with a bad guy in a public place to trade the mcguffin for a friend who they're holding hostage
These are all things that, if you do them right, they'd actually be kind of fun. That is not true of things like having a desperate hand-to-hand fight with a monster. (Which is not to say your story shouldn't have that, too. But it would not be the source of the thrill.)
What's "thrilling" is not the chance of failure, it's the actual doing of it. The doing is exciting. It would be thrilling even if success were guaranteed, because it's exciting to clamber on top of a train (Indiana Jones and the Last Crusade), or to charm your way into a stranger's home to prove you can be a spy (Spy Game), or to interview a dangerous psychopath (Silence of the Lambs), or to drive into the countryside at night with a friend to deliver ransom money (The Big Lebowski), or to have somebody you admire let you hot-rod around in their spaceship for a while (Titan A.E.).
A thrill is fun excitement.
One more word of advice: do not trust the "thriller" label that Hollywood puts on movies, or that booksellers put on books. When a commercial product has a label like "thriller" attached to it, they're just announcing whose money they want. They aren't actually sweating the details of getting the categorization right. All marketplace taxonomies are deliberately perverse, for reasons closely related to Goodhart's Law.