Magic is a very diverse mechanic that can function like many things in the real world. I always think of it as volatile. Very very few understand it and those who do cannot truly explain how it works. Alchemy, the "science" of magic, is known in golarion primarily for bombs created by it which harness it with the precision of a rampaging elephant. Wizardry, the study of magic can define it by school, and outline what each school does, but trying to explain how leads to the glazed eyes of the commonfolk. Sorcery and Bardery(?), the art of magic, can explain it as well as an artist can explain inspiration, another artist or enthusiast understands and the rest politely nod to avoid offense.
So trust probably varies as wide as the field of magic itself does. If I manage a low scale tavern in the sketchy parts of Absalom, I don't care if the wizard casts a spell, he's probably one of the uncharismatic types cooling his cheap beer with ray of frost, and probably isn't more dangerous than the guy juggling knives for his dayjob.
If I am a rich and experienced trade prince dealing with a well renouned bars, I will be offended if he doesn't try magic to grease the wheels. I have money enough to protect from most harmful spells, and authority to arrest him, so am I not good enough to use one of his cantrips?
If I am nobility hosting a dinner party, I expect the highest nobles to have abjurations, glamors, and other such things up before they enter. The richest man in town doesn't show up to a party without a bodyguard, or show up in his pajamas. He also shouldn't change in the bathroom or pull his guard's weapons for show though, so the spells should be precast.
If I have a mundane job though, like farming, I disallow unexplained magic. In the seconds it takes to cast spark my entire livelihood could go from my most prosperous year yet, to crippling povery because of a madman casting spark to burn down my wheat fields or blow up the silo.