I currently make flowcharts with mostly questions and main talks of an NPC. Also I mentally create a main personality of that NPC but if I have many NPCs to remember I typify their main characteristics and personality traits. This helps me a lot.
Notes, often with a distinct trait or tendency.*
And then add as improvisation takes you different directions. :)
Maybe the one unusual thing I do is write a distinct quote from the NPC, one that I can hear them saying (if possible, to the PCs). That tends to trigger whatever else I've developed for them.
Speaking of developing, add something unique that works as a trigger for the players to, whether it's a catchphrase, a physical trait, or an expression, i.e. that guard that likes blue a bit much or that innkeeper that always begins his stories the same way, etc. These won't necessarily be as helpful for you, but aid the players a bunch. "Ahh, Mr. Blue".
Often what seemed unimportant to me will come up again because it stood out to the players. :)
*There are plenty of lists for NPC flavor (sometimes w/ random rolling) out there, especially in older RPGing material or even material for authors, actors, etc.