Private messages (PMs) is a new feature of the new software. It's sort of a forum based email, but instead of sending via email, it sends it via the forums. When you have unread (new) PMs, you'll get the pop-up notification. You can either click on the pop-up to go to your inbox, or go there from the bottom-left of each forum page as Duncan says. Once you've read all the messages in your inbox, the pop-up will go away.
Under each member's name and avatar on each post is a "Send Message" link. Use this to send PMs to other members.
The "Send Message" function doesn't work properly in the Mozilla Firefox browser (v4.0.1): it brings up the message box, but the recipient's name is not automatically filled in as it is with Internet Explorer. My browser of choice is Firefox, and I couldn't figure out what good the Send Message feature was if I still had to type in the name. I finally got curious and did the comparison with IE. Now I see . . .