Well... Actually the starting user Karma is 0! - Yours is 4 because users have gave you 4 points (you did start with 0), and yes users "add" to another users Karma if they like what they read (what comes out from a certain user), or if users dont like what they read, they deduct a point from another users karma.
Hope that helps
