The term originated with the points earned
for various achievements by the youngest group of the Girl Scouts,
called Brownies. Brownie points are imaginary points earned by
someone for doing a good deed, and lost by doing something bad.
The Girl Scout or Girl Guide Brownies took their name from the
mythical creature, the brownie. The mythical brownies were known
for being kind and helpful and performing household chores while a
family slept. The girl-brownies are supposed to emulate this
behavior, being quietly helpful without asking for much in return.
(Historical Dictionary of American Slang argues that Brownie
points, was US army slang from WWII.)
The slang brown-noser, seems to have first popped up around 1939.
It is defined in the 1944 edition of American Speech, as: "A
person who is always asking and answering questions in class to
impress the instructor. Also a person who stays after class to try
to insinuate himself into the teacher’s good graces." This hints
at the brown-noser / brownie points connection and how the
behavior of Brownies might have overlapped with that of
brown-nosers. However, it is American military slang and alludes
to the practice of 'brown nosing', or 'arse-licking'. The older
term brown-nose is used for a person who curries favor to such an extent that
his nose seems to be up his superior’s backside.