This was originally one of the numerous
names given to ignis fatuus (Medieval Latin for “foolish fire”),
another of which is “Will O’ the Wisps”, basically the odd light
that can occasionally be seen over marshes, swamps, etc.
When you see someone carrying a lantern in a distance at night you
see is a man, but you can’t make out who exactly it is, he is
literally “man with a lantern”, a.k.a. “Jack of the Lantern” or
“Jack O’ Lantern.” This was also commonly used for a nickname for
night watchmen.
“Jack O’ Lantern” first popped up in the mid-17th century in East
Anglia, UK and spread from there through parts of England, Ireland,
and Scotland. The name likely originally derived from the practice
of calling men generically “Dick, Jack, Tom, etc.” In particular,
men who were lower class, were often called generically “Jack”
beginning around the 14th century in England.
How this name made the jump to referring to carved pumpkins with
lights inside, it has its origins in the Celtic practice of
hollowing out and carving faces into turnips and other vegetables
during Samuin (a festival where many of the traditions of Halloween
come from). After carving the vegetables, they placed candles inside
and put them in windows or carried the make-shift lanterns with them
as they walked to ward off evil spirits.
In Britain, pranksters would make these types of carved lanterns to
scare people on the road or children would carry them around during
Hallowmas while begging for soul cakes.