Lanterns were portable sources of light used to illuminate caves, tunnels, etc.
The first Grand-Master of the Templar Order in the Colonies Haytham Kenway and the Colonial Assassin Ratonhnhaké:ton used lanterns to navigate Freemason tunnels underneath Boston and New York, and to solve puzzles in order to unlock new fast-travel locations.
French Assassin Arno Dorian used lanterns to find his way around the caves underneath Saint-Denis, clearing the way of bats and insects. If Arno were to fall into water, his lantern would be disabled until he dipped it in oil.
Arno also used the Head of Saint-Denis as a lantern, which didn't have to be dipped in oil.