Hitman David Bingham bagged two goals to take Gretna to the top of Division Three and make himself the division's top scorer with 16 goals.
The first-ever capacity crowd of 2,200 saw Gretna leapfrog Peterhead at the top of the table, but for a long time the Blue Toon threatened to rain on owner Brookes Mileson's 57th birthday parade.
David Irons had cleared a Robert Linn shot off the line before Bingham got the Borderers goal, slamming the ball into the roof of the net after Paul Mathers could only push Kenny Deuchar's header into his path.
Mathers had done better in the 13th minute, blocking a Ryan Baldacchino shot with his legs, and Robert Raeside made a last-gasp challenge to deny the Gretna winger three minutes later.
But Peterhead weren't going to surrender pole position easily, and Martin Bavidge forced David Mathieson to make a brave save at his feet in the 17th minute.
Peterhead drew level from the penalty spot after out-of-sorts Andy Aitken brought down Linn inside the area. David Hagen made no mistake from the spot, beating Mathieson low to his left.
Gretna looked to have got a second goal when Deuchar played in Bingham, but Mathers saved with his feet from eight yards.
Peterhead came close themselves. Linn had a shot on the turn well saved by Mathieson on the hour,who fired just wide minutes later.
The keeper kept him out again in the 70th minute, but Jamie Buchan should have scored from the rebound but blazed his shot over the bar.
Bingham's winner came in the 88th minute. Deuchar forced a corner and Irons rose the highest to flick Gavin Skelton's in-swinging kick on to the head of Bingham to finish from close range.
Gretna boss Rowan Alexander admitted: "We stole it at the end."