Substitute Jonny Williams marked his 25th Wales appearance with his first goal, sweeping home Neco Williams' near-post cross five minutes from time.
The win was achieved without the stellar cast list of Gareth Bale, Aaron Ramsey and Joe Allen, while other key figures such as David Brooks, Joe Morrell and Kieffer Moore were also absent.
"I'm so proud of the players, it was a great performance," Wales manager Giggs said after watching his side score a late group winner for the third time in four matches.
"The players have got to keep believing in the way we play and that's not easy.
"These young players have not experienced it before, but they kept their concentration and players off the bench made an impact.
"I've said before it's just that final bit of composure. We saw it in the second half when we could have scored more goals, but we'll learn.
"It's a young team and the more you get in those positions the better your decision-making becomes.
"It's been a tough week. Three away games, four different hotels, three Covid tests and a lot of suspensions and injuries.
"We've had a lot to deal with as a group, but I was so proud of the players."
Although beaten by England in a Wembley friendly last week, Wales have not lost a competitive game since June 2019.
It is a run that has included six consecutive clean sheets, with Wales not conceding a goal in over 10 hours of football.
"We're proud of the defensive aspect and that's the team, not just the defenders or the goalkeeper," Giggs said.
"We want to press from the front and be aggressive, and we're doing that.
"The players put a real shift in. They've looked after themselves this week.
"I'm really pleased with the result, but we also played some of the best football I've seen in a long time."
Finland had taken top spot in Group B4 before Wales kicked off in Sofia by beating the Republic of Ireland 1-0 at home.
But Wales' promotion hopes to League A are firmly in their own hands with closing home games against the Republic and Finland next month.
"We've put ourselves in a good position with two home games, but we can't take it for granted," Giggs said.
"We can't be complacent. We've got to keep going and keep believing. Hopefully, we will also have a few bodies back for the November games."
Join our new commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies