Ilie Dumitrescu also got in on the act, with his first goal for the club, as Tottenham played some breathtaking football, particularly early on, and were simply a class apart from Ipswich, who were beaten 3-1.
Klinsmann scored his first goal after 15 minutes, whipping Teddy Sheringham's low cross into the roof of the net. He turned provider for Dumitrescu after 28 minutes, and scored his second 10 minutes later before Chris Kiwomya pulled one back for Ipswich.
Tottenham's performance should have brought a greater reward, but it did send confidence soaring in the team's dressing-room. That is good news for the England coach, Terry Venables, with Darren Anderton and Sheringham due to report later this week for international duty against the United States.
Anderton said: 'We can go off to join England now in confident mood. Some day we will get six or seven goals, but until then I am happy just to settle for winning. We might have had half a dozen in the first half alone. After the break we had more chances but just couldn't take them.
'We created all sorts of problems. The five of us at the front are very different players, but the way we interchange seems to come naturally. We have only played together for four games and we can only get better.'
Tottenham's performance delighted their manager, Ossie Ardiles. He said: 'It was some of the purest football I have ever seen from a club side - the sort of football I dream about. We are a beautiful team going forward. I see no reason why we can't go out and entertain like this every week.'
At Goodison Park, Colin Cooper's overhead kick in the 61st minute lifted newly promoted Nottingham Forest to the top of the table. Earlier, Andy Hinchcliffe - attempting to turn the ball behind - headed the softest of own goals to put Everton in trouble.
Paul Rideout headed in Vinny Samways' cross to cut the deficit, but Forest held on despite Steve Chettle being sent off in the 86th minute.
Everton fans booed their team off the pitch, but Mike Walker tried to deflect the criticism that fell on his side, who have just one point from four matches. Not that the Everton manager gave the impression that he thought Everton had been outclassed.
Walker said: 'I wasn't unhappy with the performance. We played quite well and gave away two bad goals. You can't legislate for things like the own goal - I just don't know what he was doing - but we created plenty of chances and didn't get any luck.
'There wasn't a lack of effort and if we'd played like that at Manchester City on Saturday we might not have been beaten. But until we stop giving away bad goals it's going to be difficult.'
The arrival of the Nigerian striker Daniel Amokachi will give Everton more options going forward, but that is likely to be balanced by the loss of Tony Cottee, who is thought to be keen to move on after being left out of the team on Tuesday night after a dispute with Walker.
While the pressure was mounting on Walker, Frank Clark, the Forest manager, said: 'The start has given us the confidence we need in this division. A few of the lads had played at this level, but it was unknown territory for most of them. Now we know we've nothing to be afraid of.'
Clark said he was pleased that his side had created more clear-cut opportunities, but admitted that the slightly desperate defence of the last few minutes was a lesson for the future.
'We created lots of chances and perhaps we should have done better with them,' he said. 'I think we have got to be more ruthless in this league. You don't get as many chances at this level and your mistakes get punished. Instead of killing the game off, as we should have, we ended up hanging on.'
Noel Whelan scored his third goal in three games as Leeds inflicted the second home defeat of the season on Crystal Palace. The teenager nodded the ball over the line in the 63rd minute after the Palace goalkeeper Nigel Martyn had parried David Wetherall's close-range header from a corner.
Palace looked to have earned themselves a point earlier after Dean Gordon cancelled out David White's 10th- minute opener. Philomen Masinga, the South African striker, had another impressive game, forcing Martyn into a couple of fine saves.
Howard Wilkinson, the Leeds manager, said: 'He is still trying to come to terms with the pace of the game here or, more correctly, the sustained pace of it. But if he carries on like this I will be delighted.'
With Whelan and Masinga in such good form, the prospects look grim for the pounds 2m striker Brian Deane. And Leeds will soon have another South African player fighting for a place in the side.
Lucas Radebe, from Kaiser Chiefs, is ready to follow in the footsteps of Masinga, who moved from Mamelodi Sundowns for pounds 275,000. Leeds are paying a similar fee for Radebe, and have already secured his British work permit.
In the First Division, Tranmere's veteran striker John Aldridge scored a second-half hat-trick in nine minutes, including a penalty, in the 4-2 defeat of Luton.
Reading also scored four times, without reply, at home to Stoke, who had Vince Overson and Wayne Biggins sent off within minutes of each other in the second half.