The Bees are unbeaten so far on their return to the top flight, having built on the home victory over Arsenal with successive away draws.
Potter’s men, though, have also made a positive start, with six points from their three matches.
After finishing 16th last season, Brighton have a view on pushing up towards the top half of the table.
To do that, however, Potter knows it will take more than just getting maximum points against the sides expected to be scrapping for their own survival.
“I don’t see any like that because every game in the Premier League is so difficult and so complicated,” said Potter.
“Brentford have come up and have a mentality and a belief in what they are doing.
“There is going to be a passionate home crowd. They have got players who deserve to be here, they have earned the right to play in the Premier League.
“We as a club, as a team, have no right to look at any opposition and think ‘this is a must win for us’.
“Every game is an opportunity to win, but it is also an opportunity to lose as well if you don’t play well and get the rub of the green.
“That is why the competition is so good – you can’t on the one hand say the Premier League is the best league in the world and then say there are some easy games in there because there are not.”
Potter, who could hand new signing Marc Cucurella his debut after the Spanish defender joined from Getafe ahead of the transfer deadline, added: “Everybody is competent. It is as simple as that.
“Everybody knows what they are doing with good players who are well organised, but we are as well.
“We have got some qualities ourselves, and it’s important that we bring that to the game.
“We are disappointed after (losing at home against) Everton, because we didn’t get to our level.
“I have said before if you don’t hit your level in the Premier League, then it is tough. That is the reality.”
Potter has been impressed with the way Brentford have taken to life back in the top flight for the first time in 74 years, continuing momentum from their promotion via the play-offs.
“I know Thomas (Frank) from my time at Swansea and I have always found them very tough to play,” said Potter.
“They have changed a little bit in terms of their formation and how they play, but they’re still the same things, like with a great organisation.
“They have done a fantastic job and are really hard to play against, hard to score against, the back five do that well and have got some clear attacking ideas.”
Potter added: “They have earned the right to be here and will compete
“They will attack the game, won’t be shy and will try to be the best version of themselves that they can be.
“We have to be really good in the game and we will go there full of respect for our opponent.”