Tottenham are searching for a ground-share after revealing their new stadium would not be ready by the time they have to vacate White Hart Lane - and the Olympic Stadium in Stratford is a possibility.
Other options include Wembley or stadium:mk, although both also present their own problems, and a situation where Spurs share with fierce rivals West Ham is now unlikely.
It was understood West Ham would be able to veto a potential move, even if Tottenham agreed a deal with the London Legacy Development Corporation, and Brady has confirmed they would be against sharing the venue.
"In reality they probably could - but only with our permission," she said when asked if another club could move in.
"No-one has asked us for our permission and if they did we would probably say no, depending on who it is - if you get my drift.
"We are the anchor tenant for the winter matches and nothing else can happen in that time without our permission and our football matches take priority over everything else.
"If there are events at the stadium we have approved we do not see any of the revenue from that."
Brady was talking at the launch of West Ham's new members' club - 'Club London' which offers fans the chance to experience what Brady labelled as an 'unrivalled hospitality experience'.
Season ticket holders and members will soon be able to visit the new Reservation Centre at Westfield Stratford City to hand-pick their new seats at the Olympic Stadium.
West Ham will leave their Upton Park home and take up a 99-year anchor residency at the showpiece venue from London's 2012 Olympic Games in time for the 2016-17 season.
For Tottenham, their future is less clear as at least one season away from their own new stadium adjacent to White Hart Lane means finding a temporary home.