The Premier League champions last week released video footage of Kompany informing his team-mates in an emotional speech after the FA Cup final victory over Watford.
Kompany, City's inspirational four-time title-winning captain, left the club in May to take up a role as player manager at Anderlecht.
He leaves a huge void as both a player and a captain after 11 years at the Etihad Stadium.
"I was crying to be honest because nobody was waiting for that," said Zinchenko.
"But that's football, nobody knows what could happen."
In his speech, Kompany urged his colleagues to go on to win the Champions League, the one trophy that now eludes the club and the current group, who won a domestic treble last term.
Zinchenko said: "Nobody knew and we were upset about the Champions League because we deserved more."
Zinchenko, 22, is primarily an attacking midfielder but has successfully reinvented himself as a left-back over the last two seasons to force his way into the side.
With no natural replacement for Benjamin Mendy, who has been hampered by injuries in that period, manager Pep Guardiola has needed to find alternatives.
Zinchenko partly puts his success in the role, particularly in the second half of last season, down to the influence of Kompany.
"I think it's good for all football players, when you can play different positions," said the Ukraine international, speaking in Shanghai during City's pre-season tour.
"I cannot tell you my best position exactly. In the national team I play in the middle, in Manchester City at left-back, so I feel comfortable both sides.
"I'm still working on it, I'm still learning a lot. Last season Vincent was helping me a lot. Outside the pitch, inside the pitch, about this position.
"I remember a few times he took me after some training sessions. He was working with me for around 40 minutes or something to try and help me.
"For me he is absolutely a life legend. I am so lucky. He can be a manager, why not? He is the smartest guy I know."
Zinchenko could begin the 2019-20 season next month as first-choice left-back. Mendy is yet to return to training as he continues knee treatment in Barcelona and Angelino has only just rejoined the club after a year at PSV Eindhoven.
Zinchenko said: "That's why everyone is here. Everyone is waiting for the chance to show their level and to be number one.
"But every single day you have to show your level in the training sessions and the games and you have to fight for your place."
Zinchenko was being linked with moves elsewhere last summer but such has been his development since that he signed a new five-year contract in June.
He said: "It's a massive pleasure to be part of this club. It was a dream, to be part of the squad, to be part of this huge family. I was dreaming about that."
Oleksandr Zinchenko was speaking at the Manchester City fanzone at PUMA's flagship store in Shanghai, during the club's pre-season tour of Asia. PA
Join our new commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies