The Tigers took control with Ellison's first shortly before half-time and sealed a hard-earned victory with his second 10 minutes into the second half.
Castleford made a flying start when stand-off Danny Orr jinked through cleverly for a third-minute try which he also converted. But Orr was injured as he scored and was taken off seven minutes later, replaced by Spencer Hargrave.
Huddersfield rallied and the Tigers were subjected to sustained pressure thanks to some accurate kicks from the Giants scrum-half, Bobbie Goulding.
The tactic paid off when Goulding's towering kick created a 23rd-minute try for winger Darren Simpson, which the half-back converted from the touchline.
Scrum-half Aaron Raper edged the Tigers back in front with a penalty goal, and the home side soon increased their lead with Ellison's first score.
Gael Tallec and Jason Flowers linked to put the wing man over, and Raper landed his second penalty on the stroke of half-time to make it 14-6.
Castleford went further in front after 50 minutes when Raper's kick led to Ellison collecting his second. A drop-goal from the full-back, Flowers, nosed them further ahead.
Huddersfield had the last word when Goulding and Andrew Tangata-Toa combined to put centre Jim Lenihan over seven minutes from time, but the score came too late.
"With a small squad of 21 it is really a case of all hands on deck," the Castleford coach, Stuart Raper, said after the match. "Our performance showed how professional our guys are. It was a very gutsy display."
His Huddersfield counterpart, Malcolm Reilly, admitted: "Castleford were quicker on the floor than us. We also missed some reasonable scoring opportunities.
"The only positive I can take is that we have no more injuries. We have to regroup.
"We had no creation and relied too heavily on Bobbie Goulding's kicking game."
Castleford: Flowers; Wells, Rogers, Eagar, Ellison; Orr, Raper; Sampson, Hepi, Sykes, Hill, Tallec, Fritz. Substitutes used: Lynch, Dobson, P Smith, Hargraves.
Huddersfield: Reilly; Simpson, Lenihan, Loughlin, Cheetham; Weston, Goulding; Fozzard, Hudson, Neill, Boughton, Storm, Tangata-Toa. Substitutes used: Bentley, Richards, Carlton, Pickavance.
Referee: S Cummings (Widnes).Reuse content