Hussey (73) and Pattinson (32) helped push the tourists, who are 1-0 up in the three-match series, to 311 all out as the West Indies finished the day on 49 for three, still trailing by 262 runs at Queen's Park Oval.
But Kemar Roach insisted the West Indies were still very much in the match.
"We're big boys. We're big men," said the seamer.
"So we're a bit behind, but so it goes in cricket. We can come back. We will come back tomorrow and play cricket the way we want to play cricket."
Hussey top-scored for Australia before falling to Narsingh Deonarine.
Rain arrived shortly before lunch, just after Hussey had brought up his fifty, and Australia added 59 runs for the loss of one wicket before lunch to take the score to 267 for six.
The showers became heavier and another 100 minutes were lost which will mean early starts for the remainder of the Test.
Hussey admitted the wicket was hard to bat on.
"The odd one is spinning a lot. The odd one is staying low. The odd one is bouncing a bit. So, again, you can't trust the conditions, you can't trust the pitch to go through with your shots," he said.
After the extended break, the West Indian team appeared to be getting more and more frustrated as Hussey and Pattinson were batting with confidence although deliveries were still regularly passing the edge.
"I think the times I got most angry with him was when he started to chase the ball and went for those big wild ones outside off stump," said Hussey of what it was like watching Pattinson battle at the other end.
"But while he was playing nice and straight and not deviating from his line I was more than happy with the way he was playing. I thought he did a fantastic job."
With the score on 297 the breakthrough finally came. It surprised everyone when Hussey didn't move his feet to a Deonarine delivery and simply slapped it to short extra cover where Kraigg Brathwaite held the chance.
Like Deonarine's other victim, Michael Clarke, Hussey couldn't believe the shot he had played. He had batted for over four hours in another fine innings.
His seventh wicket partnership with Pattinson had been worth 89 runs but the fall of the wicket opened up the tail.
As so often seems to happen, when one falls the other batsmen goes too. Just five balls later, without further addition to the score, Pattinson got a top-edge to a ball from Shane Shillingford. Despite confusion between the fielders, Darren Bravo took the skied chance.
Ben Hilfenhaus had survived a DRS lbw decision when Shillingford struck his pads. In the next over he played a lovely cut shot off Roach to the boundary but next ball his stumps were knocked back. The ball rebounded off his legs and Australia had lost their ninth wicket.
Roach hit Michael Beer on the pads and two balls later he was given out by South African umpire Marais Erasmus. The decision stood despite the batsman using the review system.
The West Indies seamer had taken a five-wicket haul for the third time in his career.
Clarke pulled a surprise move at the start of the West Indies innings by opening the bowling with left-arm off-spinner Beer.
Hilfenhaus was operating from the other end. The new ball and humid conditions allowed him to find prodigious swing.
To try and combat the movement Brathwaite was moving across his crease.
In his second over, Hilfenhaus got one to miss the inside edge and strike him on the pad. Umpire Ian Gould raised his finger and, despite Brathwaite's review, replays supported the umpire's decision.
It was consecutive ducks for Brathwaite who had failed to score for the fifth time in his fifteen Test innings.
Adrian Barath received ironic cheers from the crowd when he finally got off the mark from the 25th ball that he faced.
He never settled and Clarke's bold move to open with spin paid off in Beer's seventh over. Barath didn't pick the arm ball and was caught in front of the stumps for seven.
West Indies had lost both of their openers with only 26 on the board.
Surprisingly, Pattinson was the fourth bowler to be used by Clarke. He then struck with his very first ball.
He trapped Kieran Powell in front and it was the third lbw dismissal of the innings. He was out for 19 but replays showed that the ball had pitched just outside the line of leg stump.
Bravo (16) and Shivranine Chanderpaul (1) saw out the day.
