"Don't worry about goblins." Of course Hermione could understand what Harry was saying, but she was more worried about Harry's attitude towards the invasion of Gringotts than about the goblins. Harry should be more confident in the plan they made. After all, the three of them will take the Felixir and be fully prepared. There is no need to worry about the failure of the plan. Even if the plan does fail, they can still rely on the preparations prepared in advance. The method is to escape unscathed.