King Harald's manipulation started way before Loki in One Piece. Big Mom's flashback showed that as early as 63 years ago, Harald was being manipulated by people such as Mother Caramel.