Sri Lanka departed for Bangladesh without fielding coach Nic Pothas after he refused to be a part of the touring team over a dispute with the management.