For your workload to become realistic and achievable (assuming you are proficient at your job), you need to feel able to have honest, frank conversations with your manager. So, the RELATIONSHIP BETWEEN YOU is the most important factor

Remember that your colleagues are human beings too. Yes, even your manager, and the business owners! In the counselling world, there is a term called ‘Splitting’, which describes our tendency to only see certain parts of other people’s characters (and our own).