In the realm of matter, one and the same object can serve as a cause of happiness for some living beings, and a cause of suffering for others. Certain plants, for example, function as medicine for some creatures, but for other species they can be poisonous. From the point of view of the object itself there is no difference, but because of the physical constitution and the material state of the particular living being, that single self-same object can affect them in different ways. Then, in the sphere of our own experiences, the same holds true. A certain individual may appear to some as very friendly, kind and gentle, and so gives them feelings of happiness and pleasure. Yet to others that same person can appear harmful and wicked, and so cause them discomfort and unhappiness.
What this kind of example points to is that, although external matter may act as a cause for our experience of pain and pleasure, the principal cause that determines whether we experience happiness or suffering lies within. This is the reason why, when Buddha identified the origin of suffering, he pointed within and not outside, because he knew that the principal causes of our suffering are our own negative emotions and the actions they drive us to do.---Dzogchen: The Heart Essence of the Great Perfection