The ego cannot understand the path of joy.
Therefore, the more you are ego identified, the more difficult it is for you to follow the path of joy.
Therefore also, if you are on the path of joy, the more you identify with your ego, the more confused you become and thus suffer more. It is like keeping feet in two boats at the same time.